* [dpdk-dev] [PATCH v2 1/7] member: implement main API
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
@ 2017-09-02 1:24 ` Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 2/7] member: implement HT mode Yipeng Wang
` (6 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-02 1:24 UTC (permalink / raw)
To: dev
Cc: stephen, luca.boccassi, charlie.tai, sameh.gobriel, ren.wang,
pablo.de.lara.guarch, yipeng1.wang
Membership library is an extension and generalization of a traditional
filter (for example Bloom Filter) structure. In general, the Membership
library is a data structure that provides a "set-summary" and responds
to set-membership queries of whether a certain element belongs to a
set(s). A membership test for an element will return the set this element
belongs to or not-found if the element is never inserted into the
set-summary.
The results of the membership test is not 100% accurate. Certain
false positive or false negative probability could exist. However,
comparing to a "full-blown" complete list of elements, a "set-summary"
is memory efficient and fast on lookup.
This patch add the main API definition.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/Makefile | 2 +
lib/librte_eal/common/eal_common_log.c | 1 +
lib/librte_eal/common/include/rte_log.h | 1 +
lib/librte_member/Makefile | 49 +++
lib/librte_member/rte_member.c | 342 ++++++++++++++++++++
lib/librte_member/rte_member.h | 518 +++++++++++++++++++++++++++++++
lib/librte_member/rte_member_version.map | 16 +
7 files changed, 929 insertions(+)
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_version.map
diff --git a/lib/Makefile b/lib/Makefile
index 86caba1..c82033a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -108,6 +108,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder
DEPDIRS-librte_reorder := librte_eal librte_mempool librte_mbuf
DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump
DEPDIRS-librte_pdump := librte_eal librte_mempool librte_mbuf librte_ether
+DIRS-$(CONFIG_RTE_LIBRTE_MEMBER) += librte_member
+DEPDIRS-librte_member := librte_eal librte_hash
ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
diff --git a/lib/librte_eal/common/eal_common_log.c b/lib/librte_eal/common/eal_common_log.c
index 0e3b932..90db6a3 100644
--- a/lib/librte_eal/common/eal_common_log.c
+++ b/lib/librte_eal/common/eal_common_log.c
@@ -279,6 +279,7 @@ static const struct logtype logtype_strings[] = {
{RTE_LOGTYPE_CRYPTODEV, "cryptodev"},
{RTE_LOGTYPE_EFD, "efd"},
{RTE_LOGTYPE_EVENTDEV, "eventdev"},
+ {RTE_LOGTYPE_MEMBER, "member"},
{RTE_LOGTYPE_USER1, "user1"},
{RTE_LOGTYPE_USER2, "user2"},
{RTE_LOGTYPE_USER3, "user3"},
diff --git a/lib/librte_eal/common/include/rte_log.h b/lib/librte_eal/common/include/rte_log.h
index ec8dba7..ce1a3d0 100644
--- a/lib/librte_eal/common/include/rte_log.h
+++ b/lib/librte_eal/common/include/rte_log.h
@@ -87,6 +87,7 @@ extern struct rte_logs rte_logs;
#define RTE_LOGTYPE_CRYPTODEV 17 /**< Log related to cryptodev. */
#define RTE_LOGTYPE_EFD 18 /**< Log related to EFD. */
#define RTE_LOGTYPE_EVENTDEV 19 /**< Log related to eventdev. */
+#define RTE_LOGTYPE_MEMBER 20 /**< Log related to membership. */
/* these log types can be used in an application */
#define RTE_LOGTYPE_USER1 24 /**< User-defined log type 1. */
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
new file mode 100644
index 0000000..1a79eaa
--- /dev/null
+++ b/lib/librte_member/Makefile
@@ -0,0 +1,49 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_member.a
+
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
+
+EXPORT_MAP := rte_member_version.map
+
+LIBABIVER := 1
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+# install includes
+SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_member/rte_member.c b/lib/librte_member/rte_member.c
new file mode 100644
index 0000000..71b066d
--- /dev/null
+++ b/lib/librte_member/rte_member.c
@@ -0,0 +1,342 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+
+#include <rte_eal.h>
+#include <rte_eal_memconfig.h>
+#include <rte_memory.h>
+#include <rte_memzone.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+#include "rte_member_vbf.h"
+
+TAILQ_HEAD(rte_member_list, rte_tailq_entry);
+static struct rte_tailq_elem rte_member_tailq = {
+ .name = "RTE_MEMBER",
+};
+EAL_REGISTER_TAILQ(rte_member_tailq)
+
+
+void *
+rte_member_find_existing(const char *name)
+{
+ struct rte_member_setsum *setsum = NULL;
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(name, setsum->name, RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ if (te == NULL) {
+ rte_errno = ENOENT;
+ return NULL;
+ }
+ return setsum;
+}
+
+void
+rte_member_free(void *ss)
+{
+ struct rte_member_setsum *setsum;
+ struct rte_member_list *member_list = NULL;
+ struct rte_tailq_entry *te;
+
+ if (ss == NULL)
+ return;
+ setsum = ss;
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ if (te->data == ss)
+ break;
+ }
+ if (te == NULL) {
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return;
+ }
+ TAILQ_REMOVE(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_free_ht(setsum);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_free_vbf(setsum);
+ break;
+ default:
+ break;
+ }
+ rte_free(setsum);
+ rte_free(te);
+}
+
+
+void *
+rte_member_create(const struct rte_member_parameters *params)
+{
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list = NULL;
+ struct rte_member_setsum *setsum = NULL;
+ int ret;
+
+ if (params == NULL) {
+ rte_errno = EINVAL;
+ return NULL;
+ }
+
+ if ((params->key_len == 0)) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER,
+ "Memship create with invalid parameters\n");
+ return NULL;
+ }
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(params->name, setsum->name,
+ RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ setsum = NULL;
+ if (te != NULL) {
+ rte_errno = EEXIST;
+ te = NULL;
+ goto error_unlock_exit;
+ }
+ te = rte_zmalloc("MEMBER_TAILQ_ENTRY", sizeof(*te), 0);
+ if (te == NULL) {
+ RTE_LOG(ERR, MEMBER, "tailq entry allocation failed\n");
+ goto error_unlock_exit;
+ }
+
+ /* Create a new setsum structure */
+ setsum = (struct rte_member_setsum *) rte_zmalloc_socket(params->name,
+ sizeof(struct rte_member_setsum), RTE_CACHE_LINE_SIZE,
+ params->socket_id);
+ if (setsum == NULL) {
+ RTE_LOG(ERR, MEMBER, "Create setsummary failed\n");
+ goto error_unlock_exit;
+ }
+ setsum->type = params->type;
+ setsum->socket_id = params->socket_id;
+ setsum->key_len = params->key_len;
+ setsum->num_set = params->num_set;
+ setsum->name = params->name;
+ setsum->prim_hash_seed = params->prim_hash_seed;
+ setsum->sec_hash_seed = params->sec_hash_seed;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ ret = rte_member_create_ht(setsum, params);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ ret = rte_member_create_vbf(setsum, params);
+ break;
+ default:
+ goto error_unlock_exit;
+ }
+ if (ret < 0)
+ goto error_unlock_exit;
+ RTE_LOG(DEBUG, MEMBER, "Creating a setsummary table with mode %u\n",
+ setsum->type);
+
+ te->data = (void *)setsum;
+ TAILQ_INSERT_TAIL(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return setsum;
+
+error_unlock_exit:
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ rte_member_free(setsum);
+ return NULL;
+}
+
+
+int
+rte_member_add(const void *ss, const void *key, MEMBER_SET_TYPE set_id)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_add_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_add_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_lookup(const void *ss, const void *key,
+ MEMBER_SET_TYPE *set_id)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_lookup_bulk(const void *ss, const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || keys == NULL || set_ids == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_bulk_ht(setsum, keys, num_keys,
+ set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_bulk_vbf(setsum, keys, num_keys,
+ set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi(const void *ss, const void *key,
+ uint32_t match_per_key, MEMBER_SET_TYPE *set_id)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_ht(setsum, key, match_per_key,
+ set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_vbf(setsum, key, match_per_key,
+ set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_lookup_multi_bulk(const void *ss, const void **keys,
+ uint32_t num_keys, uint32_t max_match_per_key,
+ uint32_t *match_count, MEMBER_SET_TYPE *set_ids)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || keys == NULL || set_ids == NULL ||
+ match_count == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_bulk_ht(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_bulk_vbf(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_delete(void *ss, const void *key, MEMBER_SET_TYPE set_id)
+{
+ struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_delete_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ default:
+ return -EINVAL;
+ }
+}
+
+
+void
+rte_member_reset(const void *ss)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL)
+ return;
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_reset_ht(setsum);
+ return;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_reset_vbf(setsum);
+ return;
+ default:
+ return;
+ }
+}
diff --git a/lib/librte_member/rte_member.h b/lib/librte_member/rte_member.h
new file mode 100644
index 0000000..de44b1b
--- /dev/null
+++ b/lib/librte_member/rte_member.h
@@ -0,0 +1,518 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+
+
+/**
+ * @file
+ *
+ * RTE Membership Library
+ *
+ * The Membership Library is an extension and generalization of a traditional
+ * filter (for example Bloom Filter) structure that has multiple usages in a
+ * variety of workloads and applications. The library is used to test if a key
+ * belongs to certain sets. Two types of such "set-summary" structures are
+ * implemented: hash-table based (HT) and vector bloom filter (vBF). For HT
+ * setsummary, two subtype or modes are available, cache and non-cache modes.
+ * The table below summarize some properties of the different implementations.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ */
+
+
+/**
+ * <!--
+ * +==========+=====================+================+=========================+
+ * | type | vbf | HT-cache | HT-non-cache |
+ * +==========+=====================+==========================================+
+ * |structure | bloom-filter array | hash-table like without storing key |
+ * +----------+---------------------+------------------------------------------+
+ * |set id | limited by bf count | [1, 0x7fff] |
+ * | | up to 32. | |
+ * +----------+---------------------+------------------------------------------+
+ * |usages & | small set range, | can delete, | cache most recent keys, |
+ * |properties| user-specified | big set range, | have both false-positive|
+ * | | false-positive rate,| small false | and false-negative |
+ * | | no deletion support.| positive depend| depend on table size, |
+ * | | | on table size, | automatic overwritten. |
+ * | | | new key does | |
+ * | | | not overwrite | |
+ * | | | existing key. | |
+ * +----------+---------------------+----------------+-------------------------+
+ * -->
+ */
+
+
+#ifndef _RTE_MEMBER_H_
+#define _RTE_MEMBER_H_
+
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+#include <stdio.h>
+#include <stdint.h>
+#include <rte_hash_crc.h>
+
+/** The set ID type that stored internally in hash table based set summary. */
+typedef uint16_t MEMBER_SET_TYPE;
+/** Invalid set ID used to mean no match found. */
+#define RTE_MEMBER_NO_MATCH 0
+/** Maximum size of hash table that can be created. */
+#define RTE_MEMBER_ENTRIES_MAX (1 << 30)
+/** Maximum number of keys that can be searched as a bulk */
+#define RTE_MEMBER_LOOKUP_BULK_MAX 64
+/** Entry count per bucket in hash table based mode. */
+#define RTE_MEMBER_BUCKET_ENTRIES 16
+/** Maximum number of characters in setsum name. */
+#define RTE_MEMBER_NAMESIZE 32
+
+/** @internal Primary hash function to calculate 1st independent hash. */
+#define MEMBER_PRIM_HASH(key, key_len, seed) \
+ (uint32_t)(rte_hash_crc(key, key_len, seed))
+/** @internal Secondary hash function to calculate 2nd independent hash. */
+#define MEMBER_SEC_HASH(key, key_len, seed) \
+ (uint32_t)(rte_hash_crc(key, key_len, seed))
+
+
+struct rte_member_setsum;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameter struct used to create set summary
+ */
+struct rte_member_parameters;
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Define different set summary types
+ */
+enum rte_member_setsum_type {
+ RTE_MEMBER_TYPE_HT = 0, /**< Hash table based set summary. */
+ RTE_MEMBER_TYPE_VBF, /**< vector of bloom filters. */
+ RTE_MEMBER_NUM_TYPE
+};
+
+/** @internal compare function for different arch. */
+enum rte_member_sig_compare_function {
+ RTE_MEMBER_COMPARE_SCALAR = 0,
+ RTE_MEMBER_COMPARE_AVX2,
+ RTE_MEMBER_COMPARE_NUM
+};
+
+/** @internal setsummary structure. */
+struct rte_member_setsum {
+ enum rte_member_setsum_type type;
+ const char *name;
+ uint32_t key_len;
+ uint32_t socket_id; /* NUMA Socket ID for memory. */
+ uint32_t prim_hash_seed;
+ uint32_t sec_hash_seed;
+
+
+ /* hash table based */
+ uint32_t bucket_cnt;
+ uint32_t bucket_mask;
+ uint8_t cache;
+ enum rte_member_sig_compare_function sig_cmp_fn;
+
+ /* vector bloom filter*/
+ uint32_t num_set;
+ uint32_t bits;
+ uint32_t bit_mask;
+ uint32_t num_hashes;
+ void *table;
+};
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameters used when create the set summary table. Currently user can
+ * specify two types of setsummary: HT based and vBF. For HT based, user can
+ * specify cache or non-cache mode. Here is a table to describe some differences
+ *
+ */
+struct rte_member_parameters {
+ const char *name; /**< Name of the hash. */
+
+ /**
+ * User to specify the type of the setsummary from one of
+ * rte_member_setsum_type.
+ *
+ * HT based setsummary is implemented like a hash table. User should use
+ * this type when there are many sets.
+ *
+ * vBF setsummary is a vector of bloom filters. It is used when number
+ * of sets is not big (less than 32 for current implementation).
+ */
+ enum rte_member_setsum_type type;
+ /**
+ * If it is HT based setsummary, user to specify the subtype or mode
+ * of the setsummary. It could be cache, or non-cache mode.
+ * Set iscache to be 1 if to use as cache mode.
+ *
+ * For cache mode, keys can be evicted out of the HT setsummary. Keys
+ * with the same signature and map to the same bucket
+ * will overwrite each other in the setsummary table.
+ * This mode is useful for the case that the set-summary only
+ * needs to keep record of the recently inserted keys. Both
+ * false-negative and false-positive could happen.
+ *
+ * For non-cache mode, keys cannot be evicted out of the cache. So for
+ * this mode the setsummary will become full eventually. Keys with the
+ * same signature but map to the same bucket will still occupy multiple
+ * entries. This mode does not give false-negative result.
+ */
+ uint8_t iscache;
+
+ /**
+ * For HT setsummary, num_keys equals to the number of entries of the
+ * table. When the number of keys that inserted to the HT setsummary
+ * approaches this number, eviction could happen. For cache mode,
+ * keys could be evicted out of the table. For non-cache mode, keys will
+ * be evicted to other buckets like cuckoo hash. The table will also
+ * likely to become full before the number of inserted keys equal to the
+ * total number of entries.
+ *
+ * For vBF, num_keys equal to the expected number of keys that will
+ * be inserted into the vBF. The implementation assumes the keys are
+ * evenly distributed to each BF in vBF. This is used to calculate the
+ * number of bits we need for each BF. User does not specify the size of
+ * each BF directly because the optimal size depends on the num_keys
+ * and false positive rate.
+ */
+ uint32_t num_keys;
+
+
+ /**
+ * The length of key is used for hash calculation. Since key is not
+ * stored in set-summary, large key does not require more memory space.
+ */
+ uint32_t key_len;
+
+
+ /**
+ * num_set is only relevant to vBF based setsummary.
+ * num_set is equal to the number of BFs in vBF. For current
+ * implementation, it only supports 1,2,4,8,16,32 BFs in one vBF set
+ * summary. If other number of sets are needed, for example 5, the user
+ * should allocate the minimum available value that larger than 5,
+ * which is 8.
+ */
+ uint32_t num_set;
+
+ /**
+ * false_postive_rate is only relevant to vBF based setsummary.
+ * false_postivie_rate is the user-defined false positive rate
+ * given expected number of inserted keys (num_keys). It is used to
+ * calculate the total number of bits for each BF, and the number of
+ * hash values used during lookup and insertion. For details please
+ * refer to vBF implementation and membership library documentation.
+ *
+ * HT setsummary's false positive rate is in the order of:
+ * false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit signature.
+ * This is because two keys needs to map to same bucket and same
+ * signature to have a collision (false positive). bucket_count is equal
+ * to number of entries (num_keys) divided by entry count per bucket
+ * (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_postivie_rate is not
+ * directly set by users.
+ */
+ float false_positive_rate;
+
+ /**
+ * We use two seeds to calculate two independent hashes for each key.
+ *
+ * For HT type, one hash is used as signature, and the other is used
+ * for bucket location.
+ * For vBF type, these two hashes and their combinations are used as
+ * hash locations to index the bit array.
+ */
+ uint32_t prim_hash_seed;
+
+ /**
+ * The secondary seed should be a different value from the primary seed.
+ */
+ uint32_t sec_hash_seed;
+
+ int socket_id; /**< NUMA Socket ID for memory. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Find an existing set-summary and return a pointer to it.
+ *
+ * @param name
+ * Name of the set-summary
+ * @return
+ * Pointer to the set-summary or NULL if object not found
+ * with rte_errno set appropriately. Possible rte_errno values include:
+ * - ENOENT - value not available for return
+ */
+void *
+rte_member_find_existing(const char *name);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Create set-summary (SS).
+ *
+ * @param params
+ * parameters to initialize the setsummary
+ * @return
+ * return the pointer to the setsummary
+ * return value is NULL if the creation failed
+ */
+
+void *
+rte_member_create(const struct rte_member_parameters *params);
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup key in set-summary (SS).
+ * Single key lookup and return as soon as the first match found
+ * @param setsum
+ * pointer of a setsummary
+ * @param key
+ * pointer of the key that needs to lookup
+ * @param set_id
+ * output the set id matches the key
+ * @return
+ * return 1 for found a match and 0 for not found a match
+ */
+
+int
+rte_member_lookup(const void *setsum,
+ const void *key, MEMBER_SET_TYPE *set_id);
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * lookup bulk of keys in set-summary (SS).
+ * Each key lookup returns as soon as the first match found
+ * @param setsum
+ * Pointer of a setsummary
+ * @param keys
+ * Pointer of bulk of keys that to be lookup
+ * @param num_keys
+ * Number of keys that will be lookup
+ * @param set_ids
+ * Output set ids for all the keys to this array.
+ * User should preallocate array that can contain all results, which size is
+ * the num_keys.
+ * @return
+ * The number of keys that found a match
+ */
+
+
+int
+rte_member_lookup_bulk(const void *setsum,
+ const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids);
+
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a key in set-summary (SS) for multiple matches.
+ * The key lookup will find all matched entries (multiple match).
+ * Note that for cache mode of HT, each key can have at most one match. This is
+ * because keys with same signature that maps to same bucket will overwrite
+ * each other. So multi-match lookup should be used for vBF and non-cache HT.
+ * @param setsum
+ * pointer of a set-summary
+ * @param key
+ * The key that to be lookup
+ * @param max_match_per_key
+ * User specified maximum number of matches for each key. The function returns
+ * as soon as this number of matches found for the key.
+ * @param set_id
+ * Output set ids for all the matches of the key. User needs to preallocate
+ * the array that can contain max_match_per_key number of results.
+ * @return
+ * The number of matches that found for the key.
+ * For cache mode HT set-summary, the number should be at most 1
+ */
+
+int
+rte_member_lookup_multi(const void *setsum,
+ const void *key, uint32_t max_match_per_key,
+ MEMBER_SET_TYPE *set_id);
+
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a bulk of keys in set-summary (SS) for multiple matches each key.
+ * Each key lookup will find all matched entries (multiple match).
+ * Note that for cache mode HT, each key can have at most one match. So
+ * multi-match function is mainly used for vBF and non-cache mode HT.
+ * @param setsum
+ * pointer of a setsummary
+ * @param keys
+ * The keys that to be lookup
+ * @param num_keys
+ * The number of keys that will be lookup
+ * @param max_match_per_key
+ * The possible maximum number of matches for each key
+ * @param match_count
+ * Output the number of matches for each key in an array
+ * @param set_ids
+ * Return set ids for all the matches of all keys. User pass in a preallocated
+ * 2D array with first dimension as key index and second dimension as match
+ * index. For example set_ids[bulk_size][max_match_per_key]
+ * @return
+ * The number of keys that found one or more matches in the set-summary
+ */
+
+int
+rte_member_lookup_multi_bulk(const void *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Insert key into set-summary (SS).
+ *
+ * @param setsum
+ * pointer of a set-summary
+ * @param key
+ * the key that needs to be added
+ * @param set_id
+ * The set id associated with the key that needs to be added. Different mode
+ * supports different set_id ranges. 0 cannot be used as set_id since
+ * RTE_MEMBER_NO_MATCH by default is set as 0.
+ * For HT mode, the set_id has range as [1, 0x7FFF], MSB is reserved.
+ * For vBF mode the set id is limited by the num_set parameter when create
+ * the set-summary.
+ * @return
+ * HT (cache mode) and vBF should never fail unless the set_id is not in the
+ * valid range. In such case -EINVAL is returned.
+ * For HT (non-cache mode) it could fail with -ENOSPC error code when table is
+ * full.
+ * For success it returns different values for different modes to provide
+ * extra information for users.
+ * Return 0 for HT (cache mode) if the add does not cause
+ * eviction, return 1 otherwise. Return 0 for HT mode if success, -ENOSPC for
+ * full, and 1 if cuckoo eviction happens. Always return 0 for vBF mode.
+ */
+
+int
+rte_member_add(const void *setsum, const void *key,
+ MEMBER_SET_TYPE set_id);
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * De-allocate memory used by set-summary.
+ * @param setsum
+ * Pointer to the set summary
+ */
+void
+rte_member_free(void *setsum);
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Reset the set-summary tables. E.g. reset bits to be 0 in BF,
+ * reset set_id in each entry to be RTE_MEMBER_NO_MATCH in HT based SS.
+ * @param setsum
+ * Pointer to the set-summary
+ */
+void
+rte_member_reset(const void *setsum);
+
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Delete items from the set-summary. Note that vBF does not support deletion
+ * in current implementation. For vBF, error code of -EINVAL will be returned.
+ * @param setsum
+ * Pointer to the set-summary
+ * @param key
+ * The key to be deleted
+ * @param set_id
+ * For HT mode, we need both key and its corresponding set_id to
+ * properly delete the key. Without set_id, we may delete other keys with the
+ * same signature
+ * @return
+ * If no entry found to delete, an error code of -ENOENT could be returned
+ */
+
+int
+rte_member_delete(void *setsum, const void *key,
+ MEMBER_SET_TYPE set_id);
+
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_H_ */
diff --git a/lib/librte_member/rte_member_version.map b/lib/librte_member/rte_member_version.map
new file mode 100644
index 0000000..a5877c9
--- /dev/null
+++ b/lib/librte_member/rte_member_version.map
@@ -0,0 +1,16 @@
+DPDK_17.11 {
+ global:
+
+ rte_member_create;
+ rte_member_find_existing;
+ rte_member_lookup;
+ rte_member_lookup_bulk;
+ rte_member_lookup_multi;
+ rte_member_lookup_multi_bulk;
+ rte_member_add;
+ rte_member_free;
+ rte_member_reset;
+ rte_member_delete;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v2 2/7] member: implement HT mode
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 1/7] member: implement main API Yipeng Wang
@ 2017-09-02 1:24 ` Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 3/7] member: implement vBF mode Yipeng Wang
` (5 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-02 1:24 UTC (permalink / raw)
To: dev
Cc: stephen, luca.boccassi, charlie.tai, sameh.gobriel, ren.wang,
pablo.de.lara.guarch, yipeng1.wang
One of the set-summray structure is hash-table based
set-summary (HTSS). One example is cuckoo filter [1].
Comparing to a traditional hash table, HTSS has a much more
compact structure. For each element, only one signature and
its corresponding set ID is stored. No key comparison is required
during lookup. For the table structure, there are multiple entries
in each bucket, and the table is composed of many buckets.
Two modes are supported for HTSS, "cache" and "none-cache" modes.
The non-cache mode is more similar to traditional cuckoo filter.
When a bucket is full, one entry will be evicted to its
alternative bucket to make space for the new key. The table could
be full and then no more keys could be inserted. This mode has
false-positive rate but no false-negative. Multiple entries
with same signature could stay in the same bucket.
The "cache" mode does not evict key to its alternative bucket
when a bucket is full, an existing key will be evicted out of
the table like a cache. So the table will never reject keys when
it is full. Another property is in each bucket, there cannot be
multiple entries with same signature. The mode could have both
false-positive and false-negative probability.
This patch add the implementation of HTSS.
[1] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically
Better Than Bloom,” in Conference on emerging Networking
Experiments and Technologies, 2014.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_ht.c | 490 ++++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_ht.h | 115 +++++++++
3 files changed, 606 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 1a79eaa..ad26548 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
new file mode 100644
index 0000000..b2ae6d0
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.c
@@ -0,0 +1,490 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_errno.h>
+#include <rte_malloc.h>
+#include <rte_prefetch.h>
+#include <rte_random.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+
+
+static inline int
+insert_overwrite_search(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE set_id)
+{
+ int i;
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[bucket].sigs[i] == tmp_sig) {
+ buckets[bucket].sets[i] = set_id;
+ return 1;
+ }
+ }
+ return 0;
+}
+
+
+
+
+static inline int
+search_bucket_single(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE *set_id)
+{
+ int iter;
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket].sigs[iter] &&
+ buckets[bucket].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket].sets[iter];
+ return 1;
+ }
+ }
+ return 0;
+}
+
+
+static inline void
+search_bucket_multi(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ int iter;
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket].sigs[iter] &&
+ buckets[bucket].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket].sets[iter];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ }
+}
+
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+ uint32_t i, j;
+ uint32_t size_bucket_t;
+ uint32_t num_entries = rte_align32pow2(params->num_keys);
+
+ if ((num_entries > RTE_MEMBER_ENTRIES_MAX) ||
+ !rte_is_power_of_2(RTE_MEMBER_BUCKET_ENTRIES) ||
+ num_entries < RTE_MEMBER_BUCKET_ENTRIES) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER,
+ "Membership HT create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ uint32_t num_buckets = num_entries / RTE_MEMBER_BUCKET_ENTRIES;
+
+ size_bucket_t = sizeof(struct member_ht_bucket);
+
+ struct member_ht_bucket *buckets = rte_zmalloc_socket(NULL,
+ num_buckets * size_bucket_t,
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ if (buckets == NULL)
+ return -ENOMEM;
+
+ ss->table = buckets;
+ ss->bucket_cnt = num_buckets;
+ ss->bucket_mask = num_buckets - 1;
+ ss->cache = params->iscache;
+
+ for (i = 0; i < num_buckets; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+
+
+ RTE_LOG(DEBUG, MEMBER, "Hash table based filter created, "
+ "the table has %u entries, %u buckets\n",
+ num_buckets,
+ num_buckets / RTE_MEMBER_BUCKET_ENTRIES);
+ return 0;
+}
+
+static inline
+void get_buckets_index(const struct rte_member_setsum *ss, const void *key,
+ uint32_t *prim_bkt, uint32_t *sec_bkt, SIG_TYPE *sig)
+{
+ uint32_t first_hash = MEMBER_PRIM_HASH(key, ss->key_len,
+ ss->prim_hash_seed);
+ uint32_t sec_hash = MEMBER_SEC_HASH(&first_hash, 4, ss->sec_hash_seed);
+ *sig = first_hash & SIG_BITMASK;
+ if (ss->cache) {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (sec_hash >> 16) & ss->bucket_mask;
+ } else {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (*prim_bkt ^ *sig) & ss->bucket_mask;
+ }
+}
+
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *ss,
+ const void *key, MEMBER_SET_TYPE *set_id)
+{
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+
+ return 0;
+}
+
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, MEMBER_SET_TYPE *set_id)
+{
+ uint32_t i;
+ uint32_t ret = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ SIG_TYPE tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+
+ for (i = 0; i < num_keys; i++) {
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return ret;
+}
+
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t ret = 0;
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
+ match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+}
+
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids)
+{
+ uint32_t i;
+ uint32_t ret = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t match_cnt_t;
+ SIG_TYPE tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_t = 0;
+
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_t < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ }
+ return ret;
+}
+
+static inline int
+try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ SIG_TYPE sig, MEMBER_SET_TYPE set_id)
+{
+ int i;
+ /* If not full then insert into one slot*/
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[prim].sigs[i] = sig;
+ buckets[prim].sets[i] = set_id;
+ return 0;
+ }
+ }
+ /* if prim failed, we need to access second cache line */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[sec].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[sec].sigs[i] = sig;
+ buckets[sec].sets[i] = set_id;
+ return 0;
+ }
+ }
+ return -1;
+}
+
+
+static inline int
+try_overwrite(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ SIG_TYPE sig, MEMBER_SET_TYPE set_id)
+{
+ if (insert_overwrite_search(prim, sig, buckets, set_id) ||
+ insert_overwrite_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ return -1;
+}
+
+static inline int
+evict_from_bucket(void)
+{
+ /* for now, we randomly pick one entry to evict */
+ return rte_rand() & (RTE_MEMBER_BUCKET_ENTRIES - 1);
+}
+
+
+
+/*
+ * This function is similar to the cuckoo hash make_space function in hash
+ * library
+ */
+static inline int
+make_space_bucket(const struct rte_member_setsum *ss, uint32_t bkt_num)
+{
+
+ static unsigned int nr_pushes;
+
+ unsigned int i, j;
+ int ret;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t next_bucket_idx;
+ struct member_ht_bucket *next_bkt[RTE_MEMBER_BUCKET_ENTRIES];
+ struct member_ht_bucket *bkt = &buckets[bkt_num];
+ MEMBER_SET_TYPE flag_mask = 1U << (sizeof(MEMBER_SET_TYPE) * 8 - 1);
+ /*
+ * Push existing item (search for bucket with space in
+ * alternative locations) to its alternative location
+ */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ /* Search for space in alternative locations */
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
+ next_bkt[i] = &buckets[next_bucket_idx];
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++) {
+ if (next_bkt[i]->sets[j] == RTE_MEMBER_NO_MATCH)
+ break;
+ }
+
+ if (j != RTE_MEMBER_BUCKET_ENTRIES)
+ break;
+ }
+
+ /* Alternative location has spare room (end of recursive function) */
+ if (i != RTE_MEMBER_BUCKET_ENTRIES) {
+ next_bkt[i]->sigs[j] = bkt->sigs[i];
+ next_bkt[i]->sets[j] = bkt->sets[i];
+ return i;
+ }
+
+ /* Pick entry that has not been pushed yet */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++)
+ if ((bkt->sets[i] & flag_mask) == 0)
+ break;
+
+ /* All entries have been pushed, so entry cannot be added */
+ if (i == RTE_MEMBER_BUCKET_ENTRIES ||
+ nr_pushes > RTE_MEMBER_MAX_PUSHES)
+ return -ENOSPC;
+
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
+ /* Set flag to indicate that this entry is going to be pushed */
+ bkt->sets[i] |= flag_mask;
+
+ nr_pushes++;
+ /* Need room in alternative bucket to insert the pushed entry */
+ ret = make_space_bucket(ss, next_bucket_idx);
+ /*
+ * After recursive function.
+ * Clear flags and insert the pushed entry
+ * in its alternative location if successful,
+ * or return error
+ */
+ bkt->sets[i] &= ~flag_mask;
+ nr_pushes = 0;
+ if (ret >= 0) {
+ next_bkt[i]->sigs[ret] = bkt->sigs[i];
+ next_bkt[i]->sets[ret] = bkt->sets[i];
+ return i;
+ } else
+ return ret;
+}
+
+
+int
+rte_member_add_ht(const struct rte_member_setsum *ss,
+ const void *key, MEMBER_SET_TYPE set_id)
+{
+ int ret;
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+ MEMBER_SET_TYPE flag_mask = 1U << (sizeof(MEMBER_SET_TYPE) * 8 - 1);
+
+ if (set_id == RTE_MEMBER_NO_MATCH || (set_id & flag_mask) != 0)
+ return -EINVAL;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ /* if it is cache based filter, we try overwriting existing entry */
+ if (ss->cache) {
+ ret = try_overwrite(buckets, prim_bucket, sec_bucket, tmp_sig,
+ set_id);
+ if (ret != -1)
+ return ret;
+ }
+ /* If not full then insert into one slot*/
+ ret = try_insert(buckets, prim_bucket, sec_bucket, tmp_sig, set_id);
+ if (ret != -1)
+ return ret;
+
+ /* random pick prim or sec for recursive displacement */
+
+ uint32_t select_bucket = (tmp_sig && 1U) ? prim_bucket : sec_bucket;
+ if (ss->cache) {
+ ret = evict_from_bucket();
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ return 1;
+ }
+
+ ret = make_space_bucket(ss, select_bucket);
+ if (ret >= 0) {
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ ret = 1;
+ }
+
+ return ret;
+}
+
+
+void
+rte_member_free_ht(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+
+int
+rte_member_delete_ht(struct rte_member_setsum *ss, const void *key,
+ MEMBER_SET_TYPE set_id)
+{
+ int i;
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[prim_bucket].sigs[i] &&
+ set_id == buckets[prim_bucket].sets[i]) {
+ buckets[prim_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[sec_bucket].sigs[i] &&
+ set_id == buckets[sec_bucket].sets[i]) {
+ buckets[sec_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+ return -ENOENT;
+}
+
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *ss)
+{
+ uint32_t i, j;
+ struct member_ht_bucket *buckets = ss->table;
+ for (i = 0; i < ss->bucket_cnt; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+}
diff --git a/lib/librte_member/rte_member_ht.h b/lib/librte_member/rte_member_ht.h
new file mode 100644
index 0000000..aebf1db
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.h
@@ -0,0 +1,115 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_HT_H_
+#define _RTE_MEMBER_HT_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Maximum number of pushes for cuckoo path in HT mode. */
+#define RTE_MEMBER_MAX_PUSHES 100
+
+typedef uint16_t SIG_TYPE; /* signature size is 16 bit */
+#define SIG_BITMASK 0xFFFF /* signature mask, 16 bit */
+
+
+/* The bucket struct for ht setsum */
+struct member_ht_bucket {
+ SIG_TYPE sigs[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte signature */
+ MEMBER_SET_TYPE sets[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte set */
+} __rte_cache_aligned;
+
+
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE *set_id);
+
+
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids);
+
+
+
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id);
+
+
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids);
+
+
+
+
+int
+rte_member_add_ht(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE set_id);
+
+void
+rte_member_free_ht(struct rte_member_setsum *setsum);
+
+
+
+
+int
+rte_member_delete_ht(struct rte_member_setsum *ss, const void *key,
+ MEMBER_SET_TYPE set_id);
+
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *setsum);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+
+
+#endif /* _RTE_MEMBER_HT_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v2 3/7] member: implement vBF mode
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 1/7] member: implement main API Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 2/7] member: implement HT mode Yipeng Wang
@ 2017-09-02 1:24 ` Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 4/7] member: add AVX for HT mode Yipeng Wang
` (4 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-02 1:24 UTC (permalink / raw)
To: dev
Cc: stephen, luca.boccassi, charlie.tai, sameh.gobriel, ren.wang,
pablo.de.lara.guarch, yipeng1.wang
Bloom Filter (BF) [1] is a well-known space-efficient
probabilistic data structure that answers set membership queries.
Vector of Bloom Filters (vBF) is an extension to traditional BF
that supports multi-set membership testing. Traditional BF will
return found or not-found for each key. vBF will also return
which set the key belongs to if it is found.
Since each set requires a BF, vBF should be used when set count
is small. vBF's false positive rate could be set arbitrarily so
that its memory requirement and lookup speed is better in certain
cases comparing to HT based set-summary.
This patch adds the vBF implementation.
[1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
Errors,” Communications of the ACM, 1970.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_vbf.c | 350 +++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_vbf.h | 85 +++++++++
3 files changed, 436 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index ad26548..50275ed 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c rte_member_vbf.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_vbf.c b/lib/librte_member/rte_member_vbf.c
new file mode 100644
index 0000000..49ca69e
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.c
@@ -0,0 +1,350 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <math.h>
+#include <string.h>
+
+#include <rte_malloc.h>
+#include <rte_memory.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_vbf.h"
+
+/*
+ * vBF currently implemented as a big array.
+ * The BFs have a vertical layout. Bits in same location of all bfs will stay
+ * in the same cache line.
+ * For example, if we have 32 bloom filters, we use a uint32_t array to
+ * represent all of them. array[0] represent the first location of all the
+ * bloom filters, array[1] represents the second location of all the
+ * bloom filters, etc. The advantage of this layout is to minimize the average
+ * number of memory accesses to test all bloom filters.
+ *
+ * Currently the implementation supports vBF containing 1,2,4,8,16,32 BFs.
+ */
+
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+
+ if (params->num_set > 32 || !rte_is_power_of_2(params->num_set) ||
+ params->false_positive_rate == 0 ||
+ params->false_positive_rate > 1) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "vBF create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ /* We assume expected keys evenly distribute to all BFs */
+ uint32_t num_keys_per_bf = (params->num_keys + ss->num_set - 1) /
+ ss->num_set;
+
+ if (num_keys_per_bf == 0) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "BF expected number of key is zero\n");
+ return -EINVAL;
+ }
+ /*
+ * Note that the false positive rate is for all BFs in the vBF
+ * such that the single BF's false positive rate need to be
+ * calculated.
+ * Assume each BF's False positive rate is x. The total false positive
+ * rate is fp = 1-(1-x)^n.
+ * => x = 1 - (1-fp)^(1/n)
+ */
+
+ float x = 1 - pow((1 - params->false_positive_rate), 1.0 / ss->num_set);
+
+ if (x == 0) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "BF false positive rate is zero\n");
+ return -EINVAL;
+ }
+
+ uint32_t bits = ceil((num_keys_per_bf *
+ log(x)) / log(1.0 / (pow(2.0, log(2.0)))));
+
+ /* We round to power of 2 for performance during lookup */
+ ss->bits = rte_align32pow2(bits);
+ if (ss->bits == 0) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "BF size is zero\n");
+ return -EINVAL;
+ }
+
+ ss->num_hashes = (uint32_t)(log(2.0) * bits / num_keys_per_bf);
+ ss->bit_mask = ss->bits - 1;
+
+
+ /*
+ * Since we round the bits to power of 2, the final false positive
+ * rate will probably not be same as the user specified. We log the
+ * new value as debug message.
+ */
+ float new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ ss->num_hashes)), ss->num_hashes);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ /*
+ * reduce hash function count, until we approach the user specified
+ * false-positive rate. otherwise it is too conservative
+ */
+ int tmp_num_hash = ss->num_hashes;
+
+ while (tmp_num_hash > 1) {
+ float tmp_fp = new_fp;
+
+ tmp_num_hash--;
+ new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ tmp_num_hash)), tmp_num_hash);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ if (new_fp > params->false_positive_rate) {
+ new_fp = tmp_fp;
+ tmp_num_hash++;
+ break;
+ }
+ }
+
+ ss->num_hashes = tmp_num_hash;
+
+ RTE_LOG(DEBUG, MEMBER, "vector bloom filter created, "
+ "each bloom filter expects %u keys, needs %u bits, %u hashes, "
+ "with false positive rate set as %.5f, "
+ "The new calculated vBF false positive rate is %.5f\n",
+ num_keys_per_bf, ss->bits, ss->num_hashes, x, new_fp);
+
+ ss->table = rte_zmalloc_socket(NULL, sizeof(uint32_t) * ss->num_set *
+ (ss->bits >> 5), RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ if (ss->table == NULL)
+ return -ENOMEM;
+
+ return 0;
+}
+
+
+/*
+ * a is how many bits could be represented by one uint32_t variable.
+ * b is used for division shift
+ * shift is used for multiplication shift
+ */
+static inline uint32_t
+test_bit(uint32_t h1, uint32_t h2, uint32_t iter, uint32_t shift, uint32_t a,
+ uint32_t b, const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t n = ss->num_set;
+ uint32_t bit_loc = (h1 + iter * h2) & ss->bit_mask;
+ /*
+ * x>>b is the divide, x & (a-1) is the mod, & (1<<n-1) to mask out bits
+ * we do not need
+ */
+ return (vbf[bit_loc>>b] >> ((bit_loc & (a - 1)) << shift)) &
+ ((1ULL << n) - 1);
+}
+
+
+static inline void
+set_bit(uint32_t h1, uint32_t h2, uint32_t iter, uint32_t shift, uint32_t a,
+ uint32_t b, const struct rte_member_setsum *ss, int32_t set)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t bit_loc = (h1 + iter * h2) & ss->bit_mask;
+ vbf[bit_loc>>b] |= 1U << (((bit_loc & (a - 1)) << shift) + set - 1);
+}
+
+
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *ss, const void *key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t j;
+ uint32_t h1 = MEMBER_PRIM_HASH(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ uint32_t mask = 0xFFFFFFFF;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (j = 0; j < ss->num_hashes; j++)
+ mask &= test_bit(h1, h2, j, shift, a, b, ss);
+
+ if (mask) {
+ *set_id = __builtin_ctz(mask) + 1;
+ return 1;
+ }
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ return 0;
+}
+
+
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, MEMBER_SET_TYPE *set_ids)
+{
+ uint32_t i, k, h1, h2;
+ uint32_t ret = 0;
+ uint32_t mask;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (i = 0; i < num_keys; i++) {
+ mask = 0xFFFFFFFF;
+ h1 = MEMBER_PRIM_HASH(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ for (k = 0; k < ss->num_hashes; k++)
+ mask &= test_bit(h1, h2, k, shift, a, b, ss);
+
+ if (mask) {
+ set_ids[i] = __builtin_ctz(mask) + 1;
+ ret++;
+ } else
+ set_ids[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return ret;
+}
+
+
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t ret = 0;
+ uint32_t j;
+ uint32_t h1 = MEMBER_PRIM_HASH(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ uint32_t mask = 0xFFFFFFFF;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (j = 0; j < ss->num_hashes; j++)
+ mask &= test_bit(h1, h2, j, shift, a, b, ss);
+
+ while (mask) {
+ uint32_t loc = __builtin_ctz(mask);
+ set_id[ret] = loc + 1;
+ ret++;
+ if (ret >= match_per_key)
+ return ret;
+ mask &= ~(1U << loc);
+ }
+ return ret;
+}
+
+
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids)
+{
+ uint32_t i, k, h1, h2;
+ uint32_t ret = 0;
+ uint32_t match_cnt_t;
+ uint32_t mask;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_t = 0;
+ mask = 0xFFFFFFFF;
+ h1 = MEMBER_PRIM_HASH(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+
+ for (k = 0; k < ss->num_hashes; k++)
+ mask &= test_bit(h1, h2, k, shift, a, b, ss);
+
+ while (mask) {
+ uint32_t loc = __builtin_ctz(mask);
+ if (match_cnt_t >= match_per_key)
+ break;
+ set_ids[i * match_per_key + match_cnt_t] = loc + 1;
+ match_cnt_t++;
+ mask &= ~(1U << loc);
+ }
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ }
+ return ret;
+}
+
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *ss,
+ const void *key, MEMBER_SET_TYPE set_id)
+{
+ uint32_t i, h1, h2;
+
+ if (set_id > ss->num_set || set_id == RTE_MEMBER_NO_MATCH)
+ return -EINVAL;
+
+ h1 = MEMBER_PRIM_HASH(key, ss->key_len, ss->prim_hash_seed);
+ h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (i = 0; i < ss->num_hashes; i++)
+ set_bit(h1, h2, i, shift, a, b, ss, set_id);
+ return 0;
+}
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ memset(vbf, 0, (ss->num_set * ss->bits) >> 3);
+}
diff --git a/lib/librte_member/rte_member_vbf.h b/lib/librte_member/rte_member_vbf.h
new file mode 100644
index 0000000..3d3ee33
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.h
@@ -0,0 +1,85 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_VBF_H_
+#define _RTE_MEMBER_VBF_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE *set_id);
+
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids);
+
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id);
+
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids);
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE set_id);
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss);
+
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *setsum);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_VBF_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v2 4/7] member: add AVX for HT mode
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
` (2 preceding siblings ...)
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 3/7] member: implement vBF mode Yipeng Wang
@ 2017-09-02 1:24 ` Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 5/7] member: enable the library Yipeng Wang
` (3 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-02 1:24 UTC (permalink / raw)
To: dev
Cc: stephen, luca.boccassi, charlie.tai, sameh.gobriel, ren.wang,
pablo.de.lara.guarch, yipeng1.wang
For key search, the signatures of all entries are compared against
the signature of the key that is being looked up. Since all
signatures are contguously put in a bucket, they can be compared
with vector instructions (AVX2), achieving higher lookup performance.
This patch adds AVX2 implementation in a separate header file.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/rte_member_ht.c | 143 ++++++++++++++++++++++++++++---------
lib/librte_member/rte_member_x86.h | 111 ++++++++++++++++++++++++++++
2 files changed, 222 insertions(+), 32 deletions(-)
create mode 100644 lib/librte_member/rte_member_x86.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
index b2ae6d0..15e2534 100644
--- a/lib/librte_member/rte_member_ht.c
+++ b/lib/librte_member/rte_member_ht.c
@@ -40,6 +40,10 @@
#include "rte_member.h"
#include "rte_member_ht.h"
+#if defined(RTE_ARCH_X86)
+#include "rte_member_x86.h"
+#endif
+
static inline int
insert_overwrite_search(uint32_t bucket, SIG_TYPE tmp_sig,
@@ -135,6 +139,13 @@ rte_member_create_ht(struct rte_member_setsum *ss,
for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
}
+#if defined(RTE_ARCH_X86)
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) &&
+ RTE_MEMBER_BUCKET_ENTRIES == 16)
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_AVX2;
+ else
+#endif
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_SCALAR;
RTE_LOG(DEBUG, MEMBER, "Hash table based filter created, "
@@ -174,11 +185,23 @@ rte_member_lookup_ht(const struct rte_member_setsum *ss,
*set_id = RTE_MEMBER_NO_MATCH;
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- if (search_bucket_single(prim_bucket, tmp_sig, buckets,
- set_id) ||
- search_bucket_single(sec_bucket, tmp_sig,
- buckets, set_id))
- return 1;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single_avx(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ }
return 0;
}
@@ -203,13 +226,27 @@ rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
}
for (i = 0; i < num_keys; i++) {
- if (search_bucket_single(prim_buckets[i], tmp_sig[i],
- buckets, &set_id[i]) ||
- search_bucket_single(sec_buckets[i],
- tmp_sig[i], buckets, &set_id[i]))
- ret++;
- else
- set_id[i] = RTE_MEMBER_NO_MATCH;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]) ||
+ search_bucket_single_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
}
return ret;
}
@@ -227,12 +264,24 @@ rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
- match_per_key, set_id);
- if (ret < match_per_key)
- search_bucket_multi(sec_bucket, tmp_sig,
- buckets, &ret, match_per_key, set_id);
- return ret;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_bucket, tmp_sig, buckets,
+ &ret, match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi_avx(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+#endif
+ default:
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
+ match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+ }
}
@@ -259,16 +308,34 @@ rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
for (i = 0; i < num_keys; i++) {
match_cnt_t = 0;
- search_bucket_multi(prim_buckets[i], tmp_sig[i],
- buckets, &match_cnt_t, match_per_key,
- &set_ids[i*match_per_key]);
- if (match_cnt_t < match_per_key)
- search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_buckets[i], tmp_sig[i],
buckets, &match_cnt_t, match_per_key,
&set_ids[i*match_per_key]);
- match_count[i] = match_cnt_t;
- if (match_cnt_t != 0)
- ret++;
+ if (match_cnt_t < match_per_key)
+ search_bucket_multi_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &match_cnt_t,
+ match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ break;
+#endif
+ default:
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_t < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ }
}
return ret;
}
@@ -300,12 +367,24 @@ try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
static inline int
try_overwrite(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
- SIG_TYPE sig, MEMBER_SET_TYPE set_id)
+ SIG_TYPE sig, MEMBER_SET_TYPE set_id,
+ enum rte_member_sig_compare_function cmp_fn)
{
- if (insert_overwrite_search(prim, sig, buckets, set_id) ||
- insert_overwrite_search(sec, sig, buckets,
- set_id))
- return 0;
+ switch (cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (insert_overwrite_search_avx(prim, sig, buckets, set_id) ||
+ insert_overwrite_search_avx(sec, sig, buckets,
+ set_id))
+ return 0;
+ break;
+#endif
+ default:
+ if (insert_overwrite_search(prim, sig, buckets, set_id) ||
+ insert_overwrite_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ }
return -1;
}
@@ -411,7 +490,7 @@ rte_member_add_ht(const struct rte_member_setsum *ss,
/* if it is cache based filter, we try overwriting existing entry */
if (ss->cache) {
ret = try_overwrite(buckets, prim_bucket, sec_bucket, tmp_sig,
- set_id);
+ set_id, ss->sig_cmp_fn);
if (ret != -1)
return ret;
}
diff --git a/lib/librte_member/rte_member_x86.h b/lib/librte_member/rte_member_x86.h
new file mode 100644
index 0000000..c55f128
--- /dev/null
+++ b/lib/librte_member/rte_member_x86.h
@@ -0,0 +1,111 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_X86_H_
+#define _RTE_MEMBER_X86_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <x86intrin.h>
+
+
+#if defined(RTE_MACHINE_CPUFLAG_AVX2)
+
+
+static inline int
+insert_overwrite_search_avx(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ if (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) / 2;
+ buckets[bucket].sets[hit_idx] = set_id;
+ return 1;
+ }
+ return 0;
+}
+
+
+static inline int
+search_bucket_single_avx(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) / 2;
+ if (buckets[bucket].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket].sets[hit_idx];
+ return 1;
+ }
+ hitmask &= ~(3U << (hit_idx) * 2);
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi_avx(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) / 2;
+ if (buckets[bucket].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket].sets[hit_idx];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ hitmask &= ~(3U << (hit_idx) * 2);
+ }
+}
+#endif
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_X86_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v2 5/7] member: enable the library
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
` (3 preceding siblings ...)
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 4/7] member: add AVX for HT mode Yipeng Wang
@ 2017-09-02 1:24 ` Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 6/7] test/member: add functional and perf tests Yipeng Wang
` (2 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-02 1:24 UTC (permalink / raw)
To: dev
Cc: stephen, luca.boccassi, charlie.tai, sameh.gobriel, ren.wang,
pablo.de.lara.guarch, yipeng1.wang
This patch enables the Membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
MAINTAINERS | 7 +++++++
config/common_base | 5 +++++
lib/librte_member/Makefile | 2 ++
mk/rte.app.mk | 2 ++
4 files changed, 16 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index a0cd75e..e372edf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -734,6 +734,13 @@ F: test/test/test_meter.c
F: examples/qos_meter/
F: doc/guides/sample_app_ug/qos_metering.rst
+Membership - EXPERIMENTAL
+M: Yipeng Wang <yipeng1.wang@intel.com>
+M: Sameh Gobriel <sameh.gobriel@intel.com>
+F: lib/librte_member/
+F: doc/guides/prog_guide/member_lib.rst
+F: test/test/test_member*
+
Other libraries
---------------
diff --git a/config/common_base b/config/common_base
index 5e97a08..5e31ced 100644
--- a/config/common_base
+++ b/config/common_base
@@ -595,6 +595,11 @@ CONFIG_RTE_LIBRTE_HASH_DEBUG=n
CONFIG_RTE_LIBRTE_EFD=y
#
+# Compile librte_member
+#
+CONFIG_RTE_LIBRTE_MEMBER=y
+
+#
# Compile librte_jobstats
#
CONFIG_RTE_LIBRTE_JOBSTATS=y
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 50275ed..3bac1d0 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -37,6 +37,8 @@ LIB = librte_member.a
CFLAGS := -I$(SRCDIR) $(CFLAGS)
CFLAGS += $(WERROR_FLAGS) -O3
+LDLIBS += -lm
+
EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index c25fdd9..c79acf0 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -86,6 +86,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile
_LDLIBS-y += --whole-archive
_LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost
_LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs
_LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf
@@ -196,6 +197,7 @@ endif
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lm
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrt
_LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lm
ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lnuma
endif
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v2 6/7] test/member: add functional and perf tests
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
` (4 preceding siblings ...)
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 5/7] member: enable the library Yipeng Wang
@ 2017-09-02 1:24 ` Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 7/7] doc: add membership documentation Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-02 1:24 UTC (permalink / raw)
To: dev
Cc: stephen, luca.boccassi, charlie.tai, sameh.gobriel, ren.wang,
pablo.de.lara.guarch, yipeng1.wang
This patch adds functional and performance tests for membership
library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
test/test/Makefile | 3 +
test/test/test_member.c | 682 +++++++++++++++++++++++++++++++++++++++++++
test/test/test_member_perf.c | 643 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 1328 insertions(+)
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
diff --git a/test/test/Makefile b/test/test/Makefile
index 42d9a49..b61dde3 100644
--- a/test/test/Makefile
+++ b/test/test/Makefile
@@ -123,6 +123,9 @@ SRCS-y += test_logs.c
SRCS-y += test_memcpy.c
SRCS-y += test_memcpy_perf.c
+
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd_perf.c
diff --git a/test/test/test_member.c b/test/test/test_member.c
new file mode 100644
index 0000000..8965e81
--- /dev/null
+++ b/test/test/test_member.c
@@ -0,0 +1,682 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* This test is for membership library's simple feature test */
+
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_member.h>
+#include <rte_byteorder.h>
+#include <rte_random.h>
+#include <rte_debug.h>
+#include <rte_ip.h>
+
+#include "test.h"
+
+void *setsum_ht;
+void *setsum_cache;
+void *setsum_vbf;
+
+/* 5-tuple key type */
+struct flow_key {
+ uint32_t ip_src;
+ uint32_t ip_dst;
+ uint16_t port_src;
+ uint16_t port_dst;
+ uint8_t proto;
+} __attribute__((packed));
+
+
+/* Keys used by unit test functions */
+static struct flow_key keys[5] = {
+ {
+ .ip_src = IPv4(0x03, 0x02, 0x01, 0x00),
+ .ip_dst = IPv4(0x07, 0x06, 0x05, 0x04),
+ .port_src = 0x0908,
+ .port_dst = 0x0b0a,
+ .proto = 0x0c,
+ },
+ {
+ .ip_src = IPv4(0x13, 0x12, 0x11, 0x10),
+ .ip_dst = IPv4(0x17, 0x16, 0x15, 0x14),
+ .port_src = 0x1918,
+ .port_dst = 0x1b1a,
+ .proto = 0x1c,
+ },
+ {
+ .ip_src = IPv4(0x23, 0x22, 0x21, 0x20),
+ .ip_dst = IPv4(0x27, 0x26, 0x25, 0x24),
+ .port_src = 0x2928,
+ .port_dst = 0x2b2a,
+ .proto = 0x2c,
+ },
+ {
+ .ip_src = IPv4(0x33, 0x32, 0x31, 0x30),
+ .ip_dst = IPv4(0x37, 0x36, 0x35, 0x34),
+ .port_src = 0x3938,
+ .port_dst = 0x3b3a,
+ .proto = 0x3c,
+ },
+ {
+ .ip_src = IPv4(0x43, 0x42, 0x41, 0x40),
+ .ip_dst = IPv4(0x47, 0x46, 0x45, 0x44),
+ .port_src = 0x4948,
+ .port_dst = 0x4b4a,
+ .proto = 0x4c,
+ }
+};
+
+uint32_t test_set[5] = {1, 2, 3, 4, 5};
+
+#define ITERATIONS 3
+#define KEY_SIZE 4
+
+#define MAX_ENTRIES (1 << 16)
+uint8_t gened_keys[MAX_ENTRIES][KEY_SIZE];
+
+static struct rte_member_parameters params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+
+ /*num_set and false_positive_rate only relevant to vBF setsum*/
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+};
+
+
+/*
+ * Sequence of operations for find existing setsummary
+ *
+ * - create setsum
+ * - find existing setsum: hit
+ * - find non-existing setsum: miss
+ *
+ */
+static int
+test_member_find_existing(void)
+{
+ void *tmp_setsum = NULL, *result = NULL;
+ struct rte_member_parameters tmp_params = {
+ .name = "member_find_existing",
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ /* Create */
+ tmp_setsum = rte_member_create(&tmp_params);
+ TEST_ASSERT(tmp_setsum != NULL, "setsum creation failed");
+
+ /* Try to find existing hash table */
+ result = rte_member_find_existing("member_find_existing");
+ TEST_ASSERT(result == tmp_setsum, "could not find existing setsum");
+
+ /* Try to find non-existing hash table */
+ result = rte_member_find_existing("member_find_non_existing");
+ TEST_ASSERT(result == NULL, "found setsum that shouldn't exist");
+
+ /* Cleanup. */
+ rte_member_free(tmp_setsum);
+
+ return 0;
+}
+
+
+/*
+ * Test for bad creating parameters
+ */
+static int
+test_member_create_bad_param(void)
+{
+ void *bad_setsum = NULL;
+ struct rte_member_parameters bad_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ bad_params.name = "bad_param1";
+ bad_params.num_set = 0;
+ bad_params.type = RTE_MEMBER_TYPE_VBF;
+ /* test with 0 set for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "number of set for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param2";
+ bad_params.false_positive_rate = 0;
+ bad_params.num_set = 32;
+ /* test with 0 false positive for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "false positive rate for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param3";
+ bad_params.false_positive_rate = 0.03;
+ bad_params.num_keys = 31;
+ /* test with less than 1 key per BF for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "num_keys for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param4";
+ bad_params.type = RTE_MEMBER_TYPE_HT;
+ bad_params.num_keys = RTE_MEMBER_BUCKET_ENTRIES / 2;
+ /* test with less than 1 bucket for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with too few "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ bad_params.num_keys = RTE_MEMBER_ENTRIES_MAX + 1;
+ /* test with more than maximum entries for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with to many "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ /* test with same name should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with existed "
+ "name\n");
+ return -1;
+ }
+
+ rte_member_free(bad_setsum);
+ return 0;
+}
+
+
+/* Create test setsummaries. */
+static int test_member_create(void)
+{
+ params.key_len = sizeof(struct flow_key);
+
+ params.name = "test_member_ht";
+ params.iscache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.iscache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+ params.name = "test_member_vbf";
+ params.type = RTE_MEMBER_TYPE_VBF;
+ setsum_vbf = rte_member_create(¶ms);
+
+ if (setsum_ht == NULL || setsum_cache == NULL || setsum_vbf == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ printf("Creation of setsums success\n");
+ return 0;
+}
+
+static int test_member_insert(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_add(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[i], test_set[i]);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "insert error");
+ }
+ printf("insert key success\n");
+ return 0;
+}
+
+static int test_member_lookup(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ MEMBER_SET_TYPE set_ids_ht[5] = {0};
+ MEMBER_SET_TYPE set_ids_cache[5] = {0};
+ MEMBER_SET_TYPE set_ids_vbf[5] = {0};
+
+ uint32_t num_key_ht = 5;
+ uint32_t num_key_cache = 5;
+ uint32_t num_key_vbf = 5;
+
+ const void *key_array[5];
+
+ /* single lookup test */
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "single lookup function error");
+
+ TEST_ASSERT(set_ht == test_set[i] &&
+ set_cache == test_set[i] &&
+ set_vbf == test_set[i],
+ "single lookup set value error");
+ }
+ printf("lookup single key success\n");
+
+ /* bulk lookup test */
+ for (i = 0; i < 5; i++)
+ key_array[i] = &keys[i];
+
+ ret_ht = rte_member_lookup_bulk(setsum_ht, &key_array[0],
+ num_key_ht, set_ids_ht);
+
+ ret_cache = rte_member_lookup_bulk(setsum_cache, &key_array[0],
+ num_key_cache, set_ids_cache);
+
+ ret_vbf = rte_member_lookup_bulk(setsum_vbf, &key_array[0],
+ num_key_vbf, set_ids_vbf);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "bulk lookup function error");
+
+ for (i = 0; i < 5; i++) {
+ TEST_ASSERT((set_ids_ht[i] == test_set[i]) &&
+ (set_ids_cache[i] == test_set[i]) &&
+ (set_ids_vbf[i] == test_set[i]),
+ "bulk lookup result error");
+ }
+
+ return 0;
+}
+
+
+static int test_member_delete(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_delete(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_delete(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_delete(setsum_vbf, &keys[i], test_set[i]);
+ /* VBF does not support delete yet, so return error code */
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key deletion function error");
+ TEST_ASSERT(ret_vbf < 0,
+ "vbf does not support deletion, error");
+ }
+
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key lookup function error");
+ TEST_ASSERT(set_ht == RTE_MEMBER_NO_MATCH &&
+ ret_cache == RTE_MEMBER_NO_MATCH,
+ "key deletion failed");
+ }
+ printf("delete success\n");
+ return 0;
+}
+
+
+static int test_member_multimatch(void)
+{
+ int ret_ht, ret_vbf, ret_cache;
+ MEMBER_SET_TYPE set_ids_ht[32] = {0};
+ MEMBER_SET_TYPE set_ids_vbf[32] = {0};
+ MEMBER_SET_TYPE set_ids_cache[32] = {0};
+
+ MEMBER_SET_TYPE set_ids_ht_m[5][32] = {{0} };
+ MEMBER_SET_TYPE set_ids_vbf_m[5][32] = {{0} };
+ MEMBER_SET_TYPE set_ids_cache_m[5][32] = {{0} };
+
+ uint32_t match_count_ht[5];
+ uint32_t match_count_vbf[5];
+ uint32_t match_count_cache[5];
+
+ uint32_t num_key_ht = 5;
+ uint32_t num_key_vbf = 5;
+ uint32_t num_key_cache = 5;
+
+ const void *key_array[5];
+
+ uint32_t i, j;
+ /* same key at most inserted 2*entry_per_bucket times for HT mode */
+ for (i = 1; i < 33; i++) {
+ for (j = 0; j < 5; j++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[j], i);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[j], i);
+ ret_cache = rte_member_add(setsum_cache, &keys[j], i);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_vbf >= 0 &&
+ ret_cache >= 0,
+ "insert function error");
+ }
+ }
+
+ /* single multimatch test */
+ for (i = 0; i < 5; i++) {
+ ret_vbf = rte_member_lookup_multi(setsum_vbf, &keys[i], 32,
+ set_ids_vbf);
+ ret_ht = rte_member_lookup_multi(setsum_ht, &keys[i], 32,
+ set_ids_ht);
+ ret_cache = rte_member_lookup_multi(setsum_cache, &keys[i], 32,
+ set_ids_cache);
+ /*
+ * for cache mode, it does not support multimatch
+ * the mutimatch should work like single match
+ */
+ TEST_ASSERT(ret_ht == 32 && ret_vbf == 32 && ret_cache == 1,
+ "single lookup_multi error");
+ TEST_ASSERT(set_ids_cache[0] == 32,
+ "single lookup_multi cache error");
+
+ for (j = 1; j < 33; j++) {
+ TEST_ASSERT(set_ids_ht[j-1] == j &&
+ set_ids_vbf[j-1] == j,
+ "single multimatch lookup error");
+ }
+ }
+ printf("lookup single key for multimatch success\n");
+
+ /* bulk multimatch test */
+
+ for (i = 0; i < 5; i++)
+ key_array[i] = &keys[i];
+ ret_vbf = rte_member_lookup_multi_bulk(setsum_vbf,
+ &key_array[0], num_key_ht, 32, match_count_vbf,
+ (MEMBER_SET_TYPE *)set_ids_vbf_m);
+
+ ret_ht = rte_member_lookup_multi_bulk(setsum_ht,
+ &key_array[0], num_key_vbf, 32, match_count_ht,
+ (MEMBER_SET_TYPE *)set_ids_ht_m);
+
+ ret_cache = rte_member_lookup_multi_bulk(setsum_cache,
+ &key_array[0], num_key_cache, 32, match_count_cache,
+ (MEMBER_SET_TYPE *)set_ids_cache_m);
+
+
+ for (j = 0; j < 5; j++) {
+ TEST_ASSERT(match_count_ht[j] == 32,
+ "bulk multimatch lookup HT match count error");
+ TEST_ASSERT(match_count_vbf[j] == 32,
+ "bulk multimatch lookup vBF match count error");
+ TEST_ASSERT(match_count_cache[j] == 1,
+ "bulk multimatch lookup CACHE match count error");
+ TEST_ASSERT(set_ids_cache_m[j][0] == 32,
+ "bulk multimatch lookup CACHE set value error");
+
+ for (i = 1; i < 33; i++) {
+ TEST_ASSERT(set_ids_ht_m[j][i-1] == i,
+ "bulk multimatch lookup HT set value error");
+ TEST_ASSERT(set_ids_vbf_m[j][i-1] == i,
+ "bulk multimatch lookup vBF set value error");
+ }
+ }
+
+ printf("lookup for bulk multimatch success\n");
+
+ return 0;
+}
+
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, KEY_SIZE);
+}
+
+static void
+setup_keys_and_data(void)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ /* Reset all arrays */
+ for (i = 0; i < KEY_SIZE; i++)
+ gened_keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < MAX_ENTRIES; i++) {
+ for (j = 0; j < KEY_SIZE; j++)
+ gened_keys[i][j] = rte_rand() & 0xFF;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(gened_keys, MAX_ENTRIES, KEY_SIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < MAX_ENTRIES - 1; i++) {
+ if (memcmp(gened_keys[i], gened_keys[i + 1],
+ KEY_SIZE) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < KEY_SIZE; j++)
+ gened_keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+}
+
+
+static inline int
+add_gened_keys(void *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret >= 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &gened_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+
+static inline int
+add_gened_keys_cache(void *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret == 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &gened_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static int
+test_member_loadfactor(void)
+{
+ unsigned int j;
+ unsigned int added_keys, average_keys_added = 0;
+ int ret;
+
+ setup_keys_and_data();
+
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+
+ params.key_len = KEY_SIZE;
+ params.name = "test_member_ht";
+ params.iscache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.iscache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+
+ if (setsum_ht == NULL || setsum_cache == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ /* test HT mode */
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_gened_keys(setsum_ht, &added_keys);
+ if (ret != -ENOSPC) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_ht);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when no space(non-cache) = %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+
+ /* test cache mode */
+ added_keys = average_keys_added = 0;
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_gened_keys_cache(setsum_cache, &added_keys);
+ if (ret != 1) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_cache);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when eviction happens(cache)= %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+ return 0;
+}
+
+static void
+perform_free(void)
+{
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+}
+
+static int
+test_member(void)
+{
+ if (test_member_create_bad_param() < 0)
+ return -1;
+
+ if (test_member_find_existing() < 0)
+ return -1;
+
+ if (test_member_create() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_insert() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_lookup() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_delete() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_multimatch() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_loadfactor() < 0) {
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ return -1;
+ }
+
+ perform_free();
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_autotest, test_member);
diff --git a/test/test/test_member_perf.c b/test/test/test_member_perf.c
new file mode 100644
index 0000000..2bb3a31
--- /dev/null
+++ b/test/test/test_member_perf.c
@@ -0,0 +1,643 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <inttypes.h>
+
+#include <rte_lcore.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+#include <rte_memcpy.h>
+#include <rte_thash.h>
+#include <rte_member.h>
+
+#include "test.h"
+
+#define NUM_KEYSIZES 10
+#define NUM_SHUFFLES 10
+#define MAX_KEYSIZE 64
+#define MAX_ENTRIES (1 << 19)
+#define KEYS_TO_ADD (MAX_ENTRIES * 75 / 100) /* 75% table utilization */
+#define NUM_LOOKUPS (KEYS_TO_ADD * 5) /* Loop among keys added, several times */
+#define VBF_SET_CNT 32
+#define BURST_SIZE 64
+#define VBF_FALSE_RATE 0.03
+
+
+static unsigned int test_socket_id;
+
+enum sstype {
+ HT = 0,
+ CACHE,
+ VBF,
+ NUM_TYPE
+};
+
+enum operations {
+ ADD = 0,
+ LOOKUP,
+ LOOKUP_BULK,
+ LOOKUP_MULTI,
+ LOOKUP_MULTI_BULK,
+ DELETE,
+ LOOKUP_MISS,
+ NUM_OPERATIONS
+};
+
+
+struct member_perf_params {
+ void *setsum[NUM_TYPE];
+ uint32_t key_size;
+ unsigned int cycle;
+};
+
+
+static uint32_t hashtest_key_lens[] = {
+ /* standard key sizes */
+ 4, 8, 16, 32, 48, 64,
+ /* IPv4 SRC + DST + protocol, unpadded */
+ 9,
+ /* IPv4 5-tuple, unpadded */
+ 13,
+ /* IPv6 5-tuple, unpadded */
+ 37,
+ /* IPv6 5-tuple, padded to 8-byte boundary */
+ 40
+};
+
+/* Array to store number of cycles per operation */
+uint64_t cycles[NUM_TYPE][NUM_KEYSIZES][NUM_OPERATIONS];
+uint64_t false_data[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_bulk[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi_bulk[NUM_TYPE][NUM_KEYSIZES];
+
+uint64_t false_hit[NUM_TYPE][NUM_KEYSIZES];
+
+
+MEMBER_SET_TYPE data[NUM_TYPE][/* Array to store the data */KEYS_TO_ADD];
+
+/* Array to store all input keys */
+uint8_t keys[KEYS_TO_ADD][MAX_KEYSIZE];
+
+/* Shuffle the keys that have been added, so lookups will be totally random */
+static void
+shuffle_input_keys(struct member_perf_params *params)
+{
+ MEMBER_SET_TYPE temp_data;
+ unsigned int i, j;
+ uint32_t swap_idx;
+ uint8_t temp_key[MAX_KEYSIZE];
+
+ for (i = KEYS_TO_ADD - 1; i > 0; i--) {
+ swap_idx = rte_rand() % i;
+ memcpy(temp_key, keys[i], hashtest_key_lens[params->cycle]);
+ memcpy(keys[i], keys[swap_idx],
+ hashtest_key_lens[params->cycle]);
+ memcpy(keys[swap_idx], temp_key,
+ hashtest_key_lens[params->cycle]);
+ for (j = 0; j < NUM_TYPE; j++) {
+ temp_data = data[j][i];
+ data[j][i] = data[j][swap_idx];
+ data[j][swap_idx] = temp_data;
+ }
+ }
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, MAX_KEYSIZE);
+}
+
+struct rte_member_parameters member_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = 4, /* Length of hash key. */
+
+ /* num_set and false_positive_rate only relevant to vBF setsum*/
+ .num_set = VBF_SET_CNT,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 0,
+ .sec_hash_seed = 1,
+ .socket_id = 0, /* NUMA Socket ID for memory. */
+ };
+
+
+
+static int
+setup_keys_and_data(struct member_perf_params *params, unsigned int cycle,
+ int miss)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ params->key_size = hashtest_key_lens[cycle];
+ params->cycle = cycle;
+
+ /* Reset all arrays */
+ for (i = 0; i < params->key_size; i++)
+ keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+
+ data[HT][i] = data[CACHE][i] = (rte_rand() & 0x7FFE) + 1;
+ data[VBF][i] = rte_rand() % VBF_SET_CNT + 1;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(keys, KEYS_TO_ADD, MAX_KEYSIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < KEYS_TO_ADD - 1; i++) {
+ if (memcmp(keys[i], keys[i + 1],
+ params->key_size) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+
+ /* Shuffle the random values again */
+ shuffle_input_keys(params);
+
+ /* For testing miss lookup, we insert half and lookup the other half */
+ unsigned int entry_cnt, bf_key_cnt;
+ if (!miss) {
+ entry_cnt = MAX_ENTRIES;
+ bf_key_cnt = KEYS_TO_ADD;
+ } else {
+ entry_cnt = MAX_ENTRIES / 2;
+ bf_key_cnt = KEYS_TO_ADD / 2;
+ }
+ member_params.false_positive_rate = VBF_FALSE_RATE;
+ member_params.key_len = params->key_size;
+ member_params.socket_id = test_socket_id;
+ member_params.num_keys = entry_cnt;
+ member_params.name = "test_member_ht";
+ member_params.iscache = 0;
+ member_params.type = RTE_MEMBER_TYPE_HT;
+ params->setsum[HT] = rte_member_create(&member_params);
+ if (params->setsum[HT] == NULL)
+ fprintf(stderr, "ht create fail\n");
+
+ member_params.name = "test_member_cache";
+ member_params.iscache = 1;
+ params->setsum[CACHE] = rte_member_create(&member_params);
+ if (params->setsum[CACHE] == NULL)
+ fprintf(stderr, "CACHE create fail\n");
+
+ member_params.name = "test_member_vbf";
+ member_params.type = RTE_MEMBER_TYPE_VBF;
+ member_params.num_keys = bf_key_cnt;
+ params->setsum[VBF] = rte_member_create(&member_params);
+ if (params->setsum[VBF] == NULL)
+ fprintf(stderr, "VBF create fail\n");
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] == NULL)
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+timed_adds(struct member_perf_params *params, int type)
+{
+ const uint64_t start_tsc = rte_rdtsc();
+ unsigned int i, a;
+ int32_t ret;
+
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][ADD] = time_taken / KEYS_TO_ADD;
+ return 0;
+}
+
+static int
+timed_lookups(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+
+ false_data[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+ MEMBER_SET_TYPE result;
+ int ret;
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != data[type][j])
+ false_data[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ MEMBER_SET_TYPE result[BURST_SIZE] = {0};
+ const void *keys_burst[BURST_SIZE];
+ int ret;
+
+ false_data_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_bulk(params->setsum[type],
+ &keys_burst[0],
+ BURST_SIZE,
+ result);
+ if (ret <= 0) {
+ printf("lookup bulk has wrong return value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k] != data[type][data_idx])
+ false_data_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_BULK] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+
+static int
+timed_lookups_multimatch(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ MEMBER_SET_TYPE result[RTE_MEMBER_BUCKET_ENTRIES] = {0};
+ int ret;
+ false_data_multi[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup_multi(params->setsum[type],
+ &keys[j], RTE_MEMBER_BUCKET_ENTRIES, result);
+ if (type != CACHE && ret <= 0) {
+ printf("lookup multi has wrong return value %d,"
+ "type %d\n", ret, type);
+ }
+ if (result[0] != data[type][j])
+ false_data_multi[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+
+static int
+timed_lookups_multimatch_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ MEMBER_SET_TYPE result[BURST_SIZE][RTE_MEMBER_BUCKET_ENTRIES] = {{0} };
+ const void *keys_burst[BURST_SIZE];
+ uint32_t match_count[BURST_SIZE];
+ int ret;
+
+ false_data_multi_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_multi_bulk(
+ params->setsum[type],
+ &keys_burst[0], BURST_SIZE,
+ RTE_MEMBER_BUCKET_ENTRIES, match_count,
+ (MEMBER_SET_TYPE *)result);
+ if (ret < 0) {
+ printf("lookup multimatch bulk has wrong return"
+ " value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ if (type != CACHE && match_count[k] == 0) {
+ printf("lookup multimatch bulk get "
+ "wrong match count\n");
+ return -1;
+ }
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k][0] != data[type][data_idx])
+ false_data_multi_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI_BULK] = time_taken /
+ NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_deletes(struct member_perf_params *params, int type)
+{
+ unsigned int i;
+ int32_t ret;
+
+ if (type == VBF)
+ return 0;
+ const uint64_t start_tsc = rte_rdtsc();
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_delete(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (type != CACHE && ret < 0) {
+ printf("delete error\n");
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][DELETE] = time_taken / KEYS_TO_ADD;
+
+ return 0;
+}
+
+
+static int
+timed_miss_lookup(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ int ret;
+
+ false_hit[type][params->cycle] = 0;
+
+ for (i = 0; i < KEYS_TO_ADD / 2; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ unsigned int a;
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t start_tsc = rte_rdtsc();
+ MEMBER_SET_TYPE result;
+
+ for (i = 0; i < 2 * NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = KEYS_TO_ADD / 2; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != RTE_MEMBER_NO_MATCH)
+ false_hit[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MISS] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+
+
+static void
+perform_frees(struct member_perf_params *params)
+{
+ int i;
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] != NULL) {
+ rte_member_free(params->setsum[i]);
+ params->setsum[i] = NULL;
+ }
+ }
+}
+
+static int
+exit_with_fail(const char *testname, struct member_perf_params *params,
+ unsigned int i, unsigned int j)
+{
+ printf("<<<<<Test %s failed at keysize %d iteration %d type %d>>>>>\n",
+ testname, hashtest_key_lens[params->cycle], i, j);
+ perform_frees(params);
+ return -1;
+}
+
+static int
+run_all_tbl_perf_tests(void)
+{
+ unsigned int i, j, k;
+ struct member_perf_params params;
+
+ printf("Measuring performance, please wait\n");
+ fflush(stdout);
+
+ test_socket_id = rte_socket_id();
+
+
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 0) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+
+ if (timed_adds(¶ms, j) < 0)
+ return exit_with_fail("timed_adds", ¶ms,
+ i, j);
+
+ for (k = 0; k < NUM_SHUFFLES; k++)
+ shuffle_input_keys(¶ms);
+
+ if (timed_lookups(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups", ¶ms,
+ i, j);
+
+ if (timed_lookups_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_bulk",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi_bulk",
+ ¶ms, i, j);
+
+ if (timed_deletes(¶ms, j) < 0)
+ return exit_with_fail("timed_deletes", ¶ms,
+ i, j);
+
+ /* Print a dot to show progress on operations */
+ }
+ printf(".");
+ fflush(stdout);
+
+ perform_frees(¶ms);
+ }
+
+ /* test false postivie rate using un-inserted keys */
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 1) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+ if (timed_miss_lookup(¶ms, j) < 0)
+ return exit_with_fail("timed_miss_lookup",
+ ¶ms, i, j);
+ }
+ perform_frees(¶ms);
+ }
+
+ printf("\nResults (in CPU cycles/operation)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "Add", "Lookup", "Lookup_bulk",
+ "lookup_multi", "lookup_multi_bulk", "Delete",
+ "miss_lookup");
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ for (k = 0; k < NUM_OPERATIONS; k++)
+ printf("%-18"PRIu64, cycles[j][i][k]);
+ printf("\n");
+ }
+ }
+
+ printf("\nFalse results rate (and false positive rate)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "fr_single", "fr_bulk", "fr_multi",
+ "fr_multi_bulk", "false_positive_rate");
+ /* key size not influence False rate so just print out one key size */
+ for (i = 0; i < 1; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ printf("%-18f", (float)false_data[j][i] / NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_hit[j][i] /
+ NUM_LOOKUPS);
+ printf("\n");
+ }
+ }
+
+ return 0;
+}
+
+static int
+test_member_perf(void)
+{
+
+ if (run_all_tbl_perf_tests() < 0)
+ return -1;
+
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v2 7/7] doc: add membership documentation
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
` (5 preceding siblings ...)
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 6/7] test/member: add functional and perf tests Yipeng Wang
@ 2017-09-02 1:24 ` Yipeng Wang
2017-09-04 13:19 ` Mcnamara, John
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-09-02 1:24 UTC (permalink / raw)
To: dev
Cc: stephen, luca.boccassi, charlie.tai, sameh.gobriel, ren.wang,
pablo.de.lara.guarch, yipeng1.wang
This patch adds the documentation for membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 +++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 ++++
doc/guides/prog_guide/img/member_i6.svg | 332 +++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 440 +++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
12 files changed, 3615 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 19e0d4f..fe87e09 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -105,7 +105,8 @@ The public API headers are grouped by topics:
[LPM IPv4 route] (@ref rte_lpm.h),
[LPM IPv6 route] (@ref rte_lpm6.h),
[ACL] (@ref rte_acl.h),
- [EFD] (@ref rte_efd.h)
+ [EFD] (@ref rte_efd.h),
+ [member] (@ref rte_member.h)
- **QoS**:
[metering] (@ref rte_meter.h),
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index 823554f..b792d6d 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -58,6 +58,7 @@ INPUT = doc/api/doxy-api-index.md \
lib/librte_mempool \
lib/librte_meter \
lib/librte_metrics \
+ lib/librte_member \
lib/librte_net \
lib/librte_pdump \
lib/librte_pipeline \
diff --git a/doc/guides/prog_guide/img/member_i1.svg b/doc/guides/prog_guide/img/member_i1.svg
new file mode 100644
index 0000000..fc5f56a
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i1.svg
@@ -0,0 +1,1613 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i1.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.18709in" height="4.75757in"
+ viewBox="0 0 517.471 342.545" xml:space="preserve" color-interpolation-filters="sRGB" class="st61">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false">
+ <v:userDefs>
+ <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/>
+ <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/>
+ </v:userDefs>
+ </v:documentProperties>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:3}
+ .st3 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em;opacity:0.219608}
+ .st4 {font-size:1em}
+ .st5 {fill:none;stroke:#41719c;stroke-width:3}
+ .st6 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em}
+ .st7 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em;opacity:0.219608}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st9 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st11 {fill:none;stroke:none;stroke-width:0.25}
+ .st12 {fill:#ffffff;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:0.75em;font-weight:bold}
+ .st14 {marker-end:url(#mrkr5-63);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st15 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st16 {fill:#5b9bd5;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st17 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st18 {fill:#feffff;font-family:Calibri;font-size:0.499992em}
+ .st19 {fill:#deebf6;stroke:#c8c8c8;stroke-width:0.25}
+ .st20 {fill:#000000;font-family:Calibri;font-size:0.499992em}
+ .st21 {marker-end:url(#mrkr5-178);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:#ff0000;font-family:Calibri;font-size:0.666664em}
+ .st24 {fill:#5b9bd5;fill-opacity:0.22}
+ .st25 {stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st26 {fill:#ffffff}
+ .st27 {stroke:#0070c0;stroke-width:0.25}
+ .st28 {fill:#5b9bd5;stroke:#0070c0;stroke-width:0.25}
+ .st29 {fill:#5b9bd5;stroke:#ffffff;stroke-width:0.25}
+ .st30 {fill:#5b9bd5}
+ .st31 {stroke:#c8c8c8;stroke-width:0.25}
+ .st32 {fill:#acccea;stroke:#c8c8c8;stroke-width:0.25}
+ .st33 {fill:#5b9bd5;fill-opacity:0.22;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st34 {fill:#000000;fill-opacity:0;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st35 {fill:url(#grad30-309);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st36 {fill:url(#grad25-313);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st37 {fill:url(#grad35-317);stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st38 {fill:url(#grad36-325);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st39 {fill:url(#grad40-335);stroke:#000000;stroke-linecap:butt;stroke-width:0.130208}
+ .st40 {fill:url(#grad39-342);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st41 {fill:url(#grad40-355);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st42 {fill:none}
+ .st43 {stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st44 {stroke:#ffffff;stroke-linecap:butt;stroke-width:0.130208}
+ .st45 {fill:url(#grad30-383);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st46 {fill:url(#grad36-396);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st47 {fill:none;stroke:#c8c8c8;stroke-width:0.75}
+ .st48 {fill:#9a9a9a;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st49 {fill:url(#grad40-415);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st50 {fill:url(#grad40-419);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st51 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25}
+ .st52 {fill:url(#grad35-430);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st53 {stroke:#c8c8c8;stroke-width:0.75}
+ .st54 {stroke:#4f88bb;stroke-width:0.75}
+ .st55 {fill:#feffff;font-family:Calibri;font-size:0.416656em}
+ .st56 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st57 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st58 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:2.25}
+ .st59 {fill:none;stroke:#0070c0;stroke-width:2.25}
+ .st60 {fill:#595959;font-family:Arial;font-size:0.666664em}
+ .st61 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Patterns_And_Gradients">
+ <linearGradient id="grad30-309" v:fillPattern="30" v:foreground="#97c2e6" v:background="#4274a2" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#4274a2;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-313" v:fillPattern="25" v:foreground="#5491d3" v:background="#246ba6" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#5491d3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </linearGradient>
+ <pattern id="grad35-317" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-318)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-319)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-320)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-321)"/>
+ </pattern>
+ <linearGradient id="grad27-318" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-319" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-320" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-321" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-325" v:fillPattern="36" v:foreground="#c0dff1" v:background="#246ba6" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#c0dff1;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-335" v:fillPattern="40" v:foreground="#c8e5c8" v:background="#19bf19" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#c8e5c8;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#19bf19;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad39-342" v:fillPattern="39" v:foreground="#5599d7" v:background="#b9daf2" cx="1" cy="1" r="1">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-355" v:fillPattern="40" v:foreground="#5599d7" v:background="#214383" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#214383;stop-opacity:1"/>
+ </radialGradient>
+ <linearGradient id="grad30-383" v:fillPattern="30" v:foreground="#97c2e6" v:background="#6ba4dc" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#6ba4dc;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-396" v:fillPattern="36" v:foreground="#89bee9" v:background="#b9daf2" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#89bee9;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-415" v:fillPattern="40" v:foreground="#000000" v:background="#ffffff" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#000000;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffffff;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-419" v:fillPattern="40" v:foreground="#ffffff" v:background="#9a9a9a" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#9a9a9a;stop-opacity:1"/>
+ </radialGradient>
+ <pattern id="grad35-430" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-431)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-432)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-433)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-434)"/>
+ </pattern>
+ <linearGradient id="grad27-431" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-432" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-433" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-434" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ </defs>
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-63" class="st15" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-178" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/>
+ </v:userDefs>
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <v:layer v:name="Flowchart" v:index="0"/>
+ <g id="group165-1" transform="translate(21.7794,-24.0978)" v:mID="165" v:groupContext="group">
+ <title>Sheet.165</title>
+ <g id="group1-2" transform="translate(308.647,-25.7109)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-3" v:mID="2" v:groupContext="shape" transform="translate(11.5732,-58.1913)">
+ <title>Circle</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow2-4" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="shape3-13" v:mID="3" v:groupContext="shape" transform="translate(58.9839,-58.9839)">
+ <title>Circle.23</title>
+ <desc>List 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow3-14" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="17.73" y="318.02" class="st7" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="17.73" y="318.02" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <g id="shape4-19" v:mID="4" v:groupContext="shape">
+ <title>Circle.24</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow4-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="group5-29" transform="translate(50.7413,-4.53722)" v:mID="5" v:groupContext="group">
+ <title>Sheet.5</title>
+ <g id="shape6-30" v:mID="6" v:groupContext="shape" transform="translate(344.2,300.5) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-31" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st9"/>
+ </g>
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape7-34" v:mID="7" v:groupContext="shape" transform="translate(-0.884982,-14.7157)">
+ <title>Sheet.7</title>
+ <desc>setsum</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="12.9268" cy="336.238" width="25.86" height="12.6135"/>
+ <rect x="0" y="329.932" width="25.8535" height="12.6135" class="st11"/>
+ <text x="6.37" y="334.44" class="st12" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsu<tspan
+ x="10.49" dy="1.2em" class="st4">m</tspan></text> </g>
+ </g>
+ <g id="shape8-38" v:mID="8" v:groupContext="shape" transform="translate(72.5955,0)">
+ <title>Circle.29</title>
+ <desc>List 2 matching Criteria 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow8-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ </g>
+ <g id="group9-48" transform="translate(31.6515,-49.9094)" v:mID="9" v:groupContext="group">
+ <title>Sheet.9</title>
+ <g id="group10-49" transform="translate(99.5691,0)" v:mID="10" v:groupContext="group">
+ <title>Sheet.10</title>
+ <g id="shape11-50" v:mID="11" v:groupContext="shape" transform="translate(346.175,275.999) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st9"/>
+ </g>
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape12-54" v:mID="12" v:groupContext="shape" transform="translate(355.063,285.074) rotate(90)">
+ <title>Sheet.12</title>
+ <desc>Set Summary</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1985" cy="332.563" width="48.4" height="19.9638"/>
+ <rect x="0" y="322.581" width="48.397" height="19.9638" class="st11"/>
+ <text x="18.25" y="329.86" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Set <tspan
+ x="6.38" dy="1.2em" class="st4">Summary</tspan></text> </g>
+ </g>
+ <g id="shape13-58" v:mID="13" v:groupContext="shape" transform="translate(57.5835,-54.4467)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.9 342.55" class="st14"/>
+ </g>
+ <g id="shape14-64" v:mID="14" v:groupContext="shape" transform="translate(20.2363,-51.8439)">
+ <title>Sheet.14</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="25.3328" cy="333.471" width="50.67" height="18.1489"/>
+ <rect x="0" y="324.396" width="50.6656" height="18.1489" class="st11"/>
+ <text x="14.12" y="335.27" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape15-67" v:mID="15" v:groupContext="shape" transform="translate(5.02911,1.60865) rotate(-26.0815)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L39.25 342.55" class="st14"/>
+ </g>
+ <g id="shape16-72" v:mID="16" v:groupContext="shape" transform="translate(155.629,-33.273)">
+ <title>Sheet.16</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.34 342.55" class="st14"/>
+ </g>
+ <g id="shape17-77" v:mID="17" v:groupContext="shape" transform="translate(304.141,0.595416) rotate(25.6934)">
+ <title>Sheet.17</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L42.68 342.55" class="st14"/>
+ </g>
+ <g id="shape18-82" v:mID="18" v:groupContext="shape" transform="translate(102.642,654.842) rotate(180)">
+ <title>Sheet.18</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape19-87" v:mID="19" v:groupContext="shape" transform="translate(-15.1809,-33.9928)">
+ <title>Sheet.19</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.75" cy="338.045" width="85.5" height="9"/>
+ <rect x="0" y="333.545" width="85.5" height="9" class="st11"/>
+ <text x="5.06" y="339.85" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape20-90" v:mID="20" v:groupContext="shape" transform="translate(102.844,679.041) rotate(180)">
+ <title>Sheet.20</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape21-95" v:mID="21" v:groupContext="shape" transform="translate(-35.4309,-11.4928)">
+ <title>Sheet.21</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="337.971" width="108" height="9.14889"/>
+ <rect x="0" y="333.396" width="108" height="9.14889" class="st11"/>
+ <text x="6.36" y="339.77" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape22-98" v:mID="22" v:groupContext="shape" transform="translate(541.496,275.999) rotate(90)">
+ <title>Sheet.22</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape23-101" v:mID="23" v:groupContext="shape" transform="translate(541.496,300.198) rotate(90)">
+ <title>Sheet.23</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape24-104" v:mID="24" v:groupContext="shape" transform="translate(541.496,324.396) rotate(90)">
+ <title>Sheet.24</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ </g>
+ <g id="group25-107" transform="translate(285.961,-178.628)" v:mID="25" v:groupContext="group">
+ <title>Sheet.25</title>
+ <g id="shape26-108" v:mID="26" v:groupContext="shape" transform="translate(51.2583,-51.2583)">
+ <title>Circle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-109" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape27-112" v:mID="27" v:groupContext="shape" transform="translate(107.177,-55.9182)">
+ <title>Circle.156</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-113" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape28-116" v:mID="28" v:groupContext="shape" transform="translate(79.2174,-83.8773)">
+ <title>Circle.157</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow28-117" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape29-120" v:mID="29" v:groupContext="shape" transform="translate(153.775,-51.2583)">
+ <title>Circle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow29-121" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape30-124" v:mID="30" v:groupContext="shape" transform="translate(93.197,-18.6394)">
+ <title>Circle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow30-125" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape31-128" v:mID="31" v:groupContext="shape" transform="translate(27.4102,-57.9329) rotate(-7.12502)">
+ <title>Sheet.31</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L31.41 342.55" class="st14"/>
+ </g>
+ <g id="shape32-133" v:mID="32" v:groupContext="shape" transform="translate(182.13,-60.5772) rotate(9.46232)">
+ <title>Sheet.32</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L22.18 342.55" class="st14"/>
+ </g>
+ <g id="shape33-138" v:mID="33" v:groupContext="shape" transform="translate(47.8843,595.237) rotate(-160.346)">
+ <title>Sheet.33</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L63.11 342.55" class="st14"/>
+ </g>
+ <g id="shape34-143" v:mID="34" v:groupContext="shape" transform="translate(292.945,525.785) rotate(141.977)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L20.97 342.55" class="st14"/>
+ </g>
+ <g id="shape35-148" v:mID="35" v:groupContext="shape" transform="translate(-95.8971,591.793) rotate(-145.945)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L28.55 342.55" class="st14"/>
+ </g>
+ <g id="shape36-153" v:mID="36" v:groupContext="shape" transform="translate(37.2788,2.27374E-013)">
+ <title>Rectangle.167</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.8652" cy="335.555" width="21.74" height="13.9795"/>
+ <g id="shadow36-154" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st10"/>
+ <text x="5" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape37-158" v:mID="37" v:groupContext="shape" transform="translate(55.9182,2.27374E-013)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow37-159" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape38-163" v:mID="38" v:groupContext="shape" transform="translate(-1.65867E-013,-32.6189)">
+ <title>Rectangle.169</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.3796" cy="335.555" width="20.76" height="13.9795"/>
+ <g id="shadow38-164" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st10"/>
+ <text x="4.51" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape39-168" v:mID="39" v:groupContext="shape" transform="translate(18.6394,-32.6189)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow39-169" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape40-173" v:mID="40" v:groupContext="shape" transform="translate(197.019,626.053) rotate(161.565)">
+ <title>Sheet.40</title>
+ <path d="M0 328.31 A55.7483 27.2427 -124.2 0 0 42.37 334.19 L42.47 333.85" class="st21"/>
+ </g>
+ <g id="shape41-179" v:mID="41" v:groupContext="shape" transform="translate(154.607,584.177) rotate(161.121)">
+ <title>Sheet.41</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 319.39 A80.5593 29.9756 -101.99 0 0 41.7 325.37 L41.79 325.02" class="st21"/>
+ </g>
+ <g id="shape42-184" v:mID="42" v:groupContext="shape" transform="translate(3.02481,-66.7025)">
+ <title>Sheet.42</title>
+ <desc>Encode ID</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="19.4569" cy="335.555" width="38.92" height="13.9795"/>
+ <rect x="0" y="328.566" width="38.9138" height="13.9795" class="st11"/>
+ <text x="7.51" y="333.16" class="st23" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Encode <tspan
+ x="15.99" dy="1.2em" class="st4">ID</tspan></text> </g>
+ </g>
+ <g id="group43-188" transform="translate(12.0993,-165.858)" v:mID="43" v:groupContext="group">
+ <title>Sheet.43</title>
+ <g id="group44-189" transform="translate(7.21495,-75.757)" v:mID="44" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User</title>
+ <g id="shape45-190" v:mID="45" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.45</title>
+ <g id="shadow45-191" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape46-206" v:mID="46" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.46</title>
+ <g id="shadow46-207" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape47-210" v:mID="47" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.47</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape48-212" v:mID="48" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.48</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group49-215" transform="translate(7.21495,-47.1858)" v:mID="49" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.7</title>
+ <g id="shape50-216" v:mID="50" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.50</title>
+ <g id="shadow50-217" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape51-232" v:mID="51" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.51</title>
+ <g id="shadow51-233" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape52-236" v:mID="52" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.52</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape53-238" v:mID="53" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group54-241" transform="translate(7.21495,-18.6146)" v:mID="54" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.12</title>
+ <g id="shape55-242" v:mID="55" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.55</title>
+ <g id="shadow55-243" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape56-258" v:mID="56" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.56</title>
+ <g id="shadow56-259" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape57-262" v:mID="57" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape58-264" v:mID="58" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group59-267" transform="translate(171.161,-45.6707)" v:mID="59" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>Data Center</title>
+ <g id="shape60-268" v:mID="60" v:groupContext="shape" v:layerMember="0">
+ <title>Sheet.60</title>
+ <g id="shadow60-269" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st9"/>
+ </g>
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st10"/>
+ </g>
+ <g id="shape61-272" v:mID="61" v:groupContext="shape" v:layerMember="0" transform="translate(6.86487,-7.30475)">
+ <title>Sheet.61</title>
+ <g id="shadow61-273" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39
+ 332.44 L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69
+ L29.57 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64
+ 342.55 L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75
+ 342.55 L18.75 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69
+ L10.82 311.79 L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st9"/>
+ </g>
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39 332.44
+ L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69 L29.57
+ 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64 342.55
+ L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75 342.55 L18.75
+ 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69 L10.82 311.79
+ L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st10"/>
+ </g>
+ <g id="shape62-276" v:mID="62" v:groupContext="shape" v:layerMember="0" transform="translate(20.0835,-20.5174)">
+ <title>Sheet.62</title>
+ <g id="shadow62-277" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36
+ ZM23.46 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36
+ ZM2.27 341.36 A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z"
+ class="st24"/>
+ </g>
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36 ZM23.46
+ 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36 ZM2.27 341.36
+ A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z" class="st30"/>
+ </g>
+ <g id="shape63-282" v:mID="63" v:groupContext="shape" v:layerMember="0" transform="translate(14.2717,-12.5134)">
+ <title>Sheet.63</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <g id="shadow63-283" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17
+ 340.12 L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89
+ L43.09 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2
+ 342.55 ZM21.2 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69
+ L29.27 337.69 L29.27 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74
+ L-0 341.74 L-0 342.55 ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69
+ L8.08 337.69 L8.08 336.89 L-0 336.89 L-0 337.69 Z" class="st24"/>
+ </g>
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17 340.12
+ L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89 L43.09
+ 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2 342.55 ZM21.2
+ 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69 L29.27 337.69 L29.27
+ 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74 L-0 341.74 L-0 342.55
+ ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69 L8.08 337.69 L8.08 336.89
+ L-0 336.89 L-0 337.69 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group64-288" transform="translate(59.5234,-47.1858)" v:mID="64" v:groupContext="group">
+ <v:custProps>
+ <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false"
+ v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/>
+ <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true"
+ v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Device)"/>
+ <v:cp v:nameU="SubShapeType" v:lbl="SubShapeType" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Load balancer)"/>
+ </v:custProps>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/>
+ <v:ud v:nameU="SolSH" v:prompt="" v:val="VT14({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/>
+ <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <title>Load balancer</title>
+ <g id="shape65-289" v:mID="65" v:groupContext="shape" transform="translate(0,-1.653)">
+ <title>Sheet.65</title>
+ <g id="shadow65-290" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st24"/>
+ <path d="M0 332.02 L16.23 332.02" class="st25"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st25"/>
+ </g>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st30"/>
+ <path d="M0 332.02 L16.23 332.02" class="st31"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st31"/>
+ </g>
+ <g id="shape66-297" v:mID="66" v:groupContext="shape" transform="translate(1.81062,-2.91583)">
+ <title>Sheet.66</title>
+ <g id="shadow66-298" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44
+ L8.34 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45
+ 338.78 L10.78 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22
+ ZM10.48 335.2 L8.29 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46
+ 334.9 L10.21 334.58 L9.55 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19
+ 338.43 C4.19 339.56 5.11 340.48 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29
+ 7.38 336.37 6.25 336.37 ZM6.25 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02
+ 339.83 6.25 339.83 C5.47 339.83 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02
+ ZM2.62 338.14 L0 338.14 L0 338.71 L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73
+ 337.47 L1.95 337.47 L2.62 338.14 Z" class="st9"/>
+ </g>
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44 L8.34
+ 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45 338.78 L10.78
+ 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22 ZM10.48 335.2 L8.29
+ 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46 334.9 L10.21 334.58 L9.55
+ 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19 338.43 C4.19 339.56 5.11 340.48
+ 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29 7.38 336.37 6.25 336.37 ZM6.25
+ 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02 339.83 6.25 339.83 C5.47 339.83
+ 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02 ZM2.62 338.14 L0 338.14 L0 338.71
+ L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73 337.47 L1.95 337.47 L2.62 338.14 Z"
+ class="st32"/>
+ </g>
+ </g>
+ <g id="group67-301" transform="translate(104.617,-86.5795)" v:mID="67" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server</title>
+ <g id="shape68-302" v:mID="68" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.68</title>
+ <g id="shadow68-303" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape69-306" v:mID="69" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.69</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape70-310" v:mID="70" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.70</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape71-314" v:mID="71" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.71</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape72-322" v:mID="72" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.72</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape73-326" v:mID="73" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.73</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape74-329" v:mID="74" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.74</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape75-332" v:mID="75" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.75</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape76-336" v:mID="76" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.76</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape77-339" v:mID="77" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.77</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape78-343" v:mID="78" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.78</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape79-346" v:mID="79" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.79</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape80-349" v:mID="80" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.80</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape81-352" v:mID="81" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.81</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape82-356" v:mID="82" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.82</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape83-359" v:mID="83" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.83</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape84-362" v:mID="84" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.84</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape85-365" v:mID="85" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.85</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape86-368" v:mID="86" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.86</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape87-371" v:mID="87" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.87</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape88-374" v:mID="88" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.88</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape89-377" v:mID="89" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.89</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape90-380" v:mID="90" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.90</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape91-384" v:mID="91" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.91</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape92-387" v:mID="92" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.92</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape93-390" v:mID="93" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.93</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape94-393" v:mID="94" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.94</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape95-397" v:mID="95" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.95</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape96-400" v:mID="96" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.96</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape97-402" v:mID="97" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.97</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape98-405" v:mID="98" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.98</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape99-408" v:mID="99" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.99</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape100-410" v:mID="100" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.100</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape101-412" v:mID="101" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.101</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape102-416" v:mID="102" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.102</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape103-420" v:mID="103" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.103</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape104-427" v:mID="104" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.104</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape105-435" v:mID="105" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.105</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape106-440" v:mID="106" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.106</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="group107-443" transform="translate(104.617,-33.8201)" v:mID="107" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server.104</title>
+ <g id="shape108-444" v:mID="108" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.108</title>
+ <g id="shadow108-445" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape109-448" v:mID="109" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.109</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape110-451" v:mID="110" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.110</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape111-454" v:mID="111" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.111</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape112-457" v:mID="112" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.112</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape113-460" v:mID="113" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.113</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape114-463" v:mID="114" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.114</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape115-466" v:mID="115" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.115</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape116-469" v:mID="116" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.116</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape117-472" v:mID="117" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.117</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape118-475" v:mID="118" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.118</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape119-478" v:mID="119" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.119</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape120-481" v:mID="120" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.120</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape121-484" v:mID="121" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.121</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape122-487" v:mID="122" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.122</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape123-490" v:mID="123" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.123</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape124-493" v:mID="124" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.124</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape125-496" v:mID="125" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.125</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape126-499" v:mID="126" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.126</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape127-502" v:mID="127" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.127</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape128-505" v:mID="128" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.128</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape129-508" v:mID="129" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.129</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape130-511" v:mID="130" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.130</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape131-514" v:mID="131" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.131</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape132-517" v:mID="132" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.132</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape133-520" v:mID="133" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.133</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape134-523" v:mID="134" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.134</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape135-526" v:mID="135" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.135</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape136-529" v:mID="136" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.136</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape137-531" v:mID="137" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.137</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape138-534" v:mID="138" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.138</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape139-537" v:mID="139" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.139</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape140-539" v:mID="140" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.140</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape141-541" v:mID="141" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.141</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape142-544" v:mID="142" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.142</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape143-547" v:mID="143" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.143</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape144-554" v:mID="144" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.144</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape145-557" v:mID="145" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.145</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape146-562" v:mID="146" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.146</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="shape147-565" v:mID="147" v:groupContext="shape" transform="translate(427.321,214.49) rotate(90)">
+ <title>Cloud</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54 Z" class="st42"/>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54" class="st54"/>
+ <path d="M11.05 312.14 A8.59237 10.0375 0 0 1 5.37 311.54" class="st54"/>
+ <path d="M40.54 332.09 A8.62978 10.0812 -180 0 0 42.1 335.11" class="st54"/>
+ <path d="M73.92 326.65 A6.96633 8.13801 0 0 1 73.14 330.27" class="st54"/>
+ <path d="M89.7 308.52 A7.30994 8.5394 0 0 1 95.9 318.19" class="st54"/>
+ <path d="M103.15 297.64 A6.67364 7.79609 -180 0 0 106.25 294.02" class="st54"/>
+ <path d="M37.96 278.3 A10.2914 12.0219 -180 0 0 34.86 275.89" class="st54"/>
+ </g>
+ <g id="shape148-574" v:mID="148" v:groupContext="shape" transform="translate(110.222,-64.9346)">
+ <title>Triangle</title>
+ <desc>setsum</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="15.6995" cy="336.449" width="31.4" height="12.1933"/>
+ <g id="shadow148-575" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st9"/>
+ </g>
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st10"/>
+ <text x="8.35" y="337.95" class="st55" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsum</text> </g>
+ <g id="shape149-579" v:mID="149" v:groupContext="shape" transform="translate(292.639,20.8827) rotate(45)">
+ <title>Sheet.149</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L14.71 342.55" class="st14"/>
+ </g>
+ <g id="shape150-584" v:mID="150" v:groupContext="shape" transform="translate(43.2897,-54.1122)">
+ <title>Sheet.150</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L10.07 342.55" class="st14"/>
+ </g>
+ <g id="shape151-589" v:mID="151" v:groupContext="shape" transform="translate(-112.261,8.34531) rotate(-28.1394)">
+ <title>Sheet.151</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L18 342.55" class="st14"/>
+ </g>
+ <g id="shape152-594" v:mID="152" v:groupContext="shape">
+ <title>Sheet.152</title>
+ <desc>Clients</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="21.5" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Clients</text> </g>
+ <g id="shape153-597" v:mID="153" v:groupContext="shape" transform="translate(83.578,-9.58078)">
+ <title>Sheet.153</title>
+ <desc>Distributed Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.0677" cy="337.134" width="84.14" height="10.8224"/>
+ <rect x="0" y="331.723" width="84.1355" height="10.8224" class="st11"/>
+ <text x="13.1" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Distributed Cache</text> </g>
+ <g id="shape154-600" v:mID="154" v:groupContext="shape" transform="translate(181.983,-18.6146)">
+ <title>Sheet.154</title>
+ <desc>Web Servers</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="11.93" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Web Servers</text> </g>
+ <g id="shape155-603" v:mID="155" v:groupContext="shape" transform="translate(96.6068,630.978) rotate(180)">
+ <title>Simple Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow155-604" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ <g id="shape156-607" v:mID="156" v:groupContext="shape" transform="translate(173.159,625.567) rotate(180)">
+ <title>Simple Arrow.153</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow156-608" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ </g>
+ <g id="shape157-611" v:mID="157" v:groupContext="shape" transform="translate(0,-149.475)">
+ <title>Rectangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow157-612" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape158-615" v:mID="158" v:groupContext="shape" transform="translate(271.116,-149.475)">
+ <title>Rectangle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow158-616" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape159-619" v:mID="159" v:groupContext="shape">
+ <title>Rectangle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow159-620" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape160-623" v:mID="160" v:groupContext="shape" transform="translate(271.116,0)">
+ <title>Rectangle.160</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow160-624" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape161-627" v:mID="161" v:groupContext="shape" transform="translate(83.578,-151.241)">
+ <title>Sheet.161</title>
+ <desc>(a) Distributed Web Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow161-628" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(a) Distributed Web Cache</text> </g>
+ <g id="shape162-632" v:mID="162" v:groupContext="shape" transform="translate(319.513,-151.241)">
+ <title>Sheet.162</title>
+ <desc>(b) Detecting Routing Loops</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow162-633" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(b) Detecting Routing Loops</text> </g>
+ <g id="shape163-637" v:mID="163" v:groupContext="shape" transform="translate(77.5283,-3.35965)">
+ <title>Sheet.163</title>
+ <desc>(c) In-order Workload Scheduler</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="63.5211" cy="333.806" width="127.05" height="17.4792"/>
+ <g id="shadow163-638" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(c) In-order Workload Scheduler</text> </g>
+ <g id="shape164-642" v:mID="164" v:groupContext="shape" transform="translate(307.414,-3.35965)">
+ <title>Sheet.164</title>
+ <desc>(d) Database Semi-join Operations</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="66.2253" cy="333.806" width="132.46" height="17.4792"/>
+ <g id="shadow164-643" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(d) Database Semi-join Operations</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i2.svg b/doc/guides/prog_guide/img/member_i2.svg
new file mode 100644
index 0000000..759c654
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i2.svg
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i2.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="4.38194in" height="1.25694in"
+ viewBox="0 0 315.5 90.5" xml:space="preserve" color-interpolation-filters="sRGB" class="st6">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:none;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st3 {baseline-shift:32.4943%;font-size:0.649886em}
+ .st4 {font-size:1em}
+ .st5 {font-family:Cambria Math;font-size:1em}
+ .st6 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(0.25,-0.25)">
+ <title>Sheet.3</title>
+ <desc>False Positive Probability = (1-(1-1/m)kn)k ≃ (1-ekn/m)k</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="157.5" cy="45.5" width="315" height="90"/>
+ <rect x="0" y="0.5" width="315" height="90" class="st1"/>
+ <text x="8.28" y="49.82" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>False Positive Probability = (1-(1-1/m)<tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan dy="0.152em" class="st4">)</tspan><tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan><tspan dy="0.152em" class="st4"> </tspan><tspan
+ class="st5">≃</tspan> (1-e<tspan dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan class="st3"
+ v:baseFontSize="14">/</tspan><tspan class="st3" v:baseFontSize="14">m</tspan><tspan dy="0.152em"
+ class="st4">)</tspan><tspan dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i3.svg b/doc/guides/prog_guide/img/member_i3.svg
new file mode 100644
index 0000000..41e92cb
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i3.svg
@@ -0,0 +1,148 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i3.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ width="4.71875in" height="2.84375in" viewBox="0 0 339.75 204.75" xml:space="preserve" color-interpolation-filters="sRGB"
+ class="st14">
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22}
+ .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25}
+ .st4 {marker-end:url(#mrkr5-32);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st5 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st6 {fill:#feffff;font-family:Calibri;font-size:0.833336em}
+ .st7 {font-size:1em}
+ .st8 {fill:#deebf6;stroke:#c7c8c8;stroke-width:0.25}
+ .st9 {fill:#000000;font-family:Calibri;font-size:0.833336em}
+ .st10 {marker-end:url(#mrkr5-84);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st12 {fill:none;stroke:none;stroke-width:0.25}
+ .st13 {fill:#ff0000;font-family:Calibri;font-size:1.00001em}
+ .st14 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-32" class="st5" refX="-6.16" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-84" class="st11" refX="-5.8" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <defs id="Filters">
+ <filter id="filter_2">
+ <feGaussianBlur stdDeviation="2"/>
+ </filter>
+ </defs>
+ <g>
+ <title>Page-1</title>
+ <g id="group174-1" transform="translate(3.0294,-5.3478)">
+ <title>Sheet.174</title>
+ <g id="shape155-2" transform="translate(99,-99)">
+ <title>Circle</title>
+ <g id="shadow155-3" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape156-7" transform="translate(207,-108)">
+ <title>Circle.156</title>
+ <g id="shadow156-8" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape157-12" transform="translate(153,-162)">
+ <title>Circle.157</title>
+ <g id="shadow157-13" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape158-17" transform="translate(297,-99)">
+ <title>Circle.158</title>
+ <g id="shadow158-18" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape159-22" transform="translate(180,-36)">
+ <title>Circle.159</title>
+ <g id="shadow159-23" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape160-27" transform="translate(109.604,-115.419) rotate(-7.12502)">
+ <title>Sheet.160</title>
+ <path d="M0 204.75 L66.4 204.75" class="st4"/>
+ </g>
+ <g id="shape161-33" transform="translate(276.661,-123.214) rotate(9.46232)">
+ <title>Sheet.161</title>
+ <path d="M0 204.75 L48.58 204.75" class="st4"/>
+ </g>
+ <g id="shape162-38" transform="translate(246.135,262.572) rotate(-160.346)">
+ <title>Sheet.162</title>
+ <path d="M0 204.75 L127.63 204.75" class="st4"/>
+ </g>
+ <g id="shape163-43" transform="translate(284.391,198.775) rotate(141.977)">
+ <title>Sheet.163</title>
+ <path d="M0 204.75 L46.23 204.75" class="st4"/>
+ </g>
+ <g id="shape164-48" transform="translate(70.6118,307.655) rotate(-145.945)">
+ <title>Sheet.164</title>
+ <path d="M0 204.75 L60.88 204.75" class="st4"/>
+ </g>
+ <g id="shape167-53" transform="translate(72,0)">
+ <title>Rectangle.167</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow167-54" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape168-60" transform="translate(108,0)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <g id="shadow168-61" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape169-66" transform="translate(0,-63)">
+ <title>Rectangle.169</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow169-67" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape170-73" transform="translate(36,-63)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <g id="shadow170-74" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape171-79" transform="translate(240.248,331.493) rotate(161.565)">
+ <title>Sheet.171</title>
+ <path d="M-0 190.52 A81.3416 36.0611 -153.48 0 0 82.31 195.86 L82.49 195.55" class="st10"/>
+ </g>
+ <g id="shape172-85" transform="translate(156.426,260.029) rotate(161.565)">
+ <title>Sheet.172</title>
+ <path d="M-0 181.6 A88.1422 54.1439 -124.1 0 0 82.68 187.13 L82.83 186.81" class="st10"/>
+ </g>
+ <g id="shape173-90" transform="translate(18,-121.5)">
+ <title>Sheet.173</title>
+ <desc>Encode ID</desc>
+ <rect x="0" y="177.75" width="63" height="27" class="st12"/>
+ <text x="7.02" y="194.85" class="st13">Encode ID</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i4.svg b/doc/guides/prog_guide/img/member_i4.svg
new file mode 100644
index 0000000..a2b6f2f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i4.svg
@@ -0,0 +1,450 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i4.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.625in" height="3.125in" viewBox="0 0 549 225"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st18">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st3 {marker-end:url(#mrkr5-10);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st4 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st5 {visibility:visible}
+ .st6 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st7 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st8 {fill:none;stroke:none;stroke-width:0.25}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st10 {stroke:#5b9bd5;stroke-dasharray:1.5,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5}
+ .st11 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st12 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st14 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st15 {font-size:1em}
+ .st16 {marker-end:url(#mrkr5-162);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st17 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st18 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-10" class="st4" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-162" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group47-1" transform="translate(3.0294,-0.25)" v:mID="47" v:groupContext="group">
+ <title>Sheet.47</title>
+ <g id="shape1-2" v:mID="1" v:groupContext="shape" transform="translate(177.75,-191.922)">
+ <title>Sheet.1</title>
+ <desc>Element</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="216" width="108" height="18"/>
+ <rect x="0" y="207" width="108" height="18" class="st1"/>
+ <text x="33.77" y="219.6" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Element</text> </g>
+ <g id="shape2-5" v:mID="2" v:groupContext="shape" transform="translate(456.75,33.0781) rotate(90)">
+ <title>Sheet.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L18.65 225" class="st3"/>
+ </g>
+ <g id="shape3-11" v:mID="3" v:groupContext="shape" transform="translate(0,-67.0469)">
+ <title>Rectangle.54</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow3-12" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape4-15" v:mID="4" v:groupContext="shape" transform="translate(27,-67.0469)">
+ <title>Rectangle.55</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow4-16" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape5-19" v:mID="5" v:groupContext="shape" transform="translate(54,-67.0469)">
+ <title>Rectangle.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow5-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape6-23" v:mID="6" v:groupContext="shape" transform="translate(0,-53.5469)">
+ <title>Rectangle.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-24" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape7-27" v:mID="7" v:groupContext="shape" transform="translate(27,-53.5469)">
+ <title>Rectangle.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow7-28" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape8-31" v:mID="8" v:groupContext="shape" transform="translate(54,-53.5469)">
+ <title>Rectangle.59</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow8-32" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape9-35" v:mID="9" v:groupContext="shape" transform="translate(5.625,-72.6719)">
+ <title>Sheet.9</title>
+ <desc>BF-1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-1</text> </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(128.25,-65.0781)">
+ <title>Rectangle.74</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow10-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape11-42" v:mID="11" v:groupContext="shape" transform="translate(155.25,-65.0781)">
+ <title>Rectangle.75</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-43" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(182.25,-65.0781)">
+ <title>Rectangle.76</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow12-47" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape13-50" v:mID="13" v:groupContext="shape" transform="translate(128.25,-51.5781)">
+ <title>Rectangle.77</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow13-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape14-54" v:mID="14" v:groupContext="shape" transform="translate(155.25,-51.5781)">
+ <title>Rectangle.78</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow14-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape15-58" v:mID="15" v:groupContext="shape" transform="translate(182.25,-51.5781)">
+ <title>Rectangle.79</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow15-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape16-62" v:mID="16" v:groupContext="shape" transform="translate(301.5,-65.0781)">
+ <title>Rectangle.81</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow16-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape17-66" v:mID="17" v:groupContext="shape" transform="translate(328.5,-65.0781)">
+ <title>Rectangle.82</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow17-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape18-70" v:mID="18" v:groupContext="shape" transform="translate(355.5,-65.0781)">
+ <title>Rectangle.83</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow18-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape19-74" v:mID="19" v:groupContext="shape" transform="translate(301.5,-51.5781)">
+ <title>Rectangle.84</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow19-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape20-78" v:mID="20" v:groupContext="shape" transform="translate(328.5,-51.5781)">
+ <title>Rectangle.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow20-79" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape21-82" v:mID="21" v:groupContext="shape" transform="translate(355.5,-51.5781)">
+ <title>Rectangle.86</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow21-83" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape22-86" v:mID="22" v:groupContext="shape" transform="translate(447.75,-65.6406)">
+ <title>Rectangle.88</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow22-87" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape23-90" v:mID="23" v:groupContext="shape" transform="translate(474.75,-65.6406)">
+ <title>Rectangle.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow23-91" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape24-94" v:mID="24" v:groupContext="shape" transform="translate(501.75,-65.6406)">
+ <title>Rectangle.90</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow24-95" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape25-98" v:mID="25" v:groupContext="shape" transform="translate(447.75,-52.1406)">
+ <title>Rectangle.91</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow25-99" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape26-102" v:mID="26" v:groupContext="shape" transform="translate(474.75,-52.1406)">
+ <title>Rectangle.92</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-103" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape27-106" v:mID="27" v:groupContext="shape" transform="translate(501.75,-52.1406)">
+ <title>Rectangle.93</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-107" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape28-110" v:mID="28" v:groupContext="shape" transform="translate(213.75,-63.9531)">
+ <title>Sheet.28</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L83.25 225" class="st10"/>
+ </g>
+ <g id="shape29-113" v:mID="29" v:groupContext="shape" transform="translate(387,-63.9531)">
+ <title>Sheet.29</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L54 225" class="st10"/>
+ </g>
+ <g id="group31-116" transform="translate(184.5,-113.172)" v:mID="31" v:groupContext="group">
+ <title>Sheet.31</title>
+ <g id="shape32-117" v:mID="32" v:groupContext="shape" transform="translate(225,173.25) rotate(90)">
+ <title>Block Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow32-118" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st5">
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st11"/>
+ </g>
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st12"/>
+ </g>
+ <g id="shape33-121" v:mID="33" v:groupContext="shape" transform="translate(2.25,-24.3529)">
+ <title>Sheet.33</title>
+ <desc>h1, h2 .. hk</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="215.868" width="90" height="18.2647"/>
+ <rect x="0" y="206.735" width="90" height="18.2647" class="st8"/>
+ <text x="17.56" y="219.47" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>h1, h2 .. hk</text> </g>
+ </g>
+ <g id="shape34-124" v:mID="34" v:groupContext="shape" transform="translate(307.011,286.73) rotate(152.323)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L128.85 225" class="st3"/>
+ </g>
+ <g id="shape35-129" v:mID="35" v:groupContext="shape" transform="translate(433.272,125.452) rotate(99.7172)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L58.31 225" class="st3"/>
+ </g>
+ <g id="shape36-134" v:mID="36" v:groupContext="shape" transform="translate(407.724,-64.1459) rotate(45)">
+ <title>Sheet.36</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L79.16 225" class="st3"/>
+ </g>
+ <g id="shape37-139" v:mID="37" v:groupContext="shape" transform="translate(320.441,-127.12) rotate(15.6155)">
+ <title>Sheet.37</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L200.75 225" class="st3"/>
+ </g>
+ <g id="shape38-144" v:mID="38" v:groupContext="shape" transform="translate(132.75,-75.2588)">
+ <title>Sheet.38</title>
+ <desc>BF-2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-2</text> </g>
+ <g id="shape39-147" v:mID="39" v:groupContext="shape" transform="translate(303.75,-70.7588)">
+ <title>Sheet.39</title>
+ <desc>BF-X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.2" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-X</text> </g>
+ <g id="shape40-150" v:mID="40" v:groupContext="shape" transform="translate(447.75,-75.2588)">
+ <title>Sheet.40</title>
+ <desc>BF-L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.89" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-L</text> </g>
+ <g id="shape41-153" v:mID="41" v:groupContext="shape" transform="translate(300.375,-117)">
+ <title>Sheet.41</title>
+ <desc>Hashing for lookup/Insertion into a vector of BFs happens once</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="90" cy="202.5" width="180" height="45"/>
+ <rect x="0" y="180" width="180" height="45" class="st8"/>
+ <text x="4.6" y="198.9" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Hashing for lookup/Insertion into a <tspan
+ x="23.06" dy="1.2em" class="st15">vector of BFs happens once</tspan></text> </g>
+ <g id="shape44-157" v:mID="44" v:groupContext="shape" transform="translate(249.698,-151.505) rotate(-3.74012)">
+ <title>Sheet.44</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A93.4958 45.6256 42.23 0 1 79.38 221.66 L79.68 221.85" class="st16"/>
+ </g>
+ <g id="shape45-163" v:mID="45" v:groupContext="shape" transform="translate(30.375,0.25)">
+ <title>Sheet.45</title>
+ <desc>Lookup/Insertion is done in the series of BFs, one by one or ...</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="233.048" cy="202.5" width="466.1" height="45"/>
+ <rect x="0" y="180" width="466.096" height="45" class="st8"/>
+ <text x="4.34" y="206.1" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Lookup/Insertion is done in the series of BFs, one by one or can be optimized to do in parallel. </text> </g>
+ <g id="shape46-166" v:mID="46" v:groupContext="shape" transform="translate(123.252,-43.6868) rotate(17.0249)">
+ <title>Sheet.46</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A88.2185 43.0621 47.63 0 1 70.31 221.39 L70.6 221.6" class="st16"/>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i5.svg b/doc/guides/prog_guide/img/member_i5.svg
new file mode 100644
index 0000000..c1728cf
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i5.svg
@@ -0,0 +1,163 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i5.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="5.30481in" height="1.96146in"
+ viewBox="0 0 381.946 141.225" xml:space="preserve" color-interpolation-filters="sRGB" class="st15">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:none;stroke:none;stroke-width:0.25}
+ .st5 {fill:#ffffff;font-family:Calibri;font-size:1.16666em;font-weight:bold}
+ .st6 {marker-end:url(#mrkr5-14);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st7 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st10 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {marker-end:url(#mrkr5-63);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st12 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.22935779816514}
+ .st13 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st14 {font-size:1em}
+ .st15 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-14" class="st7" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-63" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="7.15" refX="-7.15" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-4.36,-4.36) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group1-1" transform="translate(191.995,-19.4751)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-2" v:mID="2" v:groupContext="shape" transform="translate(146.944,42.2251) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow2-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st2"/>
+ </g>
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st3"/>
+ </g>
+ <g id="shape3-6" v:mID="3" v:groupContext="shape" transform="translate(0,-34.65)">
+ <title>Sheet.3</title>
+ <desc>vBF</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="38.1251" cy="126.375" width="76.26" height="29.7"/>
+ <rect x="0" y="111.525" width="76.2502" height="29.7" class="st4"/>
+ <text x="27.68" y="130.58" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>vBF </text> </g>
+ </g>
+ <g id="shape4-9" v:mID="4" v:groupContext="shape" transform="translate(126.724,-100.475)">
+ <title>Sheet.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape5-15" v:mID="5" v:groupContext="shape" transform="translate(103.5,-101.775)">
+ <title>Sheet.5</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="39.9122" cy="135.863" width="79.83" height="10.7251"/>
+ <rect x="0" y="130.5" width="79.8244" height="10.7251" class="st4"/>
+ <text x="21.78" y="138.86" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape6-18" v:mID="6" v:groupContext="shape" transform="translate(221.726,-56.2468) rotate(-24.5123)">
+ <title>Sheet.6</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L65.42 141.23" class="st6"/>
+ </g>
+ <g id="shape7-23" v:mID="7" v:groupContext="shape" transform="translate(280.318,-68.9751)">
+ <title>Sheet.7</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape8-28" v:mID="8" v:groupContext="shape" transform="translate(338.125,-56.6022) rotate(24.1625)">
+ <title>Sheet.8</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L70.8 141.23" class="st6"/>
+ </g>
+ <g id="shape9-33" v:mID="9" v:groupContext="shape" transform="translate(197.714,217.975) rotate(180)">
+ <title>Sheet.9</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(18,-67.5)">
+ <title>Sheet.10</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="80.4201" cy="134.475" width="160.85" height="13.5"/>
+ <rect x="0" y="127.725" width="160.84" height="13.5" class="st4"/>
+ <text x="25.11" y="137.18" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape11-41" v:mID="11" v:groupContext="shape" transform="translate(198.032,253.975) rotate(180)">
+ <title>Sheet.11</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Sheet.12</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="81" cy="136.725" width="162.01" height="9"/>
+ <rect x="0" y="132.225" width="162" height="9" class="st4"/>
+ <text x="11.04" y="139.43" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape13-49" v:mID="13" v:groupContext="shape" transform="translate(494.552,22.75) rotate(90)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape14-52" v:mID="14" v:groupContext="shape" transform="translate(494.552,58.75) rotate(90)">
+ <title>Sheet.14</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape15-55" v:mID="15" v:groupContext="shape" transform="translate(494.552,94.75) rotate(90)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape17-58" v:mID="17" v:groupContext="shape" transform="translate(348.769,-25.0593) rotate(44.5185)">
+ <title>Sheet.17</title>
+ <path d="M-0 141.23 A35.1884 19.2595 167.75 0 1 42.43 138.27 L42.74 138.46" class="st11"/>
+ </g>
+ <g id="shape18-64" v:mID="18" v:groupContext="shape" transform="translate(222.188,-5.40005)">
+ <title>Sheet.18</title>
+ <desc>A BF corresponding to each worker thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="59.0625" cy="127.725" width="118.13" height="27"/>
+ <rect x="0" y="114.225" width="118.125" height="27" class="st4"/>
+ <text x="5.14" y="124.13" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>A BF corresponding to <tspan
+ x="11.19" dy="1.2em" class="st14">each worker thread</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i6.svg b/doc/guides/prog_guide/img/member_i6.svg
new file mode 100644
index 0000000..265179f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i6.svg
@@ -0,0 +1,332 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i6.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8in" height="3.625in" viewBox="0 0 576 261"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st16">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.666664em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.00001em}
+ .st9 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {marker-end:url(#mrkr5-29);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st12 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st13 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st14 {fill:#92d050;stroke:#c8c8c8;stroke-width:0.25}
+ .st15 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st16 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-29" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-9.8478)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(396.989,-54.9268)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st2"/>
+ </g>
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(248.261,-12.1936)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st2"/>
+ </g>
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(6.07514E-013,-29.0155)">
+ <title>Rectangle.10</title>
+ <desc>Signatures for target 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="225.767" width="99.49" height="70.4662"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st2"/>
+ </g>
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st3"/>
+ <text x="26.54" y="223.37" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(239.971,-20.4837)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(353.649,-19.9346)">
+ <title>Sheet.54</title>
+ <desc>Match 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 1</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(216.989,-210.652)">
+ <title>Sheet.55</title>
+ <desc>Packet Payload</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="252.71" width="99.49" height="16.5803"/>
+ <rect x="0" y="244.42" width="99.4817" height="16.5803" class="st9"/>
+ <text x="19.04" y="255.71" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet Payload</text> </g>
+ <g id="shape56-24" v:mID="56" v:groupContext="shape" transform="translate(526.665,52.2365) rotate(90)">
+ <title>Sheet.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L16.52 261" class="st11"/>
+ </g>
+ <g id="shape96-30" v:mID="96" v:groupContext="shape" transform="translate(-3.0294,-95.7818)">
+ <title>Sheet.96</title>
+ <desc>Attack Signature Length 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.75" cy="248.565" width="103.5" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.5" height="24.8704" class="st7"/>
+ <text x="4.79" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 1</text> </g>
+ <g id="group114-33" transform="translate(228.359,-134.152)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-34" transform="translate(0,-24.8704)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-35" v:mID="100" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-36" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape101-39" v:mID="101" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-40" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape102-43" v:mID="102" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-44" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape103-47" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-48" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape104-51" v:mID="104" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-52" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape105-55" v:mID="105" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-56" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-59" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-60" v:mID="108" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-61" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape109-64" v:mID="109" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-65" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape110-68" v:mID="110" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow110-69" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape111-72" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-73" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ <g id="shape112-76" v:mID="112" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-77" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape113-80" v:mID="113" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-81" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-84" v:mID="89" v:groupContext="shape" transform="translate(398.644,-116.927) rotate(24.4696)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L143.75 261" class="st11"/>
+ </g>
+ <g id="shape115-89" v:mID="115" v:groupContext="shape" transform="translate(116.062,-1.19371E-012)">
+ <title>Rectangle.115</title>
+ <desc>Signatures for target 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="211.259" width="99.49" height="99.4817"/>
+ <g id="shadow115-90" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st2"/>
+ </g>
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st3"/>
+ <text x="26.54" y="208.86" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>2</text> </g>
+ <g id="shape116-95" v:mID="116" v:groupContext="shape" transform="translate(117.989,-95.7818)">
+ <title>Sheet.116</title>
+ <desc>Attack Signature Length 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.9909" cy="248.565" width="103.99" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.982" height="24.8704" class="st7"/>
+ <text x="5.03" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 2</text> </g>
+ <g id="shape118-98" v:mID="118" v:groupContext="shape" transform="translate(392.971,-90.217)">
+ <title>Sheet.118</title>
+ <desc>Attack Signature Length L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="248.565" width="108" height="24.8704"/>
+ <rect x="0" y="236.13" width="108" height="24.8704" class="st7"/>
+ <text x="7.43" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length L</text> </g>
+ <g id="shape119-101" v:mID="119" v:groupContext="shape" transform="translate(384.909,-64.9346)">
+ <title>Sheet.119</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape120-103" v:mID="120" v:groupContext="shape" transform="translate(491.971,-64.9346)">
+ <title>Sheet.120</title>
+ <desc>Match 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 2</text> </g>
+ <g id="shape85-106" v:mID="85" v:groupContext="shape" transform="translate(478.538,12.9307) rotate(65.6291)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L109.61 261" class="st11"/>
+ </g>
+ <g id="shape117-111" v:mID="117" v:groupContext="shape" transform="translate(247.054,-91.2818)">
+ <title>Sheet.117</title>
+ <desc>Attack Signature Length X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="52.7082" cy="248.565" width="105.42" height="24.8704"/>
+ <rect x="0" y="236.13" width="105.416" height="24.8704" class="st7"/>
+ <text x="5.7" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length X</text> </g>
+ </g>
+ <g id="shape122-114" v:mID="122" v:groupContext="shape" transform="translate(315.114,-164.13)">
+ <title>Sheet.122</title>
+ <desc>HTSS</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="26.943" cy="248.565" width="53.89" height="24.8704"/>
+ <rect x="0" y="236.13" width="53.8859" height="24.8704" class="st7"/>
+ <text x="14.52" y="252.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i7.svg b/doc/guides/prog_guide/img/member_i7.svg
new file mode 100644
index 0000000..e23ae26
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i7.svg
@@ -0,0 +1,399 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i7.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8.5in" height="4.5in" viewBox="0 0 612 324"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st23">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.16666em}
+ .st9 {fill:none;stroke:#00b050;stroke-width:2.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st12 {fill:#a8d08d;stroke:#c8c8c8;stroke-width:0.25}
+ .st13 {marker-end:url(#mrkr5-83);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st14 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st15 {marker-end:url(#mrkr5-95);stroke:#92d050;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st16 {fill:#92d050;fill-opacity:1;stroke:#92d050;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st17 {fill:#00b050;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st18 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st19 {fill:none;stroke:#ff0000;stroke-width:2.25}
+ .st20 {fill:#ff0000;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st21 {marker-end:url(#mrkr5-123);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-83" class="st14" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-95" class="st16" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-123" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-32.2733)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(460.471,-62.2267)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="279" width="108" height="45" class="st2"/>
+ </g>
+ <rect x="0" y="279" width="108" height="45" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(320.452,-18.123)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="234" width="108" height="90" class="st2"/>
+ </g>
+ <rect x="0" y="234" width="108" height="90" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Rectangle.10</title>
+ <desc>Flow Keys Matching Mask 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="285.75" width="108" height="76.5"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="247.5" width="108" height="76.5" class="st2"/>
+ </g>
+ <rect x="0" y="247.5" width="108" height="76.5" class="st3"/>
+ <text x="12.56" y="282.75" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(311.452,-27.123)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="310.5" width="126" height="13.5" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(424.471,-26.2267)">
+ <title>Sheet.54</title>
+ <desc>Match</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="317.25" width="72" height="13.5"/>
+ <rect x="0" y="310.5" width="72" height="13.5" class="st7"/>
+ <text x="17.68" y="321.45" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(261,-247.163)">
+ <title>Sheet.55</title>
+ <desc>Flow ID1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st9"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID1</text> </g>
+ <g id="shape96-24" v:mID="96" v:groupContext="shape" transform="translate(0,-109.783)">
+ <title>Sheet.96</title>
+ <desc>Flow Mask 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 1</text> </g>
+ <g id="group114-27" transform="translate(247.5,-163.783)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-28" transform="translate(0,-27)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-29" v:mID="100" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-30" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape101-33" v:mID="101" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-34" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape102-37" v:mID="102" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-38" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape103-41" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-42" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape104-45" v:mID="104" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-46" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape105-49" v:mID="105" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-50" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-53" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-54" v:mID="108" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape109-58" v:mID="109" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape110-62" v:mID="110" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow110-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st12"/>
+ </g>
+ <g id="shape111-66" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape112-70" v:mID="112" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape113-74" v:mID="113" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-78" v:mID="89" v:groupContext="shape" transform="translate(413.723,393.802) rotate(146.31)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L153.9 324" class="st13"/>
+ </g>
+ <g id="shape115-84" v:mID="115" v:groupContext="shape" transform="translate(126,0)">
+ <title>Rectangle.115</title>
+ <desc>Flow Keys Matching Mask 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="270" width="108" height="108"/>
+ <g id="shadow115-85" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="216" width="108" height="108" class="st2"/>
+ </g>
+ <rect x="0" y="216" width="108" height="108" class="st3"/>
+ <text x="12.56" y="267" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>2</text> </g>
+ <g id="shape85-90" v:mID="85" v:groupContext="shape" transform="translate(635.321,91.2793) rotate(81.3573)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L143.93 324" class="st15"/>
+ </g>
+ <g id="shape56-96" v:mID="56" v:groupContext="shape" transform="translate(579.175,-64.556) rotate(64.1257)">
+ <title>Sheet.56</title>
+ <path d="M0 324 L54.31 324" class="st15"/>
+ </g>
+ </g>
+ <g id="shape122-101" v:mID="122" v:groupContext="shape" transform="translate(351,-213.444)">
+ <title>Sheet.122</title>
+ <desc>HTSS with False Negative (Cache)</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="304.722" width="90" height="38.556"/>
+ <rect x="0" y="285.444" width="90" height="38.556" class="st7"/>
+ <text x="13.29" y="301.72" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS with False <tspan
+ x="10.52" dy="1.2em" class="st5">Negative </tspan>(Cache)</text> </g>
+ <g id="shape123-105" v:mID="123" v:groupContext="shape" transform="translate(287.654,-290.556)">
+ <title>Sheet.123</title>
+ <desc>Active</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1875" cy="310.5" width="48.38" height="27"/>
+ <rect x="0" y="297" width="48.375" height="27" class="st7"/>
+ <text x="8.63" y="314.1" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Active</text> </g>
+ <g id="shape124-108" v:mID="124" v:groupContext="shape" transform="translate(278.827,-153)">
+ <title>Sheet.124</title>
+ <desc>Target for Flow ID 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36.0864" cy="310.5" width="72.18" height="27"/>
+ <rect x="0" y="297" width="72.1728" height="27" class="st9"/>
+ <text x="11.93" y="306.9" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Target for <tspan
+ x="13.54" dy="1.2em" class="st5">Flow ID </tspan>1</text> </g>
+ <g id="shape125-112" v:mID="125" v:groupContext="shape" transform="translate(155.857,-254.556)">
+ <title>Sheet.125</title>
+ <desc>Flow ID2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st19"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID2</text> </g>
+ <g id="shape126-115" v:mID="126" v:groupContext="shape" transform="translate(153,-270)">
+ <title>Sheet.126</title>
+ <desc>New/Inactive</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="40.5" cy="310.5" width="81" height="27"/>
+ <rect x="0" y="297" width="81" height="27" class="st7"/>
+ <text x="6.77" y="314.1" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New/Inactive</text> </g>
+ <g id="shape127-118" v:mID="127" v:groupContext="shape" transform="translate(251.739,-239.709) rotate(14.0795)">
+ <title>Sheet.127</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 318.73 A39.2404 18 -180 0 0 49.73 320.91 L50.07 320.78" class="st21"/>
+ </g>
+ <g id="shape128-124" v:mID="128" v:groupContext="shape" transform="translate(219.24,-229.5)">
+ <title>Sheet.128</title>
+ <desc>Miss</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="20.88" cy="310.5" width="41.76" height="27"/>
+ <rect x="0" y="297" width="41.76" height="27" class="st7"/>
+ <text x="7.81" y="314.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Miss</text> </g>
+ <g id="shape129-127" v:mID="129" v:groupContext="shape" transform="translate(147.029,-142.056)">
+ <title>Sheet.129</title>
+ <desc>Flow Mask 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 2</text> </g>
+ <g id="shape130-130" v:mID="130" v:groupContext="shape" transform="translate(166.845,-18.5004) rotate(18.2325)">
+ <title>Sheet.130</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A71.1913 104.269 -180 0 0 97.04 298.43 L97.25 298.14" class="st21"/>
+ </g>
+ <g id="shape131-135" v:mID="131" v:groupContext="shape" transform="translate(184.406,-3.04505) rotate(-3.24734)">
+ <title>Sheet.131</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A112.345 104.269 -180 0 0 154.25 297.52 L154.52 297.28" class="st21"/>
+ </g>
+ <g id="shape132-140" v:mID="132" v:groupContext="shape" transform="translate(301.368,16.888) rotate(-25.868)">
+ <title>Sheet.132</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A83.375 104.269 -180 0 0 113.91 298.14 L114.14 297.87" class="st21"/>
+ </g>
+ <g id="shape133-145" v:mID="133" v:groupContext="shape" transform="translate(345.029,-142.056)">
+ <title>Sheet.133</title>
+ <desc>Flow Mask X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.43" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask X</text> </g>
+ <g id="shape134-148" v:mID="134" v:groupContext="shape" transform="translate(481.5,-139.5)">
+ <title>Sheet.134</title>
+ <desc>Flow Mask L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="19.12" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask L</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 40f04a1..7ff5144 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -50,6 +50,7 @@ Programmer's Guide
timer_lib
hash_lib
efd_lib
+ member_lib
lpm_lib
lpm6_lib
packet_distrib_lib
@@ -191,6 +192,19 @@ Programmer's Guide
:numref:`figure_efd11` :ref:`figure_efd11`
+:numref:`figure_membership1` :ref:`figure_membership1`
+
+:numref:`figure_membership2` :ref:`figure_membership2`
+
+:numref:`figure_membership3` :ref:`figure_membership3`
+
+:numref:`figure_membership4` :ref:`figure_membership4`
+
+:numref:`figure_membership5` :ref:`figure_membership5`
+
+:numref:`figure_membership6` :ref:`figure_membership6`
+
+:numref:`figure_membership7` :ref:`figure_membership7`
**Tables**
diff --git a/doc/guides/prog_guide/member_lib.rst b/doc/guides/prog_guide/member_lib.rst
new file mode 100644
index 0000000..d8d4612
--- /dev/null
+++ b/doc/guides/prog_guide/member_lib.rst
@@ -0,0 +1,440 @@
+.. BSD LICENSE
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+.. _Member_Library:
+
+Membership Library
+=======================
+
+Introduction
+------------
+The DPDK Membership Library provides an API for DPDK applications to insert a
+new member, delete an existing member, or query the existence of a member in a
+given set, or a group of sets. For the case of a group of sets the library
+will return not only whether the element has been inserted before in one of
+the sets but also which set it belongs to. The Membership Library is an
+extension and generalization of a traditional filter structure (for example
+Bloom Filter [Member-bloom]) that has multiple usages in a wide variety of
+workloads and applications. In general, the Membership Library is a data
+structure that provides a “set-summary” on whether a member belongs to a set,
+and as discussed in details later, there are two advantages of using such a
+set-summary rather than operating on a “full-blown” complete list of elements:
+first, it has a much smaller storage requirement than storing the whole list of
+elements themselves, and secondly checking an element membership (or other
+operations) in this set-summary is much faster than checking it for the
+original full-blown complete list of elements.
+
+We use the term “Set-Summary” in this guide to refer to the space-efficient,
+probabilistic membership data structure that is provided by the library. A
+membership test for an element will return the set this element belongs to or
+the element is "not-found" with very high probability of accuracy. Set-summary
+is a fundamental data aggregation component that can be used in many network
+(and other) applications. It is a crucial structure to address performance and
+scalability issues of diverse network applications including overlay networks,
+data-centric networks, flow table summaries, network statistics and
+traffic monitoring. A set-summary is useful for applications who need to
+include a list of elements while a complete list requires too much space
+and/or too much processing cost. In these situations, the set-summary works as
+a lossy hash-based representation of a set of members. It can dramatically
+reduce space requirement and significantly improve the performance of set
+membership queries at the cost of introducing a very small membership test error
+probability.
+
+.. _figure_membership1:
+.. figure:: img/member_i1.*
+
+ Example Usages of Membership Library
+
+We believe that there are various usages for a Membership Library in a very
+large set of applications and workloads. Interested readers can refer to
+[Member-survey] for a survey of possible networking usages. The following figure
+provide a small set of examples of using the Membership Library. Sub-figure(a)
+depicts a distributed web cache architecture where a collection of proxies
+attempt to share their web caches (cached from a set of back-end web servers) to
+provide faster responses to clients, and the proxies uses the Membership
+Library to share summaries of what web pages/objects they are caching. With the
+Membership Library, a proxy receiving an \http request will inquire the
+set-summary to find its location and quickly determine whether to retrieve the
+requested web page from a nearby proxy or from a backend web server.
+Sub-figure(b) depicts another example for using the Membership Library to
+prevent routing loops which is typically done using slow TTL countdown and
+dropping packets when TTL expires. As shown in Sub-figure(b), an embedded
+set-summary in the packet header itself can be used to summarize the set of
+nodes a packet has gone through, and each node upon receiving a packet can check
+whether its id is a member of the set of visited nodes, and if it is then a
+routing loop is detected. Sub-Figure(c) presents another usage of Membership
+Library to load balance flows to worker threads with in-order guarantee where a
+set-summary is used to query if a packet belongs to an existing flow or a new
+flow. Packets belonging to a new flow are forwarded to the current least loaded
+worker thread, while those belonging to an existing flow are forwarded to the
+pre-assigned thread to guarantee in-order processing. Sub-figure(d) highlights
+yet another usage example in the database domain where a set-summary is used to
+determine joins between sets instead of creating a join by comparing each
+element of a set against the other elements in a different set, a join is done
+on the summaries since they can efficiently encode members of a given set.
+
+We are including a configurable Membership Library in DPDK to cover set
+membership functionality for both a single set and multi-set scenarios. The
+library is optimized to support the customer network applications which require
+membership checking functionality. In this guide, we will cover two set-summary
+schemes including vector of Bloom Filters and Hash-Table based
+set-summary schemes with and without false negative probability, followed by
+a brief discussion of the Membership Library API.
+
+Vector of Bloom Filters
+--------------------------
+
+Bloom Filter (BF) [Member-bloom] is a well-known space-efficient
+probabilistic data structure that answers set membership queries (test whether
+an element is a member of a set) with some probability of false positives and
+zero false negatives; a query for an element returns either it is "possibly in
+a set" (with very high probability) or "definitely not in a set".
+
+The BF is a method for representing a set of n elements (for example flow keys
+in network applications domain) to support membership queries. The idea of BF is
+to allocate a bit-vector v with m bits, which are initially all set to 0. Then
+it chooses k independent hash functions h1, h2, … hk with hash values range from
+1 to m to perform hashing calculations on each element. Every time when an
+element X being inserted into the set, the bits at positions h1(X), h2(X), …
+hk(X) in v are set to 1 (any particular bit might be set to 1 multiple times
+for multiple different inserted elements). Given a query for any element Y, the
+bits at positions h1(Y), h2(Y), ... hk(Y) are checked. If any of them is 0,
+then Y is definitely not in the set. Otherwise there is a high probability that
+Y is a member of the set with certain false positive probability. As shown in
+the next equation, the false positive probability can be made arbitrarily small
+by changing the number of hash functions (k) and the vector length (m).
+
+.. _figure_membership2:
+.. figure:: img/member_i2.*
+
+ Bloom Filter False Positive Probability
+
+Without BF, an accurate membership testing could involve a costly hash table
+lookup and full element comparison. The advantage of using a BF is to simplify
+the membership test into a series of hash calculations and memory accesses for a
+small bit-vector, which can be easily optimized. Hence the lookup throughput
+(set membership test) can be significantly faster than a normal hash table
+lookup with element comparison.
+
+.. _figure_membership3:
+.. figure:: img/member_i3.*
+
+ Detecting Routing Loops Using BF
+
+BF is used for applications that need only one set, and the
+membership of elements is checked against the BF. The example discussed
+in the above figure is one example of potential applications that uses only one
+set to capture the node IDs that have been visited so far by the packet. Each
+node will then check this embedded BF in the packet header for its own id, and
+if the BF indicates that the current node is definitely not in the set then a
+loop-free route is guaranteed.
+
+
+.. _figure_membership4:
+.. figure:: img/member_i4.*
+
+ Vector Bloom Filter (vBF) Overview
+
+To support membership test for both multiple sets and single set,
+the library implements Vector Bloom Filter (vBF) scheme.
+vBF basically composes multiple bloom filters as a vector of bloom filers.
+The membership test is conducted on all of the
+bloom filters concurrently to determine which set(s) it belongs to or none of
+them. The basic idea of vBF is shown in the above figure where an element is
+used to address multiple bloom filters concurrently and the bloom filter
+index(es) with a hit is returned.
+
+.. _figure_membership5:
+.. figure:: img/member_i5.*
+
+ vBF for Flow Scheduling to Worker Thread
+
+As previously mentioned, there are many usages of such structure. vBF is used
+for applications that needs to check membership against multiple sets
+simultaneously. The example discussed in the above figure uses a set to capture
+all flows being assigned for processing at a given worker thread. Upon receiving
+a packet the vBF is used to quickly figure out if this packet belongs to a new flow
+so as to be forwarded to the current least loaded worker thread, or otherwise it
+should be queued for an existing thread to guarantee in-order processing (i.e.
+the property of vBF to indicate right away that a given flow is a new one or
+not is critical to minimize response time latency).
+
+It should be noted that vBF can be implemented using a set of single bloom
+filters with sequential lookup of each BF. However, being able to concurrently
+search all set-summaries is a big throughput advantage. In the library, certain
+parallelism is realized by the implementation of checking all bloom filters
+together.
+
+
+Hash-Table based Set-Summaries
+---------------------------------
+
+Hash-table based set-summary (HTSS) is another scheme in the membership library.
+Cuckoo filter [Member-cfilter] is an example of HTSS.
+HTSS supports multi-set membership testing like what
+vBF does. However, while vBF is more adequate for a small number of targets, HTSS is more suitable
+and can easily outperform vBF when the number of sets is
+large, since HTSS uses a single hash table for membership testing while vBF
+requires testing a series of Bloom Filters each corresponding to one set.
+As a result, generally speaking vBF is more adequate for the case of a small limited number of sets
+while HTSS should be used with a larger number of sets. For example, based on our results comparing vBF and HTSS
+for the case of 1M keys and 5% maximum allowable false positive rate; vBF outperforms HTSS in memory footprint and
+update rate and was better or on-bar in terms of throughput for up to 16 target sets. It should be noted however,
+that actual performance results might be different and is based on the workload, configuration and the platform
+used, etc.
+
+
+.. _figure_membership6:
+.. figure:: img/member_i6.*
+
+ Using HTSS for Attack Signature Matching
+
+As shown in the above figure, attack signature matching where each set
+represents a certain signature length (for correctness of this example, an
+attack signature should not be a subset of another one) in the payload is a good
+example for using HTSS with 0% false negative (i.e., when an element returns not
+found, it has a 100% certainty that it is not a member of any set). The packet
+inspection application benefits from knowing right away that the current payload
+does not match any attack signatures in the database to establish its
+legitimacy, otherwise a deep inspection of the packet is needed.
+
+HTSS employs a similar but simpler data structure to a traditional hash table,
+and the major difference is that HTSS stores only the signatures but not the
+full keys/elements which can significantly reduce the footprint of the table.
+Along with the signature, HTSS also stores a value to indicate the target set.
+When looking up for an element, the element is hashed and the HTSS is addressed
+to retrieve the signature stored. If the signature matches then the value is
+retrieved corresponding to the index of the target set which the element belongs
+to. Because signatures can collide, HTSS can still has false positive
+probability. Furthermore, if elements are allowed to be
+overwritten or evicted when the hash table becomes full, it will also have a
+false negative probability. We discuss this case in the next section.
+
+Set-Summaries with False Negative Probability
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As previously mentioned, traditional set-summaries (e.g. Bloom Filters ) do not
+have a false negative probability, i.e., it is 100% certain when an element
+returns “not to be present” for a given set. However, the Membership Library
+also supports a set-summary probabilistic data structure based on HTSS which
+allows for false negative probability.
+
+
+In HTSS, when the hash table becomes full, keys/elements will fail to be added
+into the table and the hash table has to be resized to accommodate for these new
+elements, which can be expensive. However, if we allow new elements to overwrite
+or evict existing elements (as a cache typically does), then the resulting
+set-summary will begin to have false negative probability. This is because the
+element that was evicted from the set-summary may still be present in the target
+set. For subsequent inquiries the set-summary will falsely report the element
+not being in the set, hence having a false negative probability.
+
+The major usage of HTSS with false negative is to use it as a cache for
+distributing elements to different target sets. By allowing HTSS to evict old
+elements, the set-summary can keep track of the most recent elements
+(i.e. active) as a cache typically does. Old inactive elements (infrequently
+used elements) will automatically and eventually get evicted from the
+set-summary. It worth noting that the set-summary still has false positive
+probability, which means the application either can tolerate certain false positive
+or it has fall-back path when false positive happens.
+
+.. _figure_membership7:
+.. figure:: img/member_i7.*
+
+ Using HTSS with False Negatives for Wild Card Classification
+
+HTSS with false negative (i.e. a cache) also has its wide set of applications.
+For example wild card flow classification (e.g. ACL rules) highlighted in the
+above figure is an example of such application. In that case each target set
+represents a sub-table with rules defined by a certain flow mask. The flow masks
+are non-overlapping, and for flows matching more than one rule only the highest
+priority one is inserted in the corresponding sub-table (interested readers can
+refer to the Open vSwitch (OvS) design of Mega Flow Cache (MFC) [Member-OvS]
+for further details). Typically the rules will have a large number of distinct
+unique masks and hence, a large number of target sets each corresponding to one
+mask. Because the active set of flows varies widely based on the network
+traffic, HTSS with false negative will act as a cache for <flowid, target ACL
+sub-table> pair for the current active set of flows. When a miss occurs (as
+shown in red in the above figure) the sub-tables will be searched sequentially
+one by one for a possible match, and when found the flow key and target
+sub-table will be inserted into the set-summary (i.e. cache insertion) so
+subsequent packets from the same flow don’t incur the overhead of the
+sequential search of sub-tables.
+
+Library API Overview
+--------------------
+The design goal of the Membership Library API is to be as generic as possible to
+support all the different types of set-summaries we discussed in previous
+sections and beyond. Fundamentally, the APIs need to include creation,
+insertion, deletion, and lookup.
+
+.. Set-summary Type Query
+.. ~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. *void rte\_ms\_type\_query(enum rte\_ms\_setsum\_type\*\* types)*
+
+.. This function intends to serve as a convenient tool to query which set-summary
+.. types are supported in the current version of DPDK. Since the implementations of
+.. the set-summaries can vary, initial versions of the Membership Library may only
+.. support a subset of the set-summaries we discussed previously. Programmers could
+.. use this API to decide which type is available to use in the current version of DPDK.
+
+Set-summary Create
+~~~~~~~~~~~~~~~~~~~~~
+
+*rte\_member\_create* function is used to create a set-summary structure, the input parameter
+is a struct to pass in parameters that needed to initialize the set-summary, while the function returns the
+pointer to the created set-summary or NULL if the creation failed.
+
+The general input arguments used when creating the set-summary should include *name*
+which is the name of the created set-summary, *type* which is one of the types
+supported by the library (e.g. RTE\_MEMBER\_TYPE\_HT for HTSS or RTE\_MEMBER\_TYPE\_VBF for vBF), and *key\_len*
+which is the length of the element/key. There are other parameters
+are only used for certain type of set-summary, or has a slightly different meaning for different types of set-summary.
+For example, *num\_keys* parameter means the maximum number of entries for Hash table based set-summary.
+However, for bloom filter, this value means the expected number of keys that could be
+inserted into the bloom filter(s). The value is used to calculate the size of each
+bloom filter.
+We also pass two seeds: *prim\_hash\_seed* and
+*sec\_hash\_seed* for the primary and secondary hash functions to calculate two independent hash values.
+*socket\_id* parameter is the NUMA socket ID for the memory used to create the
+set-summary. For HTSS, another parameter *iscache* is used to indicate
+if this set-summary is a cache (i.e. with false negative probability) or not.
+For vBF, extra parameters are needed. For example, *num\_set* is the number of
+sets needed to initialize the vector bloom filters. This number is equal to the
+number of bloom filters will be created.
+*false\_pos\_rate* is the false positive rate. num\_keys and false\_pos\_rate will be used to determine
+the number of hash functions and the bloom filter size.
+
+
+Set-summary Element Insertion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. *rte\_membership\_add(const void *setsum, const void *key, MEMBERSHIP\_TARGET\_TYPE target\_id)*
+
+*rte\_member\_add* is the function to insert an element/key in a set-summary structure, if it fails an
+error is returned. For success the returned value is deferent based on the
+set-summary mode to provide extra information for the users. For vBF
+mode, a return value of 0 means a successful insert. For HTSS mode without false negative, the insert
+could fail with -ENOSPC if the table is full. With false negative (i.e. cache mode),
+for insert that does not cause any eviction (i.e. no overwriting happens to an
+existing entry) the return value is 0. For insertion that causes eviction, the return
+value is 1 to indicate such situation, but it is not an error.
+
+The input arguments for the function should include the *key* which is a pointer to the element/key that needs to
+be added to the set-summary, and *set\_id* which is the set id associated
+with the key that needs to be added.
+
+
+Set-summary Element Lookup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+
+.. *rte\_membership\_lookup(const void \*setsum, const void \*key, MEMBERSHIP\_TARGET\_TYPE \*target\_id)*
+
+*rte\_member\_lookup* looks up a single key/element in the set-summary structure. It
+returns as soon as the first match is found. The return value is 1 if a
+match is found and 0 otherwise. The arguments for the function include *key* which is a pointer to the
+element/key that needs to be looked up, and *set\_id* which is used to return the
+first target set id where the key has matched, if any.
+
+.. *rte\_membership\_lookup\_bulk(const void \*setsum, const void \*\*keys, uint32\_t num\_keys, MEMBERSHIP\_TARGET\_TYPE \*target\_ids)*
+
+*rte\_member\_lookup\_bulk* is the function to look up a bulk of keys/elements in the
+set-summary structure for their first match. Each key lookup returns as soon as the first match is found. The
+return value is the number of keys that find a match. The arguments of the
+function include *keys* which is a pointer to a bulk of keys that are to be looked up,
+*num\_keys* is the number
+of keys that will be looked up, and *set\_ids* are the return target set
+ids for the first match found for each of the input keys. *set\_ids* is an array
+needs to be sized according to the *num\_keys*. If there is no match, the set id
+for that key will be set to RTE_MEMBER_NO_MATCH.
+
+.. *rte\_membership\_lookup\_multi(const void \*setsum, const void \*key, MEMBERSHIP\_TARGET\_TYPE \*target\_id)*
+
+*rte\_member\_lookup\_multi* function looks up a single key/element in the
+set-summary structure for multiple matches. It
+returns ALL the matches (possibly more than one) found for this key when it
+is matched against all target sets (it worth noting that for cache mode HTSS,
+the current implementation matches at most one target set). The return value is
+the number of matches
+that was found for this key (for cache mode HTSS the return value
+should be at most 1). The arguments for the function include *key* which is a pointer to the
+element/key that needs to be looked up, *match\_per\_key* which is to indicate maximum number of matches
+the user expect to find for each key, and *set\_id* which is used to return all
+target set ids where the key has matched, if any. The *set\_id* array should be sized
+according to *match\_per\_key*. For vBF, maximum number of matches per key is equal
+to the number of sets. For HTSS, maximum number of matches per key is equal to two times
+entry count per bucket. *match\_per\_key* should be equal or smaller than maximum number of
+possible matches.
+
+.. *rte\_membership\_lookup\_multi\_bulk(const void \*setsum, const void \*\*keys, uint32\_t num\_keys, uint32\_t max\_match\_per\_key, uint32\_t \*match\_count, MEMBERSHIP\_TARGET\_TYPE \*target\_ids)*
+
+*rte\_membership\_lookup\_multi\_bulk* function looks up a bulk of keys/elements elements in the
+set-summary structure for multiple matches, each key lookup returns ALL the matches (possibly more
+than one) found for this key when it is matched against all target sets (cache mode HTSS
+matches at most one target set). The
+return value is the number of keys that find one or more matches in the
+set-summary structure. The arguments of the
+function include *keys* which is
+a pointer to a bulk of keys that are to be looked up, *num\_keys* is the number
+of keys that will be looked up, *match\_per\_key* is the possible
+max number of matches for each key, *match\_count* which is the returned number
+of matches for each key, and *set\_ids* are the returned target set
+ids for all matches found for each keys. *set\_ids* is 2-D array
+that for each key, a 1-D array should be sized according to *match\_per\_key*.
+*match\_per\_key* should be equal or smaller than maximum number of
+possible matches, similar to *rte\_member\_lookup\_multi*.
+
+
+Set-summary Element Delete
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. *rte\_membership\_delete(void \*setsum, const void \*key, MEMBERSHIP\_TARGET\_TYPE target\_id)*
+
+*rte\_membership\_delete* function deletes an element/key from a set-summary structure, if it fails
+an error is returned. The input arguments should include *key* which is a pointer to the
+element/key that needs to be deleted from the set-summary, and *set\_id*
+which is the set id associated with the key to delete. It worth noting that current
+implementation of vBF does not support deletion [1]_. An error code -EINVAL will be returned.
+
+.. [1] Traditional bloom filter does not support proactive deletion. Supporting proactive deletion require additional implementation and performance overhead.
+
+References
+-----------
+
+[Member-bloom] B H Bloom, "Space/Time Trade-offs in Hash Coding with Allowable Errors," Communications of the ACM, 1970.
+
+[Member-survey] A Broder and M Mitzenmacher, "Network Applications of Bloom Filters: A Survey," in Internet Mathematics, 2005.
+
+[Member-cfilter] B Fan, D G Andersen and M Kaminsky, "Cuckoo Filter: Practically Better Than Bloom," in Conference on emerging Networking Experiments and Technologies, 2014.
+
+[Member-OvS] B Pfaff, "The Design and Implementation of Open vSwitch," in NSDI, 2015.
\ No newline at end of file
diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst
index 170f4f9..4002df3 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -41,6 +41,23 @@ New Features
Also, make sure to start the actual text at the margin.
=========================================================
+* **Added Membership library (rte_member).**
+
+ Added membership library. It provides an API for DPDK applications to insert a
+ new member, delete an existing member, or query the existence of a member in a
+ given set, or a group of sets. For the case of a group of sets the library
+ will return not only whether the element has been inserted before in one of
+ the sets but also which set it belongs to.
+
+ The Membership Library is an extension and generalization of a traditional
+ filter (for example Bloom Filter) structure that has multiple usages in a wide
+ variety of workloads and applications. In general, the Membership Library is a
+ data structure that provides a “set-summary” and responds to set-membership
+ queries whether a certain member belongs to a set(s).
+
+ See the :ref:`Membership Library <Member_Library>` documentation in
+ the Programmers Guide document, for more information.
+
Resolved Issues
---------------
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v2 7/7] doc: add membership documentation
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 7/7] doc: add membership documentation Yipeng Wang
@ 2017-09-04 13:19 ` Mcnamara, John
0 siblings, 0 replies; 81+ messages in thread
From: Mcnamara, John @ 2017-09-04 13:19 UTC (permalink / raw)
To: Wang, Yipeng1, dev
Cc: stephen, luca.boccassi, Tai, Charlie, Gobriel, Sameh, Wang, Ren,
De Lara Guarch, Pablo, Wang, Yipeng1
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yipeng Wang
> Sent: Saturday, September 2, 2017 2:25 AM
> To: dev@dpdk.org
> Cc: stephen@networkplumber.org; luca.boccassi@gmail.com; Tai, Charlie
> <charlie.tai@intel.com>; Gobriel, Sameh <sameh.gobriel@intel.com>; Wang,
> Ren <ren.wang@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [dpdk-dev] [PATCH v2 7/7] doc: add membership documentation
>
> This patch adds the documentation for membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> ---
Thanks for the detailed docs and the nice images. Some comments below.
> +
> +.. _Member_Library:
> +
> +Membership Library
> +=======================
The underline should match the length of the title and should be followed by
a blank line. Make this change throughout the doc.
> +
> +Introduction
> +------------
> +The DPDK Membership Library provides an API for DPDK applications to
> insert a
> +new member, delete an existing member, or query the existence of a member
> in a
> +given set, or a group of sets. For the case of a group of sets the
> library
> +will return not only whether the element has been inserted before in one
> of
> +the sets but also which set it belongs to. The Membership Library is an
> +extension and generalization of a traditional filter structure (for
> example
> +Bloom Filter [Member-bloom]) that has multiple usages in a wide variety
> of
> +workloads and applications. In general, the Membership Library is a data
> +structure that provides a “set-summary” on whether a member belongs to a
> set,
Use standard quotes instead of smart quotes. The doc renderers will convert
them to smart quotes.
> +probability.
> +
> +.. _figure_membership1:
> +.. figure:: img/member_i1.*
> +
> + Example Usages of Membership Library
> +
> +We believe that there are various usages for a Membership Library in a
> very
> +large set of applications and workloads. Interested readers can refer to
> +[Member-survey] for a survey of possible networking usages. The following
> figure
> +provide a small set of examples of using the Membership Library. Sub-
> figure(a)
These sub-figures would be better as separate bullet sections.
There are a lot of other small fixes that I will send on to you off-line.
Reviewed-by: John McNamara <john.mcnamara@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v3 0/7] Add Membership Library
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
` (6 preceding siblings ...)
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 7/7] doc: add membership documentation Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 1/7] member: implement main API Yipeng Wang
` (7 more replies)
7 siblings, 8 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
DPDK Membership Library provides an API that can be used by many DPDK
applications to conduct one or more set-membership tests (we mention some
possible use cases below, but interested readers can refer to
[1] for a wider survey of use cases).
The basic functionalities of the Membership Library include
inserting a new member, deleting an existing member, and querying the existence
of a member. The query result would indicate with high accuracy which specific
set this member belongs to among a group of sets.
The Membership Library is an extension and generalization of traditional filter
data structures [2,3], which maintain a space-efficient “set-summary”.
There are two advantages of using such a set-summary rather than operating on a
“full-blown” complete list of elements: firstly it has a much smaller storage
requirement than storing the whole list of elements, and secondly set membership
tests (or other operations) is much more efficient than searching through the
complete list of elements.
A membership test for an element will return the set this element belongs to if
found (or return "not-found") with high accuracy. If needed, the accuracy of the
membership tests could be further increased with larger storage space.
Set-summary is a fundamental data aggregation component that can be used in many
network applications. It is a crucial structure to address performance and
scalability issues of diverse applications including overlay networks, wild card
flow classification, web-caches, load balancing, connection tracking,
data-centric networks, flow table summaries, network statistics and traffic
monitoring. Our Proof of Concept (PoC) using set-summary to optimize flow lookup
in Open vSwitch (OvS) shows a speedup of about 2-3X.
This patch set implements two types of set-summaries, i.e., hash-table based
set-summary (HTSS) and Vector Bloom Filter (vBF). HTSS supports both the
non-cache and cache modes. The non-cache mode can incur a small chance of
false-positives which is the case when the set-summary indicates a key belongs
to a given set while actually it is not. The cache mode can also have
false-negatives in addition to false-positives. False-negatives means the case
when the set-summary indicates a key does not belong to a given set while
actually it does. This happens because cache mode allows new key to evict
existing keys. vBF only has false-positives similar to the non-cache HTSS.
However, one can set the false-positive rate arbitrarily. HTSS's
false-positive rate is determined by the hash-table size and the signature size.
[1] A Broder and M Mitzenmacher, “Network Applications of Bloom Filters: A
Survey,” in Internet Mathematics, 2005.
[2] B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable Errors,”
Communications of the ACM, 1970.
[3] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically Better Than
Bloom,” in Conference on emerging Networking Experiments and Technologies, 2014.
v3:
- documentation: update documentation to incorperate John's review.
v2:
- test: add bad parameter test functions to test the creation fail case.
- test: add find existing test.
- member lib: Add num_entries check in rte_member_create_ht.
- member lib: Add multiple checks for rte_member_create_vbf to avoid divide-by-0.
- member lib: remove unnecessary ret from rte_member.c according to Stephen's
comment.
- member lib: change the vBF creation function to be not too conservative.
Previous algorithm is too conservative on false positive.
- member lib: fix uninitilize issue fail gcc-4.8 in rte_member_find_existing.
- member lib: add rte_member_find_existing to version map.
- makefile: lib/librte_member/Makefile: change include order according to
Luca's comment.
- documentation: update the programmer guide for membership library.
Yipeng Wang (7):
member: implement main API
member: implement HT mode
member: implement vBF mode
member: add AVX for HT mode
member: enable the library
test/member: add functional and perf tests
doc: add membership documentation
MAINTAINERS | 7 +
config/common_base | 5 +
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 ++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 +++
doc/guides/prog_guide/img/member_i6.svg | 332 ++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 421 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
lib/Makefile | 2 +
lib/librte_eal/common/eal_common_log.c | 1 +
lib/librte_eal/common/include/rte_log.h | 1 +
lib/librte_member/Makefile | 51 +
lib/librte_member/rte_member.c | 342 +++++++
lib/librte_member/rte_member.h | 518 ++++++++++
lib/librte_member/rte_member_ht.c | 569 +++++++++++
lib/librte_member/rte_member_ht.h | 115 +++
lib/librte_member/rte_member_vbf.c | 350 +++++++
lib/librte_member/rte_member_vbf.h | 85 ++
lib/librte_member/rte_member_version.map | 16 +
lib/librte_member/rte_member_x86.h | 111 ++
mk/rte.app.mk | 2 +
test/test/Makefile | 3 +
test/test/test_member.c | 682 +++++++++++++
test/test/test_member_perf.c | 643 ++++++++++++
30 files changed, 7099 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
create mode 100644 lib/librte_member/rte_member_version.map
create mode 100644 lib/librte_member/rte_member_x86.h
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v3 1/7] member: implement main API
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-22 10:47 ` Thomas Monjalon
2017-09-25 14:15 ` De Lara Guarch, Pablo
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 2/7] member: implement HT mode Yipeng Wang
` (6 subsequent siblings)
7 siblings, 2 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
Membership library is an extension and generalization of a traditional
filter (for example Bloom Filter) structure. In general, the Membership
library is a data structure that provides a "set-summary" and responds
to set-membership queries of whether a certain element belongs to a
set(s). A membership test for an element will return the set this element
belongs to or not-found if the element is never inserted into the
set-summary.
The results of the membership test is not 100% accurate. Certain
false positive or false negative probability could exist. However,
comparing to a "full-blown" complete list of elements, a "set-summary"
is memory efficient and fast on lookup.
This patch add the main API definition.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/Makefile | 2 +
lib/librte_eal/common/eal_common_log.c | 1 +
lib/librte_eal/common/include/rte_log.h | 1 +
lib/librte_member/Makefile | 49 +++
lib/librte_member/rte_member.c | 342 ++++++++++++++++++++
lib/librte_member/rte_member.h | 518 +++++++++++++++++++++++++++++++
lib/librte_member/rte_member_version.map | 16 +
7 files changed, 929 insertions(+)
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_version.map
diff --git a/lib/Makefile b/lib/Makefile
index 86caba1..c82033a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -108,6 +108,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder
DEPDIRS-librte_reorder := librte_eal librte_mempool librte_mbuf
DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump
DEPDIRS-librte_pdump := librte_eal librte_mempool librte_mbuf librte_ether
+DIRS-$(CONFIG_RTE_LIBRTE_MEMBER) += librte_member
+DEPDIRS-librte_member := librte_eal librte_hash
ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
diff --git a/lib/librte_eal/common/eal_common_log.c b/lib/librte_eal/common/eal_common_log.c
index 0e3b932..90db6a3 100644
--- a/lib/librte_eal/common/eal_common_log.c
+++ b/lib/librte_eal/common/eal_common_log.c
@@ -279,6 +279,7 @@ static const struct logtype logtype_strings[] = {
{RTE_LOGTYPE_CRYPTODEV, "cryptodev"},
{RTE_LOGTYPE_EFD, "efd"},
{RTE_LOGTYPE_EVENTDEV, "eventdev"},
+ {RTE_LOGTYPE_MEMBER, "member"},
{RTE_LOGTYPE_USER1, "user1"},
{RTE_LOGTYPE_USER2, "user2"},
{RTE_LOGTYPE_USER3, "user3"},
diff --git a/lib/librte_eal/common/include/rte_log.h b/lib/librte_eal/common/include/rte_log.h
index ec8dba7..ce1a3d0 100644
--- a/lib/librte_eal/common/include/rte_log.h
+++ b/lib/librte_eal/common/include/rte_log.h
@@ -87,6 +87,7 @@ extern struct rte_logs rte_logs;
#define RTE_LOGTYPE_CRYPTODEV 17 /**< Log related to cryptodev. */
#define RTE_LOGTYPE_EFD 18 /**< Log related to EFD. */
#define RTE_LOGTYPE_EVENTDEV 19 /**< Log related to eventdev. */
+#define RTE_LOGTYPE_MEMBER 20 /**< Log related to membership. */
/* these log types can be used in an application */
#define RTE_LOGTYPE_USER1 24 /**< User-defined log type 1. */
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
new file mode 100644
index 0000000..1a79eaa
--- /dev/null
+++ b/lib/librte_member/Makefile
@@ -0,0 +1,49 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_member.a
+
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
+
+EXPORT_MAP := rte_member_version.map
+
+LIBABIVER := 1
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+# install includes
+SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_member/rte_member.c b/lib/librte_member/rte_member.c
new file mode 100644
index 0000000..71b066d
--- /dev/null
+++ b/lib/librte_member/rte_member.c
@@ -0,0 +1,342 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+
+#include <rte_eal.h>
+#include <rte_eal_memconfig.h>
+#include <rte_memory.h>
+#include <rte_memzone.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+#include "rte_member_vbf.h"
+
+TAILQ_HEAD(rte_member_list, rte_tailq_entry);
+static struct rte_tailq_elem rte_member_tailq = {
+ .name = "RTE_MEMBER",
+};
+EAL_REGISTER_TAILQ(rte_member_tailq)
+
+
+void *
+rte_member_find_existing(const char *name)
+{
+ struct rte_member_setsum *setsum = NULL;
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(name, setsum->name, RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ if (te == NULL) {
+ rte_errno = ENOENT;
+ return NULL;
+ }
+ return setsum;
+}
+
+void
+rte_member_free(void *ss)
+{
+ struct rte_member_setsum *setsum;
+ struct rte_member_list *member_list = NULL;
+ struct rte_tailq_entry *te;
+
+ if (ss == NULL)
+ return;
+ setsum = ss;
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ if (te->data == ss)
+ break;
+ }
+ if (te == NULL) {
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return;
+ }
+ TAILQ_REMOVE(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_free_ht(setsum);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_free_vbf(setsum);
+ break;
+ default:
+ break;
+ }
+ rte_free(setsum);
+ rte_free(te);
+}
+
+
+void *
+rte_member_create(const struct rte_member_parameters *params)
+{
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list = NULL;
+ struct rte_member_setsum *setsum = NULL;
+ int ret;
+
+ if (params == NULL) {
+ rte_errno = EINVAL;
+ return NULL;
+ }
+
+ if ((params->key_len == 0)) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER,
+ "Memship create with invalid parameters\n");
+ return NULL;
+ }
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(params->name, setsum->name,
+ RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ setsum = NULL;
+ if (te != NULL) {
+ rte_errno = EEXIST;
+ te = NULL;
+ goto error_unlock_exit;
+ }
+ te = rte_zmalloc("MEMBER_TAILQ_ENTRY", sizeof(*te), 0);
+ if (te == NULL) {
+ RTE_LOG(ERR, MEMBER, "tailq entry allocation failed\n");
+ goto error_unlock_exit;
+ }
+
+ /* Create a new setsum structure */
+ setsum = (struct rte_member_setsum *) rte_zmalloc_socket(params->name,
+ sizeof(struct rte_member_setsum), RTE_CACHE_LINE_SIZE,
+ params->socket_id);
+ if (setsum == NULL) {
+ RTE_LOG(ERR, MEMBER, "Create setsummary failed\n");
+ goto error_unlock_exit;
+ }
+ setsum->type = params->type;
+ setsum->socket_id = params->socket_id;
+ setsum->key_len = params->key_len;
+ setsum->num_set = params->num_set;
+ setsum->name = params->name;
+ setsum->prim_hash_seed = params->prim_hash_seed;
+ setsum->sec_hash_seed = params->sec_hash_seed;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ ret = rte_member_create_ht(setsum, params);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ ret = rte_member_create_vbf(setsum, params);
+ break;
+ default:
+ goto error_unlock_exit;
+ }
+ if (ret < 0)
+ goto error_unlock_exit;
+ RTE_LOG(DEBUG, MEMBER, "Creating a setsummary table with mode %u\n",
+ setsum->type);
+
+ te->data = (void *)setsum;
+ TAILQ_INSERT_TAIL(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return setsum;
+
+error_unlock_exit:
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ rte_member_free(setsum);
+ return NULL;
+}
+
+
+int
+rte_member_add(const void *ss, const void *key, MEMBER_SET_TYPE set_id)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_add_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_add_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_lookup(const void *ss, const void *key,
+ MEMBER_SET_TYPE *set_id)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_lookup_bulk(const void *ss, const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || keys == NULL || set_ids == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_bulk_ht(setsum, keys, num_keys,
+ set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_bulk_vbf(setsum, keys, num_keys,
+ set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi(const void *ss, const void *key,
+ uint32_t match_per_key, MEMBER_SET_TYPE *set_id)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_ht(setsum, key, match_per_key,
+ set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_vbf(setsum, key, match_per_key,
+ set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_lookup_multi_bulk(const void *ss, const void **keys,
+ uint32_t num_keys, uint32_t max_match_per_key,
+ uint32_t *match_count, MEMBER_SET_TYPE *set_ids)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || keys == NULL || set_ids == NULL ||
+ match_count == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_bulk_ht(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_bulk_vbf(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+
+int
+rte_member_delete(void *ss, const void *key, MEMBER_SET_TYPE set_id)
+{
+ struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_delete_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ default:
+ return -EINVAL;
+ }
+}
+
+
+void
+rte_member_reset(const void *ss)
+{
+ const struct rte_member_setsum *setsum = ss;
+
+ if (setsum == NULL)
+ return;
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_reset_ht(setsum);
+ return;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_reset_vbf(setsum);
+ return;
+ default:
+ return;
+ }
+}
diff --git a/lib/librte_member/rte_member.h b/lib/librte_member/rte_member.h
new file mode 100644
index 0000000..de44b1b
--- /dev/null
+++ b/lib/librte_member/rte_member.h
@@ -0,0 +1,518 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+
+
+/**
+ * @file
+ *
+ * RTE Membership Library
+ *
+ * The Membership Library is an extension and generalization of a traditional
+ * filter (for example Bloom Filter) structure that has multiple usages in a
+ * variety of workloads and applications. The library is used to test if a key
+ * belongs to certain sets. Two types of such "set-summary" structures are
+ * implemented: hash-table based (HT) and vector bloom filter (vBF). For HT
+ * setsummary, two subtype or modes are available, cache and non-cache modes.
+ * The table below summarize some properties of the different implementations.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ */
+
+
+/**
+ * <!--
+ * +==========+=====================+================+=========================+
+ * | type | vbf | HT-cache | HT-non-cache |
+ * +==========+=====================+==========================================+
+ * |structure | bloom-filter array | hash-table like without storing key |
+ * +----------+---------------------+------------------------------------------+
+ * |set id | limited by bf count | [1, 0x7fff] |
+ * | | up to 32. | |
+ * +----------+---------------------+------------------------------------------+
+ * |usages & | small set range, | can delete, | cache most recent keys, |
+ * |properties| user-specified | big set range, | have both false-positive|
+ * | | false-positive rate,| small false | and false-negative |
+ * | | no deletion support.| positive depend| depend on table size, |
+ * | | | on table size, | automatic overwritten. |
+ * | | | new key does | |
+ * | | | not overwrite | |
+ * | | | existing key. | |
+ * +----------+---------------------+----------------+-------------------------+
+ * -->
+ */
+
+
+#ifndef _RTE_MEMBER_H_
+#define _RTE_MEMBER_H_
+
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+#include <stdio.h>
+#include <stdint.h>
+#include <rte_hash_crc.h>
+
+/** The set ID type that stored internally in hash table based set summary. */
+typedef uint16_t MEMBER_SET_TYPE;
+/** Invalid set ID used to mean no match found. */
+#define RTE_MEMBER_NO_MATCH 0
+/** Maximum size of hash table that can be created. */
+#define RTE_MEMBER_ENTRIES_MAX (1 << 30)
+/** Maximum number of keys that can be searched as a bulk */
+#define RTE_MEMBER_LOOKUP_BULK_MAX 64
+/** Entry count per bucket in hash table based mode. */
+#define RTE_MEMBER_BUCKET_ENTRIES 16
+/** Maximum number of characters in setsum name. */
+#define RTE_MEMBER_NAMESIZE 32
+
+/** @internal Primary hash function to calculate 1st independent hash. */
+#define MEMBER_PRIM_HASH(key, key_len, seed) \
+ (uint32_t)(rte_hash_crc(key, key_len, seed))
+/** @internal Secondary hash function to calculate 2nd independent hash. */
+#define MEMBER_SEC_HASH(key, key_len, seed) \
+ (uint32_t)(rte_hash_crc(key, key_len, seed))
+
+
+struct rte_member_setsum;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameter struct used to create set summary
+ */
+struct rte_member_parameters;
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Define different set summary types
+ */
+enum rte_member_setsum_type {
+ RTE_MEMBER_TYPE_HT = 0, /**< Hash table based set summary. */
+ RTE_MEMBER_TYPE_VBF, /**< vector of bloom filters. */
+ RTE_MEMBER_NUM_TYPE
+};
+
+/** @internal compare function for different arch. */
+enum rte_member_sig_compare_function {
+ RTE_MEMBER_COMPARE_SCALAR = 0,
+ RTE_MEMBER_COMPARE_AVX2,
+ RTE_MEMBER_COMPARE_NUM
+};
+
+/** @internal setsummary structure. */
+struct rte_member_setsum {
+ enum rte_member_setsum_type type;
+ const char *name;
+ uint32_t key_len;
+ uint32_t socket_id; /* NUMA Socket ID for memory. */
+ uint32_t prim_hash_seed;
+ uint32_t sec_hash_seed;
+
+
+ /* hash table based */
+ uint32_t bucket_cnt;
+ uint32_t bucket_mask;
+ uint8_t cache;
+ enum rte_member_sig_compare_function sig_cmp_fn;
+
+ /* vector bloom filter*/
+ uint32_t num_set;
+ uint32_t bits;
+ uint32_t bit_mask;
+ uint32_t num_hashes;
+ void *table;
+};
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameters used when create the set summary table. Currently user can
+ * specify two types of setsummary: HT based and vBF. For HT based, user can
+ * specify cache or non-cache mode. Here is a table to describe some differences
+ *
+ */
+struct rte_member_parameters {
+ const char *name; /**< Name of the hash. */
+
+ /**
+ * User to specify the type of the setsummary from one of
+ * rte_member_setsum_type.
+ *
+ * HT based setsummary is implemented like a hash table. User should use
+ * this type when there are many sets.
+ *
+ * vBF setsummary is a vector of bloom filters. It is used when number
+ * of sets is not big (less than 32 for current implementation).
+ */
+ enum rte_member_setsum_type type;
+ /**
+ * If it is HT based setsummary, user to specify the subtype or mode
+ * of the setsummary. It could be cache, or non-cache mode.
+ * Set iscache to be 1 if to use as cache mode.
+ *
+ * For cache mode, keys can be evicted out of the HT setsummary. Keys
+ * with the same signature and map to the same bucket
+ * will overwrite each other in the setsummary table.
+ * This mode is useful for the case that the set-summary only
+ * needs to keep record of the recently inserted keys. Both
+ * false-negative and false-positive could happen.
+ *
+ * For non-cache mode, keys cannot be evicted out of the cache. So for
+ * this mode the setsummary will become full eventually. Keys with the
+ * same signature but map to the same bucket will still occupy multiple
+ * entries. This mode does not give false-negative result.
+ */
+ uint8_t iscache;
+
+ /**
+ * For HT setsummary, num_keys equals to the number of entries of the
+ * table. When the number of keys that inserted to the HT setsummary
+ * approaches this number, eviction could happen. For cache mode,
+ * keys could be evicted out of the table. For non-cache mode, keys will
+ * be evicted to other buckets like cuckoo hash. The table will also
+ * likely to become full before the number of inserted keys equal to the
+ * total number of entries.
+ *
+ * For vBF, num_keys equal to the expected number of keys that will
+ * be inserted into the vBF. The implementation assumes the keys are
+ * evenly distributed to each BF in vBF. This is used to calculate the
+ * number of bits we need for each BF. User does not specify the size of
+ * each BF directly because the optimal size depends on the num_keys
+ * and false positive rate.
+ */
+ uint32_t num_keys;
+
+
+ /**
+ * The length of key is used for hash calculation. Since key is not
+ * stored in set-summary, large key does not require more memory space.
+ */
+ uint32_t key_len;
+
+
+ /**
+ * num_set is only relevant to vBF based setsummary.
+ * num_set is equal to the number of BFs in vBF. For current
+ * implementation, it only supports 1,2,4,8,16,32 BFs in one vBF set
+ * summary. If other number of sets are needed, for example 5, the user
+ * should allocate the minimum available value that larger than 5,
+ * which is 8.
+ */
+ uint32_t num_set;
+
+ /**
+ * false_postive_rate is only relevant to vBF based setsummary.
+ * false_postivie_rate is the user-defined false positive rate
+ * given expected number of inserted keys (num_keys). It is used to
+ * calculate the total number of bits for each BF, and the number of
+ * hash values used during lookup and insertion. For details please
+ * refer to vBF implementation and membership library documentation.
+ *
+ * HT setsummary's false positive rate is in the order of:
+ * false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit signature.
+ * This is because two keys needs to map to same bucket and same
+ * signature to have a collision (false positive). bucket_count is equal
+ * to number of entries (num_keys) divided by entry count per bucket
+ * (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_postivie_rate is not
+ * directly set by users.
+ */
+ float false_positive_rate;
+
+ /**
+ * We use two seeds to calculate two independent hashes for each key.
+ *
+ * For HT type, one hash is used as signature, and the other is used
+ * for bucket location.
+ * For vBF type, these two hashes and their combinations are used as
+ * hash locations to index the bit array.
+ */
+ uint32_t prim_hash_seed;
+
+ /**
+ * The secondary seed should be a different value from the primary seed.
+ */
+ uint32_t sec_hash_seed;
+
+ int socket_id; /**< NUMA Socket ID for memory. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Find an existing set-summary and return a pointer to it.
+ *
+ * @param name
+ * Name of the set-summary
+ * @return
+ * Pointer to the set-summary or NULL if object not found
+ * with rte_errno set appropriately. Possible rte_errno values include:
+ * - ENOENT - value not available for return
+ */
+void *
+rte_member_find_existing(const char *name);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Create set-summary (SS).
+ *
+ * @param params
+ * parameters to initialize the setsummary
+ * @return
+ * return the pointer to the setsummary
+ * return value is NULL if the creation failed
+ */
+
+void *
+rte_member_create(const struct rte_member_parameters *params);
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup key in set-summary (SS).
+ * Single key lookup and return as soon as the first match found
+ * @param setsum
+ * pointer of a setsummary
+ * @param key
+ * pointer of the key that needs to lookup
+ * @param set_id
+ * output the set id matches the key
+ * @return
+ * return 1 for found a match and 0 for not found a match
+ */
+
+int
+rte_member_lookup(const void *setsum,
+ const void *key, MEMBER_SET_TYPE *set_id);
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * lookup bulk of keys in set-summary (SS).
+ * Each key lookup returns as soon as the first match found
+ * @param setsum
+ * Pointer of a setsummary
+ * @param keys
+ * Pointer of bulk of keys that to be lookup
+ * @param num_keys
+ * Number of keys that will be lookup
+ * @param set_ids
+ * Output set ids for all the keys to this array.
+ * User should preallocate array that can contain all results, which size is
+ * the num_keys.
+ * @return
+ * The number of keys that found a match
+ */
+
+
+int
+rte_member_lookup_bulk(const void *setsum,
+ const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids);
+
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a key in set-summary (SS) for multiple matches.
+ * The key lookup will find all matched entries (multiple match).
+ * Note that for cache mode of HT, each key can have at most one match. This is
+ * because keys with same signature that maps to same bucket will overwrite
+ * each other. So multi-match lookup should be used for vBF and non-cache HT.
+ * @param setsum
+ * pointer of a set-summary
+ * @param key
+ * The key that to be lookup
+ * @param max_match_per_key
+ * User specified maximum number of matches for each key. The function returns
+ * as soon as this number of matches found for the key.
+ * @param set_id
+ * Output set ids for all the matches of the key. User needs to preallocate
+ * the array that can contain max_match_per_key number of results.
+ * @return
+ * The number of matches that found for the key.
+ * For cache mode HT set-summary, the number should be at most 1
+ */
+
+int
+rte_member_lookup_multi(const void *setsum,
+ const void *key, uint32_t max_match_per_key,
+ MEMBER_SET_TYPE *set_id);
+
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a bulk of keys in set-summary (SS) for multiple matches each key.
+ * Each key lookup will find all matched entries (multiple match).
+ * Note that for cache mode HT, each key can have at most one match. So
+ * multi-match function is mainly used for vBF and non-cache mode HT.
+ * @param setsum
+ * pointer of a setsummary
+ * @param keys
+ * The keys that to be lookup
+ * @param num_keys
+ * The number of keys that will be lookup
+ * @param max_match_per_key
+ * The possible maximum number of matches for each key
+ * @param match_count
+ * Output the number of matches for each key in an array
+ * @param set_ids
+ * Return set ids for all the matches of all keys. User pass in a preallocated
+ * 2D array with first dimension as key index and second dimension as match
+ * index. For example set_ids[bulk_size][max_match_per_key]
+ * @return
+ * The number of keys that found one or more matches in the set-summary
+ */
+
+int
+rte_member_lookup_multi_bulk(const void *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Insert key into set-summary (SS).
+ *
+ * @param setsum
+ * pointer of a set-summary
+ * @param key
+ * the key that needs to be added
+ * @param set_id
+ * The set id associated with the key that needs to be added. Different mode
+ * supports different set_id ranges. 0 cannot be used as set_id since
+ * RTE_MEMBER_NO_MATCH by default is set as 0.
+ * For HT mode, the set_id has range as [1, 0x7FFF], MSB is reserved.
+ * For vBF mode the set id is limited by the num_set parameter when create
+ * the set-summary.
+ * @return
+ * HT (cache mode) and vBF should never fail unless the set_id is not in the
+ * valid range. In such case -EINVAL is returned.
+ * For HT (non-cache mode) it could fail with -ENOSPC error code when table is
+ * full.
+ * For success it returns different values for different modes to provide
+ * extra information for users.
+ * Return 0 for HT (cache mode) if the add does not cause
+ * eviction, return 1 otherwise. Return 0 for HT mode if success, -ENOSPC for
+ * full, and 1 if cuckoo eviction happens. Always return 0 for vBF mode.
+ */
+
+int
+rte_member_add(const void *setsum, const void *key,
+ MEMBER_SET_TYPE set_id);
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * De-allocate memory used by set-summary.
+ * @param setsum
+ * Pointer to the set summary
+ */
+void
+rte_member_free(void *setsum);
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Reset the set-summary tables. E.g. reset bits to be 0 in BF,
+ * reset set_id in each entry to be RTE_MEMBER_NO_MATCH in HT based SS.
+ * @param setsum
+ * Pointer to the set-summary
+ */
+void
+rte_member_reset(const void *setsum);
+
+
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Delete items from the set-summary. Note that vBF does not support deletion
+ * in current implementation. For vBF, error code of -EINVAL will be returned.
+ * @param setsum
+ * Pointer to the set-summary
+ * @param key
+ * The key to be deleted
+ * @param set_id
+ * For HT mode, we need both key and its corresponding set_id to
+ * properly delete the key. Without set_id, we may delete other keys with the
+ * same signature
+ * @return
+ * If no entry found to delete, an error code of -ENOENT could be returned
+ */
+
+int
+rte_member_delete(void *setsum, const void *key,
+ MEMBER_SET_TYPE set_id);
+
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_H_ */
diff --git a/lib/librte_member/rte_member_version.map b/lib/librte_member/rte_member_version.map
new file mode 100644
index 0000000..a5877c9
--- /dev/null
+++ b/lib/librte_member/rte_member_version.map
@@ -0,0 +1,16 @@
+DPDK_17.11 {
+ global:
+
+ rte_member_create;
+ rte_member_find_existing;
+ rte_member_lookup;
+ rte_member_lookup_bulk;
+ rte_member_lookup_multi;
+ rte_member_lookup_multi_bulk;
+ rte_member_add;
+ rte_member_free;
+ rte_member_reset;
+ rte_member_delete;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v3 1/7] member: implement main API
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 1/7] member: implement main API Yipeng Wang
@ 2017-09-22 10:47 ` Thomas Monjalon
2017-09-25 14:15 ` De Lara Guarch, Pablo
1 sibling, 0 replies; 81+ messages in thread
From: Thomas Monjalon @ 2017-09-22 10:47 UTC (permalink / raw)
To: Yipeng Wang
Cc: dev, charlie.tai, sameh.gobriel, ren.wang, john.mcnamara, olivier.matz
06/09/2017 01:59, Yipeng Wang:
> --- a/lib/librte_eal/common/eal_common_log.c
> +++ b/lib/librte_eal/common/eal_common_log.c
> @@ -279,6 +279,7 @@ static const struct logtype logtype_strings[] = {
> {RTE_LOGTYPE_CRYPTODEV, "cryptodev"},
> {RTE_LOGTYPE_EFD, "efd"},
> {RTE_LOGTYPE_EVENTDEV, "eventdev"},
> + {RTE_LOGTYPE_MEMBER, "member"},
> {RTE_LOGTYPE_USER1, "user1"},
> {RTE_LOGTYPE_USER2, "user2"},
> {RTE_LOGTYPE_USER3, "user3"},
The static logtypes should be removed.
Please use the new dynamic log types.
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v3 1/7] member: implement main API
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 1/7] member: implement main API Yipeng Wang
2017-09-22 10:47 ` Thomas Monjalon
@ 2017-09-25 14:15 ` De Lara Guarch, Pablo
1 sibling, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-09-25 14:15 UTC (permalink / raw)
To: Wang, Yipeng1, dev
Cc: Tai, Charlie, Gobriel, Sameh, Wang, Ren, Wang, Yipeng1, Mcnamara, John
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yipeng Wang
> Sent: Wednesday, September 6, 2017 1:00 AM
> To: dev@dpdk.org
> Cc: Tai, Charlie <charlie.tai@intel.com>; Gobriel, Sameh
> <sameh.gobriel@intel.com>; Wang, Ren <ren.wang@intel.com>; Wang,
> Yipeng1 <yipeng1.wang@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>
> Subject: [dpdk-dev] [PATCH v3 1/7] member: implement main API
>
> Membership library is an extension and generalization of a traditional filter
> (for example Bloom Filter) structure. In general, the Membership library is a
> data structure that provides a "set-summary" and responds to set-
> membership queries of whether a certain element belongs to a set(s). A
> membership test for an element will return the set this element belongs to
> or not-found if the element is never inserted into the set-summary.
>
> The results of the membership test is not 100% accurate. Certain false
Is -> are.
> positive or false negative probability could exist. However, comparing to a
> "full-blown" complete list of elements, a "set-summary"
> is memory efficient and fast on lookup.
>
> This patch add the main API definition.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> ---
> lib/Makefile | 2 +
> lib/librte_eal/common/eal_common_log.c | 1 +
> lib/librte_eal/common/include/rte_log.h | 1 +
> lib/librte_member/Makefile | 49 +++
> lib/librte_member/rte_member.c | 342 ++++++++++++++++++++
> lib/librte_member/rte_member.h | 518
> +++++++++++++++++++++++++++++++
> lib/librte_member/rte_member_version.map | 16 +
> 7 files changed, 929 insertions(+)
> create mode 100644 lib/librte_member/Makefile create mode 100644
> lib/librte_member/rte_member.c create mode 100644
> lib/librte_member/rte_member.h create mode 100644
> lib/librte_member/rte_member_version.map
>
...
> diff --git a/lib/librte_member/rte_member.c
> b/lib/librte_member/rte_member.c new file mode 100644 index
> 0000000..71b066d
> --- /dev/null
> +++ b/lib/librte_member/rte_member.c
...
> +
> +#include <string.h>
> +
> +#include <rte_eal.h>
> +#include <rte_eal_memconfig.h>
> +#include <rte_memory.h>
> +#include <rte_memzone.h>
> +#include <rte_malloc.h>
> +#include <rte_errno.h>
> +
> +#include "rte_member.h"
> +#include "rte_member_ht.h"
> +#include "rte_member_vbf.h"
> +
> +TAILQ_HEAD(rte_member_list, rte_tailq_entry); static struct
> +rte_tailq_elem rte_member_tailq = {
> + .name = "RTE_MEMBER",
> +};
> +EAL_REGISTER_TAILQ(rte_member_tailq)
> +
> +
> +void *
> +rte_member_find_existing(const char *name) {
> + struct rte_member_setsum *setsum = NULL;
> + struct rte_tailq_entry *te;
> + struct rte_member_list *member_list;
> +
> + member_list = RTE_TAILQ_CAST(rte_member_tailq.head,
> rte_member_list);
> +
> + rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
> + TAILQ_FOREACH(te, member_list, next) {
> + setsum = (struct rte_member_setsum *) te->data;
> + if (strncmp(name, setsum->name,
> RTE_MEMBER_NAMESIZE) == 0)
> + break;
> + }
> + rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
> +
> + if (te == NULL) {
> + rte_errno = ENOENT;
> + return NULL;
> + }
> + return setsum;
> +}
> +
> +void
> +rte_member_free(void *ss)
Why not using directly "struct rte_member_setsum", so you can use it directly?
This applies to other functions that are using "void *".
I see that the content of this structure is internal, but you can still use a pointer to that structure.
...
> +void *
> +rte_member_create(const struct rte_member_parameters *params) {
> + struct rte_tailq_entry *te;
> + struct rte_member_list *member_list = NULL;
> + struct rte_member_setsum *setsum = NULL;
> + int ret;
> +
> + if (params == NULL) {
> + rte_errno = EINVAL;
> + return NULL;
> + }
> +
> + if ((params->key_len == 0)) {
> + rte_errno = EINVAL;
> + RTE_LOG(ERR, MEMBER,
> + "Memship create with invalid parameters\n");
rte_member_create has invalid parameters?
> diff --git a/lib/librte_member/rte_member.h
> b/lib/librte_member/rte_member.h new file mode 100644 index
> 0000000..de44b1b
> --- /dev/null
> +++ b/lib/librte_member/rte_member.h
> @@ -0,0 +1,518 @@
...
> +enum rte_member_setsum_type {
> + RTE_MEMBER_TYPE_HT = 0, /**< Hash table based set summary.
> */
> + RTE_MEMBER_TYPE_VBF, /**< vector of bloom filters. */
"vector" -> "Vector".
> + RTE_MEMBER_NUM_TYPE
> +};
> +
> +/** @internal compare function for different arch. */ enum
> +rte_member_sig_compare_function {
> + RTE_MEMBER_COMPARE_SCALAR = 0,
> + RTE_MEMBER_COMPARE_AVX2,
> + RTE_MEMBER_COMPARE_NUM
> +};
> +
> +/** @internal setsummary structure. */
> +struct rte_member_setsum {
> + enum rte_member_setsum_type type;
> + const char *name;
> + uint32_t key_len;
> + uint32_t socket_id; /* NUMA Socket ID for memory. */
Either comment all the fields or do not do it (for this case, because the
structure is internal, otherwise, all fields would require a comment).
Also, remember that they should all start with capital letters.
> + uint32_t prim_hash_seed;
> + uint32_t sec_hash_seed;
> +
> +
> + /* hash table based */
> + uint32_t bucket_cnt;
> + uint32_t bucket_mask;
> + uint8_t cache;
> + enum rte_member_sig_compare_function sig_cmp_fn;
...
> +struct rte_member_parameters {
...
> + uint8_t iscache;
As said in the documentation patch, change to "is_cache".
> +
> + /**
> + * For HT setsummary, num_keys equals to the number of entries
> of the
> + * table. When the number of keys that inserted to the HT
> setsummary
"number of keys inserted in the HT summary".
> + * approaches this number, eviction could happen. For cache mode,
> + * keys could be evicted out of the table. For non-cache mode, keys
> will
> + * be evicted to other buckets like cuckoo hash. The table will also
> + * likely to become full before the number of inserted keys equal to
> the
> + * total number of entries.
> + *
> + * For vBF, num_keys equal to the expected number of keys that
> will
> + * be inserted into the vBF. The implementation assumes the keys
> are
> + * evenly distributed to each BF in vBF. This is used to calculate the
> + * number of bits we need for each BF. User does not specify the
> size of
> + * each BF directly because the optimal size depends on the
> num_keys
> + * and false positive rate.
> + */
> + uint32_t num_keys;
> +
> +
Leave just one blank line between fields.
> + /**
> + * The length of key is used for hash calculation. Since key is not
> + * stored in set-summary, large key does not require more memory
> space.
> + */
> + uint32_t key_len;
> +
> +
> + /**
> + * num_set is only relevant to vBF based setsummary.
> + * num_set is equal to the number of BFs in vBF. For current
> + * implementation, it only supports 1,2,4,8,16,32 BFs in one vBF set
> + * summary. If other number of sets are needed, for example 5, the
> user
> + * should allocate the minimum available value that larger than 5,
> + * which is 8.
> + */
> + uint32_t num_set;
> +
> + /**
> + * false_postive_rate is only relevant to vBF based setsummary.
> + * false_postivie_rate is the user-defined false positive rate
> + * given expected number of inserted keys (num_keys). It is used to
> + * calculate the total number of bits for each BF, and the number of
> + * hash values used during lookup and insertion. For details please
> + * refer to vBF implementation and membership library
> documentation.
> + *
> + * HT setsummary's false positive rate is in the order of:
> + * false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit
> signature.
> + * This is because two keys needs to map to same bucket and same
> + * signature to have a collision (false positive). bucket_count is
> equal
> + * to number of entries (num_keys) divided by entry count per
> bucket
> + * (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_postivie_rate
> is not
> + * directly set by users.
> + */
Typo in "positive" in several places in the comment above.
Also, clarify that "false_positive_rate" is not directly set by users for HT mode.
> + float false_positive_rate;
...
> + * Lookup key in set-summary (SS).
> + * Single key lookup and return as soon as the first match found
Leave a blank line before parameters.
> + * @param setsum
> + * pointer of a setsummary
> + * @param key
> + * pointer of the key that needs to lookup
Better something like this "Pointer of the key to be looked up"
> + * @param set_id
> + * output the set id matches the key
> + * @return
> + * return 1 for found a match and 0 for not found a match
> + */
> +
Definitions of the fields should start with capital letter.
> +int
> +rte_member_lookup(const void *setsum,
> + const void *key, MEMBER_SET_TYPE *set_id);
> +
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * lookup bulk of keys in set-summary (SS).
"Lookup bulk"
> + * Each key lookup returns as soon as the first match found
> + * @param setsum
> + * Pointer of a setsummary
> + * @param keys
> + * Pointer of bulk of keys that to be lookup
"Pointer of bulk of keys to be looked up".
Check for this in other parts of the code.
> + * @param num_keys
> + * Number of keys that will be lookup
> + * @param set_ids
> + * Output set ids for all the keys to this array.
> + * User should preallocate array that can contain all results, which size is
> + * the num_keys.
> + * @return
> + * The number of keys that found a match
> + */
> +
> +
> +int
> +rte_member_lookup_bulk(const void *setsum,
> + const void **keys, uint32_t num_keys,
> + MEMBER_SET_TYPE *set_ids);
> +
> +
> +
> +
Too many blank spaces. Generally leave one between functions.
Check for this in the rest of the files.
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Lookup a bulk of keys in set-summary (SS) for multiple matches each
> key.
> + * Each key lookup will find all matched entries (multiple match).
> + * Note that for cache mode HT, each key can have at most one match. So
> + * multi-match function is mainly used for vBF and non-cache mode HT.
> + * @param setsum
> + * pointer of a setsummary
> + * @param keys
> + * The keys that to be lookup
> + * @param num_keys
> + * The number of keys that will be lookup
> + * @param max_match_per_key
> + * The possible maximum number of matches for each key
> + * @param match_count
> + * Output the number of matches for each key in an array
> + * @param set_ids
> + * Return set ids for all the matches of all keys. User pass in a
> preallocated
"User passes in"
> + * 2D array with first dimension as key index and second dimension as
> match
> + * index. For example set_ids[bulk_size][max_match_per_key]
> + * @return
> + * The number of keys that found one or more matches in the set-
> summary
> + */
> +
> +int
> +rte_member_lookup_multi_bulk(const void *setsum,
> + const void **keys, uint32_t num_keys,
> + uint32_t max_match_per_key,
> + uint32_t *match_count,
> + MEMBER_SET_TYPE *set_ids);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Insert key into set-summary (SS).
> + *
> + * @param setsum
> + * pointer of a set-summary
Start with capital letter.
> + * @param key
> + * the key that needs to be added
> + * @param set_id
> + * The set id associated with the key that needs to be added. Different
> mode
> + * supports different set_id ranges. 0 cannot be used as set_id since
> + * RTE_MEMBER_NO_MATCH by default is set as 0.
> + * For HT mode, the set_id has range as [1, 0x7FFF], MSB is reserved.
> + * For vBF mode the set id is limited by the num_set parameter when
> create
> + * the set-summary.
> + * @return
> + * HT (cache mode) and vBF should never fail unless the set_id is not in
> the
> + * valid range. In such case -EINVAL is returned.
> + * For HT (non-cache mode) it could fail with -ENOSPC error code when
> table is
> + * full.
> + * For success it returns different values for different modes to provide
> + * extra information for users.
> + * Return 0 for HT (cache mode) if the add does not cause
> + * eviction, return 1 otherwise. Return 0 for HT mode if success, -ENOSPC
For HT non-cache mode?
> for
> + * full, and 1 if cuckoo eviction happens. Always return 0 for vBF mode.
> + */
> +
> +int
> +rte_member_add(const void *setsum, const void *key,
> + MEMBER_SET_TYPE set_id);
> +
> +
...
> diff --git a/lib/librte_member/rte_member_version.map
> b/lib/librte_member/rte_member_version.map
> new file mode 100644
> index 0000000..a5877c9
> --- /dev/null
> +++ b/lib/librte_member/rte_member_version.map
> @@ -0,0 +1,16 @@
> +DPDK_17.11 {
> + global:
> +
> + rte_member_create;
> + rte_member_find_existing;
> + rte_member_lookup;
> + rte_member_lookup_bulk;
> + rte_member_lookup_multi;
> + rte_member_lookup_multi_bulk;
> + rte_member_add;
> + rte_member_free;
> + rte_member_reset;
> + rte_member_delete;
This list must be in alphabetical order.
> +
> + local: *;
> +};
> --
> 2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v3 2/7] member: implement HT mode
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 1/7] member: implement main API Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 3/7] member: implement vBF mode Yipeng Wang
` (5 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
One of the set-summray structure is hash-table based
set-summary (HTSS). One example is cuckoo filter [1].
Comparing to a traditional hash table, HTSS has a much more
compact structure. For each element, only one signature and
its corresponding set ID is stored. No key comparison is required
during lookup. For the table structure, there are multiple entries
in each bucket, and the table is composed of many buckets.
Two modes are supported for HTSS, "cache" and "none-cache" modes.
The non-cache mode is more similar to traditional cuckoo filter.
When a bucket is full, one entry will be evicted to its
alternative bucket to make space for the new key. The table could
be full and then no more keys could be inserted. This mode has
false-positive rate but no false-negative. Multiple entries
with same signature could stay in the same bucket.
The "cache" mode does not evict key to its alternative bucket
when a bucket is full, an existing key will be evicted out of
the table like a cache. So the table will never reject keys when
it is full. Another property is in each bucket, there cannot be
multiple entries with same signature. The mode could have both
false-positive and false-negative probability.
This patch add the implementation of HTSS.
[1] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically
Better Than Bloom,” in Conference on emerging Networking
Experiments and Technologies, 2014.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_ht.c | 490 ++++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_ht.h | 115 +++++++++
3 files changed, 606 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 1a79eaa..ad26548 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
new file mode 100644
index 0000000..b2ae6d0
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.c
@@ -0,0 +1,490 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_errno.h>
+#include <rte_malloc.h>
+#include <rte_prefetch.h>
+#include <rte_random.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+
+
+static inline int
+insert_overwrite_search(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE set_id)
+{
+ int i;
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[bucket].sigs[i] == tmp_sig) {
+ buckets[bucket].sets[i] = set_id;
+ return 1;
+ }
+ }
+ return 0;
+}
+
+
+
+
+static inline int
+search_bucket_single(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE *set_id)
+{
+ int iter;
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket].sigs[iter] &&
+ buckets[bucket].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket].sets[iter];
+ return 1;
+ }
+ }
+ return 0;
+}
+
+
+static inline void
+search_bucket_multi(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ int iter;
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket].sigs[iter] &&
+ buckets[bucket].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket].sets[iter];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ }
+}
+
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+ uint32_t i, j;
+ uint32_t size_bucket_t;
+ uint32_t num_entries = rte_align32pow2(params->num_keys);
+
+ if ((num_entries > RTE_MEMBER_ENTRIES_MAX) ||
+ !rte_is_power_of_2(RTE_MEMBER_BUCKET_ENTRIES) ||
+ num_entries < RTE_MEMBER_BUCKET_ENTRIES) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER,
+ "Membership HT create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ uint32_t num_buckets = num_entries / RTE_MEMBER_BUCKET_ENTRIES;
+
+ size_bucket_t = sizeof(struct member_ht_bucket);
+
+ struct member_ht_bucket *buckets = rte_zmalloc_socket(NULL,
+ num_buckets * size_bucket_t,
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ if (buckets == NULL)
+ return -ENOMEM;
+
+ ss->table = buckets;
+ ss->bucket_cnt = num_buckets;
+ ss->bucket_mask = num_buckets - 1;
+ ss->cache = params->iscache;
+
+ for (i = 0; i < num_buckets; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+
+
+ RTE_LOG(DEBUG, MEMBER, "Hash table based filter created, "
+ "the table has %u entries, %u buckets\n",
+ num_buckets,
+ num_buckets / RTE_MEMBER_BUCKET_ENTRIES);
+ return 0;
+}
+
+static inline
+void get_buckets_index(const struct rte_member_setsum *ss, const void *key,
+ uint32_t *prim_bkt, uint32_t *sec_bkt, SIG_TYPE *sig)
+{
+ uint32_t first_hash = MEMBER_PRIM_HASH(key, ss->key_len,
+ ss->prim_hash_seed);
+ uint32_t sec_hash = MEMBER_SEC_HASH(&first_hash, 4, ss->sec_hash_seed);
+ *sig = first_hash & SIG_BITMASK;
+ if (ss->cache) {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (sec_hash >> 16) & ss->bucket_mask;
+ } else {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (*prim_bkt ^ *sig) & ss->bucket_mask;
+ }
+}
+
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *ss,
+ const void *key, MEMBER_SET_TYPE *set_id)
+{
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+
+ return 0;
+}
+
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, MEMBER_SET_TYPE *set_id)
+{
+ uint32_t i;
+ uint32_t ret = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ SIG_TYPE tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+
+ for (i = 0; i < num_keys; i++) {
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return ret;
+}
+
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t ret = 0;
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
+ match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+}
+
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids)
+{
+ uint32_t i;
+ uint32_t ret = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t match_cnt_t;
+ SIG_TYPE tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX] = {0};
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_t = 0;
+
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_t < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ }
+ return ret;
+}
+
+static inline int
+try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ SIG_TYPE sig, MEMBER_SET_TYPE set_id)
+{
+ int i;
+ /* If not full then insert into one slot*/
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[prim].sigs[i] = sig;
+ buckets[prim].sets[i] = set_id;
+ return 0;
+ }
+ }
+ /* if prim failed, we need to access second cache line */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[sec].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[sec].sigs[i] = sig;
+ buckets[sec].sets[i] = set_id;
+ return 0;
+ }
+ }
+ return -1;
+}
+
+
+static inline int
+try_overwrite(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ SIG_TYPE sig, MEMBER_SET_TYPE set_id)
+{
+ if (insert_overwrite_search(prim, sig, buckets, set_id) ||
+ insert_overwrite_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ return -1;
+}
+
+static inline int
+evict_from_bucket(void)
+{
+ /* for now, we randomly pick one entry to evict */
+ return rte_rand() & (RTE_MEMBER_BUCKET_ENTRIES - 1);
+}
+
+
+
+/*
+ * This function is similar to the cuckoo hash make_space function in hash
+ * library
+ */
+static inline int
+make_space_bucket(const struct rte_member_setsum *ss, uint32_t bkt_num)
+{
+
+ static unsigned int nr_pushes;
+
+ unsigned int i, j;
+ int ret;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t next_bucket_idx;
+ struct member_ht_bucket *next_bkt[RTE_MEMBER_BUCKET_ENTRIES];
+ struct member_ht_bucket *bkt = &buckets[bkt_num];
+ MEMBER_SET_TYPE flag_mask = 1U << (sizeof(MEMBER_SET_TYPE) * 8 - 1);
+ /*
+ * Push existing item (search for bucket with space in
+ * alternative locations) to its alternative location
+ */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ /* Search for space in alternative locations */
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
+ next_bkt[i] = &buckets[next_bucket_idx];
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++) {
+ if (next_bkt[i]->sets[j] == RTE_MEMBER_NO_MATCH)
+ break;
+ }
+
+ if (j != RTE_MEMBER_BUCKET_ENTRIES)
+ break;
+ }
+
+ /* Alternative location has spare room (end of recursive function) */
+ if (i != RTE_MEMBER_BUCKET_ENTRIES) {
+ next_bkt[i]->sigs[j] = bkt->sigs[i];
+ next_bkt[i]->sets[j] = bkt->sets[i];
+ return i;
+ }
+
+ /* Pick entry that has not been pushed yet */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++)
+ if ((bkt->sets[i] & flag_mask) == 0)
+ break;
+
+ /* All entries have been pushed, so entry cannot be added */
+ if (i == RTE_MEMBER_BUCKET_ENTRIES ||
+ nr_pushes > RTE_MEMBER_MAX_PUSHES)
+ return -ENOSPC;
+
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
+ /* Set flag to indicate that this entry is going to be pushed */
+ bkt->sets[i] |= flag_mask;
+
+ nr_pushes++;
+ /* Need room in alternative bucket to insert the pushed entry */
+ ret = make_space_bucket(ss, next_bucket_idx);
+ /*
+ * After recursive function.
+ * Clear flags and insert the pushed entry
+ * in its alternative location if successful,
+ * or return error
+ */
+ bkt->sets[i] &= ~flag_mask;
+ nr_pushes = 0;
+ if (ret >= 0) {
+ next_bkt[i]->sigs[ret] = bkt->sigs[i];
+ next_bkt[i]->sets[ret] = bkt->sets[i];
+ return i;
+ } else
+ return ret;
+}
+
+
+int
+rte_member_add_ht(const struct rte_member_setsum *ss,
+ const void *key, MEMBER_SET_TYPE set_id)
+{
+ int ret;
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+ MEMBER_SET_TYPE flag_mask = 1U << (sizeof(MEMBER_SET_TYPE) * 8 - 1);
+
+ if (set_id == RTE_MEMBER_NO_MATCH || (set_id & flag_mask) != 0)
+ return -EINVAL;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ /* if it is cache based filter, we try overwriting existing entry */
+ if (ss->cache) {
+ ret = try_overwrite(buckets, prim_bucket, sec_bucket, tmp_sig,
+ set_id);
+ if (ret != -1)
+ return ret;
+ }
+ /* If not full then insert into one slot*/
+ ret = try_insert(buckets, prim_bucket, sec_bucket, tmp_sig, set_id);
+ if (ret != -1)
+ return ret;
+
+ /* random pick prim or sec for recursive displacement */
+
+ uint32_t select_bucket = (tmp_sig && 1U) ? prim_bucket : sec_bucket;
+ if (ss->cache) {
+ ret = evict_from_bucket();
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ return 1;
+ }
+
+ ret = make_space_bucket(ss, select_bucket);
+ if (ret >= 0) {
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ ret = 1;
+ }
+
+ return ret;
+}
+
+
+void
+rte_member_free_ht(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+
+int
+rte_member_delete_ht(struct rte_member_setsum *ss, const void *key,
+ MEMBER_SET_TYPE set_id)
+{
+ int i;
+ uint32_t prim_bucket, sec_bucket;
+ SIG_TYPE tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[prim_bucket].sigs[i] &&
+ set_id == buckets[prim_bucket].sets[i]) {
+ buckets[prim_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[sec_bucket].sigs[i] &&
+ set_id == buckets[sec_bucket].sets[i]) {
+ buckets[sec_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+ return -ENOENT;
+}
+
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *ss)
+{
+ uint32_t i, j;
+ struct member_ht_bucket *buckets = ss->table;
+ for (i = 0; i < ss->bucket_cnt; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+}
diff --git a/lib/librte_member/rte_member_ht.h b/lib/librte_member/rte_member_ht.h
new file mode 100644
index 0000000..aebf1db
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.h
@@ -0,0 +1,115 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_HT_H_
+#define _RTE_MEMBER_HT_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Maximum number of pushes for cuckoo path in HT mode. */
+#define RTE_MEMBER_MAX_PUSHES 100
+
+typedef uint16_t SIG_TYPE; /* signature size is 16 bit */
+#define SIG_BITMASK 0xFFFF /* signature mask, 16 bit */
+
+
+/* The bucket struct for ht setsum */
+struct member_ht_bucket {
+ SIG_TYPE sigs[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte signature */
+ MEMBER_SET_TYPE sets[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte set */
+} __rte_cache_aligned;
+
+
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE *set_id);
+
+
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids);
+
+
+
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id);
+
+
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids);
+
+
+
+
+int
+rte_member_add_ht(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE set_id);
+
+void
+rte_member_free_ht(struct rte_member_setsum *setsum);
+
+
+
+
+int
+rte_member_delete_ht(struct rte_member_setsum *ss, const void *key,
+ MEMBER_SET_TYPE set_id);
+
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *setsum);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+
+
+#endif /* _RTE_MEMBER_HT_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v3 3/7] member: implement vBF mode
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 1/7] member: implement main API Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 2/7] member: implement HT mode Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 4/7] member: add AVX for HT mode Yipeng Wang
` (4 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
Bloom Filter (BF) [1] is a well-known space-efficient
probabilistic data structure that answers set membership queries.
Vector of Bloom Filters (vBF) is an extension to traditional BF
that supports multi-set membership testing. Traditional BF will
return found or not-found for each key. vBF will also return
which set the key belongs to if it is found.
Since each set requires a BF, vBF should be used when set count
is small. vBF's false positive rate could be set arbitrarily so
that its memory requirement and lookup speed is better in certain
cases comparing to HT based set-summary.
This patch adds the vBF implementation.
[1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
Errors,” Communications of the ACM, 1970.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_vbf.c | 350 +++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_vbf.h | 85 +++++++++
3 files changed, 436 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index ad26548..50275ed 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c rte_member_vbf.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_vbf.c b/lib/librte_member/rte_member_vbf.c
new file mode 100644
index 0000000..49ca69e
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.c
@@ -0,0 +1,350 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <math.h>
+#include <string.h>
+
+#include <rte_malloc.h>
+#include <rte_memory.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_vbf.h"
+
+/*
+ * vBF currently implemented as a big array.
+ * The BFs have a vertical layout. Bits in same location of all bfs will stay
+ * in the same cache line.
+ * For example, if we have 32 bloom filters, we use a uint32_t array to
+ * represent all of them. array[0] represent the first location of all the
+ * bloom filters, array[1] represents the second location of all the
+ * bloom filters, etc. The advantage of this layout is to minimize the average
+ * number of memory accesses to test all bloom filters.
+ *
+ * Currently the implementation supports vBF containing 1,2,4,8,16,32 BFs.
+ */
+
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+
+ if (params->num_set > 32 || !rte_is_power_of_2(params->num_set) ||
+ params->false_positive_rate == 0 ||
+ params->false_positive_rate > 1) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "vBF create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ /* We assume expected keys evenly distribute to all BFs */
+ uint32_t num_keys_per_bf = (params->num_keys + ss->num_set - 1) /
+ ss->num_set;
+
+ if (num_keys_per_bf == 0) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "BF expected number of key is zero\n");
+ return -EINVAL;
+ }
+ /*
+ * Note that the false positive rate is for all BFs in the vBF
+ * such that the single BF's false positive rate need to be
+ * calculated.
+ * Assume each BF's False positive rate is x. The total false positive
+ * rate is fp = 1-(1-x)^n.
+ * => x = 1 - (1-fp)^(1/n)
+ */
+
+ float x = 1 - pow((1 - params->false_positive_rate), 1.0 / ss->num_set);
+
+ if (x == 0) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "BF false positive rate is zero\n");
+ return -EINVAL;
+ }
+
+ uint32_t bits = ceil((num_keys_per_bf *
+ log(x)) / log(1.0 / (pow(2.0, log(2.0)))));
+
+ /* We round to power of 2 for performance during lookup */
+ ss->bits = rte_align32pow2(bits);
+ if (ss->bits == 0) {
+ rte_errno = EINVAL;
+ RTE_LOG(ERR, MEMBER, "BF size is zero\n");
+ return -EINVAL;
+ }
+
+ ss->num_hashes = (uint32_t)(log(2.0) * bits / num_keys_per_bf);
+ ss->bit_mask = ss->bits - 1;
+
+
+ /*
+ * Since we round the bits to power of 2, the final false positive
+ * rate will probably not be same as the user specified. We log the
+ * new value as debug message.
+ */
+ float new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ ss->num_hashes)), ss->num_hashes);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ /*
+ * reduce hash function count, until we approach the user specified
+ * false-positive rate. otherwise it is too conservative
+ */
+ int tmp_num_hash = ss->num_hashes;
+
+ while (tmp_num_hash > 1) {
+ float tmp_fp = new_fp;
+
+ tmp_num_hash--;
+ new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ tmp_num_hash)), tmp_num_hash);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ if (new_fp > params->false_positive_rate) {
+ new_fp = tmp_fp;
+ tmp_num_hash++;
+ break;
+ }
+ }
+
+ ss->num_hashes = tmp_num_hash;
+
+ RTE_LOG(DEBUG, MEMBER, "vector bloom filter created, "
+ "each bloom filter expects %u keys, needs %u bits, %u hashes, "
+ "with false positive rate set as %.5f, "
+ "The new calculated vBF false positive rate is %.5f\n",
+ num_keys_per_bf, ss->bits, ss->num_hashes, x, new_fp);
+
+ ss->table = rte_zmalloc_socket(NULL, sizeof(uint32_t) * ss->num_set *
+ (ss->bits >> 5), RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ if (ss->table == NULL)
+ return -ENOMEM;
+
+ return 0;
+}
+
+
+/*
+ * a is how many bits could be represented by one uint32_t variable.
+ * b is used for division shift
+ * shift is used for multiplication shift
+ */
+static inline uint32_t
+test_bit(uint32_t h1, uint32_t h2, uint32_t iter, uint32_t shift, uint32_t a,
+ uint32_t b, const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t n = ss->num_set;
+ uint32_t bit_loc = (h1 + iter * h2) & ss->bit_mask;
+ /*
+ * x>>b is the divide, x & (a-1) is the mod, & (1<<n-1) to mask out bits
+ * we do not need
+ */
+ return (vbf[bit_loc>>b] >> ((bit_loc & (a - 1)) << shift)) &
+ ((1ULL << n) - 1);
+}
+
+
+static inline void
+set_bit(uint32_t h1, uint32_t h2, uint32_t iter, uint32_t shift, uint32_t a,
+ uint32_t b, const struct rte_member_setsum *ss, int32_t set)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t bit_loc = (h1 + iter * h2) & ss->bit_mask;
+ vbf[bit_loc>>b] |= 1U << (((bit_loc & (a - 1)) << shift) + set - 1);
+}
+
+
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *ss, const void *key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t j;
+ uint32_t h1 = MEMBER_PRIM_HASH(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ uint32_t mask = 0xFFFFFFFF;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (j = 0; j < ss->num_hashes; j++)
+ mask &= test_bit(h1, h2, j, shift, a, b, ss);
+
+ if (mask) {
+ *set_id = __builtin_ctz(mask) + 1;
+ return 1;
+ }
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ return 0;
+}
+
+
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, MEMBER_SET_TYPE *set_ids)
+{
+ uint32_t i, k, h1, h2;
+ uint32_t ret = 0;
+ uint32_t mask;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (i = 0; i < num_keys; i++) {
+ mask = 0xFFFFFFFF;
+ h1 = MEMBER_PRIM_HASH(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ for (k = 0; k < ss->num_hashes; k++)
+ mask &= test_bit(h1, h2, k, shift, a, b, ss);
+
+ if (mask) {
+ set_ids[i] = __builtin_ctz(mask) + 1;
+ ret++;
+ } else
+ set_ids[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return ret;
+}
+
+
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t ret = 0;
+ uint32_t j;
+ uint32_t h1 = MEMBER_PRIM_HASH(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ uint32_t mask = 0xFFFFFFFF;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (j = 0; j < ss->num_hashes; j++)
+ mask &= test_bit(h1, h2, j, shift, a, b, ss);
+
+ while (mask) {
+ uint32_t loc = __builtin_ctz(mask);
+ set_id[ret] = loc + 1;
+ ret++;
+ if (ret >= match_per_key)
+ return ret;
+ mask &= ~(1U << loc);
+ }
+ return ret;
+}
+
+
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids)
+{
+ uint32_t i, k, h1, h2;
+ uint32_t ret = 0;
+ uint32_t match_cnt_t;
+ uint32_t mask;
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_t = 0;
+ mask = 0xFFFFFFFF;
+ h1 = MEMBER_PRIM_HASH(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+
+ for (k = 0; k < ss->num_hashes; k++)
+ mask &= test_bit(h1, h2, k, shift, a, b, ss);
+
+ while (mask) {
+ uint32_t loc = __builtin_ctz(mask);
+ if (match_cnt_t >= match_per_key)
+ break;
+ set_ids[i * match_per_key + match_cnt_t] = loc + 1;
+ match_cnt_t++;
+ mask &= ~(1U << loc);
+ }
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ }
+ return ret;
+}
+
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *ss,
+ const void *key, MEMBER_SET_TYPE set_id)
+{
+ uint32_t i, h1, h2;
+
+ if (set_id > ss->num_set || set_id == RTE_MEMBER_NO_MATCH)
+ return -EINVAL;
+
+ h1 = MEMBER_PRIM_HASH(key, ss->key_len, ss->prim_hash_seed);
+ h2 = MEMBER_SEC_HASH(&h1, 4, ss->sec_hash_seed);
+ uint32_t shift = __builtin_ctz(ss->num_set);
+ uint32_t a = 32 >> shift;
+ uint32_t b = __builtin_ctz(a);
+
+ for (i = 0; i < ss->num_hashes; i++)
+ set_bit(h1, h2, i, shift, a, b, ss, set_id);
+ return 0;
+}
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ memset(vbf, 0, (ss->num_set * ss->bits) >> 3);
+}
diff --git a/lib/librte_member/rte_member_vbf.h b/lib/librte_member/rte_member_vbf.h
new file mode 100644
index 0000000..3d3ee33
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.h
@@ -0,0 +1,85 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_VBF_H_
+#define _RTE_MEMBER_VBF_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE *set_id);
+
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ MEMBER_SET_TYPE *set_ids);
+
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id);
+
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ MEMBER_SET_TYPE *set_ids);
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *setsum,
+ const void *key, MEMBER_SET_TYPE set_id);
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss);
+
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *setsum);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_VBF_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v3 4/7] member: add AVX for HT mode
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
` (2 preceding siblings ...)
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 3/7] member: implement vBF mode Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 5/7] member: enable the library Yipeng Wang
` (3 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
For key search, the signatures of all entries are compared against
the signature of the key that is being looked up. Since all
signatures are contguously put in a bucket, they can be compared
with vector instructions (AVX2), achieving higher lookup performance.
This patch adds AVX2 implementation in a separate header file.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/rte_member_ht.c | 143 ++++++++++++++++++++++++++++---------
lib/librte_member/rte_member_x86.h | 111 ++++++++++++++++++++++++++++
2 files changed, 222 insertions(+), 32 deletions(-)
create mode 100644 lib/librte_member/rte_member_x86.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
index b2ae6d0..15e2534 100644
--- a/lib/librte_member/rte_member_ht.c
+++ b/lib/librte_member/rte_member_ht.c
@@ -40,6 +40,10 @@
#include "rte_member.h"
#include "rte_member_ht.h"
+#if defined(RTE_ARCH_X86)
+#include "rte_member_x86.h"
+#endif
+
static inline int
insert_overwrite_search(uint32_t bucket, SIG_TYPE tmp_sig,
@@ -135,6 +139,13 @@ rte_member_create_ht(struct rte_member_setsum *ss,
for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
}
+#if defined(RTE_ARCH_X86)
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) &&
+ RTE_MEMBER_BUCKET_ENTRIES == 16)
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_AVX2;
+ else
+#endif
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_SCALAR;
RTE_LOG(DEBUG, MEMBER, "Hash table based filter created, "
@@ -174,11 +185,23 @@ rte_member_lookup_ht(const struct rte_member_setsum *ss,
*set_id = RTE_MEMBER_NO_MATCH;
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- if (search_bucket_single(prim_bucket, tmp_sig, buckets,
- set_id) ||
- search_bucket_single(sec_bucket, tmp_sig,
- buckets, set_id))
- return 1;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single_avx(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ }
return 0;
}
@@ -203,13 +226,27 @@ rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
}
for (i = 0; i < num_keys; i++) {
- if (search_bucket_single(prim_buckets[i], tmp_sig[i],
- buckets, &set_id[i]) ||
- search_bucket_single(sec_buckets[i],
- tmp_sig[i], buckets, &set_id[i]))
- ret++;
- else
- set_id[i] = RTE_MEMBER_NO_MATCH;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]) ||
+ search_bucket_single_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
}
return ret;
}
@@ -227,12 +264,24 @@ rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
- match_per_key, set_id);
- if (ret < match_per_key)
- search_bucket_multi(sec_bucket, tmp_sig,
- buckets, &ret, match_per_key, set_id);
- return ret;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_bucket, tmp_sig, buckets,
+ &ret, match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi_avx(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+#endif
+ default:
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
+ match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+ }
}
@@ -259,16 +308,34 @@ rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
for (i = 0; i < num_keys; i++) {
match_cnt_t = 0;
- search_bucket_multi(prim_buckets[i], tmp_sig[i],
- buckets, &match_cnt_t, match_per_key,
- &set_ids[i*match_per_key]);
- if (match_cnt_t < match_per_key)
- search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_buckets[i], tmp_sig[i],
buckets, &match_cnt_t, match_per_key,
&set_ids[i*match_per_key]);
- match_count[i] = match_cnt_t;
- if (match_cnt_t != 0)
- ret++;
+ if (match_cnt_t < match_per_key)
+ search_bucket_multi_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &match_cnt_t,
+ match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ break;
+#endif
+ default:
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_t < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_t, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ }
}
return ret;
}
@@ -300,12 +367,24 @@ try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
static inline int
try_overwrite(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
- SIG_TYPE sig, MEMBER_SET_TYPE set_id)
+ SIG_TYPE sig, MEMBER_SET_TYPE set_id,
+ enum rte_member_sig_compare_function cmp_fn)
{
- if (insert_overwrite_search(prim, sig, buckets, set_id) ||
- insert_overwrite_search(sec, sig, buckets,
- set_id))
- return 0;
+ switch (cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (insert_overwrite_search_avx(prim, sig, buckets, set_id) ||
+ insert_overwrite_search_avx(sec, sig, buckets,
+ set_id))
+ return 0;
+ break;
+#endif
+ default:
+ if (insert_overwrite_search(prim, sig, buckets, set_id) ||
+ insert_overwrite_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ }
return -1;
}
@@ -411,7 +490,7 @@ rte_member_add_ht(const struct rte_member_setsum *ss,
/* if it is cache based filter, we try overwriting existing entry */
if (ss->cache) {
ret = try_overwrite(buckets, prim_bucket, sec_bucket, tmp_sig,
- set_id);
+ set_id, ss->sig_cmp_fn);
if (ret != -1)
return ret;
}
diff --git a/lib/librte_member/rte_member_x86.h b/lib/librte_member/rte_member_x86.h
new file mode 100644
index 0000000..c55f128
--- /dev/null
+++ b/lib/librte_member/rte_member_x86.h
@@ -0,0 +1,111 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_X86_H_
+#define _RTE_MEMBER_X86_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <x86intrin.h>
+
+
+#if defined(RTE_MACHINE_CPUFLAG_AVX2)
+
+
+static inline int
+insert_overwrite_search_avx(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ if (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) / 2;
+ buckets[bucket].sets[hit_idx] = set_id;
+ return 1;
+ }
+ return 0;
+}
+
+
+static inline int
+search_bucket_single_avx(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) / 2;
+ if (buckets[bucket].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket].sets[hit_idx];
+ return 1;
+ }
+ hitmask &= ~(3U << (hit_idx) * 2);
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi_avx(uint32_t bucket, SIG_TYPE tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ MEMBER_SET_TYPE *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) / 2;
+ if (buckets[bucket].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket].sets[hit_idx];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ hitmask &= ~(3U << (hit_idx) * 2);
+ }
+}
+#endif
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_X86_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v3 5/7] member: enable the library
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
` (3 preceding siblings ...)
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 4/7] member: add AVX for HT mode Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-22 10:48 ` Thomas Monjalon
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 6/7] test/member: add functional and perf tests Yipeng Wang
` (2 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
This patch enables the Membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
MAINTAINERS | 7 +++++++
config/common_base | 5 +++++
lib/librte_member/Makefile | 2 ++
mk/rte.app.mk | 2 ++
4 files changed, 16 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index a0cd75e..e372edf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -734,6 +734,13 @@ F: test/test/test_meter.c
F: examples/qos_meter/
F: doc/guides/sample_app_ug/qos_metering.rst
+Membership - EXPERIMENTAL
+M: Yipeng Wang <yipeng1.wang@intel.com>
+M: Sameh Gobriel <sameh.gobriel@intel.com>
+F: lib/librte_member/
+F: doc/guides/prog_guide/member_lib.rst
+F: test/test/test_member*
+
Other libraries
---------------
diff --git a/config/common_base b/config/common_base
index 5e97a08..5e31ced 100644
--- a/config/common_base
+++ b/config/common_base
@@ -595,6 +595,11 @@ CONFIG_RTE_LIBRTE_HASH_DEBUG=n
CONFIG_RTE_LIBRTE_EFD=y
#
+# Compile librte_member
+#
+CONFIG_RTE_LIBRTE_MEMBER=y
+
+#
# Compile librte_jobstats
#
CONFIG_RTE_LIBRTE_JOBSTATS=y
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 50275ed..3bac1d0 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -37,6 +37,8 @@ LIB = librte_member.a
CFLAGS := -I$(SRCDIR) $(CFLAGS)
CFLAGS += $(WERROR_FLAGS) -O3
+LDLIBS += -lm
+
EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index c25fdd9..c79acf0 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -86,6 +86,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile
_LDLIBS-y += --whole-archive
_LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost
_LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs
_LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf
@@ -196,6 +197,7 @@ endif
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lm
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrt
_LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lm
ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lnuma
endif
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v3 5/7] member: enable the library
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 5/7] member: enable the library Yipeng Wang
@ 2017-09-22 10:48 ` Thomas Monjalon
0 siblings, 0 replies; 81+ messages in thread
From: Thomas Monjalon @ 2017-09-22 10:48 UTC (permalink / raw)
To: Yipeng Wang; +Cc: dev, charlie.tai, sameh.gobriel, ren.wang, john.mcnamara
06/09/2017 01:59, Yipeng Wang:
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -734,6 +734,13 @@ F: test/test/test_meter.c
> F: examples/qos_meter/
> F: doc/guides/sample_app_ug/qos_metering.rst
>
> +Membership - EXPERIMENTAL
> +M: Yipeng Wang <yipeng1.wang@intel.com>
> +M: Sameh Gobriel <sameh.gobriel@intel.com>
> +F: lib/librte_member/
> +F: doc/guides/prog_guide/member_lib.rst
> +F: test/test/test_member*
I know it is not obvious but this section is sorted in alphabetical order.
Please move between Hashes and LPM. Thanks
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v3 6/7] test/member: add functional and perf tests
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
` (4 preceding siblings ...)
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 5/7] member: enable the library Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
This patch adds functional and performance tests for membership
library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
test/test/Makefile | 3 +
test/test/test_member.c | 682 +++++++++++++++++++++++++++++++++++++++++++
test/test/test_member_perf.c | 643 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 1328 insertions(+)
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
diff --git a/test/test/Makefile b/test/test/Makefile
index 42d9a49..b61dde3 100644
--- a/test/test/Makefile
+++ b/test/test/Makefile
@@ -123,6 +123,9 @@ SRCS-y += test_logs.c
SRCS-y += test_memcpy.c
SRCS-y += test_memcpy_perf.c
+
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd_perf.c
diff --git a/test/test/test_member.c b/test/test/test_member.c
new file mode 100644
index 0000000..8965e81
--- /dev/null
+++ b/test/test/test_member.c
@@ -0,0 +1,682 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* This test is for membership library's simple feature test */
+
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_member.h>
+#include <rte_byteorder.h>
+#include <rte_random.h>
+#include <rte_debug.h>
+#include <rte_ip.h>
+
+#include "test.h"
+
+void *setsum_ht;
+void *setsum_cache;
+void *setsum_vbf;
+
+/* 5-tuple key type */
+struct flow_key {
+ uint32_t ip_src;
+ uint32_t ip_dst;
+ uint16_t port_src;
+ uint16_t port_dst;
+ uint8_t proto;
+} __attribute__((packed));
+
+
+/* Keys used by unit test functions */
+static struct flow_key keys[5] = {
+ {
+ .ip_src = IPv4(0x03, 0x02, 0x01, 0x00),
+ .ip_dst = IPv4(0x07, 0x06, 0x05, 0x04),
+ .port_src = 0x0908,
+ .port_dst = 0x0b0a,
+ .proto = 0x0c,
+ },
+ {
+ .ip_src = IPv4(0x13, 0x12, 0x11, 0x10),
+ .ip_dst = IPv4(0x17, 0x16, 0x15, 0x14),
+ .port_src = 0x1918,
+ .port_dst = 0x1b1a,
+ .proto = 0x1c,
+ },
+ {
+ .ip_src = IPv4(0x23, 0x22, 0x21, 0x20),
+ .ip_dst = IPv4(0x27, 0x26, 0x25, 0x24),
+ .port_src = 0x2928,
+ .port_dst = 0x2b2a,
+ .proto = 0x2c,
+ },
+ {
+ .ip_src = IPv4(0x33, 0x32, 0x31, 0x30),
+ .ip_dst = IPv4(0x37, 0x36, 0x35, 0x34),
+ .port_src = 0x3938,
+ .port_dst = 0x3b3a,
+ .proto = 0x3c,
+ },
+ {
+ .ip_src = IPv4(0x43, 0x42, 0x41, 0x40),
+ .ip_dst = IPv4(0x47, 0x46, 0x45, 0x44),
+ .port_src = 0x4948,
+ .port_dst = 0x4b4a,
+ .proto = 0x4c,
+ }
+};
+
+uint32_t test_set[5] = {1, 2, 3, 4, 5};
+
+#define ITERATIONS 3
+#define KEY_SIZE 4
+
+#define MAX_ENTRIES (1 << 16)
+uint8_t gened_keys[MAX_ENTRIES][KEY_SIZE];
+
+static struct rte_member_parameters params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+
+ /*num_set and false_positive_rate only relevant to vBF setsum*/
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+};
+
+
+/*
+ * Sequence of operations for find existing setsummary
+ *
+ * - create setsum
+ * - find existing setsum: hit
+ * - find non-existing setsum: miss
+ *
+ */
+static int
+test_member_find_existing(void)
+{
+ void *tmp_setsum = NULL, *result = NULL;
+ struct rte_member_parameters tmp_params = {
+ .name = "member_find_existing",
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ /* Create */
+ tmp_setsum = rte_member_create(&tmp_params);
+ TEST_ASSERT(tmp_setsum != NULL, "setsum creation failed");
+
+ /* Try to find existing hash table */
+ result = rte_member_find_existing("member_find_existing");
+ TEST_ASSERT(result == tmp_setsum, "could not find existing setsum");
+
+ /* Try to find non-existing hash table */
+ result = rte_member_find_existing("member_find_non_existing");
+ TEST_ASSERT(result == NULL, "found setsum that shouldn't exist");
+
+ /* Cleanup. */
+ rte_member_free(tmp_setsum);
+
+ return 0;
+}
+
+
+/*
+ * Test for bad creating parameters
+ */
+static int
+test_member_create_bad_param(void)
+{
+ void *bad_setsum = NULL;
+ struct rte_member_parameters bad_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ bad_params.name = "bad_param1";
+ bad_params.num_set = 0;
+ bad_params.type = RTE_MEMBER_TYPE_VBF;
+ /* test with 0 set for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "number of set for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param2";
+ bad_params.false_positive_rate = 0;
+ bad_params.num_set = 32;
+ /* test with 0 false positive for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "false positive rate for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param3";
+ bad_params.false_positive_rate = 0.03;
+ bad_params.num_keys = 31;
+ /* test with less than 1 key per BF for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "num_keys for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param4";
+ bad_params.type = RTE_MEMBER_TYPE_HT;
+ bad_params.num_keys = RTE_MEMBER_BUCKET_ENTRIES / 2;
+ /* test with less than 1 bucket for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with too few "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ bad_params.num_keys = RTE_MEMBER_ENTRIES_MAX + 1;
+ /* test with more than maximum entries for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with to many "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ /* test with same name should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with existed "
+ "name\n");
+ return -1;
+ }
+
+ rte_member_free(bad_setsum);
+ return 0;
+}
+
+
+/* Create test setsummaries. */
+static int test_member_create(void)
+{
+ params.key_len = sizeof(struct flow_key);
+
+ params.name = "test_member_ht";
+ params.iscache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.iscache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+ params.name = "test_member_vbf";
+ params.type = RTE_MEMBER_TYPE_VBF;
+ setsum_vbf = rte_member_create(¶ms);
+
+ if (setsum_ht == NULL || setsum_cache == NULL || setsum_vbf == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ printf("Creation of setsums success\n");
+ return 0;
+}
+
+static int test_member_insert(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_add(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[i], test_set[i]);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "insert error");
+ }
+ printf("insert key success\n");
+ return 0;
+}
+
+static int test_member_lookup(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ MEMBER_SET_TYPE set_ids_ht[5] = {0};
+ MEMBER_SET_TYPE set_ids_cache[5] = {0};
+ MEMBER_SET_TYPE set_ids_vbf[5] = {0};
+
+ uint32_t num_key_ht = 5;
+ uint32_t num_key_cache = 5;
+ uint32_t num_key_vbf = 5;
+
+ const void *key_array[5];
+
+ /* single lookup test */
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "single lookup function error");
+
+ TEST_ASSERT(set_ht == test_set[i] &&
+ set_cache == test_set[i] &&
+ set_vbf == test_set[i],
+ "single lookup set value error");
+ }
+ printf("lookup single key success\n");
+
+ /* bulk lookup test */
+ for (i = 0; i < 5; i++)
+ key_array[i] = &keys[i];
+
+ ret_ht = rte_member_lookup_bulk(setsum_ht, &key_array[0],
+ num_key_ht, set_ids_ht);
+
+ ret_cache = rte_member_lookup_bulk(setsum_cache, &key_array[0],
+ num_key_cache, set_ids_cache);
+
+ ret_vbf = rte_member_lookup_bulk(setsum_vbf, &key_array[0],
+ num_key_vbf, set_ids_vbf);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "bulk lookup function error");
+
+ for (i = 0; i < 5; i++) {
+ TEST_ASSERT((set_ids_ht[i] == test_set[i]) &&
+ (set_ids_cache[i] == test_set[i]) &&
+ (set_ids_vbf[i] == test_set[i]),
+ "bulk lookup result error");
+ }
+
+ return 0;
+}
+
+
+static int test_member_delete(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_delete(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_delete(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_delete(setsum_vbf, &keys[i], test_set[i]);
+ /* VBF does not support delete yet, so return error code */
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key deletion function error");
+ TEST_ASSERT(ret_vbf < 0,
+ "vbf does not support deletion, error");
+ }
+
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key lookup function error");
+ TEST_ASSERT(set_ht == RTE_MEMBER_NO_MATCH &&
+ ret_cache == RTE_MEMBER_NO_MATCH,
+ "key deletion failed");
+ }
+ printf("delete success\n");
+ return 0;
+}
+
+
+static int test_member_multimatch(void)
+{
+ int ret_ht, ret_vbf, ret_cache;
+ MEMBER_SET_TYPE set_ids_ht[32] = {0};
+ MEMBER_SET_TYPE set_ids_vbf[32] = {0};
+ MEMBER_SET_TYPE set_ids_cache[32] = {0};
+
+ MEMBER_SET_TYPE set_ids_ht_m[5][32] = {{0} };
+ MEMBER_SET_TYPE set_ids_vbf_m[5][32] = {{0} };
+ MEMBER_SET_TYPE set_ids_cache_m[5][32] = {{0} };
+
+ uint32_t match_count_ht[5];
+ uint32_t match_count_vbf[5];
+ uint32_t match_count_cache[5];
+
+ uint32_t num_key_ht = 5;
+ uint32_t num_key_vbf = 5;
+ uint32_t num_key_cache = 5;
+
+ const void *key_array[5];
+
+ uint32_t i, j;
+ /* same key at most inserted 2*entry_per_bucket times for HT mode */
+ for (i = 1; i < 33; i++) {
+ for (j = 0; j < 5; j++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[j], i);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[j], i);
+ ret_cache = rte_member_add(setsum_cache, &keys[j], i);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_vbf >= 0 &&
+ ret_cache >= 0,
+ "insert function error");
+ }
+ }
+
+ /* single multimatch test */
+ for (i = 0; i < 5; i++) {
+ ret_vbf = rte_member_lookup_multi(setsum_vbf, &keys[i], 32,
+ set_ids_vbf);
+ ret_ht = rte_member_lookup_multi(setsum_ht, &keys[i], 32,
+ set_ids_ht);
+ ret_cache = rte_member_lookup_multi(setsum_cache, &keys[i], 32,
+ set_ids_cache);
+ /*
+ * for cache mode, it does not support multimatch
+ * the mutimatch should work like single match
+ */
+ TEST_ASSERT(ret_ht == 32 && ret_vbf == 32 && ret_cache == 1,
+ "single lookup_multi error");
+ TEST_ASSERT(set_ids_cache[0] == 32,
+ "single lookup_multi cache error");
+
+ for (j = 1; j < 33; j++) {
+ TEST_ASSERT(set_ids_ht[j-1] == j &&
+ set_ids_vbf[j-1] == j,
+ "single multimatch lookup error");
+ }
+ }
+ printf("lookup single key for multimatch success\n");
+
+ /* bulk multimatch test */
+
+ for (i = 0; i < 5; i++)
+ key_array[i] = &keys[i];
+ ret_vbf = rte_member_lookup_multi_bulk(setsum_vbf,
+ &key_array[0], num_key_ht, 32, match_count_vbf,
+ (MEMBER_SET_TYPE *)set_ids_vbf_m);
+
+ ret_ht = rte_member_lookup_multi_bulk(setsum_ht,
+ &key_array[0], num_key_vbf, 32, match_count_ht,
+ (MEMBER_SET_TYPE *)set_ids_ht_m);
+
+ ret_cache = rte_member_lookup_multi_bulk(setsum_cache,
+ &key_array[0], num_key_cache, 32, match_count_cache,
+ (MEMBER_SET_TYPE *)set_ids_cache_m);
+
+
+ for (j = 0; j < 5; j++) {
+ TEST_ASSERT(match_count_ht[j] == 32,
+ "bulk multimatch lookup HT match count error");
+ TEST_ASSERT(match_count_vbf[j] == 32,
+ "bulk multimatch lookup vBF match count error");
+ TEST_ASSERT(match_count_cache[j] == 1,
+ "bulk multimatch lookup CACHE match count error");
+ TEST_ASSERT(set_ids_cache_m[j][0] == 32,
+ "bulk multimatch lookup CACHE set value error");
+
+ for (i = 1; i < 33; i++) {
+ TEST_ASSERT(set_ids_ht_m[j][i-1] == i,
+ "bulk multimatch lookup HT set value error");
+ TEST_ASSERT(set_ids_vbf_m[j][i-1] == i,
+ "bulk multimatch lookup vBF set value error");
+ }
+ }
+
+ printf("lookup for bulk multimatch success\n");
+
+ return 0;
+}
+
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, KEY_SIZE);
+}
+
+static void
+setup_keys_and_data(void)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ /* Reset all arrays */
+ for (i = 0; i < KEY_SIZE; i++)
+ gened_keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < MAX_ENTRIES; i++) {
+ for (j = 0; j < KEY_SIZE; j++)
+ gened_keys[i][j] = rte_rand() & 0xFF;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(gened_keys, MAX_ENTRIES, KEY_SIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < MAX_ENTRIES - 1; i++) {
+ if (memcmp(gened_keys[i], gened_keys[i + 1],
+ KEY_SIZE) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < KEY_SIZE; j++)
+ gened_keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+}
+
+
+static inline int
+add_gened_keys(void *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret >= 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &gened_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+
+static inline int
+add_gened_keys_cache(void *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret == 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &gened_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static int
+test_member_loadfactor(void)
+{
+ unsigned int j;
+ unsigned int added_keys, average_keys_added = 0;
+ int ret;
+
+ setup_keys_and_data();
+
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+
+ params.key_len = KEY_SIZE;
+ params.name = "test_member_ht";
+ params.iscache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.iscache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+
+ if (setsum_ht == NULL || setsum_cache == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ /* test HT mode */
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_gened_keys(setsum_ht, &added_keys);
+ if (ret != -ENOSPC) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_ht);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when no space(non-cache) = %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+
+ /* test cache mode */
+ added_keys = average_keys_added = 0;
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_gened_keys_cache(setsum_cache, &added_keys);
+ if (ret != 1) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_cache);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when eviction happens(cache)= %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+ return 0;
+}
+
+static void
+perform_free(void)
+{
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+}
+
+static int
+test_member(void)
+{
+ if (test_member_create_bad_param() < 0)
+ return -1;
+
+ if (test_member_find_existing() < 0)
+ return -1;
+
+ if (test_member_create() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_insert() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_lookup() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_delete() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_multimatch() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_loadfactor() < 0) {
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ return -1;
+ }
+
+ perform_free();
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_autotest, test_member);
diff --git a/test/test/test_member_perf.c b/test/test/test_member_perf.c
new file mode 100644
index 0000000..2bb3a31
--- /dev/null
+++ b/test/test/test_member_perf.c
@@ -0,0 +1,643 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <inttypes.h>
+
+#include <rte_lcore.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+#include <rte_memcpy.h>
+#include <rte_thash.h>
+#include <rte_member.h>
+
+#include "test.h"
+
+#define NUM_KEYSIZES 10
+#define NUM_SHUFFLES 10
+#define MAX_KEYSIZE 64
+#define MAX_ENTRIES (1 << 19)
+#define KEYS_TO_ADD (MAX_ENTRIES * 75 / 100) /* 75% table utilization */
+#define NUM_LOOKUPS (KEYS_TO_ADD * 5) /* Loop among keys added, several times */
+#define VBF_SET_CNT 32
+#define BURST_SIZE 64
+#define VBF_FALSE_RATE 0.03
+
+
+static unsigned int test_socket_id;
+
+enum sstype {
+ HT = 0,
+ CACHE,
+ VBF,
+ NUM_TYPE
+};
+
+enum operations {
+ ADD = 0,
+ LOOKUP,
+ LOOKUP_BULK,
+ LOOKUP_MULTI,
+ LOOKUP_MULTI_BULK,
+ DELETE,
+ LOOKUP_MISS,
+ NUM_OPERATIONS
+};
+
+
+struct member_perf_params {
+ void *setsum[NUM_TYPE];
+ uint32_t key_size;
+ unsigned int cycle;
+};
+
+
+static uint32_t hashtest_key_lens[] = {
+ /* standard key sizes */
+ 4, 8, 16, 32, 48, 64,
+ /* IPv4 SRC + DST + protocol, unpadded */
+ 9,
+ /* IPv4 5-tuple, unpadded */
+ 13,
+ /* IPv6 5-tuple, unpadded */
+ 37,
+ /* IPv6 5-tuple, padded to 8-byte boundary */
+ 40
+};
+
+/* Array to store number of cycles per operation */
+uint64_t cycles[NUM_TYPE][NUM_KEYSIZES][NUM_OPERATIONS];
+uint64_t false_data[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_bulk[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi_bulk[NUM_TYPE][NUM_KEYSIZES];
+
+uint64_t false_hit[NUM_TYPE][NUM_KEYSIZES];
+
+
+MEMBER_SET_TYPE data[NUM_TYPE][/* Array to store the data */KEYS_TO_ADD];
+
+/* Array to store all input keys */
+uint8_t keys[KEYS_TO_ADD][MAX_KEYSIZE];
+
+/* Shuffle the keys that have been added, so lookups will be totally random */
+static void
+shuffle_input_keys(struct member_perf_params *params)
+{
+ MEMBER_SET_TYPE temp_data;
+ unsigned int i, j;
+ uint32_t swap_idx;
+ uint8_t temp_key[MAX_KEYSIZE];
+
+ for (i = KEYS_TO_ADD - 1; i > 0; i--) {
+ swap_idx = rte_rand() % i;
+ memcpy(temp_key, keys[i], hashtest_key_lens[params->cycle]);
+ memcpy(keys[i], keys[swap_idx],
+ hashtest_key_lens[params->cycle]);
+ memcpy(keys[swap_idx], temp_key,
+ hashtest_key_lens[params->cycle]);
+ for (j = 0; j < NUM_TYPE; j++) {
+ temp_data = data[j][i];
+ data[j][i] = data[j][swap_idx];
+ data[j][swap_idx] = temp_data;
+ }
+ }
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, MAX_KEYSIZE);
+}
+
+struct rte_member_parameters member_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = 4, /* Length of hash key. */
+
+ /* num_set and false_positive_rate only relevant to vBF setsum*/
+ .num_set = VBF_SET_CNT,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 0,
+ .sec_hash_seed = 1,
+ .socket_id = 0, /* NUMA Socket ID for memory. */
+ };
+
+
+
+static int
+setup_keys_and_data(struct member_perf_params *params, unsigned int cycle,
+ int miss)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ params->key_size = hashtest_key_lens[cycle];
+ params->cycle = cycle;
+
+ /* Reset all arrays */
+ for (i = 0; i < params->key_size; i++)
+ keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+
+ data[HT][i] = data[CACHE][i] = (rte_rand() & 0x7FFE) + 1;
+ data[VBF][i] = rte_rand() % VBF_SET_CNT + 1;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(keys, KEYS_TO_ADD, MAX_KEYSIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < KEYS_TO_ADD - 1; i++) {
+ if (memcmp(keys[i], keys[i + 1],
+ params->key_size) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+
+ /* Shuffle the random values again */
+ shuffle_input_keys(params);
+
+ /* For testing miss lookup, we insert half and lookup the other half */
+ unsigned int entry_cnt, bf_key_cnt;
+ if (!miss) {
+ entry_cnt = MAX_ENTRIES;
+ bf_key_cnt = KEYS_TO_ADD;
+ } else {
+ entry_cnt = MAX_ENTRIES / 2;
+ bf_key_cnt = KEYS_TO_ADD / 2;
+ }
+ member_params.false_positive_rate = VBF_FALSE_RATE;
+ member_params.key_len = params->key_size;
+ member_params.socket_id = test_socket_id;
+ member_params.num_keys = entry_cnt;
+ member_params.name = "test_member_ht";
+ member_params.iscache = 0;
+ member_params.type = RTE_MEMBER_TYPE_HT;
+ params->setsum[HT] = rte_member_create(&member_params);
+ if (params->setsum[HT] == NULL)
+ fprintf(stderr, "ht create fail\n");
+
+ member_params.name = "test_member_cache";
+ member_params.iscache = 1;
+ params->setsum[CACHE] = rte_member_create(&member_params);
+ if (params->setsum[CACHE] == NULL)
+ fprintf(stderr, "CACHE create fail\n");
+
+ member_params.name = "test_member_vbf";
+ member_params.type = RTE_MEMBER_TYPE_VBF;
+ member_params.num_keys = bf_key_cnt;
+ params->setsum[VBF] = rte_member_create(&member_params);
+ if (params->setsum[VBF] == NULL)
+ fprintf(stderr, "VBF create fail\n");
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] == NULL)
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+timed_adds(struct member_perf_params *params, int type)
+{
+ const uint64_t start_tsc = rte_rdtsc();
+ unsigned int i, a;
+ int32_t ret;
+
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][ADD] = time_taken / KEYS_TO_ADD;
+ return 0;
+}
+
+static int
+timed_lookups(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+
+ false_data[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+ MEMBER_SET_TYPE result;
+ int ret;
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != data[type][j])
+ false_data[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ MEMBER_SET_TYPE result[BURST_SIZE] = {0};
+ const void *keys_burst[BURST_SIZE];
+ int ret;
+
+ false_data_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_bulk(params->setsum[type],
+ &keys_burst[0],
+ BURST_SIZE,
+ result);
+ if (ret <= 0) {
+ printf("lookup bulk has wrong return value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k] != data[type][data_idx])
+ false_data_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_BULK] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+
+static int
+timed_lookups_multimatch(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ MEMBER_SET_TYPE result[RTE_MEMBER_BUCKET_ENTRIES] = {0};
+ int ret;
+ false_data_multi[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup_multi(params->setsum[type],
+ &keys[j], RTE_MEMBER_BUCKET_ENTRIES, result);
+ if (type != CACHE && ret <= 0) {
+ printf("lookup multi has wrong return value %d,"
+ "type %d\n", ret, type);
+ }
+ if (result[0] != data[type][j])
+ false_data_multi[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+
+static int
+timed_lookups_multimatch_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ MEMBER_SET_TYPE result[BURST_SIZE][RTE_MEMBER_BUCKET_ENTRIES] = {{0} };
+ const void *keys_burst[BURST_SIZE];
+ uint32_t match_count[BURST_SIZE];
+ int ret;
+
+ false_data_multi_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_multi_bulk(
+ params->setsum[type],
+ &keys_burst[0], BURST_SIZE,
+ RTE_MEMBER_BUCKET_ENTRIES, match_count,
+ (MEMBER_SET_TYPE *)result);
+ if (ret < 0) {
+ printf("lookup multimatch bulk has wrong return"
+ " value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ if (type != CACHE && match_count[k] == 0) {
+ printf("lookup multimatch bulk get "
+ "wrong match count\n");
+ return -1;
+ }
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k][0] != data[type][data_idx])
+ false_data_multi_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI_BULK] = time_taken /
+ NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_deletes(struct member_perf_params *params, int type)
+{
+ unsigned int i;
+ int32_t ret;
+
+ if (type == VBF)
+ return 0;
+ const uint64_t start_tsc = rte_rdtsc();
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_delete(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (type != CACHE && ret < 0) {
+ printf("delete error\n");
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][DELETE] = time_taken / KEYS_TO_ADD;
+
+ return 0;
+}
+
+
+static int
+timed_miss_lookup(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ int ret;
+
+ false_hit[type][params->cycle] = 0;
+
+ for (i = 0; i < KEYS_TO_ADD / 2; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ unsigned int a;
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t start_tsc = rte_rdtsc();
+ MEMBER_SET_TYPE result;
+
+ for (i = 0; i < 2 * NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = KEYS_TO_ADD / 2; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != RTE_MEMBER_NO_MATCH)
+ false_hit[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MISS] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+
+
+static void
+perform_frees(struct member_perf_params *params)
+{
+ int i;
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] != NULL) {
+ rte_member_free(params->setsum[i]);
+ params->setsum[i] = NULL;
+ }
+ }
+}
+
+static int
+exit_with_fail(const char *testname, struct member_perf_params *params,
+ unsigned int i, unsigned int j)
+{
+ printf("<<<<<Test %s failed at keysize %d iteration %d type %d>>>>>\n",
+ testname, hashtest_key_lens[params->cycle], i, j);
+ perform_frees(params);
+ return -1;
+}
+
+static int
+run_all_tbl_perf_tests(void)
+{
+ unsigned int i, j, k;
+ struct member_perf_params params;
+
+ printf("Measuring performance, please wait\n");
+ fflush(stdout);
+
+ test_socket_id = rte_socket_id();
+
+
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 0) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+
+ if (timed_adds(¶ms, j) < 0)
+ return exit_with_fail("timed_adds", ¶ms,
+ i, j);
+
+ for (k = 0; k < NUM_SHUFFLES; k++)
+ shuffle_input_keys(¶ms);
+
+ if (timed_lookups(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups", ¶ms,
+ i, j);
+
+ if (timed_lookups_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_bulk",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi_bulk",
+ ¶ms, i, j);
+
+ if (timed_deletes(¶ms, j) < 0)
+ return exit_with_fail("timed_deletes", ¶ms,
+ i, j);
+
+ /* Print a dot to show progress on operations */
+ }
+ printf(".");
+ fflush(stdout);
+
+ perform_frees(¶ms);
+ }
+
+ /* test false postivie rate using un-inserted keys */
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 1) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+ if (timed_miss_lookup(¶ms, j) < 0)
+ return exit_with_fail("timed_miss_lookup",
+ ¶ms, i, j);
+ }
+ perform_frees(¶ms);
+ }
+
+ printf("\nResults (in CPU cycles/operation)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "Add", "Lookup", "Lookup_bulk",
+ "lookup_multi", "lookup_multi_bulk", "Delete",
+ "miss_lookup");
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ for (k = 0; k < NUM_OPERATIONS; k++)
+ printf("%-18"PRIu64, cycles[j][i][k]);
+ printf("\n");
+ }
+ }
+
+ printf("\nFalse results rate (and false positive rate)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "fr_single", "fr_bulk", "fr_multi",
+ "fr_multi_bulk", "false_positive_rate");
+ /* key size not influence False rate so just print out one key size */
+ for (i = 0; i < 1; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ printf("%-18f", (float)false_data[j][i] / NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_hit[j][i] /
+ NUM_LOOKUPS);
+ printf("\n");
+ }
+ }
+
+ return 0;
+}
+
+static int
+test_member_perf(void)
+{
+
+ if (run_all_tbl_perf_tests() < 0)
+ return -1;
+
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
` (5 preceding siblings ...)
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 6/7] test/member: add functional and perf tests Yipeng Wang
@ 2017-09-05 23:59 ` Yipeng Wang
2017-09-18 18:42 ` Mcnamara, John
2017-09-25 12:30 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
7 siblings, 2 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-05 23:59 UTC (permalink / raw)
To: dev; +Cc: charlie.tai, sameh.gobriel, ren.wang, yipeng1.wang, john.mcnamara
This patch adds the documentation for membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: John McNamara <john.mcnamara@intel.com>
---
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 +++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 ++++
doc/guides/prog_guide/img/member_i6.svg | 332 +++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 421 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
12 files changed, 3596 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 19e0d4f..fe87e09 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -105,7 +105,8 @@ The public API headers are grouped by topics:
[LPM IPv4 route] (@ref rte_lpm.h),
[LPM IPv6 route] (@ref rte_lpm6.h),
[ACL] (@ref rte_acl.h),
- [EFD] (@ref rte_efd.h)
+ [EFD] (@ref rte_efd.h),
+ [member] (@ref rte_member.h)
- **QoS**:
[metering] (@ref rte_meter.h),
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index 823554f..b792d6d 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -58,6 +58,7 @@ INPUT = doc/api/doxy-api-index.md \
lib/librte_mempool \
lib/librte_meter \
lib/librte_metrics \
+ lib/librte_member \
lib/librte_net \
lib/librte_pdump \
lib/librte_pipeline \
diff --git a/doc/guides/prog_guide/img/member_i1.svg b/doc/guides/prog_guide/img/member_i1.svg
new file mode 100644
index 0000000..fc5f56a
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i1.svg
@@ -0,0 +1,1613 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i1.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.18709in" height="4.75757in"
+ viewBox="0 0 517.471 342.545" xml:space="preserve" color-interpolation-filters="sRGB" class="st61">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false">
+ <v:userDefs>
+ <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/>
+ <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/>
+ </v:userDefs>
+ </v:documentProperties>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:3}
+ .st3 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em;opacity:0.219608}
+ .st4 {font-size:1em}
+ .st5 {fill:none;stroke:#41719c;stroke-width:3}
+ .st6 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em}
+ .st7 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em;opacity:0.219608}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st9 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st11 {fill:none;stroke:none;stroke-width:0.25}
+ .st12 {fill:#ffffff;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:0.75em;font-weight:bold}
+ .st14 {marker-end:url(#mrkr5-63);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st15 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st16 {fill:#5b9bd5;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st17 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st18 {fill:#feffff;font-family:Calibri;font-size:0.499992em}
+ .st19 {fill:#deebf6;stroke:#c8c8c8;stroke-width:0.25}
+ .st20 {fill:#000000;font-family:Calibri;font-size:0.499992em}
+ .st21 {marker-end:url(#mrkr5-178);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:#ff0000;font-family:Calibri;font-size:0.666664em}
+ .st24 {fill:#5b9bd5;fill-opacity:0.22}
+ .st25 {stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st26 {fill:#ffffff}
+ .st27 {stroke:#0070c0;stroke-width:0.25}
+ .st28 {fill:#5b9bd5;stroke:#0070c0;stroke-width:0.25}
+ .st29 {fill:#5b9bd5;stroke:#ffffff;stroke-width:0.25}
+ .st30 {fill:#5b9bd5}
+ .st31 {stroke:#c8c8c8;stroke-width:0.25}
+ .st32 {fill:#acccea;stroke:#c8c8c8;stroke-width:0.25}
+ .st33 {fill:#5b9bd5;fill-opacity:0.22;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st34 {fill:#000000;fill-opacity:0;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st35 {fill:url(#grad30-309);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st36 {fill:url(#grad25-313);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st37 {fill:url(#grad35-317);stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st38 {fill:url(#grad36-325);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st39 {fill:url(#grad40-335);stroke:#000000;stroke-linecap:butt;stroke-width:0.130208}
+ .st40 {fill:url(#grad39-342);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st41 {fill:url(#grad40-355);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st42 {fill:none}
+ .st43 {stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st44 {stroke:#ffffff;stroke-linecap:butt;stroke-width:0.130208}
+ .st45 {fill:url(#grad30-383);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st46 {fill:url(#grad36-396);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st47 {fill:none;stroke:#c8c8c8;stroke-width:0.75}
+ .st48 {fill:#9a9a9a;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st49 {fill:url(#grad40-415);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st50 {fill:url(#grad40-419);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st51 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25}
+ .st52 {fill:url(#grad35-430);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st53 {stroke:#c8c8c8;stroke-width:0.75}
+ .st54 {stroke:#4f88bb;stroke-width:0.75}
+ .st55 {fill:#feffff;font-family:Calibri;font-size:0.416656em}
+ .st56 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st57 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st58 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:2.25}
+ .st59 {fill:none;stroke:#0070c0;stroke-width:2.25}
+ .st60 {fill:#595959;font-family:Arial;font-size:0.666664em}
+ .st61 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Patterns_And_Gradients">
+ <linearGradient id="grad30-309" v:fillPattern="30" v:foreground="#97c2e6" v:background="#4274a2" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#4274a2;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-313" v:fillPattern="25" v:foreground="#5491d3" v:background="#246ba6" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#5491d3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </linearGradient>
+ <pattern id="grad35-317" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-318)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-319)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-320)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-321)"/>
+ </pattern>
+ <linearGradient id="grad27-318" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-319" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-320" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-321" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-325" v:fillPattern="36" v:foreground="#c0dff1" v:background="#246ba6" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#c0dff1;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-335" v:fillPattern="40" v:foreground="#c8e5c8" v:background="#19bf19" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#c8e5c8;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#19bf19;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad39-342" v:fillPattern="39" v:foreground="#5599d7" v:background="#b9daf2" cx="1" cy="1" r="1">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-355" v:fillPattern="40" v:foreground="#5599d7" v:background="#214383" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#214383;stop-opacity:1"/>
+ </radialGradient>
+ <linearGradient id="grad30-383" v:fillPattern="30" v:foreground="#97c2e6" v:background="#6ba4dc" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#6ba4dc;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-396" v:fillPattern="36" v:foreground="#89bee9" v:background="#b9daf2" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#89bee9;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-415" v:fillPattern="40" v:foreground="#000000" v:background="#ffffff" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#000000;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffffff;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-419" v:fillPattern="40" v:foreground="#ffffff" v:background="#9a9a9a" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#9a9a9a;stop-opacity:1"/>
+ </radialGradient>
+ <pattern id="grad35-430" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-431)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-432)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-433)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-434)"/>
+ </pattern>
+ <linearGradient id="grad27-431" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-432" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-433" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-434" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ </defs>
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-63" class="st15" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-178" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/>
+ </v:userDefs>
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <v:layer v:name="Flowchart" v:index="0"/>
+ <g id="group165-1" transform="translate(21.7794,-24.0978)" v:mID="165" v:groupContext="group">
+ <title>Sheet.165</title>
+ <g id="group1-2" transform="translate(308.647,-25.7109)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-3" v:mID="2" v:groupContext="shape" transform="translate(11.5732,-58.1913)">
+ <title>Circle</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow2-4" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="shape3-13" v:mID="3" v:groupContext="shape" transform="translate(58.9839,-58.9839)">
+ <title>Circle.23</title>
+ <desc>List 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow3-14" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="17.73" y="318.02" class="st7" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="17.73" y="318.02" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <g id="shape4-19" v:mID="4" v:groupContext="shape">
+ <title>Circle.24</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow4-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="group5-29" transform="translate(50.7413,-4.53722)" v:mID="5" v:groupContext="group">
+ <title>Sheet.5</title>
+ <g id="shape6-30" v:mID="6" v:groupContext="shape" transform="translate(344.2,300.5) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-31" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st9"/>
+ </g>
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape7-34" v:mID="7" v:groupContext="shape" transform="translate(-0.884982,-14.7157)">
+ <title>Sheet.7</title>
+ <desc>setsum</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="12.9268" cy="336.238" width="25.86" height="12.6135"/>
+ <rect x="0" y="329.932" width="25.8535" height="12.6135" class="st11"/>
+ <text x="6.37" y="334.44" class="st12" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsu<tspan
+ x="10.49" dy="1.2em" class="st4">m</tspan></text> </g>
+ </g>
+ <g id="shape8-38" v:mID="8" v:groupContext="shape" transform="translate(72.5955,0)">
+ <title>Circle.29</title>
+ <desc>List 2 matching Criteria 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow8-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ </g>
+ <g id="group9-48" transform="translate(31.6515,-49.9094)" v:mID="9" v:groupContext="group">
+ <title>Sheet.9</title>
+ <g id="group10-49" transform="translate(99.5691,0)" v:mID="10" v:groupContext="group">
+ <title>Sheet.10</title>
+ <g id="shape11-50" v:mID="11" v:groupContext="shape" transform="translate(346.175,275.999) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st9"/>
+ </g>
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape12-54" v:mID="12" v:groupContext="shape" transform="translate(355.063,285.074) rotate(90)">
+ <title>Sheet.12</title>
+ <desc>Set Summary</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1985" cy="332.563" width="48.4" height="19.9638"/>
+ <rect x="0" y="322.581" width="48.397" height="19.9638" class="st11"/>
+ <text x="18.25" y="329.86" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Set <tspan
+ x="6.38" dy="1.2em" class="st4">Summary</tspan></text> </g>
+ </g>
+ <g id="shape13-58" v:mID="13" v:groupContext="shape" transform="translate(57.5835,-54.4467)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.9 342.55" class="st14"/>
+ </g>
+ <g id="shape14-64" v:mID="14" v:groupContext="shape" transform="translate(20.2363,-51.8439)">
+ <title>Sheet.14</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="25.3328" cy="333.471" width="50.67" height="18.1489"/>
+ <rect x="0" y="324.396" width="50.6656" height="18.1489" class="st11"/>
+ <text x="14.12" y="335.27" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape15-67" v:mID="15" v:groupContext="shape" transform="translate(5.02911,1.60865) rotate(-26.0815)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L39.25 342.55" class="st14"/>
+ </g>
+ <g id="shape16-72" v:mID="16" v:groupContext="shape" transform="translate(155.629,-33.273)">
+ <title>Sheet.16</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.34 342.55" class="st14"/>
+ </g>
+ <g id="shape17-77" v:mID="17" v:groupContext="shape" transform="translate(304.141,0.595416) rotate(25.6934)">
+ <title>Sheet.17</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L42.68 342.55" class="st14"/>
+ </g>
+ <g id="shape18-82" v:mID="18" v:groupContext="shape" transform="translate(102.642,654.842) rotate(180)">
+ <title>Sheet.18</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape19-87" v:mID="19" v:groupContext="shape" transform="translate(-15.1809,-33.9928)">
+ <title>Sheet.19</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.75" cy="338.045" width="85.5" height="9"/>
+ <rect x="0" y="333.545" width="85.5" height="9" class="st11"/>
+ <text x="5.06" y="339.85" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape20-90" v:mID="20" v:groupContext="shape" transform="translate(102.844,679.041) rotate(180)">
+ <title>Sheet.20</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape21-95" v:mID="21" v:groupContext="shape" transform="translate(-35.4309,-11.4928)">
+ <title>Sheet.21</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="337.971" width="108" height="9.14889"/>
+ <rect x="0" y="333.396" width="108" height="9.14889" class="st11"/>
+ <text x="6.36" y="339.77" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape22-98" v:mID="22" v:groupContext="shape" transform="translate(541.496,275.999) rotate(90)">
+ <title>Sheet.22</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape23-101" v:mID="23" v:groupContext="shape" transform="translate(541.496,300.198) rotate(90)">
+ <title>Sheet.23</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape24-104" v:mID="24" v:groupContext="shape" transform="translate(541.496,324.396) rotate(90)">
+ <title>Sheet.24</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ </g>
+ <g id="group25-107" transform="translate(285.961,-178.628)" v:mID="25" v:groupContext="group">
+ <title>Sheet.25</title>
+ <g id="shape26-108" v:mID="26" v:groupContext="shape" transform="translate(51.2583,-51.2583)">
+ <title>Circle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-109" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape27-112" v:mID="27" v:groupContext="shape" transform="translate(107.177,-55.9182)">
+ <title>Circle.156</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-113" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape28-116" v:mID="28" v:groupContext="shape" transform="translate(79.2174,-83.8773)">
+ <title>Circle.157</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow28-117" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape29-120" v:mID="29" v:groupContext="shape" transform="translate(153.775,-51.2583)">
+ <title>Circle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow29-121" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape30-124" v:mID="30" v:groupContext="shape" transform="translate(93.197,-18.6394)">
+ <title>Circle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow30-125" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape31-128" v:mID="31" v:groupContext="shape" transform="translate(27.4102,-57.9329) rotate(-7.12502)">
+ <title>Sheet.31</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L31.41 342.55" class="st14"/>
+ </g>
+ <g id="shape32-133" v:mID="32" v:groupContext="shape" transform="translate(182.13,-60.5772) rotate(9.46232)">
+ <title>Sheet.32</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L22.18 342.55" class="st14"/>
+ </g>
+ <g id="shape33-138" v:mID="33" v:groupContext="shape" transform="translate(47.8843,595.237) rotate(-160.346)">
+ <title>Sheet.33</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L63.11 342.55" class="st14"/>
+ </g>
+ <g id="shape34-143" v:mID="34" v:groupContext="shape" transform="translate(292.945,525.785) rotate(141.977)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L20.97 342.55" class="st14"/>
+ </g>
+ <g id="shape35-148" v:mID="35" v:groupContext="shape" transform="translate(-95.8971,591.793) rotate(-145.945)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L28.55 342.55" class="st14"/>
+ </g>
+ <g id="shape36-153" v:mID="36" v:groupContext="shape" transform="translate(37.2788,2.27374E-013)">
+ <title>Rectangle.167</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.8652" cy="335.555" width="21.74" height="13.9795"/>
+ <g id="shadow36-154" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st10"/>
+ <text x="5" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape37-158" v:mID="37" v:groupContext="shape" transform="translate(55.9182,2.27374E-013)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow37-159" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape38-163" v:mID="38" v:groupContext="shape" transform="translate(-1.65867E-013,-32.6189)">
+ <title>Rectangle.169</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.3796" cy="335.555" width="20.76" height="13.9795"/>
+ <g id="shadow38-164" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st10"/>
+ <text x="4.51" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape39-168" v:mID="39" v:groupContext="shape" transform="translate(18.6394,-32.6189)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow39-169" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape40-173" v:mID="40" v:groupContext="shape" transform="translate(197.019,626.053) rotate(161.565)">
+ <title>Sheet.40</title>
+ <path d="M0 328.31 A55.7483 27.2427 -124.2 0 0 42.37 334.19 L42.47 333.85" class="st21"/>
+ </g>
+ <g id="shape41-179" v:mID="41" v:groupContext="shape" transform="translate(154.607,584.177) rotate(161.121)">
+ <title>Sheet.41</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 319.39 A80.5593 29.9756 -101.99 0 0 41.7 325.37 L41.79 325.02" class="st21"/>
+ </g>
+ <g id="shape42-184" v:mID="42" v:groupContext="shape" transform="translate(3.02481,-66.7025)">
+ <title>Sheet.42</title>
+ <desc>Encode ID</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="19.4569" cy="335.555" width="38.92" height="13.9795"/>
+ <rect x="0" y="328.566" width="38.9138" height="13.9795" class="st11"/>
+ <text x="7.51" y="333.16" class="st23" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Encode <tspan
+ x="15.99" dy="1.2em" class="st4">ID</tspan></text> </g>
+ </g>
+ <g id="group43-188" transform="translate(12.0993,-165.858)" v:mID="43" v:groupContext="group">
+ <title>Sheet.43</title>
+ <g id="group44-189" transform="translate(7.21495,-75.757)" v:mID="44" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User</title>
+ <g id="shape45-190" v:mID="45" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.45</title>
+ <g id="shadow45-191" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape46-206" v:mID="46" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.46</title>
+ <g id="shadow46-207" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape47-210" v:mID="47" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.47</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape48-212" v:mID="48" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.48</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group49-215" transform="translate(7.21495,-47.1858)" v:mID="49" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.7</title>
+ <g id="shape50-216" v:mID="50" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.50</title>
+ <g id="shadow50-217" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape51-232" v:mID="51" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.51</title>
+ <g id="shadow51-233" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape52-236" v:mID="52" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.52</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape53-238" v:mID="53" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group54-241" transform="translate(7.21495,-18.6146)" v:mID="54" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.12</title>
+ <g id="shape55-242" v:mID="55" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.55</title>
+ <g id="shadow55-243" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape56-258" v:mID="56" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.56</title>
+ <g id="shadow56-259" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape57-262" v:mID="57" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape58-264" v:mID="58" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group59-267" transform="translate(171.161,-45.6707)" v:mID="59" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>Data Center</title>
+ <g id="shape60-268" v:mID="60" v:groupContext="shape" v:layerMember="0">
+ <title>Sheet.60</title>
+ <g id="shadow60-269" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st9"/>
+ </g>
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st10"/>
+ </g>
+ <g id="shape61-272" v:mID="61" v:groupContext="shape" v:layerMember="0" transform="translate(6.86487,-7.30475)">
+ <title>Sheet.61</title>
+ <g id="shadow61-273" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39
+ 332.44 L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69
+ L29.57 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64
+ 342.55 L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75
+ 342.55 L18.75 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69
+ L10.82 311.79 L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st9"/>
+ </g>
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39 332.44
+ L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69 L29.57
+ 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64 342.55
+ L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75 342.55 L18.75
+ 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69 L10.82 311.79
+ L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st10"/>
+ </g>
+ <g id="shape62-276" v:mID="62" v:groupContext="shape" v:layerMember="0" transform="translate(20.0835,-20.5174)">
+ <title>Sheet.62</title>
+ <g id="shadow62-277" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36
+ ZM23.46 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36
+ ZM2.27 341.36 A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z"
+ class="st24"/>
+ </g>
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36 ZM23.46
+ 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36 ZM2.27 341.36
+ A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z" class="st30"/>
+ </g>
+ <g id="shape63-282" v:mID="63" v:groupContext="shape" v:layerMember="0" transform="translate(14.2717,-12.5134)">
+ <title>Sheet.63</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <g id="shadow63-283" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17
+ 340.12 L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89
+ L43.09 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2
+ 342.55 ZM21.2 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69
+ L29.27 337.69 L29.27 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74
+ L-0 341.74 L-0 342.55 ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69
+ L8.08 337.69 L8.08 336.89 L-0 336.89 L-0 337.69 Z" class="st24"/>
+ </g>
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17 340.12
+ L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89 L43.09
+ 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2 342.55 ZM21.2
+ 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69 L29.27 337.69 L29.27
+ 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74 L-0 341.74 L-0 342.55
+ ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69 L8.08 337.69 L8.08 336.89
+ L-0 336.89 L-0 337.69 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group64-288" transform="translate(59.5234,-47.1858)" v:mID="64" v:groupContext="group">
+ <v:custProps>
+ <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false"
+ v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/>
+ <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true"
+ v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Device)"/>
+ <v:cp v:nameU="SubShapeType" v:lbl="SubShapeType" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Load balancer)"/>
+ </v:custProps>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/>
+ <v:ud v:nameU="SolSH" v:prompt="" v:val="VT14({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/>
+ <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <title>Load balancer</title>
+ <g id="shape65-289" v:mID="65" v:groupContext="shape" transform="translate(0,-1.653)">
+ <title>Sheet.65</title>
+ <g id="shadow65-290" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st24"/>
+ <path d="M0 332.02 L16.23 332.02" class="st25"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st25"/>
+ </g>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st30"/>
+ <path d="M0 332.02 L16.23 332.02" class="st31"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st31"/>
+ </g>
+ <g id="shape66-297" v:mID="66" v:groupContext="shape" transform="translate(1.81062,-2.91583)">
+ <title>Sheet.66</title>
+ <g id="shadow66-298" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44
+ L8.34 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45
+ 338.78 L10.78 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22
+ ZM10.48 335.2 L8.29 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46
+ 334.9 L10.21 334.58 L9.55 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19
+ 338.43 C4.19 339.56 5.11 340.48 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29
+ 7.38 336.37 6.25 336.37 ZM6.25 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02
+ 339.83 6.25 339.83 C5.47 339.83 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02
+ ZM2.62 338.14 L0 338.14 L0 338.71 L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73
+ 337.47 L1.95 337.47 L2.62 338.14 Z" class="st9"/>
+ </g>
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44 L8.34
+ 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45 338.78 L10.78
+ 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22 ZM10.48 335.2 L8.29
+ 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46 334.9 L10.21 334.58 L9.55
+ 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19 338.43 C4.19 339.56 5.11 340.48
+ 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29 7.38 336.37 6.25 336.37 ZM6.25
+ 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02 339.83 6.25 339.83 C5.47 339.83
+ 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02 ZM2.62 338.14 L0 338.14 L0 338.71
+ L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73 337.47 L1.95 337.47 L2.62 338.14 Z"
+ class="st32"/>
+ </g>
+ </g>
+ <g id="group67-301" transform="translate(104.617,-86.5795)" v:mID="67" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server</title>
+ <g id="shape68-302" v:mID="68" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.68</title>
+ <g id="shadow68-303" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape69-306" v:mID="69" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.69</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape70-310" v:mID="70" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.70</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape71-314" v:mID="71" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.71</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape72-322" v:mID="72" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.72</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape73-326" v:mID="73" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.73</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape74-329" v:mID="74" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.74</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape75-332" v:mID="75" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.75</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape76-336" v:mID="76" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.76</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape77-339" v:mID="77" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.77</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape78-343" v:mID="78" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.78</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape79-346" v:mID="79" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.79</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape80-349" v:mID="80" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.80</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape81-352" v:mID="81" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.81</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape82-356" v:mID="82" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.82</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape83-359" v:mID="83" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.83</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape84-362" v:mID="84" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.84</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape85-365" v:mID="85" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.85</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape86-368" v:mID="86" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.86</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape87-371" v:mID="87" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.87</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape88-374" v:mID="88" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.88</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape89-377" v:mID="89" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.89</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape90-380" v:mID="90" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.90</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape91-384" v:mID="91" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.91</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape92-387" v:mID="92" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.92</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape93-390" v:mID="93" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.93</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape94-393" v:mID="94" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.94</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape95-397" v:mID="95" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.95</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape96-400" v:mID="96" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.96</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape97-402" v:mID="97" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.97</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape98-405" v:mID="98" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.98</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape99-408" v:mID="99" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.99</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape100-410" v:mID="100" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.100</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape101-412" v:mID="101" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.101</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape102-416" v:mID="102" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.102</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape103-420" v:mID="103" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.103</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape104-427" v:mID="104" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.104</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape105-435" v:mID="105" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.105</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape106-440" v:mID="106" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.106</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="group107-443" transform="translate(104.617,-33.8201)" v:mID="107" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server.104</title>
+ <g id="shape108-444" v:mID="108" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.108</title>
+ <g id="shadow108-445" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape109-448" v:mID="109" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.109</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape110-451" v:mID="110" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.110</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape111-454" v:mID="111" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.111</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape112-457" v:mID="112" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.112</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape113-460" v:mID="113" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.113</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape114-463" v:mID="114" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.114</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape115-466" v:mID="115" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.115</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape116-469" v:mID="116" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.116</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape117-472" v:mID="117" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.117</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape118-475" v:mID="118" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.118</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape119-478" v:mID="119" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.119</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape120-481" v:mID="120" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.120</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape121-484" v:mID="121" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.121</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape122-487" v:mID="122" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.122</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape123-490" v:mID="123" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.123</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape124-493" v:mID="124" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.124</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape125-496" v:mID="125" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.125</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape126-499" v:mID="126" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.126</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape127-502" v:mID="127" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.127</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape128-505" v:mID="128" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.128</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape129-508" v:mID="129" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.129</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape130-511" v:mID="130" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.130</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape131-514" v:mID="131" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.131</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape132-517" v:mID="132" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.132</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape133-520" v:mID="133" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.133</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape134-523" v:mID="134" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.134</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape135-526" v:mID="135" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.135</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape136-529" v:mID="136" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.136</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape137-531" v:mID="137" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.137</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape138-534" v:mID="138" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.138</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape139-537" v:mID="139" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.139</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape140-539" v:mID="140" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.140</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape141-541" v:mID="141" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.141</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape142-544" v:mID="142" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.142</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape143-547" v:mID="143" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.143</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape144-554" v:mID="144" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.144</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape145-557" v:mID="145" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.145</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape146-562" v:mID="146" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.146</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="shape147-565" v:mID="147" v:groupContext="shape" transform="translate(427.321,214.49) rotate(90)">
+ <title>Cloud</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54 Z" class="st42"/>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54" class="st54"/>
+ <path d="M11.05 312.14 A8.59237 10.0375 0 0 1 5.37 311.54" class="st54"/>
+ <path d="M40.54 332.09 A8.62978 10.0812 -180 0 0 42.1 335.11" class="st54"/>
+ <path d="M73.92 326.65 A6.96633 8.13801 0 0 1 73.14 330.27" class="st54"/>
+ <path d="M89.7 308.52 A7.30994 8.5394 0 0 1 95.9 318.19" class="st54"/>
+ <path d="M103.15 297.64 A6.67364 7.79609 -180 0 0 106.25 294.02" class="st54"/>
+ <path d="M37.96 278.3 A10.2914 12.0219 -180 0 0 34.86 275.89" class="st54"/>
+ </g>
+ <g id="shape148-574" v:mID="148" v:groupContext="shape" transform="translate(110.222,-64.9346)">
+ <title>Triangle</title>
+ <desc>setsum</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="15.6995" cy="336.449" width="31.4" height="12.1933"/>
+ <g id="shadow148-575" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st9"/>
+ </g>
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st10"/>
+ <text x="8.35" y="337.95" class="st55" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsum</text> </g>
+ <g id="shape149-579" v:mID="149" v:groupContext="shape" transform="translate(292.639,20.8827) rotate(45)">
+ <title>Sheet.149</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L14.71 342.55" class="st14"/>
+ </g>
+ <g id="shape150-584" v:mID="150" v:groupContext="shape" transform="translate(43.2897,-54.1122)">
+ <title>Sheet.150</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L10.07 342.55" class="st14"/>
+ </g>
+ <g id="shape151-589" v:mID="151" v:groupContext="shape" transform="translate(-112.261,8.34531) rotate(-28.1394)">
+ <title>Sheet.151</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L18 342.55" class="st14"/>
+ </g>
+ <g id="shape152-594" v:mID="152" v:groupContext="shape">
+ <title>Sheet.152</title>
+ <desc>Clients</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="21.5" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Clients</text> </g>
+ <g id="shape153-597" v:mID="153" v:groupContext="shape" transform="translate(83.578,-9.58078)">
+ <title>Sheet.153</title>
+ <desc>Distributed Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.0677" cy="337.134" width="84.14" height="10.8224"/>
+ <rect x="0" y="331.723" width="84.1355" height="10.8224" class="st11"/>
+ <text x="13.1" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Distributed Cache</text> </g>
+ <g id="shape154-600" v:mID="154" v:groupContext="shape" transform="translate(181.983,-18.6146)">
+ <title>Sheet.154</title>
+ <desc>Web Servers</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="11.93" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Web Servers</text> </g>
+ <g id="shape155-603" v:mID="155" v:groupContext="shape" transform="translate(96.6068,630.978) rotate(180)">
+ <title>Simple Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow155-604" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ <g id="shape156-607" v:mID="156" v:groupContext="shape" transform="translate(173.159,625.567) rotate(180)">
+ <title>Simple Arrow.153</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow156-608" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ </g>
+ <g id="shape157-611" v:mID="157" v:groupContext="shape" transform="translate(0,-149.475)">
+ <title>Rectangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow157-612" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape158-615" v:mID="158" v:groupContext="shape" transform="translate(271.116,-149.475)">
+ <title>Rectangle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow158-616" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape159-619" v:mID="159" v:groupContext="shape">
+ <title>Rectangle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow159-620" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape160-623" v:mID="160" v:groupContext="shape" transform="translate(271.116,0)">
+ <title>Rectangle.160</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow160-624" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape161-627" v:mID="161" v:groupContext="shape" transform="translate(83.578,-151.241)">
+ <title>Sheet.161</title>
+ <desc>(a) Distributed Web Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow161-628" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(a) Distributed Web Cache</text> </g>
+ <g id="shape162-632" v:mID="162" v:groupContext="shape" transform="translate(319.513,-151.241)">
+ <title>Sheet.162</title>
+ <desc>(b) Detecting Routing Loops</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow162-633" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(b) Detecting Routing Loops</text> </g>
+ <g id="shape163-637" v:mID="163" v:groupContext="shape" transform="translate(77.5283,-3.35965)">
+ <title>Sheet.163</title>
+ <desc>(c) In-order Workload Scheduler</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="63.5211" cy="333.806" width="127.05" height="17.4792"/>
+ <g id="shadow163-638" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(c) In-order Workload Scheduler</text> </g>
+ <g id="shape164-642" v:mID="164" v:groupContext="shape" transform="translate(307.414,-3.35965)">
+ <title>Sheet.164</title>
+ <desc>(d) Database Semi-join Operations</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="66.2253" cy="333.806" width="132.46" height="17.4792"/>
+ <g id="shadow164-643" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(d) Database Semi-join Operations</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i2.svg b/doc/guides/prog_guide/img/member_i2.svg
new file mode 100644
index 0000000..759c654
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i2.svg
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i2.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="4.38194in" height="1.25694in"
+ viewBox="0 0 315.5 90.5" xml:space="preserve" color-interpolation-filters="sRGB" class="st6">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:none;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st3 {baseline-shift:32.4943%;font-size:0.649886em}
+ .st4 {font-size:1em}
+ .st5 {font-family:Cambria Math;font-size:1em}
+ .st6 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(0.25,-0.25)">
+ <title>Sheet.3</title>
+ <desc>False Positive Probability = (1-(1-1/m)kn)k ≃ (1-ekn/m)k</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="157.5" cy="45.5" width="315" height="90"/>
+ <rect x="0" y="0.5" width="315" height="90" class="st1"/>
+ <text x="8.28" y="49.82" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>False Positive Probability = (1-(1-1/m)<tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan dy="0.152em" class="st4">)</tspan><tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan><tspan dy="0.152em" class="st4"> </tspan><tspan
+ class="st5">≃</tspan> (1-e<tspan dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan class="st3"
+ v:baseFontSize="14">/</tspan><tspan class="st3" v:baseFontSize="14">m</tspan><tspan dy="0.152em"
+ class="st4">)</tspan><tspan dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i3.svg b/doc/guides/prog_guide/img/member_i3.svg
new file mode 100644
index 0000000..41e92cb
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i3.svg
@@ -0,0 +1,148 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i3.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ width="4.71875in" height="2.84375in" viewBox="0 0 339.75 204.75" xml:space="preserve" color-interpolation-filters="sRGB"
+ class="st14">
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22}
+ .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25}
+ .st4 {marker-end:url(#mrkr5-32);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st5 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st6 {fill:#feffff;font-family:Calibri;font-size:0.833336em}
+ .st7 {font-size:1em}
+ .st8 {fill:#deebf6;stroke:#c7c8c8;stroke-width:0.25}
+ .st9 {fill:#000000;font-family:Calibri;font-size:0.833336em}
+ .st10 {marker-end:url(#mrkr5-84);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st12 {fill:none;stroke:none;stroke-width:0.25}
+ .st13 {fill:#ff0000;font-family:Calibri;font-size:1.00001em}
+ .st14 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-32" class="st5" refX="-6.16" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-84" class="st11" refX="-5.8" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <defs id="Filters">
+ <filter id="filter_2">
+ <feGaussianBlur stdDeviation="2"/>
+ </filter>
+ </defs>
+ <g>
+ <title>Page-1</title>
+ <g id="group174-1" transform="translate(3.0294,-5.3478)">
+ <title>Sheet.174</title>
+ <g id="shape155-2" transform="translate(99,-99)">
+ <title>Circle</title>
+ <g id="shadow155-3" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape156-7" transform="translate(207,-108)">
+ <title>Circle.156</title>
+ <g id="shadow156-8" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape157-12" transform="translate(153,-162)">
+ <title>Circle.157</title>
+ <g id="shadow157-13" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape158-17" transform="translate(297,-99)">
+ <title>Circle.158</title>
+ <g id="shadow158-18" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape159-22" transform="translate(180,-36)">
+ <title>Circle.159</title>
+ <g id="shadow159-23" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape160-27" transform="translate(109.604,-115.419) rotate(-7.12502)">
+ <title>Sheet.160</title>
+ <path d="M0 204.75 L66.4 204.75" class="st4"/>
+ </g>
+ <g id="shape161-33" transform="translate(276.661,-123.214) rotate(9.46232)">
+ <title>Sheet.161</title>
+ <path d="M0 204.75 L48.58 204.75" class="st4"/>
+ </g>
+ <g id="shape162-38" transform="translate(246.135,262.572) rotate(-160.346)">
+ <title>Sheet.162</title>
+ <path d="M0 204.75 L127.63 204.75" class="st4"/>
+ </g>
+ <g id="shape163-43" transform="translate(284.391,198.775) rotate(141.977)">
+ <title>Sheet.163</title>
+ <path d="M0 204.75 L46.23 204.75" class="st4"/>
+ </g>
+ <g id="shape164-48" transform="translate(70.6118,307.655) rotate(-145.945)">
+ <title>Sheet.164</title>
+ <path d="M0 204.75 L60.88 204.75" class="st4"/>
+ </g>
+ <g id="shape167-53" transform="translate(72,0)">
+ <title>Rectangle.167</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow167-54" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape168-60" transform="translate(108,0)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <g id="shadow168-61" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape169-66" transform="translate(0,-63)">
+ <title>Rectangle.169</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow169-67" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape170-73" transform="translate(36,-63)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <g id="shadow170-74" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape171-79" transform="translate(240.248,331.493) rotate(161.565)">
+ <title>Sheet.171</title>
+ <path d="M-0 190.52 A81.3416 36.0611 -153.48 0 0 82.31 195.86 L82.49 195.55" class="st10"/>
+ </g>
+ <g id="shape172-85" transform="translate(156.426,260.029) rotate(161.565)">
+ <title>Sheet.172</title>
+ <path d="M-0 181.6 A88.1422 54.1439 -124.1 0 0 82.68 187.13 L82.83 186.81" class="st10"/>
+ </g>
+ <g id="shape173-90" transform="translate(18,-121.5)">
+ <title>Sheet.173</title>
+ <desc>Encode ID</desc>
+ <rect x="0" y="177.75" width="63" height="27" class="st12"/>
+ <text x="7.02" y="194.85" class="st13">Encode ID</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i4.svg b/doc/guides/prog_guide/img/member_i4.svg
new file mode 100644
index 0000000..a2b6f2f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i4.svg
@@ -0,0 +1,450 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i4.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.625in" height="3.125in" viewBox="0 0 549 225"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st18">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st3 {marker-end:url(#mrkr5-10);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st4 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st5 {visibility:visible}
+ .st6 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st7 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st8 {fill:none;stroke:none;stroke-width:0.25}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st10 {stroke:#5b9bd5;stroke-dasharray:1.5,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5}
+ .st11 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st12 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st14 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st15 {font-size:1em}
+ .st16 {marker-end:url(#mrkr5-162);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st17 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st18 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-10" class="st4" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-162" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group47-1" transform="translate(3.0294,-0.25)" v:mID="47" v:groupContext="group">
+ <title>Sheet.47</title>
+ <g id="shape1-2" v:mID="1" v:groupContext="shape" transform="translate(177.75,-191.922)">
+ <title>Sheet.1</title>
+ <desc>Element</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="216" width="108" height="18"/>
+ <rect x="0" y="207" width="108" height="18" class="st1"/>
+ <text x="33.77" y="219.6" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Element</text> </g>
+ <g id="shape2-5" v:mID="2" v:groupContext="shape" transform="translate(456.75,33.0781) rotate(90)">
+ <title>Sheet.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L18.65 225" class="st3"/>
+ </g>
+ <g id="shape3-11" v:mID="3" v:groupContext="shape" transform="translate(0,-67.0469)">
+ <title>Rectangle.54</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow3-12" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape4-15" v:mID="4" v:groupContext="shape" transform="translate(27,-67.0469)">
+ <title>Rectangle.55</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow4-16" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape5-19" v:mID="5" v:groupContext="shape" transform="translate(54,-67.0469)">
+ <title>Rectangle.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow5-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape6-23" v:mID="6" v:groupContext="shape" transform="translate(0,-53.5469)">
+ <title>Rectangle.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-24" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape7-27" v:mID="7" v:groupContext="shape" transform="translate(27,-53.5469)">
+ <title>Rectangle.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow7-28" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape8-31" v:mID="8" v:groupContext="shape" transform="translate(54,-53.5469)">
+ <title>Rectangle.59</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow8-32" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape9-35" v:mID="9" v:groupContext="shape" transform="translate(5.625,-72.6719)">
+ <title>Sheet.9</title>
+ <desc>BF-1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-1</text> </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(128.25,-65.0781)">
+ <title>Rectangle.74</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow10-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape11-42" v:mID="11" v:groupContext="shape" transform="translate(155.25,-65.0781)">
+ <title>Rectangle.75</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-43" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(182.25,-65.0781)">
+ <title>Rectangle.76</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow12-47" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape13-50" v:mID="13" v:groupContext="shape" transform="translate(128.25,-51.5781)">
+ <title>Rectangle.77</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow13-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape14-54" v:mID="14" v:groupContext="shape" transform="translate(155.25,-51.5781)">
+ <title>Rectangle.78</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow14-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape15-58" v:mID="15" v:groupContext="shape" transform="translate(182.25,-51.5781)">
+ <title>Rectangle.79</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow15-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape16-62" v:mID="16" v:groupContext="shape" transform="translate(301.5,-65.0781)">
+ <title>Rectangle.81</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow16-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape17-66" v:mID="17" v:groupContext="shape" transform="translate(328.5,-65.0781)">
+ <title>Rectangle.82</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow17-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape18-70" v:mID="18" v:groupContext="shape" transform="translate(355.5,-65.0781)">
+ <title>Rectangle.83</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow18-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape19-74" v:mID="19" v:groupContext="shape" transform="translate(301.5,-51.5781)">
+ <title>Rectangle.84</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow19-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape20-78" v:mID="20" v:groupContext="shape" transform="translate(328.5,-51.5781)">
+ <title>Rectangle.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow20-79" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape21-82" v:mID="21" v:groupContext="shape" transform="translate(355.5,-51.5781)">
+ <title>Rectangle.86</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow21-83" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape22-86" v:mID="22" v:groupContext="shape" transform="translate(447.75,-65.6406)">
+ <title>Rectangle.88</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow22-87" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape23-90" v:mID="23" v:groupContext="shape" transform="translate(474.75,-65.6406)">
+ <title>Rectangle.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow23-91" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape24-94" v:mID="24" v:groupContext="shape" transform="translate(501.75,-65.6406)">
+ <title>Rectangle.90</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow24-95" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape25-98" v:mID="25" v:groupContext="shape" transform="translate(447.75,-52.1406)">
+ <title>Rectangle.91</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow25-99" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape26-102" v:mID="26" v:groupContext="shape" transform="translate(474.75,-52.1406)">
+ <title>Rectangle.92</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-103" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape27-106" v:mID="27" v:groupContext="shape" transform="translate(501.75,-52.1406)">
+ <title>Rectangle.93</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-107" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape28-110" v:mID="28" v:groupContext="shape" transform="translate(213.75,-63.9531)">
+ <title>Sheet.28</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L83.25 225" class="st10"/>
+ </g>
+ <g id="shape29-113" v:mID="29" v:groupContext="shape" transform="translate(387,-63.9531)">
+ <title>Sheet.29</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L54 225" class="st10"/>
+ </g>
+ <g id="group31-116" transform="translate(184.5,-113.172)" v:mID="31" v:groupContext="group">
+ <title>Sheet.31</title>
+ <g id="shape32-117" v:mID="32" v:groupContext="shape" transform="translate(225,173.25) rotate(90)">
+ <title>Block Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow32-118" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st5">
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st11"/>
+ </g>
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st12"/>
+ </g>
+ <g id="shape33-121" v:mID="33" v:groupContext="shape" transform="translate(2.25,-24.3529)">
+ <title>Sheet.33</title>
+ <desc>h1, h2 .. hk</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="215.868" width="90" height="18.2647"/>
+ <rect x="0" y="206.735" width="90" height="18.2647" class="st8"/>
+ <text x="17.56" y="219.47" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>h1, h2 .. hk</text> </g>
+ </g>
+ <g id="shape34-124" v:mID="34" v:groupContext="shape" transform="translate(307.011,286.73) rotate(152.323)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L128.85 225" class="st3"/>
+ </g>
+ <g id="shape35-129" v:mID="35" v:groupContext="shape" transform="translate(433.272,125.452) rotate(99.7172)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L58.31 225" class="st3"/>
+ </g>
+ <g id="shape36-134" v:mID="36" v:groupContext="shape" transform="translate(407.724,-64.1459) rotate(45)">
+ <title>Sheet.36</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L79.16 225" class="st3"/>
+ </g>
+ <g id="shape37-139" v:mID="37" v:groupContext="shape" transform="translate(320.441,-127.12) rotate(15.6155)">
+ <title>Sheet.37</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L200.75 225" class="st3"/>
+ </g>
+ <g id="shape38-144" v:mID="38" v:groupContext="shape" transform="translate(132.75,-75.2588)">
+ <title>Sheet.38</title>
+ <desc>BF-2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-2</text> </g>
+ <g id="shape39-147" v:mID="39" v:groupContext="shape" transform="translate(303.75,-70.7588)">
+ <title>Sheet.39</title>
+ <desc>BF-X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.2" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-X</text> </g>
+ <g id="shape40-150" v:mID="40" v:groupContext="shape" transform="translate(447.75,-75.2588)">
+ <title>Sheet.40</title>
+ <desc>BF-L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.89" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-L</text> </g>
+ <g id="shape41-153" v:mID="41" v:groupContext="shape" transform="translate(300.375,-117)">
+ <title>Sheet.41</title>
+ <desc>Hashing for lookup/Insertion into a vector of BFs happens once</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="90" cy="202.5" width="180" height="45"/>
+ <rect x="0" y="180" width="180" height="45" class="st8"/>
+ <text x="4.6" y="198.9" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Hashing for lookup/Insertion into a <tspan
+ x="23.06" dy="1.2em" class="st15">vector of BFs happens once</tspan></text> </g>
+ <g id="shape44-157" v:mID="44" v:groupContext="shape" transform="translate(249.698,-151.505) rotate(-3.74012)">
+ <title>Sheet.44</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A93.4958 45.6256 42.23 0 1 79.38 221.66 L79.68 221.85" class="st16"/>
+ </g>
+ <g id="shape45-163" v:mID="45" v:groupContext="shape" transform="translate(30.375,0.25)">
+ <title>Sheet.45</title>
+ <desc>Lookup/Insertion is done in the series of BFs, one by one or ...</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="233.048" cy="202.5" width="466.1" height="45"/>
+ <rect x="0" y="180" width="466.096" height="45" class="st8"/>
+ <text x="4.34" y="206.1" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Lookup/Insertion is done in the series of BFs, one by one or can be optimized to do in parallel. </text> </g>
+ <g id="shape46-166" v:mID="46" v:groupContext="shape" transform="translate(123.252,-43.6868) rotate(17.0249)">
+ <title>Sheet.46</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A88.2185 43.0621 47.63 0 1 70.31 221.39 L70.6 221.6" class="st16"/>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i5.svg b/doc/guides/prog_guide/img/member_i5.svg
new file mode 100644
index 0000000..c1728cf
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i5.svg
@@ -0,0 +1,163 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i5.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="5.30481in" height="1.96146in"
+ viewBox="0 0 381.946 141.225" xml:space="preserve" color-interpolation-filters="sRGB" class="st15">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:none;stroke:none;stroke-width:0.25}
+ .st5 {fill:#ffffff;font-family:Calibri;font-size:1.16666em;font-weight:bold}
+ .st6 {marker-end:url(#mrkr5-14);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st7 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st10 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {marker-end:url(#mrkr5-63);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st12 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.22935779816514}
+ .st13 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st14 {font-size:1em}
+ .st15 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-14" class="st7" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-63" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="7.15" refX="-7.15" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-4.36,-4.36) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group1-1" transform="translate(191.995,-19.4751)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-2" v:mID="2" v:groupContext="shape" transform="translate(146.944,42.2251) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow2-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st2"/>
+ </g>
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st3"/>
+ </g>
+ <g id="shape3-6" v:mID="3" v:groupContext="shape" transform="translate(0,-34.65)">
+ <title>Sheet.3</title>
+ <desc>vBF</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="38.1251" cy="126.375" width="76.26" height="29.7"/>
+ <rect x="0" y="111.525" width="76.2502" height="29.7" class="st4"/>
+ <text x="27.68" y="130.58" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>vBF </text> </g>
+ </g>
+ <g id="shape4-9" v:mID="4" v:groupContext="shape" transform="translate(126.724,-100.475)">
+ <title>Sheet.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape5-15" v:mID="5" v:groupContext="shape" transform="translate(103.5,-101.775)">
+ <title>Sheet.5</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="39.9122" cy="135.863" width="79.83" height="10.7251"/>
+ <rect x="0" y="130.5" width="79.8244" height="10.7251" class="st4"/>
+ <text x="21.78" y="138.86" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape6-18" v:mID="6" v:groupContext="shape" transform="translate(221.726,-56.2468) rotate(-24.5123)">
+ <title>Sheet.6</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L65.42 141.23" class="st6"/>
+ </g>
+ <g id="shape7-23" v:mID="7" v:groupContext="shape" transform="translate(280.318,-68.9751)">
+ <title>Sheet.7</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape8-28" v:mID="8" v:groupContext="shape" transform="translate(338.125,-56.6022) rotate(24.1625)">
+ <title>Sheet.8</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L70.8 141.23" class="st6"/>
+ </g>
+ <g id="shape9-33" v:mID="9" v:groupContext="shape" transform="translate(197.714,217.975) rotate(180)">
+ <title>Sheet.9</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(18,-67.5)">
+ <title>Sheet.10</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="80.4201" cy="134.475" width="160.85" height="13.5"/>
+ <rect x="0" y="127.725" width="160.84" height="13.5" class="st4"/>
+ <text x="25.11" y="137.18" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape11-41" v:mID="11" v:groupContext="shape" transform="translate(198.032,253.975) rotate(180)">
+ <title>Sheet.11</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Sheet.12</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="81" cy="136.725" width="162.01" height="9"/>
+ <rect x="0" y="132.225" width="162" height="9" class="st4"/>
+ <text x="11.04" y="139.43" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape13-49" v:mID="13" v:groupContext="shape" transform="translate(494.552,22.75) rotate(90)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape14-52" v:mID="14" v:groupContext="shape" transform="translate(494.552,58.75) rotate(90)">
+ <title>Sheet.14</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape15-55" v:mID="15" v:groupContext="shape" transform="translate(494.552,94.75) rotate(90)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape17-58" v:mID="17" v:groupContext="shape" transform="translate(348.769,-25.0593) rotate(44.5185)">
+ <title>Sheet.17</title>
+ <path d="M-0 141.23 A35.1884 19.2595 167.75 0 1 42.43 138.27 L42.74 138.46" class="st11"/>
+ </g>
+ <g id="shape18-64" v:mID="18" v:groupContext="shape" transform="translate(222.188,-5.40005)">
+ <title>Sheet.18</title>
+ <desc>A BF corresponding to each worker thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="59.0625" cy="127.725" width="118.13" height="27"/>
+ <rect x="0" y="114.225" width="118.125" height="27" class="st4"/>
+ <text x="5.14" y="124.13" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>A BF corresponding to <tspan
+ x="11.19" dy="1.2em" class="st14">each worker thread</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i6.svg b/doc/guides/prog_guide/img/member_i6.svg
new file mode 100644
index 0000000..265179f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i6.svg
@@ -0,0 +1,332 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i6.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8in" height="3.625in" viewBox="0 0 576 261"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st16">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.666664em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.00001em}
+ .st9 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {marker-end:url(#mrkr5-29);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st12 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st13 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st14 {fill:#92d050;stroke:#c8c8c8;stroke-width:0.25}
+ .st15 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st16 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-29" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-9.8478)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(396.989,-54.9268)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st2"/>
+ </g>
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(248.261,-12.1936)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st2"/>
+ </g>
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(6.07514E-013,-29.0155)">
+ <title>Rectangle.10</title>
+ <desc>Signatures for target 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="225.767" width="99.49" height="70.4662"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st2"/>
+ </g>
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st3"/>
+ <text x="26.54" y="223.37" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(239.971,-20.4837)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(353.649,-19.9346)">
+ <title>Sheet.54</title>
+ <desc>Match 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 1</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(216.989,-210.652)">
+ <title>Sheet.55</title>
+ <desc>Packet Payload</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="252.71" width="99.49" height="16.5803"/>
+ <rect x="0" y="244.42" width="99.4817" height="16.5803" class="st9"/>
+ <text x="19.04" y="255.71" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet Payload</text> </g>
+ <g id="shape56-24" v:mID="56" v:groupContext="shape" transform="translate(526.665,52.2365) rotate(90)">
+ <title>Sheet.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L16.52 261" class="st11"/>
+ </g>
+ <g id="shape96-30" v:mID="96" v:groupContext="shape" transform="translate(-3.0294,-95.7818)">
+ <title>Sheet.96</title>
+ <desc>Attack Signature Length 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.75" cy="248.565" width="103.5" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.5" height="24.8704" class="st7"/>
+ <text x="4.79" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 1</text> </g>
+ <g id="group114-33" transform="translate(228.359,-134.152)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-34" transform="translate(0,-24.8704)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-35" v:mID="100" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-36" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape101-39" v:mID="101" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-40" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape102-43" v:mID="102" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-44" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape103-47" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-48" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape104-51" v:mID="104" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-52" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape105-55" v:mID="105" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-56" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-59" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-60" v:mID="108" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-61" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape109-64" v:mID="109" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-65" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape110-68" v:mID="110" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow110-69" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape111-72" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-73" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ <g id="shape112-76" v:mID="112" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-77" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape113-80" v:mID="113" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-81" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-84" v:mID="89" v:groupContext="shape" transform="translate(398.644,-116.927) rotate(24.4696)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L143.75 261" class="st11"/>
+ </g>
+ <g id="shape115-89" v:mID="115" v:groupContext="shape" transform="translate(116.062,-1.19371E-012)">
+ <title>Rectangle.115</title>
+ <desc>Signatures for target 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="211.259" width="99.49" height="99.4817"/>
+ <g id="shadow115-90" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st2"/>
+ </g>
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st3"/>
+ <text x="26.54" y="208.86" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>2</text> </g>
+ <g id="shape116-95" v:mID="116" v:groupContext="shape" transform="translate(117.989,-95.7818)">
+ <title>Sheet.116</title>
+ <desc>Attack Signature Length 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.9909" cy="248.565" width="103.99" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.982" height="24.8704" class="st7"/>
+ <text x="5.03" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 2</text> </g>
+ <g id="shape118-98" v:mID="118" v:groupContext="shape" transform="translate(392.971,-90.217)">
+ <title>Sheet.118</title>
+ <desc>Attack Signature Length L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="248.565" width="108" height="24.8704"/>
+ <rect x="0" y="236.13" width="108" height="24.8704" class="st7"/>
+ <text x="7.43" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length L</text> </g>
+ <g id="shape119-101" v:mID="119" v:groupContext="shape" transform="translate(384.909,-64.9346)">
+ <title>Sheet.119</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape120-103" v:mID="120" v:groupContext="shape" transform="translate(491.971,-64.9346)">
+ <title>Sheet.120</title>
+ <desc>Match 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 2</text> </g>
+ <g id="shape85-106" v:mID="85" v:groupContext="shape" transform="translate(478.538,12.9307) rotate(65.6291)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L109.61 261" class="st11"/>
+ </g>
+ <g id="shape117-111" v:mID="117" v:groupContext="shape" transform="translate(247.054,-91.2818)">
+ <title>Sheet.117</title>
+ <desc>Attack Signature Length X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="52.7082" cy="248.565" width="105.42" height="24.8704"/>
+ <rect x="0" y="236.13" width="105.416" height="24.8704" class="st7"/>
+ <text x="5.7" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length X</text> </g>
+ </g>
+ <g id="shape122-114" v:mID="122" v:groupContext="shape" transform="translate(315.114,-164.13)">
+ <title>Sheet.122</title>
+ <desc>HTSS</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="26.943" cy="248.565" width="53.89" height="24.8704"/>
+ <rect x="0" y="236.13" width="53.8859" height="24.8704" class="st7"/>
+ <text x="14.52" y="252.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i7.svg b/doc/guides/prog_guide/img/member_i7.svg
new file mode 100644
index 0000000..e23ae26
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i7.svg
@@ -0,0 +1,399 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i7.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8.5in" height="4.5in" viewBox="0 0 612 324"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st23">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.16666em}
+ .st9 {fill:none;stroke:#00b050;stroke-width:2.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st12 {fill:#a8d08d;stroke:#c8c8c8;stroke-width:0.25}
+ .st13 {marker-end:url(#mrkr5-83);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st14 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st15 {marker-end:url(#mrkr5-95);stroke:#92d050;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st16 {fill:#92d050;fill-opacity:1;stroke:#92d050;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st17 {fill:#00b050;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st18 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st19 {fill:none;stroke:#ff0000;stroke-width:2.25}
+ .st20 {fill:#ff0000;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st21 {marker-end:url(#mrkr5-123);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-83" class="st14" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-95" class="st16" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-123" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-32.2733)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(460.471,-62.2267)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="279" width="108" height="45" class="st2"/>
+ </g>
+ <rect x="0" y="279" width="108" height="45" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(320.452,-18.123)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="234" width="108" height="90" class="st2"/>
+ </g>
+ <rect x="0" y="234" width="108" height="90" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Rectangle.10</title>
+ <desc>Flow Keys Matching Mask 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="285.75" width="108" height="76.5"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="247.5" width="108" height="76.5" class="st2"/>
+ </g>
+ <rect x="0" y="247.5" width="108" height="76.5" class="st3"/>
+ <text x="12.56" y="282.75" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(311.452,-27.123)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="310.5" width="126" height="13.5" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(424.471,-26.2267)">
+ <title>Sheet.54</title>
+ <desc>Match</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="317.25" width="72" height="13.5"/>
+ <rect x="0" y="310.5" width="72" height="13.5" class="st7"/>
+ <text x="17.68" y="321.45" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(261,-247.163)">
+ <title>Sheet.55</title>
+ <desc>Flow ID1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st9"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID1</text> </g>
+ <g id="shape96-24" v:mID="96" v:groupContext="shape" transform="translate(0,-109.783)">
+ <title>Sheet.96</title>
+ <desc>Flow Mask 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 1</text> </g>
+ <g id="group114-27" transform="translate(247.5,-163.783)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-28" transform="translate(0,-27)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-29" v:mID="100" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-30" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape101-33" v:mID="101" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-34" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape102-37" v:mID="102" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-38" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape103-41" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-42" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape104-45" v:mID="104" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-46" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape105-49" v:mID="105" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-50" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-53" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-54" v:mID="108" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape109-58" v:mID="109" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape110-62" v:mID="110" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow110-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st12"/>
+ </g>
+ <g id="shape111-66" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape112-70" v:mID="112" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape113-74" v:mID="113" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-78" v:mID="89" v:groupContext="shape" transform="translate(413.723,393.802) rotate(146.31)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L153.9 324" class="st13"/>
+ </g>
+ <g id="shape115-84" v:mID="115" v:groupContext="shape" transform="translate(126,0)">
+ <title>Rectangle.115</title>
+ <desc>Flow Keys Matching Mask 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="270" width="108" height="108"/>
+ <g id="shadow115-85" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="216" width="108" height="108" class="st2"/>
+ </g>
+ <rect x="0" y="216" width="108" height="108" class="st3"/>
+ <text x="12.56" y="267" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>2</text> </g>
+ <g id="shape85-90" v:mID="85" v:groupContext="shape" transform="translate(635.321,91.2793) rotate(81.3573)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L143.93 324" class="st15"/>
+ </g>
+ <g id="shape56-96" v:mID="56" v:groupContext="shape" transform="translate(579.175,-64.556) rotate(64.1257)">
+ <title>Sheet.56</title>
+ <path d="M0 324 L54.31 324" class="st15"/>
+ </g>
+ </g>
+ <g id="shape122-101" v:mID="122" v:groupContext="shape" transform="translate(351,-213.444)">
+ <title>Sheet.122</title>
+ <desc>HTSS with False Negative (Cache)</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="304.722" width="90" height="38.556"/>
+ <rect x="0" y="285.444" width="90" height="38.556" class="st7"/>
+ <text x="13.29" y="301.72" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS with False <tspan
+ x="10.52" dy="1.2em" class="st5">Negative </tspan>(Cache)</text> </g>
+ <g id="shape123-105" v:mID="123" v:groupContext="shape" transform="translate(287.654,-290.556)">
+ <title>Sheet.123</title>
+ <desc>Active</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1875" cy="310.5" width="48.38" height="27"/>
+ <rect x="0" y="297" width="48.375" height="27" class="st7"/>
+ <text x="8.63" y="314.1" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Active</text> </g>
+ <g id="shape124-108" v:mID="124" v:groupContext="shape" transform="translate(278.827,-153)">
+ <title>Sheet.124</title>
+ <desc>Target for Flow ID 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36.0864" cy="310.5" width="72.18" height="27"/>
+ <rect x="0" y="297" width="72.1728" height="27" class="st9"/>
+ <text x="11.93" y="306.9" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Target for <tspan
+ x="13.54" dy="1.2em" class="st5">Flow ID </tspan>1</text> </g>
+ <g id="shape125-112" v:mID="125" v:groupContext="shape" transform="translate(155.857,-254.556)">
+ <title>Sheet.125</title>
+ <desc>Flow ID2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st19"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID2</text> </g>
+ <g id="shape126-115" v:mID="126" v:groupContext="shape" transform="translate(153,-270)">
+ <title>Sheet.126</title>
+ <desc>New/Inactive</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="40.5" cy="310.5" width="81" height="27"/>
+ <rect x="0" y="297" width="81" height="27" class="st7"/>
+ <text x="6.77" y="314.1" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New/Inactive</text> </g>
+ <g id="shape127-118" v:mID="127" v:groupContext="shape" transform="translate(251.739,-239.709) rotate(14.0795)">
+ <title>Sheet.127</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 318.73 A39.2404 18 -180 0 0 49.73 320.91 L50.07 320.78" class="st21"/>
+ </g>
+ <g id="shape128-124" v:mID="128" v:groupContext="shape" transform="translate(219.24,-229.5)">
+ <title>Sheet.128</title>
+ <desc>Miss</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="20.88" cy="310.5" width="41.76" height="27"/>
+ <rect x="0" y="297" width="41.76" height="27" class="st7"/>
+ <text x="7.81" y="314.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Miss</text> </g>
+ <g id="shape129-127" v:mID="129" v:groupContext="shape" transform="translate(147.029,-142.056)">
+ <title>Sheet.129</title>
+ <desc>Flow Mask 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 2</text> </g>
+ <g id="shape130-130" v:mID="130" v:groupContext="shape" transform="translate(166.845,-18.5004) rotate(18.2325)">
+ <title>Sheet.130</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A71.1913 104.269 -180 0 0 97.04 298.43 L97.25 298.14" class="st21"/>
+ </g>
+ <g id="shape131-135" v:mID="131" v:groupContext="shape" transform="translate(184.406,-3.04505) rotate(-3.24734)">
+ <title>Sheet.131</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A112.345 104.269 -180 0 0 154.25 297.52 L154.52 297.28" class="st21"/>
+ </g>
+ <g id="shape132-140" v:mID="132" v:groupContext="shape" transform="translate(301.368,16.888) rotate(-25.868)">
+ <title>Sheet.132</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A83.375 104.269 -180 0 0 113.91 298.14 L114.14 297.87" class="st21"/>
+ </g>
+ <g id="shape133-145" v:mID="133" v:groupContext="shape" transform="translate(345.029,-142.056)">
+ <title>Sheet.133</title>
+ <desc>Flow Mask X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.43" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask X</text> </g>
+ <g id="shape134-148" v:mID="134" v:groupContext="shape" transform="translate(481.5,-139.5)">
+ <title>Sheet.134</title>
+ <desc>Flow Mask L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="19.12" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask L</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 40f04a1..7ff5144 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -50,6 +50,7 @@ Programmer's Guide
timer_lib
hash_lib
efd_lib
+ member_lib
lpm_lib
lpm6_lib
packet_distrib_lib
@@ -191,6 +192,19 @@ Programmer's Guide
:numref:`figure_efd11` :ref:`figure_efd11`
+:numref:`figure_membership1` :ref:`figure_membership1`
+
+:numref:`figure_membership2` :ref:`figure_membership2`
+
+:numref:`figure_membership3` :ref:`figure_membership3`
+
+:numref:`figure_membership4` :ref:`figure_membership4`
+
+:numref:`figure_membership5` :ref:`figure_membership5`
+
+:numref:`figure_membership6` :ref:`figure_membership6`
+
+:numref:`figure_membership7` :ref:`figure_membership7`
**Tables**
diff --git a/doc/guides/prog_guide/member_lib.rst b/doc/guides/prog_guide/member_lib.rst
new file mode 100644
index 0000000..2b32535
--- /dev/null
+++ b/doc/guides/prog_guide/member_lib.rst
@@ -0,0 +1,421 @@
+.. BSD LICENSE
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+.. _member_library:
+
+Membership Library
+==================
+
+Introduction
+------------
+
+The DPDK Membership Library provides an API for DPDK applications to insert a
+new member, delete an existing member, or query the existence of a member in a
+given set, or a group of sets. For the case of a group of sets the library
+will return not only whether the element has been inserted before in one of
+the sets but also which set it belongs to. The Membership Library is an
+extension and generalization of a traditional filter structure (for example
+Bloom Filter [Member-bloom]) that has multiple usages in a wide variety of
+workloads and applications. In general, the Membership Library is a data
+structure that provides a "set-summary" on whether a member belongs to a set,
+and as discussed in detail later, there are two advantages of using such a
+set-summary rather than operating on a "full-blown" complete list of elements:
+first, it has a much smaller storage requirement than storing the whole list of
+elements themselves, and secondly checking an element membership (or other
+operations) in this set-summary is much faster than checking it for the
+original full-blown complete list of elements.
+
+We use the term "Set-Summary" in this guide to refer to the space-efficient,
+probabilistic membership data structure that is provided by the library. A
+membership test for an element will return the set this element belongs to or
+that the element is "not-found" with very high probability of accuracy. Set-summary
+is a fundamental data aggregation component that can be used in many network
+(and other) applications. It is a crucial structure to address performance and
+scalability issues of diverse network applications including overlay networks,
+data-centric networks, flow table summaries, network statistics and
+traffic monitoring. A set-summary is useful for applications who need to
+include a list of elements while a complete list requires too much space
+and/or too much processing cost. In these situations, the set-summary works as
+a lossy hash-based representation of a set of members. It can dramatically
+reduce space requirement and significantly improve the performance of set
+membership queries at the cost of introducing a very small membership test error
+probability.
+
+.. _figure_membership1:
+.. figure:: img/member_i1.*
+
+ Example Usages of Membership Library
+
+There are various usages for a Membership Library in a very
+large set of applications and workloads. Interested readers can refer to
+[Member-survey] for a survey of possible networking usages. The above figure
+provide a small set of examples of using the Membership Library:
+
+* Sub-figure (a)
+ depicts a distributed web cache architecture where a collection of proxies
+ attempt to share their web caches (cached from a set of back-end web servers) to
+ provide faster responses to clients, and the proxies use the Membership
+ Library to share summaries of what web pages/objects they are caching. With the
+ Membership Library, a proxy receiving an http request will inquire the
+ set-summary to find its location and quickly determine whether to retrieve the
+ requested web page from a nearby proxy or from a backend web server.
+
+* Sub-figure (b) depicts another example for using the Membership Library to
+ prevent routing loops which is typically done using slow TTL countdown and
+ dropping packets when TTL expires. As shown in Sub-figure (b), an embedded
+ set-summary in the packet header itself can be used to summarize the set of
+ nodes a packet has gone through, and each node upon receiving a packet can check
+ whether its id is a member of the set of visited nodes, and if it is, then a
+ routing loop is detected.
+
+* Sub-Figure (c) presents another usage of the Membership
+ Library to load-balance flows to worker threads with in-order guarantee where a
+ set-summary is used to query if a packet belongs to an existing flow or a new
+ flow. Packets belonging to a new flow are forwarded to the current least loaded
+ worker thread, while those belonging to an existing flow are forwarded to the
+ pre-assigned thread to guarantee in-order processing.
+
+* Sub-figure (d) highlights
+ yet another usage example in the database domain where a set-summary is used to
+ determine joins between sets instead of creating a join by comparing each
+ element of a set against the other elements in a different set, a join is done
+ on the summaries since they can efficiently encode members of a given set.
+
+We are including a configurable Membership Library in DPDK to cover set
+membership functionality for both a single set and multi-set scenarios. The
+library is optimized to support the customer network applications which require
+membership checking functionality. In this guide, we will cover two set-summary
+schemes including a vector of Bloom Filters and Hash-Table based
+set-summary schemes with and without false negative probability, followed by
+a brief discussion of the Membership Library API.
+
+Vector of Bloom Filters
+-----------------------
+
+Bloom Filter (BF) [Member-bloom] is a well-known space-efficient
+probabilistic data structure that answers set membership queries (test whether
+an element is a member of a set) with some probability of false positives and
+zero false negatives; a query for an element returns either it is "possibly in
+a set" (with very high probability) or "definitely not in a set".
+
+The BF is a method for representing a set of ``n`` elements (for example flow keys
+in network applications domain) to support membership queries. The idea of BF is
+to allocate a bit-vector ``v`` with ``m`` bits, which are initially all set to 0. Then
+it chooses ``k`` independent hash functions ``h1``, ``h2``, ... ``hk`` with hash values range from
+``1`` to ``m`` to perform hashing calculations on each element. Every time when an
+element ``X`` being inserted into the set, the bits at positions ``h1(X)``, ``h2(X)``, ...
+``hk(X)`` in ``v`` are set to 1 (any particular bit might be set to 1 multiple times
+for multiple different inserted elements). Given a query for any element ``Y``, the
+bits at positions ``h1(Y)``, ``h2(Y)``, ... ``hk(Y)`` are checked. If any of them is 0,
+then Y is definitely not in the set. Otherwise there is a high probability that
+Y is a member of the set with certain false positive probability. As shown in
+the next equation, the false positive probability can be made arbitrarily small
+by changing the number of hash functions (``k``) and the vector length (``m``).
+
+.. _figure_membership2:
+.. figure:: img/member_i2.*
+
+ Bloom Filter False Positive Probability
+
+Without BF, an accurate membership testing could involve a costly hash table
+lookup and full element comparison. The advantage of using a BF is to simplify
+the membership test into a series of hash calculations and memory accesses for a
+small bit-vector, which can be easily optimized. Hence the lookup throughput
+(set membership test) can be significantly faster than a normal hash table
+lookup with element comparison.
+
+.. _figure_membership3:
+.. figure:: img/member_i3.*
+
+ Detecting Routing Loops Using BF
+
+BF is used for applications that need only one set, and the
+membership of elements is checked against the BF. The example discussed
+in the above figure is one example of potential applications that uses only one
+set to capture the node IDs that have been visited so far by the packet. Each
+node will then check this embedded BF in the packet header for its own id, and
+if the BF indicates that the current node is definitely not in the set then a
+loop-free route is guaranteed.
+
+
+.. _figure_membership4:
+.. figure:: img/member_i4.*
+
+ Vector Bloom Filter (vBF) Overview
+
+To support membership test for both multiple sets and a single set,
+the library implements a Vector Bloom Filter (vBF) scheme.
+vBF basically composes multiple bloom filters into a vector of bloom filers.
+The membership test is conducted on all of the
+bloom filters concurrently to determine which set(s) it belongs to or none of
+them. The basic idea of vBF is shown in the above figure where an element is
+used to address multiple bloom filters concurrently and the bloom filter
+index(es) with a hit is returned.
+
+.. _figure_membership5:
+.. figure:: img/member_i5.*
+
+ vBF for Flow Scheduling to Worker Thread
+
+As previously mentioned, there are many usages of such structures. vBF is used
+for applications that needs to check membership against multiple sets
+simultaneously. The example shown in the above figure uses a set to capture
+all flows being assigned for processing at a given worker thread. Upon receiving
+a packet the vBF is used to quickly figure out if this packet belongs to a new flow
+so as to be forwarded to the current least loaded worker thread, or otherwise it
+should be queued for an existing thread to guarantee in-order processing (i.e.
+the property of vBF to indicate right away that a given flow is a new one or
+not is critical to minimize response time latency).
+
+It should be noted that vBF can be implemented using a set of single bloom
+filters with sequential lookup of each BF. However, being able to concurrently
+search all set-summaries is a big throughput advantage. In the library, certain
+parallelism is realized by the implementation of checking all bloom filters
+together.
+
+
+Hash-Table based Set-Summaries
+------------------------------
+
+Hash-table based set-summary (HTSS) is another scheme in the membership library.
+Cuckoo filter [Member-cfilter] is an example of HTSS.
+HTSS supports multi-set membership testing like
+vBF does. However, while vBF is better for a small number of targets, HTSS is more suitable
+and can easily outperform vBF when the number of sets is
+large, since HTSS uses a single hash table for membership testing while vBF
+requires testing a series of Bloom Filters each corresponding to one set.
+As a result, generally speaking vBF is more adequate for the case of a small limited number of sets
+while HTSS should be used with a larger number of sets.
+
+.. _figure_membership6:
+.. figure:: img/member_i6.*
+
+ Using HTSS for Attack Signature Matching
+
+As shown in the above figure, attack signature matching where each set
+represents a certain signature length (for correctness of this example, an
+attack signature should not be a subset of another one) in the payload is a good
+example for using HTSS with 0% false negative (i.e., when an element returns not
+found, it has a 100% certainty that it is not a member of any set). The packet
+inspection application benefits from knowing right away that the current payload
+does not match any attack signatures in the database to establish its
+legitimacy, otherwise a deep inspection of the packet is needed.
+
+HTSS employs a similar but simpler data structure to a traditional hash table,
+and the major difference is that HTSS stores only the signatures but not the
+full keys/elements which can significantly reduce the footprint of the table.
+Along with the signature, HTSS also stores a value to indicate the target set.
+When looking up an element, the element is hashed and the HTSS is addressed
+to retrieve the signature stored. If the signature matches then the value is
+retrieved corresponding to the index of the target set which the element belongs
+to. Because signatures can collide, HTSS can still has false positive
+probability. Furthermore, if elements are allowed to be
+overwritten or evicted when the hash table becomes full, it will also have a
+false negative probability. We discuss this case in the next section.
+
+Set-Summaries with False Negative Probability
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As previously mentioned, traditional set-summaries (e.g. Bloom Filters) do not
+have a false negative probability, i.e., it is 100% certain when an element
+returns "not to be present" for a given set. However, the Membership Library
+also supports a set-summary probabilistic data structure based on HTSS which
+allows for false negative probability.
+
+In HTSS, when the hash table becomes full, keys/elements will fail to be added
+into the table and the hash table has to be resized to accommodate for these new
+elements, which can be expensive. However, if we allow new elements to overwrite
+or evict existing elements (as a cache typically does), then the resulting
+set-summary will begin to have false negative probability. This is because the
+element that was evicted from the set-summary may still be present in the target
+set. For subsequent inquiries the set-summary will falsely report the element
+not being in the set, hence having a false negative probability.
+
+The major usage of HTSS with false negative is to use it as a cache for
+distributing elements to different target sets. By allowing HTSS to evict old
+elements, the set-summary can keep track of the most recent elements
+(i.e. active) as a cache typically does. Old inactive elements (infrequently
+used elements) will automatically and eventually get evicted from the
+set-summary. It worth noting that the set-summary still has false positive
+probability, which means the application either can tolerate certain false positive
+or it has fall-back path when false positive happens.
+
+.. _figure_membership7:
+.. figure:: img/member_i7.*
+
+ Using HTSS with False Negatives for Wild Card Classification
+
+HTSS with false negative (i.e. a cache) also has its wide set of applications.
+For example wild card flow classification (e.g. ACL rules) highlighted in the
+above figure is an example of such application. In that case each target set
+represents a sub-table with rules defined by a certain flow mask. The flow masks
+are non-overlapping, and for flows matching more than one rule only the highest
+priority one is inserted in the corresponding sub-table (interested readers can
+refer to the Open vSwitch (OvS) design of Mega Flow Cache (MFC) [Member-OvS]
+for further details). Typically the rules will have a large number of distinct
+unique masks and hence, a large number of target sets each corresponding to one
+mask. Because the active set of flows varies widely based on the network
+traffic, HTSS with false negative will act as a cache for <flowid, target ACL
+sub-table> pair for the current active set of flows. When a miss occurs (as
+shown in red in the above figure) the sub-tables will be searched sequentially
+one by one for a possible match, and when found the flow key and target
+sub-table will be inserted into the set-summary (i.e. cache insertion) so
+subsequent packets from the same flow don’t incur the overhead of the
+sequential search of sub-tables.
+
+Library API Overview
+--------------------
+
+The design goal of the Membership Library API is to be as generic as possible to
+support all the different types of set-summaries we discussed in previous
+sections and beyond. Fundamentally, the APIs need to include creation,
+insertion, deletion, and lookup.
+
+
+Set-summary Create
+~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_create()`` function is used to create a set-summary structure, the input parameter
+is a struct to pass in parameters that needed to initialize the set-summary, while the function returns the
+pointer to the created set-summary or ``NULL`` if the creation failed.
+
+The general input arguments used when creating the set-summary should include ``name``
+which is the name of the created set-summary, *type* which is one of the types
+supported by the library (e.g. ``RTE_MEMBER_TYPE_HT`` for HTSS or ``RTE_MEMBER_TYPE_VBF`` for vBF), and ``key_len``
+which is the length of the element/key. There are other parameters
+are only used for certain type of set-summary, or which have a slightly different meaning for different types of set-summary.
+For example, ``num_keys`` parameter means the maximum number of entries for Hash table based set-summary.
+However, for bloom filter, this value means the expected number of keys that could be
+inserted into the bloom filter(s). The value is used to calculate the size of each
+bloom filter.
+
+We also pass two seeds: ``prim_hash_seed`` and
+``sec_hash_seed`` for the primary and secondary hash functions to calculate two independent hash values.
+``socket_id`` parameter is the NUMA socket ID for the memory used to create the
+set-summary. For HTSS, another parameter ``iscache`` is used to indicate
+if this set-summary is a cache (i.e. with false negative probability) or not.
+For vBF, extra parameters are needed. For example, ``num_set`` is the number of
+sets needed to initialize the vector bloom filters. This number is equal to the
+number of bloom filters will be created.
+``false_pos_rate`` is the false positive rate. num_keys and false_pos_rate will be used to determine
+the number of hash functions and the bloom filter size.
+
+
+Set-summary Element Insertion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_add()`` function is use to insert an element/key into a set-summary structure. If it fails an
+error is returned. For success the returned value is dpendednt on the
+set-summary mode to provide extra information for the users. For vBF
+mode, a return value of 0 means a successful insert. For HTSS mode without false negative, the insert
+could fail with ``-ENOSPC`` if the table is full. With false negative (i.e. cache mode),
+for insert that does not cause any eviction (i.e. no overwriting happens to an
+existing entry) the return value is 0. For insertion that causes eviction, the return
+value is 1 to indicate such situation, but it is not an error.
+
+The input arguments for the function should include the ``key`` which is a pointer to the element/key that needs to
+be added to the set-summary, and ``set_id`` which is the set id associated
+with the key that needs to be added.
+
+
+Set-summary Element Lookup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_lookup()`` function looks up a single key/element in the set-summary structure. It
+returns as soon as the first match is found. The return value is 1 if a
+match is found and 0 otherwise. The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, and ``set_id`` which is used to return the
+first target set id where the key has matched, if any.
+
+The ``rte_member_lookup_bulk()`` function is used to look up a bulk of keys/elements in the
+set-summary structure for their first match. Each key lookup returns as soon as the first match is found. The
+return value is the number of keys that find a match. The arguments of the
+function include ``keys`` which is a pointer to a bulk of keys that are to be looked up,
+``num_keys`` is the number
+of keys that will be looked up, and ``set_ids`` are the return target set
+ids for the first match found for each of the input keys. ``set_ids`` is an array
+needs to be sized according to the ``num_keys``. If there is no match, the set id
+for that key will be set to RTE_MEMBER_NO_MATCH.
+
+The ``rte_member_lookup_multi()`` function looks up a single key/element in the
+set-summary structure for multiple matches. It
+returns ALL the matches (possibly more than one) found for this key when it
+is matched against all target sets (it worth noting that for cache mode HTSS,
+the current implementation matches at most one target set). The return value is
+the number of matches
+that was found for this key (for cache mode HTSS the return value
+should be at most 1). The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, ``match_per_key`` which is to indicate maximum number of matches
+the user expect to find for each key, and ``set_id`` which is used to return all
+target set ids where the key has matched, if any. The ``set_id`` array should be sized
+according to ``match_per_key``. For vBF, maximum number of matches per key is equal
+to the number of sets. For HTSS, maximum number of matches per key is equal to two times
+entry count per bucket. ``match_per_key`` should be equal or smaller than maximum number of
+possible matches.
+
+The ``rte_membership_lookup_multi_bulk()`` function looks up a bulk of keys/elements elements in the
+set-summary structure for multiple matches, each key lookup returns ALL the matches (possibly more
+than one) found for this key when it is matched against all target sets (cache mode HTSS
+matches at most one target set). The
+return value is the number of keys that find one or more matches in the
+set-summary structure. The arguments of the
+function include ``keys`` which is
+a pointer to a bulk of keys that are to be looked up, ``num_keys`` is the number
+of keys that will be looked up, ``match_per_key`` is the possible
+max number of matches for each key, ``match_count`` which is the returned number
+of matches for each key, and ``set_ids`` are the returned target set
+ids for all matches found for each keys. ``set_ids`` is 2-D array
+that for each key, a 1-D array should be sized according to ``match_per_key``.
+``match_per_key`` should be equal or smaller than maximum number of
+possible matches, similar to ``rte_member_lookup_multi``.
+
+
+Set-summary Element Delete
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_membership_delete()`` function deletes an element/key from a set-summary structure, if it fails
+an error is returned. The input arguments should include ``key`` which is a pointer to the
+element/key that needs to be deleted from the set-summary, and ``set_id``
+which is the set id associated with the key to delete. It worth noting that current
+implementation of vBF does not support deletion [1]_. An error code ``-EINVAL`` will be returned.
+
+.. [1] Traditional bloom filter does not support proactive deletion. Supporting proactive deletion require additional implementation and performance overhead.
+
+References
+-----------
+
+[Member-bloom] B H Bloom, "Space/Time Trade-offs in Hash Coding with Allowable Errors," Communications of the ACM, 1970.
+
+[Member-survey] A Broder and M Mitzenmacher, "Network Applications of Bloom Filters: A Survey," in Internet Mathematics, 2005.
+
+[Member-cfilter] B Fan, D G Andersen and M Kaminsky, "Cuckoo Filter: Practically Better Than Bloom," in Conference on emerging Networking Experiments and Technologies, 2014.
+
+[Member-OvS] B Pfaff, "The Design and Implementation of Open vSwitch," in NSDI, 2015.
diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst
index 170f4f9..4002df3 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -41,6 +41,23 @@ New Features
Also, make sure to start the actual text at the margin.
=========================================================
+* **Added Membership library (rte_member).**
+
+ Added membership library. It provides an API for DPDK applications to insert a
+ new member, delete an existing member, or query the existence of a member in a
+ given set, or a group of sets. For the case of a group of sets the library
+ will return not only whether the element has been inserted before in one of
+ the sets but also which set it belongs to.
+
+ The Membership Library is an extension and generalization of a traditional
+ filter (for example Bloom Filter) structure that has multiple usages in a wide
+ variety of workloads and applications. In general, the Membership Library is a
+ data structure that provides a “set-summary” and responds to set-membership
+ queries whether a certain member belongs to a set(s).
+
+ See the :ref:`Membership Library <Member_Library>` documentation in
+ the Programmers Guide document, for more information.
+
Resolved Issues
---------------
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation Yipeng Wang
@ 2017-09-18 18:42 ` Mcnamara, John
2017-09-25 12:30 ` De Lara Guarch, Pablo
1 sibling, 0 replies; 81+ messages in thread
From: Mcnamara, John @ 2017-09-18 18:42 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: Tai, Charlie, Gobriel, Sameh, Wang, Ren
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Wednesday, September 6, 2017 1:00 AM
> To: dev@dpdk.org
> Cc: Tai, Charlie <charlie.tai@intel.com>; Gobriel, Sameh
> <sameh.gobriel@intel.com>; Wang, Ren <ren.wang@intel.com>; Wang, Yipeng1
> <yipeng1.wang@intel.com>; Mcnamara, John <john.mcnamara@intel.com>
> Subject: [PATCH v3 7/7] doc: add membership documentation
>
> This patch adds the documentation for membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> Reviewed-by: John McNamara <john.mcnamara@intel.com>
> +Set-summary Element Insertion
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``rte_member_add()`` function is use to insert an element/key into a
> set-summary structure. If it fails an
> +error is returned. For success the returned value is dpendednt on the
Minor typo: s/dpendednt/dependent/
Apart from that:
Acked-by: John McNamara <john.mcnamara@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation Yipeng Wang
2017-09-18 18:42 ` Mcnamara, John
@ 2017-09-25 12:30 ` De Lara Guarch, Pablo
1 sibling, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-09-25 12:30 UTC (permalink / raw)
To: Wang, Yipeng1, dev
Cc: Tai, Charlie, Gobriel, Sameh, Wang, Ren, Wang, Yipeng1, Mcnamara, John
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yipeng Wang
> Sent: Wednesday, September 6, 2017 1:00 AM
> To: dev@dpdk.org
> Cc: Tai, Charlie <charlie.tai@intel.com>; Gobriel, Sameh
> <sameh.gobriel@intel.com>; Wang, Ren <ren.wang@intel.com>; Wang,
> Yipeng1 <yipeng1.wang@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>
> Subject: [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation
>
> This patch adds the documentation for membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> Reviewed-by: John McNamara <john.mcnamara@intel.com>
...
> diff --git a/doc/guides/prog_guide/member_lib.rst
> b/doc/guides/prog_guide/member_lib.rst
> new file mode 100644
> index 0000000..2b32535
> --- /dev/null
> +++ b/doc/guides/prog_guide/member_lib.rst
> +Membership Library
> +==================
> +
> +Introduction
> +------------
> +
> +The DPDK Membership Library provides an API for DPDK applications to
> insert a
> +new member, delete an existing member, or query the existence of a
> member in a
> +given set, or a group of sets. For the case of a group of sets the library
"group of sets, the library".
...
> +We are including a configurable Membership Library in DPDK to cover set
This looks like more like a cover letter than part of programmer's guide.
Reword this loke "This Membership Library covers set membership...",
and avoid "include... in DPDK", as it is implicit.
> +membership functionality for both a single set and multi-set scenarios.
> The
> +library is optimized to support the customer network applications which
> require
> +membership checking functionality. In this guide, we will cover two set-
> summary
> +schemes including a vector of Bloom Filters and Hash-Table based
> +set-summary schemes with and without false negative probability,
> followed by
> +a brief discussion of the Membership Library API.
> +
> +Vector of Bloom Filters
> +-----------------------
> +
> +Bloom Filter (BF) [Member-bloom] is a well-known space-efficient
> +probabilistic data structure that answers set membership queries (test
> whether
> +an element is a member of a set) with some probability of false positives
> and
> +zero false negatives; a query for an element returns either it is "possibly in
> +a set" (with very high probability) or "definitely not in a set".
> +
> +The BF is a method for representing a set of ``n`` elements (for example
> flow keys
> +in network applications domain) to support membership queries. The idea
> of BF is
> +to allocate a bit-vector ``v`` with ``m`` bits, which are initially all set to 0.
> Then
> +it chooses ``k`` independent hash functions ``h1``, ``h2``, ... ``hk`` with
> hash values range from
> +``1`` to ``m`` to perform hashing calculations on each element.
From "0" to "m-1"?
...
> +
> + Vector Bloom Filter (vBF) Overview
> +
> +To support membership test for both multiple sets and a single set,
> +the library implements a Vector Bloom Filter (vBF) scheme.
> +vBF basically composes multiple bloom filters into a vector of bloom filers.
> +The membership test is conducted on all of the
> +bloom filters concurrently to determine which set(s) it belongs to or none
> of
> +them. The basic idea of vBF is shown in the above figure where an element
> is
> +used to address multiple bloom filters concurrently and the bloom filter
> +index(es) with a hit is returned.
> +
> +.. _figure_membership5:
> +.. figure:: img/member_i5.*
> +
> + vBF for Flow Scheduling to Worker Thread
> +
> +As previously mentioned, there are many usages of such structures. vBF is
> used
> +for applications that needs to check membership against multiple sets
> +simultaneously.
"needs" -> "need".
...
> +The major usage of HTSS with false negative is to use it as a cache for
> +distributing elements to different target sets. By allowing HTSS to evict old
> +elements, the set-summary can keep track of the most recent elements
> +(i.e. active) as a cache typically does. Old inactive elements (infrequently
> +used elements) will automatically and eventually get evicted from the
> +set-summary. It worth noting that the set-summary still has false positive
"It is worth".
...
> +We also pass two seeds: ``prim_hash_seed`` and
> +``sec_hash_seed`` for the primary and secondary hash functions to
> calculate two independent hash values.
> +``socket_id`` parameter is the NUMA socket ID for the memory used to
> create the
> +set-summary. For HTSS, another parameter ``iscache`` is used to indicate
In order to keep consistency, I would use "is_cache".
> +if this set-summary is a cache (i.e. with false negative probability) or not.
> +For vBF, extra parameters are needed. For example, ``num_set`` is the
> number of
> +sets needed to initialize the vector bloom filters. This number is equal to
> the
> +number of bloom filters will be created.
> +``false_pos_rate`` is the false positive rate. num_keys and false_pos_rate
> will be used to determine
> +the number of hash functions and the bloom filter size.
> +
> +
> +Set-summary Element Insertion
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``rte_member_add()`` function is use to insert an element/key into a
> set-summary structure. If it fails an
> +error is returned. For success the returned value is dpendednt on the
> +set-summary mode to provide extra information for the users. For vBF
> +mode, a return value of 0 means a successful insert. For HTSS mode
> without false negative, the insert
> +could fail with ``-ENOSPC`` if the table is full. With false negative (i.e. cache
> mode),
> +for insert that does not cause any eviction (i.e. no overwriting happens to
> an
> +existing entry) the return value is 0. For insertion that causes eviction, the
> return
> +value is 1 to indicate such situation, but it is not an error.
> +
> +The input arguments for the function should include the ``key`` which is a
> pointer to the element/key that needs to
> +be added to the set-summary, and ``set_id`` which is the set id associated
> +with the key that needs to be added.
> +
> +
> +Set-summary Element Lookup
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Remove unneeded last "~" (I see 4 extra at the end).
> +
> +The ``rte_member_lookup()`` function looks up a single key/element in
> the set-summary structure. It
> +returns as soon as the first match is found. The return value is 1 if a
> +match is found and 0 otherwise. The arguments for the function include
> ``key`` which is a pointer to the
> +element/key that needs to be looked up, and ``set_id`` which is used to
> return the
> +first target set id where the key has matched, if any.
> +
> +The ``rte_member_lookup_bulk()`` function is used to look up a bulk of
> keys/elements in the
> +set-summary structure for their first match. Each key lookup returns as
> soon as the first match is found. The
> +return value is the number of keys that find a match. The arguments of the
> +function include ``keys`` which is a pointer to a bulk of keys that are to be
> looked up,
> +``num_keys`` is the number
> +of keys that will be looked up, and ``set_ids`` are the return target set
> +ids for the first match found for each of the input keys. ``set_ids`` is an
> array
> +needs to be sized according to the ``num_keys``. If there is no match, the
> set id
> +for that key will be set to RTE_MEMBER_NO_MATCH.
> +
> +The ``rte_member_lookup_multi()`` function looks up a single
> key/element in the
> +set-summary structure for multiple matches. It
> +returns ALL the matches (possibly more than one) found for this key when
> it
> +is matched against all target sets (it worth noting that for cache mode
> HTSS,
"it is worth"
> +the current implementation matches at most one target set). The return
> value is
> +the number of matches
> +that was found for this key (for cache mode HTSS the return value
> +should be at most 1). The arguments for the function include ``key`` which
> is a pointer to the
> +element/key that needs to be looked up, ``match_per_key`` which is to
Better to use "max_match_per_key". Will comment more in other patches.
> indicate maximum number of matches
> +the user expect to find for each key, and ``set_id`` which is used to return
> all
"user expects to".
> +target set ids where the key has matched, if any. The ``set_id`` array
> should be sized
> +according to ``match_per_key``. For vBF, maximum number of matches
> per key is equal
"For vBF, the maximum number".
> +to the number of sets. For HTSS, maximum number of matches per key is
> equal to two times
> +entry count per bucket. ``match_per_key`` should be equal or smaller than
"two time the entry count".
> maximum number of
> +possible matches.
> +
> +The ``rte_membership_lookup_multi_bulk()`` function looks up a bulk of
> keys/elements elements in the
> +set-summary structure for multiple matches, each key lookup returns ALL
> the matches (possibly more
> +than one) found for this key when it is matched against all target sets
> (cache mode HTSS
> +matches at most one target set). The
> +return value is the number of keys that find one or more matches in the
> +set-summary structure. The arguments of the
> +function include ``keys`` which is
> +a pointer to a bulk of keys that are to be looked up, ``num_keys`` is the
> number
> +of keys that will be looked up, ``match_per_key`` is the possible
> +max number of matches for each key, ``match_count`` which is the
Use "maximum number", no need to abbreviate here.
> returned number
> +of matches for each key, and ``set_ids`` are the returned target set
> +ids for all matches found for each keys. ``set_ids`` is 2-D array
> +that for each key, a 1-D array should be sized according to
> ``match_per_key``.
I would reword it a bit, to improve readability. For instance:
"set_ids" is a 2-D array, containing a 1-D array per key,
which should be sized according to "match_per_key".
> +``match_per_key`` should be equal or smaller than maximum number of
> +possible matches, similar to ``rte_member_lookup_multi``.
> +
> +
> +Set-summary Element Delete
> +~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``rte_membership_delete()`` function deletes an element/key from a
> set-summary structure, if it fails
> +an error is returned. The input arguments should include ``key`` which is a
> pointer to the
> +element/key that needs to be deleted from the set-summary, and
> ``set_id``
> +which is the set id associated with the key to delete. It worth noting that
> current
"It is worth".
> +implementation of vBF does not support deletion [1]_. An error code ``-
> EINVAL`` will be returned.
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v4 0/7] Add Membership Library
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
` (6 preceding siblings ...)
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 1/7] member: implement main API Yipeng Wang
` (7 more replies)
7 siblings, 8 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
DPDK Membership Library provides an API that can be used by many DPDK
applications to conduct one or more set-membership tests (we mention some
possible use cases below, but interested readers can refer to
[1] for a wider survey of use cases).
The basic functionalities of the Membership Library include
inserting a new member, deleting an existing member, and querying the existence
of a member. The query result would indicate with high accuracy which specific
set this member belongs to among a group of sets.
The Membership Library is an extension and generalization of traditional filter
data structures [2,3], which maintain a space-efficient “set-summary”.
There are two advantages of using such a set-summary rather than operating on a
“full-blown” complete list of elements: firstly it has a much smaller storage
requirement than storing the whole list of elements, and secondly set membership
tests (or other operations) is much more efficient than searching through the
complete list of elements.
A membership test for an element will return the set this element belongs to if
found (or return "not-found") with high accuracy. If needed, the accuracy of the
membership tests could be further increased with larger storage space.
Set-summary is a fundamental data aggregation component that can be used in many
network applications. It is a crucial structure to address performance and
scalability issues of diverse applications including overlay networks, wild card
flow classification, web-caches, load balancing, connection tracking,
data-centric networks, flow table summaries, network statistics and traffic
monitoring. Our Proof of Concept (PoC) using set-summary to optimize flow lookup
in Open vSwitch (OvS) shows a speedup of about 2-3X.
This patch set implements two types of set-summaries, i.e., hash-table based
set-summary (HTSS) and Vector Bloom Filter (vBF). HTSS supports both the
non-cache and cache modes. The non-cache mode can incur a small chance of
false-positives which is the case when the set-summary indicates a key belongs
to a given set while actually it is not. The cache mode can also have
false-negatives in addition to false-positives. False-negatives means the case
when the set-summary indicates a key does not belong to a given set while
actually it does. This happens because cache mode allows new key to evict
existing keys. vBF only has false-positives similar to the non-cache HTSS.
However, one can set the false-positive rate arbitrarily. HTSS's
false-positive rate is determined by the hash-table size and the signature size.
[1] A Broder and M Mitzenmacher, “Network Applications of Bloom Filters: A
Survey,” in Internet Mathematics, 2005.
[2] B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable Errors,”
Communications of the ACM, 1970.
[3] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically Better Than
Bloom,” in Conference on emerging Networking Experiments and Technologies, 2014.
v4:
- member lib: change two hash function macros to be one. crc or jhash will be
chosen depending on the architecture.
- member lib: use dynamic log type instead of static one by Thomas' comment.
- member lib: code cleanups (remove unnecessary code, change variable name,
coding style, etc).
- member lib: reorder members in setsummary struct to be more cache friendly.
- member lib: setsummary struct member "name" should be char array rather than
char pointer. Fixed.
- member lib: add check for two hash seeds to make sure they are not equal.
- member lib: vBF lookup speed optimization.
- member lib: revise comments in various places as Pablo suggested.
- member lib: change "void *" to "struct rte_member_setsum *" in public API as
Pablo suggested. test case is modified accordingly to directly use the struct.
- member lib: change order in rte_member_version.map as Pablo suggested.
- MAINTAINERS: change the maintainer block order according to Thomas' comment.
- test: add printf to show which section expects error messages. Change set
count to 16 from 32 for performance test.
- documentation: revise several places according to Pablo's and John's comments.
v3:
- documentation: update documentation to incorporate John's review.
v2:
- test: add bad parameter test functions to test the creation fail case.
- test: add find existing test.
- member lib: Add num_entries check in rte_member_create_ht.
- member lib: Add multiple checks for rte_member_create_vbf to avoid divide-by-0.
- member lib: remove unnecessary ret from rte_member.c according to Stephen's
comment.
- member lib: change the vBF creation function to be not too conservative.
Previous algorithm is too conservative on false positive.
- member lib: fix uninitialized issue fail gcc-4.8 in rte_member_find_existing.
- member lib: add rte_member_find_existing to version map.
- makefile: lib/librte_member/Makefile: change include order according to
Luca's comment.
- documentation: update the programmer guide for membership library.
Yipeng Wang (7):
member: implement main API
member: implement HT mode
member: implement vBF mode
member: add AVX for HT mode
member: enable the library
test/member: add functional and perf tests
doc: add membership documentation
MAINTAINERS | 8 +-
config/common_base | 5 +
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 ++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 +++
doc/guides/prog_guide/img/member_i6.svg | 332 ++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 420 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
lib/Makefile | 2 +
lib/librte_member/Makefile | 51 +
lib/librte_member/rte_member.c | 336 +++++++
lib/librte_member/rte_member.h | 507 ++++++++++
lib/librte_member/rte_member_ht.c | 553 ++++++++++
lib/librte_member/rte_member_ht.h | 94 ++
lib/librte_member/rte_member_vbf.c | 349 +++++++
lib/librte_member/rte_member_vbf.h | 79 ++
lib/librte_member/rte_member_version.map | 16 +
lib/librte_member/rte_member_x86.h | 107 ++
mk/rte.app.mk | 2 +
test/test/Makefile | 3 +
test/test/test_member.c | 676 +++++++++++++
test/test/test_member_perf.c | 630 ++++++++++++
28 files changed, 7012 insertions(+), 2 deletions(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
create mode 100644 lib/librte_member/rte_member_version.map
create mode 100644 lib/librte_member/rte_member_x86.h
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v4 1/7] member: implement main API
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-10-02 10:04 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 2/7] member: implement HT mode Yipeng Wang
` (6 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
Membership library is an extension and generalization of a traditional
filter (for example Bloom Filter) structure. In general, the Membership
library is a data structure that provides a "set-summary" and responds
to set-membership queries of whether a certain element belongs to a
set(s). A membership test for an element will return the set this element
belongs to or not-found if the element is never inserted into the
set-summary.
The results of the membership test are not 100% accurate. Certain
false positive or false negative probability could exist. However,
comparing to a "full-blown" complete list of elements, a "set-summary"
is memory efficient and fast on lookup.
This patch adds the main API definition.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/Makefile | 2 +
lib/librte_member/Makefile | 49 +++
lib/librte_member/rte_member.c | 336 ++++++++++++++++++++
lib/librte_member/rte_member.h | 507 +++++++++++++++++++++++++++++++
lib/librte_member/rte_member_version.map | 16 +
5 files changed, 910 insertions(+)
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_version.map
diff --git a/lib/Makefile b/lib/Makefile
index 86caba1..c82033a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -108,6 +108,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder
DEPDIRS-librte_reorder := librte_eal librte_mempool librte_mbuf
DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump
DEPDIRS-librte_pdump := librte_eal librte_mempool librte_mbuf librte_ether
+DIRS-$(CONFIG_RTE_LIBRTE_MEMBER) += librte_member
+DEPDIRS-librte_member := librte_eal librte_hash
ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
new file mode 100644
index 0000000..1a79eaa
--- /dev/null
+++ b/lib/librte_member/Makefile
@@ -0,0 +1,49 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_member.a
+
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
+
+EXPORT_MAP := rte_member_version.map
+
+LIBABIVER := 1
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+# install includes
+SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_member/rte_member.c b/lib/librte_member/rte_member.c
new file mode 100644
index 0000000..fee75a3
--- /dev/null
+++ b/lib/librte_member/rte_member.c
@@ -0,0 +1,336 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+
+#include <rte_eal.h>
+#include <rte_eal_memconfig.h>
+#include <rte_memory.h>
+#include <rte_memzone.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+#include "rte_member_vbf.h"
+
+int librte_member_logtype;
+
+TAILQ_HEAD(rte_member_list, rte_tailq_entry);
+static struct rte_tailq_elem rte_member_tailq = {
+ .name = "RTE_MEMBER",
+};
+EAL_REGISTER_TAILQ(rte_member_tailq)
+
+struct rte_member_setsum *
+rte_member_find_existing(const char *name)
+{
+ struct rte_member_setsum *setsum = NULL;
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(name, setsum->name, RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ if (te == NULL) {
+ rte_errno = ENOENT;
+ return NULL;
+ }
+ return setsum;
+}
+
+void
+rte_member_free(struct rte_member_setsum *setsum)
+{
+ struct rte_member_list *member_list;
+ struct rte_tailq_entry *te;
+
+ if (setsum == NULL)
+ return;
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ if (te->data == (void *)setsum)
+ break;
+ }
+ if (te == NULL) {
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return;
+ }
+ TAILQ_REMOVE(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_free_ht(setsum);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_free_vbf(setsum);
+ break;
+ default:
+ break;
+ }
+ rte_free(setsum);
+ rte_free(te);
+}
+
+struct rte_member_setsum *
+rte_member_create(const struct rte_member_parameters *params)
+{
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+ struct rte_member_setsum *setsum;
+ int ret;
+
+ if (params == NULL) {
+ rte_errno = EINVAL;
+ return NULL;
+ }
+
+ if (params->key_len == 0 ||
+ params->prim_hash_seed == params->sec_hash_seed) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "Memship create with invalid parameters\n");
+ return NULL;
+ }
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(params->name, setsum->name,
+ RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ setsum = NULL;
+ if (te != NULL) {
+ rte_errno = EEXIST;
+ te = NULL;
+ goto error_unlock_exit;
+ }
+ te = rte_zmalloc("MEMBER_TAILQ_ENTRY", sizeof(*te), 0);
+ if (te == NULL) {
+ RTE_MEMBER_LOG(ERR, "tailq entry allocation failed\n");
+ goto error_unlock_exit;
+ }
+
+ /* Create a new setsum structure */
+ setsum = (struct rte_member_setsum *) rte_zmalloc_socket(params->name,
+ sizeof(struct rte_member_setsum), RTE_CACHE_LINE_SIZE,
+ params->socket_id);
+ if (setsum == NULL) {
+ RTE_MEMBER_LOG(ERR, "Create setsummary failed\n");
+ goto error_unlock_exit;
+ }
+ snprintf(setsum->name, sizeof(setsum->name), "%s", params->name);
+ setsum->type = params->type;
+ setsum->socket_id = params->socket_id;
+ setsum->key_len = params->key_len;
+ setsum->num_set = params->num_set;
+ setsum->prim_hash_seed = params->prim_hash_seed;
+ setsum->sec_hash_seed = params->sec_hash_seed;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ ret = rte_member_create_ht(setsum, params);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ ret = rte_member_create_vbf(setsum, params);
+ break;
+ default:
+ goto error_unlock_exit;
+ }
+ if (ret < 0)
+ goto error_unlock_exit;
+
+ RTE_MEMBER_LOG(DEBUG, "Creating a setsummary table with "
+ "mode %u\n", setsum->type);
+
+ te->data = (void *)setsum;
+ TAILQ_INSERT_TAIL(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return setsum;
+
+error_unlock_exit:
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ rte_member_free(setsum);
+ return NULL;
+}
+
+int
+rte_member_add(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id)
+{
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_add_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_add_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t *set_id)
+{
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids)
+{
+ if (setsum == NULL || keys == NULL || set_ids == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_bulk_ht(setsum, keys, num_keys,
+ set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_bulk_vbf(setsum, keys, num_keys,
+ set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi(const struct rte_member_setsum *setsum, const void *key,
+ uint32_t match_per_key, member_set_t *set_id)
+{
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_ht(setsum, key, match_per_key,
+ set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_vbf(setsum, key, match_per_key,
+ set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key, uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ if (setsum == NULL || keys == NULL || set_ids == NULL ||
+ match_count == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_bulk_ht(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_bulk_vbf(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_delete(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id)
+{
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_delete_ht(setsum, key, set_id);
+ /* current vBF implementation does not support delete function */
+ case RTE_MEMBER_TYPE_VBF:
+ default:
+ return -EINVAL;
+ }
+}
+
+void
+rte_member_reset(const struct rte_member_setsum *setsum)
+{
+ if (setsum == NULL)
+ return;
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_reset_ht(setsum);
+ return;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_reset_vbf(setsum);
+ return;
+ default:
+ return;
+ }
+}
+
+RTE_INIT(librte_member_init_log);
+
+static void
+librte_member_init_log(void)
+{
+ librte_member_logtype = rte_log_register("librte.member");
+ if (librte_member_logtype >= 0)
+ rte_log_set_level(librte_member_logtype, RTE_LOG_DEBUG);
+}
diff --git a/lib/librte_member/rte_member.h b/lib/librte_member/rte_member.h
new file mode 100644
index 0000000..48c6de7
--- /dev/null
+++ b/lib/librte_member/rte_member.h
@@ -0,0 +1,507 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/**
+ * @file
+ *
+ * RTE Membership Library
+ *
+ * The Membership Library is an extension and generalization of a traditional
+ * filter (for example Bloom Filter) structure that has multiple usages in a
+ * variety of workloads and applications. The library is used to test if a key
+ * belongs to certain sets. Two types of such "set-summary" structures are
+ * implemented: hash-table based (HT) and vector bloom filter (vBF). For HT
+ * setsummary, two subtype or modes are available, cache and non-cache modes.
+ * The table below summarize some properties of the different implementations.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ */
+
+/**
+ * <!--
+ * +==========+=====================+================+=========================+
+ * | type | vbf | HT-cache | HT-non-cache |
+ * +==========+=====================+==========================================+
+ * |structure | bloom-filter array | hash-table like without storing key |
+ * +----------+---------------------+------------------------------------------+
+ * |set id | limited by bf count | [1, 0x7fff] |
+ * | | up to 32. | |
+ * +----------+---------------------+------------------------------------------+
+ * |usages & | small set range, | can delete, | cache most recent keys, |
+ * |properties| user-specified | big set range, | have both false-positive|
+ * | | false-positive rate,| small false | and false-negative |
+ * | | no deletion support.| positive depend| depend on table size, |
+ * | | | on table size, | automatic overwritten. |
+ * | | | new key does | |
+ * | | | not overwrite | |
+ * | | | existing key. | |
+ * +----------+---------------------+----------------+-------------------------+
+ * -->
+ */
+
+#ifndef _RTE_MEMBER_H_
+#define _RTE_MEMBER_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+
+/** The set ID type that stored internally in hash table based set summary. */
+typedef uint16_t member_set_t;
+/** Invalid set ID used to mean no match found. */
+#define RTE_MEMBER_NO_MATCH 0
+/** Maximum size of hash table that can be created. */
+#define RTE_MEMBER_ENTRIES_MAX (1 << 30)
+/** Maximum number of keys that can be searched as a bulk */
+#define RTE_MEMBER_LOOKUP_BULK_MAX 64
+/** Entry count per bucket in hash table based mode. */
+#define RTE_MEMBER_BUCKET_ENTRIES 16
+/** Maximum number of characters in setsum name. */
+#define RTE_MEMBER_NAMESIZE 32
+
+/** @internal Hash function used by membership library. */
+#if defined(RTE_ARCH_X86) || defined(RTE_MACHINE_CPUFLAG_CRC32)
+#include <rte_hash_crc.h>
+#define MEMBER_HASH_FUNC rte_hash_crc
+#else
+#include <rte_jhash.h>
+#define MEMBER_HASH_FUNC rte_jhash
+#endif
+
+extern int librte_member_logtype;
+
+#define RTE_MEMBER_LOG(level, fmt, args...) \
+rte_log(RTE_LOG_ ## level, librte_member_logtype, "%s(): " fmt, \
+ __func__, ## args)
+
+/** @internal setsummary structure. */
+struct rte_member_setsum;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameter struct used to create set summary
+ */
+struct rte_member_parameters;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Define different set summary types
+ */
+enum rte_member_setsum_type {
+ RTE_MEMBER_TYPE_HT = 0, /**< Hash table based set summary. */
+ RTE_MEMBER_TYPE_VBF, /**< Vector of bloom filters. */
+ RTE_MEMBER_NUM_TYPE
+};
+
+/** @internal compare function for different arch. */
+enum rte_member_sig_compare_function {
+ RTE_MEMBER_COMPARE_SCALAR = 0,
+ RTE_MEMBER_COMPARE_AVX2,
+ RTE_MEMBER_COMPARE_NUM
+};
+
+/** @internal setsummary structure. */
+struct rte_member_setsum {
+ enum rte_member_setsum_type type; /* Type of the set summary. */
+ uint32_t key_len; /* Length of key. */
+ uint32_t prim_hash_seed; /* Primary hash function seed. */
+ uint32_t sec_hash_seed; /* Secondary hash function seed. */
+
+ /* Hash table based. */
+ uint32_t bucket_cnt; /* Number of buckets. */
+ uint32_t bucket_mask; /* Bit mask to get bucket index. */
+ /* For runtime selecting AVX, scalar, etc for signature comparison. */
+ enum rte_member_sig_compare_function sig_cmp_fn;
+ uint8_t cache; /* If it is cache mode for ht based. */
+
+ /* Vector bloom filter. */
+ uint32_t num_set; /* Number of set (bf) in vbf. */
+ uint32_t bits; /* Number of bits in each bf. */
+ uint32_t bit_mask; /* Bit mask to get bit location in bf. */
+ uint32_t num_hashes; /* Number of hash values to index bf. */
+
+ uint32_t mul_shift; /* vbf internal variable used during bit test. */
+ uint32_t div_shift; /* vbf internal variable used during bit test. */
+
+ void *table; /* This is the handler of hash table or vBF array. */
+
+
+ /* Second cache line should start here. */
+ uint32_t socket_id; /* NUMA Socket ID for memory. */
+ char name[RTE_MEMBER_NAMESIZE]; /* Name of this set summary. */
+} __rte_cache_aligned;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameters used when create the set summary table. Currently user can
+ * specify two types of setsummary: HT based and vBF. For HT based, user can
+ * specify cache or non-cache mode. Here is a table to describe some differences
+ *
+ */
+struct rte_member_parameters {
+ const char *name; /**< Name of the hash. */
+
+ /**
+ * User to specify the type of the setsummary from one of
+ * rte_member_setsum_type.
+ *
+ * HT based setsummary is implemented like a hash table. User should use
+ * this type when there are many sets.
+ *
+ * vBF setsummary is a vector of bloom filters. It is used when number
+ * of sets is not big (less than 32 for current implementation).
+ */
+ enum rte_member_setsum_type type;
+
+ /**
+ * If it is HT based setsummary, user to specify the subtype or mode
+ * of the setsummary. It could be cache, or non-cache mode.
+ * Set iscache to be 1 if to use as cache mode.
+ *
+ * For cache mode, keys can be evicted out of the HT setsummary. Keys
+ * with the same signature and map to the same bucket
+ * will overwrite each other in the setsummary table.
+ * This mode is useful for the case that the set-summary only
+ * needs to keep record of the recently inserted keys. Both
+ * false-negative and false-positive could happen.
+ *
+ * For non-cache mode, keys cannot be evicted out of the cache. So for
+ * this mode the setsummary will become full eventually. Keys with the
+ * same signature but map to the same bucket will still occupy multiple
+ * entries. This mode does not give false-negative result.
+ */
+ uint8_t is_cache;
+
+ /**
+ * For HT setsummary, num_keys equals to the number of entries of the
+ * table. When the number of keys that inserted in the HT setsummary
+ * approaches this number, eviction could happen. For cache mode,
+ * keys could be evicted out of the table. For non-cache mode, keys will
+ * be evicted to other buckets like cuckoo hash. The table will also
+ * likely to become full before the number of inserted keys equal to the
+ * total number of entries.
+ *
+ * For vBF, num_keys equal to the expected number of keys that will
+ * be inserted into the vBF. The implementation assumes the keys are
+ * evenly distributed to each BF in vBF. This is used to calculate the
+ * number of bits we need for each BF. User does not specify the size of
+ * each BF directly because the optimal size depends on the num_keys
+ * and false positive rate.
+ */
+ uint32_t num_keys;
+
+ /**
+ * The length of key is used for hash calculation. Since key is not
+ * stored in set-summary, large key does not require more memory space.
+ */
+ uint32_t key_len;
+
+ /**
+ * num_set is only relevant to vBF based setsummary.
+ * num_set is equal to the number of BFs in vBF. For current
+ * implementation, it only supports 1,2,4,8,16,32 BFs in one vBF set
+ * summary. If other number of sets are needed, for example 5, the user
+ * should allocate the minimum available value that larger than 5,
+ * which is 8.
+ */
+ uint32_t num_set;
+
+ /**
+ * false_positive_rate is only relevant to vBF based setsummary.
+ * false_positive_rate is the user-defined false positive rate
+ * given expected number of inserted keys (num_keys). It is used to
+ * calculate the total number of bits for each BF, and the number of
+ * hash values used during lookup and insertion. For details please
+ * refer to vBF implementation and membership library documentation.
+ * Note that this parameter is not directly set by users for HT mode.
+ *
+ * HT setsummary's false positive rate is in the order of:
+ * false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit signature.
+ * This is because two keys needs to map to same bucket and same
+ * signature to have a collision (false positive). bucket_count is equal
+ * to number of entries (num_keys) divided by entry count per bucket
+ * (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_positive_rate is not
+ * directly set by users.
+ */
+ float false_positive_rate;
+
+ /**
+ * We use two seeds to calculate two independent hashes for each key.
+ *
+ * For HT type, one hash is used as signature, and the other is used
+ * for bucket location.
+ * For vBF type, these two hashes and their combinations are used as
+ * hash locations to index the bit array.
+ */
+ uint32_t prim_hash_seed;
+
+ /**
+ * The secondary seed should be a different value from the primary seed.
+ */
+ uint32_t sec_hash_seed;
+
+ int socket_id; /**< NUMA Socket ID for memory. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Find an existing set-summary and return a pointer to it.
+ *
+ * @param name
+ * Name of the set-summary.
+ * @return
+ * Pointer to the set-summary or NULL if object not found
+ * with rte_errno set appropriately. Possible rte_errno values include:
+ * - ENOENT - value not available for return
+ */
+struct rte_member_setsum *
+rte_member_find_existing(const char *name);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Create set-summary (SS).
+ *
+ * @param params
+ * Parameters to initialize the setsummary.
+ * @return
+ * Return the pointer to the setsummary.
+ * Return value is NULL if the creation failed.
+ */
+struct rte_member_setsum *
+rte_member_create(const struct rte_member_parameters *params);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup key in set-summary (SS).
+ * Single key lookup and return as soon as the first match found
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param key
+ * Pointer of the key to be looked up.
+ * @param set_id
+ * Output the set id matches the key.
+ * @return
+ * Return 1 for found a match and 0 for not found a match.
+ */
+int
+rte_member_lookup(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t *set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * lookup bulk of keys in set-summary (SS).
+ * Each key lookup returns as soon as the first match found
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param keys
+ * Pointer of the bulk of keys to be looked up.
+ * @param num_keys
+ * Number of keys that will be lookup.
+ * @param set_ids
+ * Output set ids for all the keys to this array.
+ * User should preallocate array that can contain all results, which size is
+ * the num_keys.
+ * @return
+ * The number of keys that found a match.
+ */
+int
+rte_member_lookup_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a key in set-summary (SS) for multiple matches.
+ * The key lookup will find all matched entries (multiple match).
+ * Note that for cache mode of HT, each key can have at most one match. This is
+ * because keys with same signature that maps to same bucket will overwrite
+ * each other. So multi-match lookup should be used for vBF and non-cache HT.
+ *
+ * @param setsum
+ * Pointer of a set-summary.
+ * @param key
+ * Pointer of the key that to be looked up.
+ * @param max_match_per_key
+ * User specified maximum number of matches for each key. The function returns
+ * as soon as this number of matches found for the key.
+ * @param set_id
+ * Output set ids for all the matches of the key. User needs to preallocate
+ * the array that can contain max_match_per_key number of results.
+ * @return
+ * The number of matches that found for the key.
+ * For cache mode HT set-summary, the number should be at most 1.
+ */
+int
+rte_member_lookup_multi(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t max_match_per_key,
+ member_set_t *set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a bulk of keys in set-summary (SS) for multiple matches each key.
+ * Each key lookup will find all matched entries (multiple match).
+ * Note that for cache mode HT, each key can have at most one match. So
+ * multi-match function is mainly used for vBF and non-cache mode HT.
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param keys
+ * Pointer of the keys to be looked up.
+ * @param num_keys
+ * The number of keys that will be lookup.
+ * @param max_match_per_key
+ * The possible maximum number of matches for each key.
+ * @param match_count
+ * Output the number of matches for each key in an array.
+ * @param set_ids
+ * Return set ids for all the matches of all keys. Users pass in a
+ * preallocated 2D array with first dimension as key index and second
+ * dimension as match index. For example set_ids[bulk_size][max_match_per_key]
+ * @return
+ * The number of keys that found one or more matches in the set-summary.
+ */
+int
+rte_member_lookup_multi_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Insert key into set-summary (SS).
+ *
+ * @param setsum
+ * Pointer of a set-summary.
+ * @param key
+ * Pointer of the key to be added.
+ * @param set_id
+ * The set id associated with the key that needs to be added. Different mode
+ * supports different set_id ranges. 0 cannot be used as set_id since
+ * RTE_MEMBER_NO_MATCH by default is set as 0.
+ * For HT mode, the set_id has range as [1, 0x7FFF], MSB is reserved.
+ * For vBF mode the set id is limited by the num_set parameter when create
+ * the set-summary.
+ * @return
+ * HT (cache mode) and vBF should never fail unless the set_id is not in the
+ * valid range. In such case -EINVAL is returned.
+ * For HT (non-cache mode) it could fail with -ENOSPC error code when table is
+ * full.
+ * For success it returns different values for different modes to provide
+ * extra information for users.
+ * Return 0 for HT (cache mode) if the add does not cause
+ * eviction, return 1 otherwise. Return 0 for non-cache mode if success,
+ * -ENOSPC for full, and 1 if cuckoo eviction happens.
+ * Always returns 0 for vBF mode.
+ */
+int
+rte_member_add(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * De-allocate memory used by set-summary.
+ *
+ * @param setsum
+ * Pointer to the set summary.
+ */
+void
+rte_member_free(struct rte_member_setsum *setsum);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Reset the set-summary tables. E.g. reset bits to be 0 in BF,
+ * reset set_id in each entry to be RTE_MEMBER_NO_MATCH in HT based SS.
+ *
+ * @param setsum
+ * Pointer to the set-summary.
+ */
+void
+rte_member_reset(const struct rte_member_setsum *setsum);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Delete items from the set-summary. Note that vBF does not support deletion
+ * in current implementation. For vBF, error code of -EINVAL will be returned.
+ *
+ * @param setsum
+ * Pointer to the set-summary.
+ * @param key
+ * Pointer of the key to be deleted.
+ * @param set_id
+ * For HT mode, we need both key and its corresponding set_id to
+ * properly delete the key. Without set_id, we may delete other keys with the
+ * same signature.
+ * @return
+ * If no entry found to delete, an error code of -ENOENT could be returned.
+ */
+int
+rte_member_delete(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_H_ */
diff --git a/lib/librte_member/rte_member_version.map b/lib/librte_member/rte_member_version.map
new file mode 100644
index 0000000..019e4cd
--- /dev/null
+++ b/lib/librte_member/rte_member_version.map
@@ -0,0 +1,16 @@
+DPDK_17.11 {
+ global:
+
+ rte_member_add;
+ rte_member_create;
+ rte_member_delete;
+ rte_member_find_existing;
+ rte_member_free;
+ rte_member_lookup;
+ rte_member_lookup_bulk;
+ rte_member_lookup_multi;
+ rte_member_lookup_multi_bulk;
+ rte_member_reset;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v4 1/7] member: implement main API
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 1/7] member: implement main API Yipeng Wang
@ 2017-10-02 10:04 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-02 10:04 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
Hi Yipeng,
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Wednesday, September 27, 2017 6:40 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v4 1/7] member: implement main API
>
> Membership library is an extension and generalization of a traditional filter
> (for example Bloom Filter) structure. In general, the Membership library is a
> data structure that provides a "set-summary" and responds to set-
> membership queries of whether a certain element belongs to a set(s). A
> membership test for an element will return the set this element belongs to
> or not-found if the element is never inserted into the set-summary.
>
> The results of the membership test are not 100% accurate. Certain false
> positive or false negative probability could exist. However, comparing to a
> "full-blown" complete list of elements, a "set-summary"
> is memory efficient and fast on lookup.
>
> This patch adds the main API definition.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
A few comments on changes that you didn't make in the v4.
Thanks,
Pablo
...
> +
> +struct rte_member_setsum *
> +rte_member_create(const struct rte_member_parameters *params) {
> + struct rte_tailq_entry *te;
> + struct rte_member_list *member_list;
> + struct rte_member_setsum *setsum;
> + int ret;
> +
> + if (params == NULL) {
> + rte_errno = EINVAL;
> + return NULL;
> + }
> +
> + if (params->key_len == 0 ||
> + params->prim_hash_seed == params-
> >sec_hash_seed) {
> + rte_errno = EINVAL;
> + RTE_MEMBER_LOG(ERR, "Memship create with invalid
> parameters\n");
Do not use "Memship". Change to " rte_member_create has invalid parameters"?
Or something else that you want, but not Memship.
> + return NULL;
> + }
> +
...
> +struct rte_member_parameters {
> + const char *name; /**< Name of the hash. */
> +
> + /**
> + * User to specify the type of the setsummary from one of
> + * rte_member_setsum_type.
> + *
> + * HT based setsummary is implemented like a hash table. User
> should use
> + * this type when there are many sets.
> + *
> + * vBF setsummary is a vector of bloom filters. It is used when
> number
> + * of sets is not big (less than 32 for current implementation).
> + */
> + enum rte_member_setsum_type type;
> +
> + /**
> + * If it is HT based setsummary, user to specify the subtype or mode
> + * of the setsummary. It could be cache, or non-cache mode.
> + * Set iscache to be 1 if to use as cache mode.
Change to "is_cache".
> + *
> + * For cache mode, keys can be evicted out of the HT setsummary.
> Keys
> + * with the same signature and map to the same bucket
> + * will overwrite each other in the setsummary table.
> + * This mode is useful for the case that the set-summary only
> + * needs to keep record of the recently inserted keys. Both
> + * false-negative and false-positive could happen.
> + *
> + * For non-cache mode, keys cannot be evicted out of the cache. So
> for
> + * this mode the setsummary will become full eventually. Keys with
> the
> + * same signature but map to the same bucket will still occupy
> multiple
> + * entries. This mode does not give false-negative result.
> + */
> + uint8_t is_cache;
> +
> + /**
> + * For HT setsummary, num_keys equals to the number of entries
> of the
> + * table. When the number of keys that inserted in the HT
> setsummary
"number of keys inserted in the HT summary". Or "were inserted".
> + * approaches this number, eviction could happen. For cache mode,
> + * keys could be evicted out of the table. For non-cache mode, keys
> will
...
> + /**
> + * false_positive_rate is only relevant to vBF based setsummary.
> + * false_positive_rate is the user-defined false positive rate
> + * given expected number of inserted keys (num_keys). It is used to
> + * calculate the total number of bits for each BF, and the number of
> + * hash values used during lookup and insertion. For details please
> + * refer to vBF implementation and membership library
> documentation.
> + * Note that this parameter is not directly set by users for HT mode.
> + *
> + * HT setsummary's false positive rate is in the order of:
> + * false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit
> signature.
> + * This is because two keys needs to map to same bucket and same
> + * signature to have a collision (false positive). bucket_count is
> equal
> + * to number of entries (num_keys) divided by entry count per
> bucket
> + * (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_positive_rate
> is not
> + * directly set by users.
Unless I understood it incorrectly, I think you should clarify that
this field is not set for HT mode, but it is for vBF, right?
> + */
> + float false_positive_rate;
> +
> + /**
> + * We use two seeds to calculate two independent hashes for each
> key.
> + *
> + * For HT type, one hash is used as signature, and the other is used
> + * for bucket location.
> + * For vBF type, these two hashes and their combinations are used
> as
> + * hash locations to index the bit array.
> + */
> + uint32_t prim_hash_seed;
> +
> + /**
> + * The secondary seed should be a different value from the primary
> seed.
> + */
> + uint32_t sec_hash_seed;
> +
> + int socket_id; /**< NUMA Socket ID for memory.
> */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Find an existing set-summary and return a pointer to it.
> + *
> + * @param name
> + * Name of the set-summary.
> + * @return
> + * Pointer to the set-summary or NULL if object not found
> + * with rte_errno set appropriately. Possible rte_errno values include:
> + * - ENOENT - value not available for return
> + */
> +struct rte_member_setsum *
> +rte_member_find_existing(const char *name);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Create set-summary (SS).
> + *
> + * @param params
> + * Parameters to initialize the setsummary.
> + * @return
> + * Return the pointer to the setsummary.
> + * Return value is NULL if the creation failed.
> + */
> +struct rte_member_setsum *
> +rte_member_create(const struct rte_member_parameters *params);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Lookup key in set-summary (SS).
> + * Single key lookup and return as soon as the first match found
> + *
> + * @param setsum
> + * Pointer of a setsummary.
> + * @param key
> + * Pointer of the key to be looked up.
> + * @param set_id
> + * Output the set id matches the key.
> + * @return
> + * Return 1 for found a match and 0 for not found a match.
> + */
> +int
> +rte_member_lookup(const struct rte_member_setsum *setsum, const
> void *key,
> + member_set_t *set_id);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * lookup bulk of keys in set-summary (SS).
"Lookup bulk"
> + * Each key lookup returns as soon as the first match found
> + *
> + * @param setsum
> + * Pointer of a setsummary.
> + * @param keys
> + * Pointer of the bulk of keys to be looked up.
> + * @param num_keys
> + * Number of keys that will be lookup.
> + * @param set_ids
> + * Output set ids for all the keys to this array.
> + * User should preallocate array that can contain all results, which size is
> + * the num_keys.
> + * @return
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v4 2/7] member: implement HT mode
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 1/7] member: implement main API Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-10-02 13:30 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 3/7] member: implement vBF mode Yipeng Wang
` (5 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
One of the set-summary structures is hash-table based
set-summary (HTSS). One example is cuckoo filter [1].
Comparing to a traditional hash table, HTSS has a much more
compact structure. For each element, only one signature and
its corresponding set ID is stored. No key comparison is required
during lookup. For the table structure, there are multiple entries
in each bucket, and the table is composed of many buckets.
Two modes are supported for HTSS, "cache" and "none-cache" modes.
The non-cache mode is more similar to traditional cuckoo filter.
When a bucket is full, one entry will be evicted to its
alternative bucket to make space for the new key. The table could
be full and then no more keys could be inserted. This mode has
false-positive rate but no false-negative. Multiple entries
with same signature could stay in the same bucket.
The "cache" mode does not evict key to its alternative bucket
when a bucket is full, an existing key will be evicted out of
the table like a cache. Thus, the table will never reject keys when
it is full. Another property is in each bucket, there cannot be
multiple entries with same signature. The mode could have both
false-positive and false-negative probability.
This patch adds the implementation of HTSS.
[1] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically
Better Than Bloom,” in Conference on emerging Networking
Experiments and Technologies, 2014.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_ht.c | 474 ++++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_ht.h | 94 ++++++++
3 files changed, 569 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 1a79eaa..ad26548 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
new file mode 100644
index 0000000..55672a4
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.c
@@ -0,0 +1,474 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_errno.h>
+#include <rte_malloc.h>
+#include <rte_prefetch.h>
+#include <rte_random.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+
+static inline int
+insert_overwrite_search(uint32_t bucket, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t set_id)
+{
+ int i;
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[bucket].sigs[i] == tmp_sig) {
+ buckets[bucket].sets[i] = set_id;
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static inline int
+search_bucket_single(uint32_t bucket, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t *set_id)
+{
+ int iter;
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket].sigs[iter] &&
+ buckets[bucket].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket].sets[iter];
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi(uint32_t bucket, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ int iter;
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket].sigs[iter] &&
+ buckets[bucket].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket].sets[iter];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ }
+}
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+ uint32_t i, j;
+ uint32_t size_bucket_t;
+ uint32_t num_entries = rte_align32pow2(params->num_keys);
+
+ if ((num_entries > RTE_MEMBER_ENTRIES_MAX) ||
+ !rte_is_power_of_2(RTE_MEMBER_BUCKET_ENTRIES) ||
+ num_entries < RTE_MEMBER_BUCKET_ENTRIES) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR,
+ "Membership HT create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ uint32_t num_buckets = num_entries / RTE_MEMBER_BUCKET_ENTRIES;
+
+ size_bucket_t = sizeof(struct member_ht_bucket);
+
+ struct member_ht_bucket *buckets = rte_zmalloc_socket(NULL,
+ num_buckets * size_bucket_t,
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ if (buckets == NULL) {
+ RTE_MEMBER_LOG(ERR, "memory allocation failed for HT "
+ "setsummary\n");
+ return -ENOMEM;
+ }
+
+ ss->table = buckets;
+ ss->bucket_cnt = num_buckets;
+ ss->bucket_mask = num_buckets - 1;
+ ss->cache = params->is_cache;
+
+ for (i = 0; i < num_buckets; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+
+ RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
+ "the table has %u entries, %u buckets\n",
+ num_buckets,
+ num_buckets / RTE_MEMBER_BUCKET_ENTRIES);
+ return 0;
+}
+
+static inline
+void get_buckets_index(const struct rte_member_setsum *ss, const void *key,
+ uint32_t *prim_bkt, uint32_t *sec_bkt, member_sig_t *sig)
+{
+ uint32_t first_hash = MEMBER_HASH_FUNC(key, ss->key_len,
+ ss->prim_hash_seed);
+ uint32_t sec_hash = MEMBER_HASH_FUNC(&first_hash, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ *sig = first_hash;
+ if (ss->cache) {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (sec_hash >> 16) & ss->bucket_mask;
+ } else {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (*prim_bkt ^ *sig) & ss->bucket_mask;
+ }
+}
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *ss,
+ const void *key, member_set_t *set_id)
+{
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+
+ return 0;
+}
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, member_set_t *set_id)
+{
+ uint32_t i;
+ uint32_t ret = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+
+ for (i = 0; i < num_keys; i++) {
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return ret;
+}
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t ret = 0;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
+ match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+}
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ uint32_t i;
+ uint32_t ret = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t match_cnt_tmp;
+ member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_tmp = 0;
+
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ ret++;
+ }
+ return ret;
+}
+
+static inline int
+try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ member_sig_t sig, member_set_t set_id)
+{
+ int i;
+ /* If not full then insert into one slot*/
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[prim].sigs[i] = sig;
+ buckets[prim].sets[i] = set_id;
+ return 0;
+ }
+ }
+ /* if prim failed, we need to access second cache line */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[sec].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[sec].sigs[i] = sig;
+ buckets[sec].sets[i] = set_id;
+ return 0;
+ }
+ }
+ return -1;
+}
+
+static inline int
+try_overwrite(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ member_sig_t sig, member_set_t set_id)
+{
+ if (insert_overwrite_search(prim, sig, buckets, set_id) ||
+ insert_overwrite_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ return -1;
+}
+
+static inline int
+evict_from_bucket(void)
+{
+ /* for now, we randomly pick one entry to evict */
+ return rte_rand() & (RTE_MEMBER_BUCKET_ENTRIES - 1);
+}
+
+/*
+ * This function is similar to the cuckoo hash make_space function in hash
+ * library
+ */
+static inline int
+make_space_bucket(const struct rte_member_setsum *ss, uint32_t bkt_num)
+{
+ static unsigned int nr_pushes;
+ unsigned int i, j;
+ int ret;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t next_bucket_idx;
+ struct member_ht_bucket *next_bkt[RTE_MEMBER_BUCKET_ENTRIES];
+ struct member_ht_bucket *bkt = &buckets[bkt_num];
+ member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
+ /*
+ * Push existing item (search for bucket with space in
+ * alternative locations) to its alternative location
+ */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ /* Search for space in alternative locations */
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
+ next_bkt[i] = &buckets[next_bucket_idx];
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++) {
+ if (next_bkt[i]->sets[j] == RTE_MEMBER_NO_MATCH)
+ break;
+ }
+
+ if (j != RTE_MEMBER_BUCKET_ENTRIES)
+ break;
+ }
+
+ /* Alternative location has spare room (end of recursive function) */
+ if (i != RTE_MEMBER_BUCKET_ENTRIES) {
+ next_bkt[i]->sigs[j] = bkt->sigs[i];
+ next_bkt[i]->sets[j] = bkt->sets[i];
+ return i;
+ }
+
+ /* Pick entry that has not been pushed yet */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++)
+ if ((bkt->sets[i] & flag_mask) == 0)
+ break;
+
+ /* All entries have been pushed, so entry cannot be added */
+ if (i == RTE_MEMBER_BUCKET_ENTRIES ||
+ nr_pushes > RTE_MEMBER_MAX_PUSHES)
+ return -ENOSPC;
+
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
+ /* Set flag to indicate that this entry is going to be pushed */
+ bkt->sets[i] |= flag_mask;
+
+ nr_pushes++;
+ /* Need room in alternative bucket to insert the pushed entry */
+ ret = make_space_bucket(ss, next_bucket_idx);
+ /*
+ * After recursive function.
+ * Clear flags and insert the pushed entry
+ * in its alternative location if successful,
+ * or return error
+ */
+ bkt->sets[i] &= ~flag_mask;
+ nr_pushes = 0;
+ if (ret >= 0) {
+ next_bkt[i]->sigs[ret] = bkt->sigs[i];
+ next_bkt[i]->sets[ret] = bkt->sets[i];
+ return i;
+ } else
+ return ret;
+}
+
+int
+rte_member_add_ht(const struct rte_member_setsum *ss,
+ const void *key, member_set_t set_id)
+{
+ int ret;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+ member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
+
+ if (set_id == RTE_MEMBER_NO_MATCH || (set_id & flag_mask) != 0)
+ return -EINVAL;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ /* if it is cache based filter, we try overwriting existing entry */
+ if (ss->cache) {
+ ret = try_overwrite(buckets, prim_bucket, sec_bucket, tmp_sig,
+ set_id);
+ if (ret != -1)
+ return ret;
+ }
+ /* If not full then insert into one slot*/
+ ret = try_insert(buckets, prim_bucket, sec_bucket, tmp_sig, set_id);
+ if (ret != -1)
+ return ret;
+
+ /* random pick prim or sec for recursive displacement */
+
+ uint32_t select_bucket = (tmp_sig && 1U) ? prim_bucket : sec_bucket;
+ if (ss->cache) {
+ ret = evict_from_bucket();
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ return 1;
+ }
+
+ ret = make_space_bucket(ss, select_bucket);
+ if (ret >= 0) {
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ ret = 1;
+ }
+
+ return ret;
+}
+
+void
+rte_member_free_ht(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+int
+rte_member_delete_ht(const struct rte_member_setsum *ss, const void *key,
+ member_set_t set_id)
+{
+ int i;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[prim_bucket].sigs[i] &&
+ set_id == buckets[prim_bucket].sets[i]) {
+ buckets[prim_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[sec_bucket].sigs[i] &&
+ set_id == buckets[sec_bucket].sets[i]) {
+ buckets[sec_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+ return -ENOENT;
+}
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *ss)
+{
+ uint32_t i, j;
+ struct member_ht_bucket *buckets = ss->table;
+ for (i = 0; i < ss->bucket_cnt; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+}
diff --git a/lib/librte_member/rte_member_ht.h b/lib/librte_member/rte_member_ht.h
new file mode 100644
index 0000000..3148a49
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.h
@@ -0,0 +1,94 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_HT_H_
+#define _RTE_MEMBER_HT_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Maximum number of pushes for cuckoo path in HT mode. */
+#define RTE_MEMBER_MAX_PUSHES 50
+
+typedef uint16_t member_sig_t; /* signature size is 16 bit */
+
+/* The bucket struct for ht setsum */
+struct member_ht_bucket {
+ member_sig_t sigs[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte signature */
+ member_set_t sets[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte set */
+} __rte_cache_aligned;
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+int
+rte_member_add_ht(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t set_id);
+
+void
+rte_member_free_ht(struct rte_member_setsum *setsum);
+
+int
+rte_member_delete_ht(const struct rte_member_setsum *ss, const void *key,
+ member_set_t set_id);
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *setsum);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_HT_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v4 2/7] member: implement HT mode
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 2/7] member: implement HT mode Yipeng Wang
@ 2017-10-02 13:30 ` De Lara Guarch, Pablo
2017-10-03 1:18 ` Wang, Yipeng1
0 siblings, 1 reply; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-02 13:30 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Wednesday, September 27, 2017 6:40 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v4 2/7] member: implement HT mode
>
...
> diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile index
> 1a79eaa..ad26548 100644
> --- a/lib/librte_member/Makefile
> +++ b/lib/librte_member/Makefile
> @@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
> LIBABIVER := 1
>
> # all source are stored in SRCS-y
> -SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
> +SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
> rte_member_ht.c
> # install includes
> SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
>
> diff --git a/lib/librte_member/rte_member_ht.c
> b/lib/librte_member/rte_member_ht.c
> new file mode 100644
> index 0000000..55672a4
> --- /dev/null
> +++ b/lib/librte_member/rte_member_ht.c
...
> +
> +static inline int
> +insert_overwrite_search(uint32_t bucket, member_sig_t tmp_sig,
> + struct member_ht_bucket *buckets,
> + member_set_t set_id)
I would call "bucket", "bucket_id", for better understanding.
This comment also applies to other parts of the code (e.g. prim_buckets).
> +{
> + int i;
> + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> + if (buckets[bucket].sigs[i] == tmp_sig) {
> + buckets[bucket].sets[i] = set_id;
> + return 1;
> + }
Is this function used to update an existing entry?
At first, I thought that this was evicting another entry, when the bucket was full,
but it looks like it is updating the set_id of an existing entry.
If this is the case, I would change the function name and add a comment explaining this.
> + }
> + return 0;
> +}
> +
> +static inline int
> +search_bucket_single(uint32_t bucket, member_sig_t tmp_sig,
> + struct member_ht_bucket *buckets,
> + member_set_t *set_id)
> +{
> + int iter;
Iter should be "unsigned int" (or similar, maybe "uint8_t")
> + for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
> + if (tmp_sig == buckets[bucket].sigs[iter] &&
> + buckets[bucket].sets[iter] !=
> + RTE_MEMBER_NO_MATCH) {
> + *set_id = buckets[bucket].sets[iter];
> + return 1;
> + }
> + }
> + return 0;
> +}
> +
> +static inline void
> +search_bucket_multi(uint32_t bucket, member_sig_t tmp_sig,
> + struct member_ht_bucket *buckets,
> + uint32_t *counter,
> + uint32_t match_per_key,
Better change to "matches_per_key".
> + member_set_t *set_id)
> +{
> + int iter;
> + for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
> + if (tmp_sig == buckets[bucket].sigs[iter] &&
> + buckets[bucket].sets[iter] !=
> + RTE_MEMBER_NO_MATCH) {
> + set_id[*counter] = buckets[bucket].sets[iter];
> + (*counter)++;
> + if (*counter >= match_per_key)
> + return;
> + }
> + }
> +}
> +
> +int
> +rte_member_create_ht(struct rte_member_setsum *ss,
> + const struct rte_member_parameters *params) {
...
> +
> + RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
> + "the table has %u entries, %u buckets\n",
> + num_buckets,
> + num_buckets / RTE_MEMBER_BUCKET_ENTRIES);
Shouldn't this be "num_buckets * RTE_MEMBER_BUCKET_ENTRIES" and "num_buckets"?
> + return 0;
> +}
> +
> +static inline
> +void get_buckets_index(const struct rte_member_setsum *ss, const void
> *key,
> + uint32_t *prim_bkt, uint32_t *sec_bkt, member_sig_t
> *sig) {
"static inline void" should be in the same line.
> + uint32_t first_hash = MEMBER_HASH_FUNC(key, ss->key_len,
> + ss->prim_hash_seed);
> + uint32_t sec_hash = MEMBER_HASH_FUNC(&first_hash,
> sizeof(uint32_t),
> + ss->sec_hash_seed);
> + *sig = first_hash;
> + if (ss->cache) {
> + *prim_bkt = sec_hash & ss->bucket_mask;
Is this correct? Using the secondary hash to acces the primary bucket?
Why is the cache case different from the non-cache case?
I think this function deserves some comments to explain all this calculations.
> + *sec_bkt = (sec_hash >> 16) & ss->bucket_mask;
> + } else {
> + *prim_bkt = sec_hash & ss->bucket_mask;
> + *sec_bkt = (*prim_bkt ^ *sig) & ss->bucket_mask;
> + }
> +}
> +
> +int
> +rte_member_lookup_ht(const struct rte_member_setsum *ss,
> + const void *key, member_set_t *set_id) {
> + uint32_t prim_bucket, sec_bucket;
> + member_sig_t tmp_sig;
> + struct member_ht_bucket *buckets = ss->table;
> +
> +
Remove extra blank line.
> + *set_id = RTE_MEMBER_NO_MATCH;
> + get_buckets_index(ss, key, &prim_bucket, &sec_bucket,
> &tmp_sig);
> +
> + if (search_bucket_single(prim_bucket, tmp_sig, buckets,
> + set_id) ||
> + search_bucket_single(sec_bucket, tmp_sig,
> + buckets, set_id))
> + return 1;
> +
> + return 0;
> +}
> +
> +uint32_t
> +rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
> + const void **keys, uint32_t num_keys, member_set_t
> *set_id) {
> + uint32_t i;
> + uint32_t ret = 0;
Better change to something more meaningful, like "nr_matches".
> + struct member_ht_bucket *buckets = ss->table;
> + member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
> + uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
> + uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
> +
> + for (i = 0; i < num_keys; i++) {
> + get_buckets_index(ss, keys[i], &prim_buckets[i],
> + &sec_buckets[i], &tmp_sig[i]);
> + rte_prefetch0(&buckets[prim_buckets[i]]);
> + rte_prefetch0(&buckets[sec_buckets[i]]);
> + }
> +
> + for (i = 0; i < num_keys; i++) {
> + if (search_bucket_single(prim_buckets[i], tmp_sig[i],
> + buckets, &set_id[i]) ||
> + search_bucket_single(sec_buckets[i],
> + tmp_sig[i], buckets, &set_id[i]))
> + ret++;
> + else
> + set_id[i] = RTE_MEMBER_NO_MATCH;
> + }
> + return ret;
> +}
> +
> +uint32_t
> +rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
> + const void *key, uint32_t match_per_key,
> + member_set_t *set_id)
> +{
> + uint32_t ret = 0;
Better change to something more meaningful, like "nr_matches".
This applies to the following functions.
> + uint32_t prim_bucket, sec_bucket;
> + member_sig_t tmp_sig;
> + struct member_ht_bucket *buckets = ss->table;
> +
> + get_buckets_index(ss, key, &prim_bucket, &sec_bucket,
> &tmp_sig);
> +
> + search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
> + match_per_key, set_id);
> + if (ret < match_per_key)
> + search_bucket_multi(sec_bucket, tmp_sig,
> + buckets, &ret, match_per_key, set_id);
> + return ret;
> +}
> +
...
> +static inline int
> +try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t
> sec,
> + member_sig_t sig, member_set_t set_id) {
> + int i;
> + /* If not full then insert into one slot*/
Extra space at the end of the comment, before */
> + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> + if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
> + buckets[prim].sigs[i] = sig;
> + buckets[prim].sets[i] = set_id;
> + return 0;
> + }
> + }
> + /* if prim failed, we need to access second cache line */
Second bucket, instead of second cache line? Also, check that all comments
start with capital letters.
> + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> + if (buckets[sec].sets[i] == RTE_MEMBER_NO_MATCH) {
> + buckets[sec].sigs[i] = sig;
> + buckets[sec].sets[i] = set_id;
> + return 0;
> + }
> + }
> + return -1;
> +}
> +
> +static inline int
> +try_overwrite(struct member_ht_bucket *buckets, uint32_t prim,
> uint32_t sec,
> + member_sig_t sig, member_set_t set_id) {
> + if (insert_overwrite_search(prim, sig, buckets, set_id) ||
> + insert_overwrite_search(sec, sig, buckets,
> + set_id))
> + return 0;
> + return -1;
> +}
> +
> +static inline int
> +evict_from_bucket(void)
> +{
> + /* for now, we randomly pick one entry to evict */
> + return rte_rand() & (RTE_MEMBER_BUCKET_ENTRIES - 1); }
> +
> +/*
> + * This function is similar to the cuckoo hash make_space function in
> +hash
> + * library
> + */
> +static inline int
> +make_space_bucket(const struct rte_member_setsum *ss, uint32_t
> bkt_num)
> +{
> + static unsigned int nr_pushes;
A patch has been sent to change this static variable to be non-static,
in the hash library. Since this is following a similar implementation,
it will need the same change:
http://dpdk.org/dev/patchwork/patch/29104/
> + unsigned int i, j;
> + int ret;
> + struct member_ht_bucket *buckets = ss->table;
> + uint32_t next_bucket_idx;
> + struct member_ht_bucket
> *next_bkt[RTE_MEMBER_BUCKET_ENTRIES];
> + struct member_ht_bucket *bkt = &buckets[bkt_num];
> + member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
Include a comment about this flag_mask
(MSB is set to indicate if an entry has been already pushed).
> + /*
> + * Push existing item (search for bucket with space in
> + * alternative locations) to its alternative location
> + */
> + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> + /* Search for space in alternative locations */
> + next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss-
> >bucket_mask;
> + next_bkt[i] = &buckets[next_bucket_idx];
> + for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++) {
> + if (next_bkt[i]->sets[j] ==
> RTE_MEMBER_NO_MATCH)
> + break;
> + }
> +
> + if (j != RTE_MEMBER_BUCKET_ENTRIES)
> + break;
> + }
> +
> + /* Alternative location has spare room (end of recursive function)
> */
> + if (i != RTE_MEMBER_BUCKET_ENTRIES) {
> + next_bkt[i]->sigs[j] = bkt->sigs[i];
> + next_bkt[i]->sets[j] = bkt->sets[i];
> + return i;
Since you want to return 1 when there is an eviction, I think you can return 1 here,
no need to return the index in the bucket.
> + }
> +
> + /* Pick entry that has not been pushed yet */
> + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++)
> + if ((bkt->sets[i] & flag_mask) == 0)
> + break;
> +
> + /* All entries have been pushed, so entry cannot be added */
> + if (i == RTE_MEMBER_BUCKET_ENTRIES ||
> + nr_pushes > RTE_MEMBER_MAX_PUSHES)
> + return -ENOSPC;
> +
> + next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
> + /* Set flag to indicate that this entry is going to be pushed */
> + bkt->sets[i] |= flag_mask;
> +
> + nr_pushes++;
> + /* Need room in alternative bucket to insert the pushed entry */
> + ret = make_space_bucket(ss, next_bucket_idx);
> + /*
> + * After recursive function.
> + * Clear flags and insert the pushed entry
> + * in its alternative location if successful,
> + * or return error
> + */
> + bkt->sets[i] &= ~flag_mask;
> + nr_pushes = 0;
> + if (ret >= 0) {
> + next_bkt[i]->sigs[ret] = bkt->sigs[i];
> + next_bkt[i]->sets[ret] = bkt->sets[i];
> + return i;
Same about return 1 here.
> + } else
> + return ret;
> +}
> +
> +int
> +rte_member_add_ht(const struct rte_member_setsum *ss,
> + const void *key, member_set_t set_id) {
> + int ret;
> + uint32_t prim_bucket, sec_bucket;
> + member_sig_t tmp_sig;
> + struct member_ht_bucket *buckets = ss->table;
> + member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
> +
> + if (set_id == RTE_MEMBER_NO_MATCH || (set_id & flag_mask) !=
> 0)
> + return -EINVAL;
> +
> + get_buckets_index(ss, key, &prim_bucket, &sec_bucket,
> &tmp_sig);
> +
> + /* if it is cache based filter, we try overwriting existing entry */
> + if (ss->cache) {
> + ret = try_overwrite(buckets, prim_bucket, sec_bucket,
> tmp_sig,
> + set_id);
If the comment that I made above (in the insert_overwrite_search function)
is true and this function is used to update the set_id of an entry,
can this be used also in non-cache mode?
> + if (ret != -1)
> + return ret;
> + }
> + /* If not full then insert into one slot*/
Extra space before */.
> + ret = try_insert(buckets, prim_bucket, sec_bucket, tmp_sig, set_id);
> + if (ret != -1)
> + return ret;
> +
> + /* random pick prim or sec for recursive displacement */
> +
> + uint32_t select_bucket = (tmp_sig && 1U) ? prim_bucket :
> sec_bucket;
> + if (ss->cache) {
> + ret = evict_from_bucket();
> + buckets[select_bucket].sigs[ret] = tmp_sig;
> + buckets[select_bucket].sets[ret] = set_id;
> + return 1;
> + }
> +
> + ret = make_space_bucket(ss, select_bucket);
If this only return a negative value when failure or 1 when success,
you can check for ret == 1 and not set ret = 1.
> + if (ret >= 0) {
> + buckets[select_bucket].sigs[ret] = tmp_sig;
> + buckets[select_bucket].sets[ret] = set_id;
> + ret = 1;
> + }
> +
> + return ret;
> +}
> +
> +void
> +rte_member_free_ht(struct rte_member_setsum *ss) {
> + rte_free(ss->table);
> +}
> +
...
> +
> +void
> +rte_member_reset_ht(const struct rte_member_setsum *ss) {
> + uint32_t i, j;
> + struct member_ht_bucket *buckets = ss->table;
To keep to consistency, I would leave a blank line between the variables declarations
and the rest of the function implementation.
> + for (i = 0; i < ss->bucket_cnt; i++) {
> + for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
> + buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
> + }
> +}
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v4 2/7] member: implement HT mode
2017-10-02 13:30 ` De Lara Guarch, Pablo
@ 2017-10-03 1:18 ` Wang, Yipeng1
0 siblings, 0 replies; 81+ messages in thread
From: Wang, Yipeng1 @ 2017-10-03 1:18 UTC (permalink / raw)
To: De Lara Guarch, Pablo, dev
Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
Thank you very much Pablo.
I agree with you on most of your comments and I will address them soon.
But please find some explanations inlined for certain questions you raised.
> -----Original Message-----
> From: De Lara Guarch, Pablo
> Sent: Monday, October 2, 2017 6:31 AM
> To: Wang, Yipeng1 <yipeng1.wang@intel.com>; dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>
> Subject: RE: [PATCH v4 2/7] member: implement HT mode
>
>
>
> > -----Original Message-----
> > From: Wang, Yipeng1
> > Sent: Wednesday, September 27, 2017 6:40 PM
> > To: dev@dpdk.org
> > Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> > Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> > <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> > <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> > Subject: [PATCH v4 2/7] member: implement HT mode
> >
>
> ...
>
> > diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
> index
> > 1a79eaa..ad26548 100644
> > --- a/lib/librte_member/Makefile
> > +++ b/lib/librte_member/Makefile
> > @@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
> > LIBABIVER := 1
> >
> > # all source are stored in SRCS-y
> > -SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
> > +SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
> > rte_member_ht.c
> > # install includes
> > SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
> >
> > diff --git a/lib/librte_member/rte_member_ht.c
> > b/lib/librte_member/rte_member_ht.c
> > new file mode 100644
> > index 0000000..55672a4
> > --- /dev/null
> > +++ b/lib/librte_member/rte_member_ht.c
>
> ...
>
> > +
> > +static inline int
> > +insert_overwrite_search(uint32_t bucket, member_sig_t tmp_sig,
> > + struct member_ht_bucket *buckets,
> > + member_set_t set_id)
>
> I would call "bucket", "bucket_id", for better understanding.
> This comment also applies to other parts of the code (e.g. prim_buckets).
>
> > +{
> > + int i;
> > + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> > + if (buckets[bucket].sigs[i] == tmp_sig) {
> > + buckets[bucket].sets[i] = set_id;
> > + return 1;
> > + }
>
> Is this function used to update an existing entry?
> At first, I thought that this was evicting another entry, when the bucket was
> full,
> but it looks like it is updating the set_id of an existing entry.
> If this is the case, I would change the function name and add a comment
> explaining this.
>
> > + }
> > + return 0;
> > +}
> > +
> > +static inline int
> > +search_bucket_single(uint32_t bucket, member_sig_t tmp_sig,
> > + struct member_ht_bucket *buckets,
> > + member_set_t *set_id)
> > +{
> > + int iter;
>
> Iter should be "unsigned int" (or similar, maybe "uint8_t")
>
> > + for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
> > + if (tmp_sig == buckets[bucket].sigs[iter] &&
> > + buckets[bucket].sets[iter] !=
> > + RTE_MEMBER_NO_MATCH) {
> > + *set_id = buckets[bucket].sets[iter];
> > + return 1;
> > + }
> > + }
> > + return 0;
> > +}
> > +
> > +static inline void
> > +search_bucket_multi(uint32_t bucket, member_sig_t tmp_sig,
> > + struct member_ht_bucket *buckets,
> > + uint32_t *counter,
> > + uint32_t match_per_key,
>
> Better change to "matches_per_key".
>
> > + member_set_t *set_id)
> > +{
> > + int iter;
> > + for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
> > + if (tmp_sig == buckets[bucket].sigs[iter] &&
> > + buckets[bucket].sets[iter] !=
> > + RTE_MEMBER_NO_MATCH) {
> > + set_id[*counter] = buckets[bucket].sets[iter];
> > + (*counter)++;
> > + if (*counter >= match_per_key)
> > + return;
> > + }
> > + }
> > +}
> > +
> > +int
> > +rte_member_create_ht(struct rte_member_setsum *ss,
> > + const struct rte_member_parameters *params) {
>
> ...
>
> > +
> > + RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
> > + "the table has %u entries, %u buckets\n",
> > + num_buckets,
> > + num_buckets / RTE_MEMBER_BUCKET_ENTRIES);
>
> Shouldn't this be "num_buckets * RTE_MEMBER_BUCKET_ENTRIES" and
> "num_buckets"?
>
> > + return 0;
> > +}
> > +
> > +static inline
> > +void get_buckets_index(const struct rte_member_setsum *ss, const void
> > *key,
> > + uint32_t *prim_bkt, uint32_t *sec_bkt, member_sig_t
> > *sig) {
>
> "static inline void" should be in the same line.
>
> > + uint32_t first_hash = MEMBER_HASH_FUNC(key, ss->key_len,
> > + ss->prim_hash_seed);
> > + uint32_t sec_hash = MEMBER_HASH_FUNC(&first_hash,
> > sizeof(uint32_t),
> > + ss->sec_hash_seed);
> > + *sig = first_hash;
> > + if (ss->cache) {
> > + *prim_bkt = sec_hash & ss->bucket_mask;
>
> Is this correct? Using the secondary hash to acces the primary bucket?
> Why is the cache case different from the non-cache case?
> I think this function deserves some comments to explain all this calculations.
>
[Wang, Yipeng]
We use the first hash value for the signature, and the second hash
value to derive the primary and secondary bucket locations.
For the non-cache mode, we use xor hashing which is called partial-key cuckoo hashing
proposed by B. Fan, et al's paper
"Cuckoo Filter: Practically Better Than Bloom". Since the non-cache mode
is similar to cuckoo filter, the partial key hash enables the derivation of alternative
bucket location by only using signatures without full key or storing alternative signature.
For cache mode, we do not need to get alternative bucket (just overwritten), so
we use higher/lower bits for more independent hash values.
I will add a comment to explain in next version.
> > + *sec_bkt = (sec_hash >> 16) & ss->bucket_mask;
> > + } else {
> > + *prim_bkt = sec_hash & ss->bucket_mask;
> > + *sec_bkt = (*prim_bkt ^ *sig) & ss->bucket_mask;
> > + }
> > +}
> > +
> > +int
> > +rte_member_lookup_ht(const struct rte_member_setsum *ss,
> > + const void *key, member_set_t *set_id) {
> > + uint32_t prim_bucket, sec_bucket;
> > + member_sig_t tmp_sig;
> > + struct member_ht_bucket *buckets = ss->table;
> > +
> > +
>
> Remove extra blank line.
>
> > + *set_id = RTE_MEMBER_NO_MATCH;
> > + get_buckets_index(ss, key, &prim_bucket, &sec_bucket,
> > &tmp_sig);
> > +
> > + if (search_bucket_single(prim_bucket, tmp_sig, buckets,
> > + set_id) ||
> > + search_bucket_single(sec_bucket, tmp_sig,
> > + buckets, set_id))
> > + return 1;
> > +
> > + return 0;
> > +}
> > +
> > +uint32_t
> > +rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
> > + const void **keys, uint32_t num_keys, member_set_t
> > *set_id) {
> > + uint32_t i;
> > + uint32_t ret = 0;
>
> Better change to something more meaningful, like "nr_matches".
>
> > + struct member_ht_bucket *buckets = ss->table;
> > + member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
> > + uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
> > + uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
> > +
> > + for (i = 0; i < num_keys; i++) {
> > + get_buckets_index(ss, keys[i], &prim_buckets[i],
> > + &sec_buckets[i], &tmp_sig[i]);
> > + rte_prefetch0(&buckets[prim_buckets[i]]);
> > + rte_prefetch0(&buckets[sec_buckets[i]]);
> > + }
> > +
> > + for (i = 0; i < num_keys; i++) {
> > + if (search_bucket_single(prim_buckets[i], tmp_sig[i],
> > + buckets, &set_id[i]) ||
> > + search_bucket_single(sec_buckets[i],
> > + tmp_sig[i], buckets, &set_id[i]))
> > + ret++;
> > + else
> > + set_id[i] = RTE_MEMBER_NO_MATCH;
> > + }
> > + return ret;
> > +}
> > +
> > +uint32_t
> > +rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
> > + const void *key, uint32_t match_per_key,
> > + member_set_t *set_id)
> > +{
> > + uint32_t ret = 0;
>
> Better change to something more meaningful, like "nr_matches".
> This applies to the following functions.
>
> > + uint32_t prim_bucket, sec_bucket;
> > + member_sig_t tmp_sig;
> > + struct member_ht_bucket *buckets = ss->table;
> > +
> > + get_buckets_index(ss, key, &prim_bucket, &sec_bucket,
> > &tmp_sig);
> > +
> > + search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
> > + match_per_key, set_id);
> > + if (ret < match_per_key)
> > + search_bucket_multi(sec_bucket, tmp_sig,
> > + buckets, &ret, match_per_key, set_id);
> > + return ret;
> > +}
> > +
>
> ...
>
> > +static inline int
> > +try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t
> > sec,
> > + member_sig_t sig, member_set_t set_id) {
> > + int i;
> > + /* If not full then insert into one slot*/
>
> Extra space at the end of the comment, before */
> > + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> > + if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
> > + buckets[prim].sigs[i] = sig;
> > + buckets[prim].sets[i] = set_id;
> > + return 0;
> > + }
> > + }
> > + /* if prim failed, we need to access second cache line */
>
> Second bucket, instead of second cache line? Also, check that all comments
> start with capital letters.
>
> > + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> > + if (buckets[sec].sets[i] == RTE_MEMBER_NO_MATCH) {
> > + buckets[sec].sigs[i] = sig;
> > + buckets[sec].sets[i] = set_id;
> > + return 0;
> > + }
> > + }
> > + return -1;
> > +}
> > +
> > +static inline int
> > +try_overwrite(struct member_ht_bucket *buckets, uint32_t prim,
> > uint32_t sec,
> > + member_sig_t sig, member_set_t set_id) {
> > + if (insert_overwrite_search(prim, sig, buckets, set_id) ||
> > + insert_overwrite_search(sec, sig, buckets,
> > + set_id))
> > + return 0;
> > + return -1;
> > +}
> > +
> > +static inline int
> > +evict_from_bucket(void)
> > +{
> > + /* for now, we randomly pick one entry to evict */
> > + return rte_rand() & (RTE_MEMBER_BUCKET_ENTRIES - 1); }
> > +
> > +/*
> > + * This function is similar to the cuckoo hash make_space function in
> > +hash
> > + * library
> > + */
> > +static inline int
> > +make_space_bucket(const struct rte_member_setsum *ss, uint32_t
> > bkt_num)
> > +{
> > + static unsigned int nr_pushes;
>
> A patch has been sent to change this static variable to be non-static,
> in the hash library. Since this is following a similar implementation,
> it will need the same change:
> http://dpdk.org/dev/patchwork/patch/29104/
>
> > + unsigned int i, j;
> > + int ret;
> > + struct member_ht_bucket *buckets = ss->table;
> > + uint32_t next_bucket_idx;
> > + struct member_ht_bucket
> > *next_bkt[RTE_MEMBER_BUCKET_ENTRIES];
> > + struct member_ht_bucket *bkt = &buckets[bkt_num];
> > + member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
>
> Include a comment about this flag_mask
> (MSB is set to indicate if an entry has been already pushed).
>
> > + /*
> > + * Push existing item (search for bucket with space in
> > + * alternative locations) to its alternative location
> > + */
> > + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> > + /* Search for space in alternative locations */
> > + next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss-
> > >bucket_mask;
> > + next_bkt[i] = &buckets[next_bucket_idx];
> > + for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++) {
> > + if (next_bkt[i]->sets[j] ==
> > RTE_MEMBER_NO_MATCH)
> > + break;
> > + }
> > +
> > + if (j != RTE_MEMBER_BUCKET_ENTRIES)
> > + break;
> > + }
> > +
> > + /* Alternative location has spare room (end of recursive function)
> > */
> > + if (i != RTE_MEMBER_BUCKET_ENTRIES) {
> > + next_bkt[i]->sigs[j] = bkt->sigs[i];
> > + next_bkt[i]->sets[j] = bkt->sets[i];
> > + return i;
>
> Since you want to return 1 when there is an eviction, I think you can return 1
> here,
> no need to return the index in the bucket.
[Wang, Yipeng]
We return the index in the bucket to keep the recursive function going right?
I.e. the make_space needs the returned index to displace the entry.
>
> > + }
> > +
> > + /* Pick entry that has not been pushed yet */
> > + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++)
> > + if ((bkt->sets[i] & flag_mask) == 0)
> > + break;
> > +
> > + /* All entries have been pushed, so entry cannot be added */
> > + if (i == RTE_MEMBER_BUCKET_ENTRIES ||
> > + nr_pushes > RTE_MEMBER_MAX_PUSHES)
> > + return -ENOSPC;
> > +
> > + next_bucket_idx = (bkt->sigs[i] ^ bkt_num) & ss->bucket_mask;
> > + /* Set flag to indicate that this entry is going to be pushed */
> > + bkt->sets[i] |= flag_mask;
> > +
> > + nr_pushes++;
> > + /* Need room in alternative bucket to insert the pushed entry */
> > + ret = make_space_bucket(ss, next_bucket_idx);
> > + /*
> > + * After recursive function.
> > + * Clear flags and insert the pushed entry
> > + * in its alternative location if successful,
> > + * or return error
> > + */
> > + bkt->sets[i] &= ~flag_mask;
> > + nr_pushes = 0;
> > + if (ret >= 0) {
> > + next_bkt[i]->sigs[ret] = bkt->sigs[i];
> > + next_bkt[i]->sets[ret] = bkt->sets[i];
> > + return i;
>
> Same about return 1 here.
>
> > + } else
> > + return ret;
> > +}
> > +
> > +int
> > +rte_member_add_ht(const struct rte_member_setsum *ss,
> > + const void *key, member_set_t set_id) {
> > + int ret;
> > + uint32_t prim_bucket, sec_bucket;
> > + member_sig_t tmp_sig;
> > + struct member_ht_bucket *buckets = ss->table;
> > + member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
> > +
> > + if (set_id == RTE_MEMBER_NO_MATCH || (set_id & flag_mask) !=
> > 0)
> > + return -EINVAL;
> > +
> > + get_buckets_index(ss, key, &prim_bucket, &sec_bucket,
> > &tmp_sig);
> > +
> > + /* if it is cache based filter, we try overwriting existing entry */
> > + if (ss->cache) {
> > + ret = try_overwrite(buckets, prim_bucket, sec_bucket,
> > tmp_sig,
> > + set_id);
>
> If the comment that I made above (in the insert_overwrite_search function)
> is true and this function is used to update the set_id of an entry,
> can this be used also in non-cache mode?
>
[Wang, Yipeng]
I changed the name to update_entry_search. For non-cache mode, we do not do
update even if the key finds an entry with the same signature during add.
This is because for non-cache mode, we do not allow false negative, keys
Overwriting each other may incur false negative.
I will add more explanation in comments.
> > + if (ret != -1)
> > + return ret;
> > + }
> > + /* If not full then insert into one slot*/
>
> Extra space before */.
>
> > + ret = try_insert(buckets, prim_bucket, sec_bucket, tmp_sig, set_id);
> > + if (ret != -1)
> > + return ret;
> > +
> > + /* random pick prim or sec for recursive displacement */
> > +
> > + uint32_t select_bucket = (tmp_sig && 1U) ? prim_bucket :
> > sec_bucket;
> > + if (ss->cache) {
> > + ret = evict_from_bucket();
> > + buckets[select_bucket].sigs[ret] = tmp_sig;
> > + buckets[select_bucket].sets[ret] = set_id;
> > + return 1;
> > + }
> > +
> > + ret = make_space_bucket(ss, select_bucket);
>
> If this only return a negative value when failure or 1 when success,
> you can check for ret == 1 and not set ret = 1.
>
[Wang, Yipeng]
Please see my comments above, if we return 1 for make_space, will the
recursive process be broken?
> > + if (ret >= 0) {
> > + buckets[select_bucket].sigs[ret] = tmp_sig;
> > + buckets[select_bucket].sets[ret] = set_id;
> > + ret = 1;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +void
> > +rte_member_free_ht(struct rte_member_setsum *ss) {
> > + rte_free(ss->table);
> > +}
> > +
>
> ...
>
> > +
> > +void
> > +rte_member_reset_ht(const struct rte_member_setsum *ss) {
> > + uint32_t i, j;
> > + struct member_ht_bucket *buckets = ss->table;
>
> To keep to consistency, I would leave a blank line between the variables
> declarations
> and the rest of the function implementation.
>
> > + for (i = 0; i < ss->bucket_cnt; i++) {
> > + for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
> > + buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
> > + }
> > +}
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v4 3/7] member: implement vBF mode
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 1/7] member: implement main API Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 2/7] member: implement HT mode Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-10-02 15:44 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 4/7] member: add AVX for HT mode Yipeng Wang
` (4 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
Bloom Filter (BF) [1] is a well-known space-efficient
probabilistic data structure that answers set membership queries.
Vector of Bloom Filters (vBF) is an extension to traditional BF
that supports multi-set membership testing. Traditional BF will
return found or not-found for each key. vBF will also return
which set the key belongs to if it is found.
Since each set requires a BF, vBF should be used when set count
is small. vBF's false positive rate could be set appropriately so
that its memory requirement and lookup speed is better in certain
cases comparing to HT based set-summary.
This patch adds the vBF implementation.
[1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
Errors,” Communications of the ACM, 1970.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_vbf.c | 349 +++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_vbf.h | 79 +++++++++
3 files changed, 429 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index ad26548..50275ed 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c rte_member_vbf.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_vbf.c b/lib/librte_member/rte_member_vbf.c
new file mode 100644
index 0000000..cbfa18e
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.c
@@ -0,0 +1,349 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <math.h>
+#include <string.h>
+
+#include <rte_malloc.h>
+#include <rte_memory.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_vbf.h"
+
+/*
+ * vBF currently implemented as a big array.
+ * The BFs have a vertical layout. Bits in same location of all bfs will stay
+ * in the same cache line.
+ * For example, if we have 32 bloom filters, we use a uint32_t array to
+ * represent all of them. array[0] represent the first location of all the
+ * bloom filters, array[1] represents the second location of all the
+ * bloom filters, etc. The advantage of this layout is to minimize the average
+ * number of memory accesses to test all bloom filters.
+ *
+ * Currently the implementation supports vBF containing 1,2,4,8,16,32 BFs.
+ */
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+
+ if (params->num_set > 32 || !rte_is_power_of_2(params->num_set) ||
+ params->num_keys == 0 ||
+ params->false_positive_rate == 0 ||
+ params->false_positive_rate > 1) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "vBF create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ /* We assume expected keys evenly distribute to all BFs */
+ uint32_t num_keys_per_bf = 1 + (params->num_keys - 1) / ss->num_set;
+
+ /*
+ * Note that the false positive rate is for all BFs in the vBF
+ * such that the single BF's false positive rate needs to be
+ * calculated.
+ * Assume each BF's False positive rate is x. The total false positive
+ * rate is fp = 1-(1-x)^n.
+ * => x = 1 - (1-fp)^(1/n)
+ */
+
+ float x = 1 - pow((1 - params->false_positive_rate), 1.0 / ss->num_set);
+
+ if (x == 0) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "BF false positive rate is too small\n");
+ return -EINVAL;
+ }
+
+ uint32_t bits = ceil((num_keys_per_bf *
+ log(x)) / log(1.0 / (pow(2.0, log(2.0)))));
+
+ /* We round to power of 2 for performance during lookup */
+ ss->bits = rte_align32pow2(bits);
+
+ ss->num_hashes = (uint32_t)(log(2.0) * bits / num_keys_per_bf);
+ ss->bit_mask = ss->bits - 1;
+
+
+ /*
+ * Since we round the bits to power of 2, the final false positive
+ * rate will probably not be same as the user specified. We log the
+ * new value as debug message.
+ */
+ float new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ ss->num_hashes)), ss->num_hashes);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ /*
+ * reduce hash function count, until we approach the user specified
+ * false-positive rate. otherwise it is too conservative
+ */
+ int tmp_num_hash = ss->num_hashes;
+
+ while (tmp_num_hash > 1) {
+ float tmp_fp = new_fp;
+
+ tmp_num_hash--;
+ new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ tmp_num_hash)), tmp_num_hash);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ if (new_fp > params->false_positive_rate) {
+ new_fp = tmp_fp;
+ tmp_num_hash++;
+ break;
+ }
+ }
+
+ ss->num_hashes = tmp_num_hash;
+
+ RTE_MEMBER_LOG(DEBUG, "vector bloom filter created, "
+ "each bloom filter expects %u keys, needs %u bits, %u hashes, "
+ "with false positive rate set as %.5f, "
+ "The new calculated vBF false positive rate is %.5f\n",
+ num_keys_per_bf, ss->bits, ss->num_hashes, x, new_fp);
+
+ ss->table = rte_zmalloc_socket(NULL, ss->num_set * (ss->bits >> 3),
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ /*
+ * To avoid multiplication and division:
+ * mul_shift is used for multiplication shift during bit test
+ * div_shift is used for division shift, to be divided by number of bits
+ * represented by a uint32_t variable
+ */
+ ss->mul_shift = __builtin_ctzl(ss->num_set);
+ ss->div_shift = __builtin_ctzl(32 >> ss->mul_shift);
+
+ if (ss->table == NULL)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static inline uint32_t
+test_bit(uint32_t bit_loc, const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t n = ss->num_set;
+ uint32_t div_shift = ss->div_shift;
+ uint32_t mul_shift = ss->mul_shift;
+ /*
+ * a is how many bits in one BF are represented by one 32bit
+ * variable.
+ */
+ uint32_t a = 32 >> mul_shift;
+ /*
+ * x>>b is the divide, x & (a-1) is the mod, & (1<<n-1) to mask out bits
+ * we do not need
+ */
+ return (vbf[bit_loc>>div_shift] >> ((bit_loc & (a - 1)) << mul_shift)) &
+ ((1ULL << n) - 1);
+}
+
+static inline void
+set_bit(uint32_t bit_loc, const struct rte_member_setsum *ss, int32_t set)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t div_shift = ss->div_shift;
+ uint32_t mul_shift = ss->mul_shift;
+ uint32_t a = 32 >> mul_shift;
+
+ vbf[bit_loc>>div_shift] |= 1U << (((bit_loc & (a - 1)) << mul_shift) +
+ set - 1);
+}
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *ss, const void *key,
+ member_set_t *set_id)
+{
+ uint32_t j;
+ uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ uint32_t mask = ~0;
+ uint32_t bit_loc;
+
+ for (j = 0; j < ss->num_hashes; j++) {
+ bit_loc = (h1 + j * h2) & ss->bit_mask;
+ mask &= test_bit(bit_loc, ss);
+ }
+
+ if (mask) {
+ *set_id = __builtin_ctzl(mask) + 1;
+ return 1;
+ }
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ return 0;
+}
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, member_set_t *set_ids)
+{
+ uint32_t i, k;
+ uint32_t ret = 0;
+ uint32_t mask[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t h1[RTE_MEMBER_LOOKUP_BULK_MAX], h2[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t bit_loc;
+
+ for (i = 0; i < num_keys; i++)
+ h1[i] = MEMBER_HASH_FUNC(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ for (i = 0; i < num_keys; i++)
+ h2[i] = MEMBER_HASH_FUNC(&h1[i], sizeof(uint32_t),
+ ss->sec_hash_seed);
+ for (i = 0; i < num_keys; i++) {
+ mask[i] = ~0;
+ for (k = 0; k < ss->num_hashes; k++) {
+ bit_loc = (h1[i] + k * h2[i]) & ss->bit_mask;
+ mask[i] &= test_bit(bit_loc, ss);
+ }
+ }
+ for (i = 0; i < num_keys; i++) {
+ if (mask[i]) {
+ set_ids[i] = __builtin_ctzl(mask[i]) + 1;
+ ret++;
+ } else
+ set_ids[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return ret;
+}
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t ret = 0;
+ uint32_t j;
+ uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ uint32_t mask = ~0;
+ uint32_t bit_loc;
+
+ for (j = 0; j < ss->num_hashes; j++) {
+ bit_loc = (h1 + j * h2) & ss->bit_mask;
+ mask &= test_bit(bit_loc, ss);
+ }
+ while (mask) {
+ uint32_t loc = __builtin_ctzl(mask);
+ set_id[ret] = loc + 1;
+ ret++;
+ if (ret >= match_per_key)
+ return ret;
+ mask &= ~(1U << loc);
+ }
+ return ret;
+}
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ uint32_t i, k;
+ uint32_t ret = 0;
+ uint32_t match_cnt_t;
+ uint32_t mask[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t h1[RTE_MEMBER_LOOKUP_BULK_MAX], h2[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t bit_loc;
+
+ for (i = 0; i < num_keys; i++)
+ h1[i] = MEMBER_HASH_FUNC(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ for (i = 0; i < num_keys; i++)
+ h2[i] = MEMBER_HASH_FUNC(&h1[i], sizeof(uint32_t),
+ ss->sec_hash_seed);
+ for (i = 0; i < num_keys; i++) {
+ mask[i] = ~0;
+ for (k = 0; k < ss->num_hashes; k++) {
+ bit_loc = (h1[i] + k * h2[i]) & ss->bit_mask;
+ mask[i] &= test_bit(bit_loc, ss);
+ }
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_t = 0;
+ while (mask[i]) {
+ uint32_t loc = __builtin_ctzl(mask[i]);
+ set_ids[i * match_per_key + match_cnt_t] = loc + 1;
+ match_cnt_t++;
+ if (match_cnt_t >= match_per_key)
+ break;
+ mask[i] &= ~(1U << loc);
+ }
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ ret++;
+ }
+ return ret;
+}
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *ss,
+ const void *key, member_set_t set_id)
+{
+ uint32_t i, h1, h2;
+ uint32_t bit_loc;
+
+ if (set_id > ss->num_set || set_id == RTE_MEMBER_NO_MATCH)
+ return -EINVAL;
+
+ h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t), ss->sec_hash_seed);
+
+ for (i = 0; i < ss->num_hashes; i++) {
+ bit_loc = (h1 + i * h2) & ss->bit_mask;
+ set_bit(bit_loc, ss, set_id);
+ }
+ return 0;
+}
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ memset(vbf, 0, (ss->num_set * ss->bits) >> 3);
+}
diff --git a/lib/librte_member/rte_member_vbf.h b/lib/librte_member/rte_member_vbf.h
new file mode 100644
index 0000000..16144ff
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.h
@@ -0,0 +1,79 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_VBF_H_
+#define _RTE_MEMBER_VBF_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t set_id);
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss);
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *setsum);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_VBF_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v4 3/7] member: implement vBF mode
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 3/7] member: implement vBF mode Yipeng Wang
@ 2017-10-02 15:44 ` De Lara Guarch, Pablo
2017-10-03 1:24 ` Wang, Yipeng1
0 siblings, 1 reply; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-02 15:44 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Wednesday, September 27, 2017 6:41 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v4 3/7] member: implement vBF mode
>
> Bloom Filter (BF) [1] is a well-known space-efficient probabilistic data
> structure that answers set membership queries.
> Vector of Bloom Filters (vBF) is an extension to traditional BF that supports
> multi-set membership testing. Traditional BF will return found or not-found
> for each key. vBF will also return which set the key belongs to if it is found.
>
> Since each set requires a BF, vBF should be used when set count is small.
> vBF's false positive rate could be set appropriately so that its memory
> requirement and lookup speed is better in certain cases comparing to HT
> based set-summary.
>
> This patch adds the vBF implementation.
>
> [1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
> Errors,” Communications of the ACM, 1970.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
...
> diff --git a/lib/librte_member/rte_member_vbf.c
> b/lib/librte_member/rte_member_vbf.c
...
> +int
> +rte_member_create_vbf(struct rte_member_setsum *ss,
> + const struct rte_member_parameters *params) {
> +
> + if (params->num_set > 32 || !rte_is_power_of_2(params-
> >num_set) ||
Magic number. Define a macro instead.
> + params->num_keys == 0 ||
> + params->false_positive_rate == 0 ||
> + params->false_positive_rate > 1) {
> + rte_errno = EINVAL;
> + RTE_MEMBER_LOG(ERR, "vBF create with invalid
> parameters\n");
> + return -EINVAL;
...
> +
> + /*
> + * reduce hash function count, until we approach the user specified
> + * false-positive rate. otherwise it is too conservative
Watch out for capital letters at the start of the comment and after a full stop.
> + */
> + int tmp_num_hash = ss->num_hashes;
> +
> + while (tmp_num_hash > 1) {
> + float tmp_fp = new_fp;
> +
> + tmp_num_hash--;
> + new_fp = pow((1 - pow((1 - 1.0 / ss->bits),
> num_keys_per_bf *
> + tmp_num_hash)), tmp_num_hash);
> + new_fp = 1 - pow((1 - new_fp), ss->num_set);
> +
> + if (new_fp > params->false_positive_rate) {
> + new_fp = tmp_fp;
> + tmp_num_hash++;
> + break;
> + }
> + }
> +
> + ss->num_hashes = tmp_num_hash;
> +
> + RTE_MEMBER_LOG(DEBUG, "vector bloom filter created, "
> + "each bloom filter expects %u keys, needs %u bits, %u
> hashes, "
> + "with false positive rate set as %.5f, "
> + "The new calculated vBF false positive rate is %.5f\n",
> + num_keys_per_bf, ss->bits, ss->num_hashes, x, new_fp);
Use a more descriptive variable name for "x".
> +
> + ss->table = rte_zmalloc_socket(NULL, ss->num_set * (ss->bits >> 3),
> + RTE_CACHE_LINE_SIZE, ss-
> >socket_id);
> +
> + /*
> + * To avoid multiplication and division:
> + * mul_shift is used for multiplication shift during bit test
> + * div_shift is used for division shift, to be divided by number of bits
> + * represented by a uint32_t variable
> + */
> + ss->mul_shift = __builtin_ctzl(ss->num_set);
> + ss->div_shift = __builtin_ctzl(32 >> ss->mul_shift);
> +
> + if (ss->table == NULL)
> + return -ENOMEM;
I would move this check just after the malloc call.
> +
> + return 0;
> +}
> +
> +static inline uint32_t
> +test_bit(uint32_t bit_loc, const struct rte_member_setsum *ss) {
> + uint32_t *vbf = ss->table;
> + uint32_t n = ss->num_set;
> + uint32_t div_shift = ss->div_shift;
> + uint32_t mul_shift = ss->mul_shift;
> + /*
> + * a is how many bits in one BF are represented by one 32bit
> + * variable.
> + */
> + uint32_t a = 32 >> mul_shift;
> + /*
> + * x>>b is the divide, x & (a-1) is the mod, & (1<<n-1) to mask out
> bits
> + * we do not need
> + */
> + return (vbf[bit_loc>>div_shift] >> ((bit_loc & (a - 1)) << mul_shift))
Add spaces around ">>".
> &
> + ((1ULL << n) - 1);
> +}
> +
> +static inline void
> +set_bit(uint32_t bit_loc, const struct rte_member_setsum *ss, int32_t
> +set) {
> + uint32_t *vbf = ss->table;
> + uint32_t div_shift = ss->div_shift;
> + uint32_t mul_shift = ss->mul_shift;
> + uint32_t a = 32 >> mul_shift;
> +
> + vbf[bit_loc>>div_shift] |= 1U << (((bit_loc & (a - 1)) << mul_shift) +
> + set - 1);
Same as above.
> +}
> +
> +int
> +rte_member_lookup_vbf(const struct rte_member_setsum *ss, const
> void *key,
> + member_set_t *set_id)
> +{
> + uint32_t j;
> + uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss-
> >prim_hash_seed);
> + uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
> + ss->sec_hash_seed);
> + uint32_t mask = ~0;
> + uint32_t bit_loc;
> +
> + for (j = 0; j < ss->num_hashes; j++) {
> + bit_loc = (h1 + j * h2) & ss->bit_mask;
> + mask &= test_bit(bit_loc, ss);
> + }
> +
> + if (mask) {
> + *set_id = __builtin_ctzl(mask) + 1;
> + return 1;
Wouldn't it be better to return 0 when there is a hit and -ENOENT when there is not?
> + }
> +
> + *set_id = RTE_MEMBER_NO_MATCH;
> + return 0;
> +}
> +
> +uint32_t
> +rte_member_lookup_bulk_vbf(const struct rte_member_setsum *ss,
> + const void **keys, uint32_t num_keys, member_set_t
> *set_ids) {
> + uint32_t i, k;
> + uint32_t ret = 0;
Change variable name to "nr_matches/hits" or similar.
Same in the next functions.
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v4 3/7] member: implement vBF mode
2017-10-02 15:44 ` De Lara Guarch, Pablo
@ 2017-10-03 1:24 ` Wang, Yipeng1
0 siblings, 0 replies; 81+ messages in thread
From: Wang, Yipeng1 @ 2017-10-03 1:24 UTC (permalink / raw)
To: De Lara Guarch, Pablo, dev
Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
Please see one comment inlined:
> -----Original Message-----
> From: De Lara Guarch, Pablo
> Sent: Monday, October 2, 2017 8:44 AM
> To: Wang, Yipeng1 <yipeng1.wang@intel.com>; dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>
> Subject: RE: [PATCH v4 3/7] member: implement vBF mode
>
>
>
> > -----Original Message-----
> > From: Wang, Yipeng1
> > Sent: Wednesday, September 27, 2017 6:41 PM
> > To: dev@dpdk.org
> > Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> > Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> > <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> > <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> > Subject: [PATCH v4 3/7] member: implement vBF mode
> >
> > Bloom Filter (BF) [1] is a well-known space-efficient probabilistic data
> > structure that answers set membership queries.
> > Vector of Bloom Filters (vBF) is an extension to traditional BF that supports
> > multi-set membership testing. Traditional BF will return found or not-found
> > for each key. vBF will also return which set the key belongs to if it is found.
> >
> > Since each set requires a BF, vBF should be used when set count is small.
> > vBF's false positive rate could be set appropriately so that its memory
> > requirement and lookup speed is better in certain cases comparing to HT
> > based set-summary.
> >
> > This patch adds the vBF implementation.
> >
> > [1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
> > Errors,” Communications of the ACM, 1970.
> >
> > Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
>
> ...
>
> > diff --git a/lib/librte_member/rte_member_vbf.c
> > b/lib/librte_member/rte_member_vbf.c
>
> ...
>
> > +int
> > +rte_member_create_vbf(struct rte_member_setsum *ss,
> > + const struct rte_member_parameters *params) {
> > +
> > + if (params->num_set > 32 || !rte_is_power_of_2(params-
> > >num_set) ||
>
> Magic number. Define a macro instead.
>
> > + params->num_keys == 0 ||
> > + params->false_positive_rate == 0 ||
> > + params->false_positive_rate > 1) {
> > + rte_errno = EINVAL;
> > + RTE_MEMBER_LOG(ERR, "vBF create with invalid
> > parameters\n");
> > + return -EINVAL;
>
> ...
>
> > +
> > + /*
> > + * reduce hash function count, until we approach the user specified
> > + * false-positive rate. otherwise it is too conservative
>
> Watch out for capital letters at the start of the comment and after a full stop.
>
> > + */
> > + int tmp_num_hash = ss->num_hashes;
> > +
> > + while (tmp_num_hash > 1) {
> > + float tmp_fp = new_fp;
> > +
> > + tmp_num_hash--;
> > + new_fp = pow((1 - pow((1 - 1.0 / ss->bits),
> > num_keys_per_bf *
> > + tmp_num_hash)), tmp_num_hash);
> > + new_fp = 1 - pow((1 - new_fp), ss->num_set);
> > +
> > + if (new_fp > params->false_positive_rate) {
> > + new_fp = tmp_fp;
> > + tmp_num_hash++;
> > + break;
> > + }
> > + }
> > +
> > + ss->num_hashes = tmp_num_hash;
> > +
> > + RTE_MEMBER_LOG(DEBUG, "vector bloom filter created, "
> > + "each bloom filter expects %u keys, needs %u bits, %u
> > hashes, "
> > + "with false positive rate set as %.5f, "
> > + "The new calculated vBF false positive rate is %.5f\n",
> > + num_keys_per_bf, ss->bits, ss->num_hashes, x, new_fp);
>
> Use a more descriptive variable name for "x".
>
> > +
> > + ss->table = rte_zmalloc_socket(NULL, ss->num_set * (ss->bits >> 3),
> > + RTE_CACHE_LINE_SIZE, ss-
> > >socket_id);
> > +
> > + /*
> > + * To avoid multiplication and division:
> > + * mul_shift is used for multiplication shift during bit test
> > + * div_shift is used for division shift, to be divided by number of bits
> > + * represented by a uint32_t variable
> > + */
> > + ss->mul_shift = __builtin_ctzl(ss->num_set);
> > + ss->div_shift = __builtin_ctzl(32 >> ss->mul_shift);
> > +
> > + if (ss->table == NULL)
> > + return -ENOMEM;
>
> I would move this check just after the malloc call.
>
> > +
> > + return 0;
> > +}
> > +
> > +static inline uint32_t
> > +test_bit(uint32_t bit_loc, const struct rte_member_setsum *ss) {
> > + uint32_t *vbf = ss->table;
> > + uint32_t n = ss->num_set;
> > + uint32_t div_shift = ss->div_shift;
> > + uint32_t mul_shift = ss->mul_shift;
> > + /*
> > + * a is how many bits in one BF are represented by one 32bit
> > + * variable.
> > + */
> > + uint32_t a = 32 >> mul_shift;
> > + /*
> > + * x>>b is the divide, x & (a-1) is the mod, & (1<<n-1) to mask out
> > bits
> > + * we do not need
> > + */
> > + return (vbf[bit_loc>>div_shift] >> ((bit_loc & (a - 1)) << mul_shift))
>
> Add spaces around ">>".
>
> > &
> > + ((1ULL << n) - 1);
> > +}
> > +
> > +static inline void
> > +set_bit(uint32_t bit_loc, const struct rte_member_setsum *ss, int32_t
> > +set) {
> > + uint32_t *vbf = ss->table;
> > + uint32_t div_shift = ss->div_shift;
> > + uint32_t mul_shift = ss->mul_shift;
> > + uint32_t a = 32 >> mul_shift;
> > +
> > + vbf[bit_loc>>div_shift] |= 1U << (((bit_loc & (a - 1)) << mul_shift) +
> > + set - 1);
>
> Same as above.
>
> > +}
> > +
> > +int
> > +rte_member_lookup_vbf(const struct rte_member_setsum *ss, const
> > void *key,
> > + member_set_t *set_id)
> > +{
> > + uint32_t j;
> > + uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss-
> > >prim_hash_seed);
> > + uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
> > + ss->sec_hash_seed);
> > + uint32_t mask = ~0;
> > + uint32_t bit_loc;
> > +
> > + for (j = 0; j < ss->num_hashes; j++) {
> > + bit_loc = (h1 + j * h2) & ss->bit_mask;
> > + mask &= test_bit(bit_loc, ss);
> > + }
> > +
> > + if (mask) {
> > + *set_id = __builtin_ctzl(mask) + 1;
> > + return 1;
>
> Wouldn't it be better to return 0 when there is a hit and -ENOENT when
> there is not?
[Wang, Yipeng]
I tried to follow the convention of other lookup functions that the return value
Is kind of the number of matches that found. So 1 means found 1 and 0 means not
Found. I defined the public API rte_member_lookup to also returns 1 for found.
> > + }
> > +
> > + *set_id = RTE_MEMBER_NO_MATCH;
> > + return 0;
> > +}
> > +
> > +uint32_t
> > +rte_member_lookup_bulk_vbf(const struct rte_member_setsum *ss,
> > + const void **keys, uint32_t num_keys, member_set_t
> > *set_ids) {
> > + uint32_t i, k;
> > + uint32_t ret = 0;
>
> Change variable name to "nr_matches/hits" or similar.
> Same in the next functions.
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v4 4/7] member: add AVX for HT mode
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
` (2 preceding siblings ...)
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 3/7] member: implement vBF mode Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 5/7] member: enable the library Yipeng Wang
` (3 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
For key search, the signatures of all entries are compared against
the signature of the key that is being looked up. Since all
signatures are contiguously put in a bucket, they can be compared
with vector instructions (AVX2), achieving higher lookup performance.
This patch adds AVX2 implementation in a separate header file.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/rte_member_ht.c | 145 ++++++++++++++++++++++++++++---------
lib/librte_member/rte_member_x86.h | 107 +++++++++++++++++++++++++++
2 files changed, 219 insertions(+), 33 deletions(-)
create mode 100644 lib/librte_member/rte_member_x86.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
index 55672a4..ef1541a 100644
--- a/lib/librte_member/rte_member_ht.c
+++ b/lib/librte_member/rte_member_ht.c
@@ -40,6 +40,10 @@
#include "rte_member.h"
#include "rte_member_ht.h"
+#if defined(RTE_ARCH_X86)
+#include "rte_member_x86.h"
+#endif
+
static inline int
insert_overwrite_search(uint32_t bucket, member_sig_t tmp_sig,
struct member_ht_bucket *buckets,
@@ -132,6 +136,13 @@ rte_member_create_ht(struct rte_member_setsum *ss,
for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
}
+#if defined(RTE_ARCH_X86)
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) &&
+ RTE_MEMBER_BUCKET_ENTRIES == 16)
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_AVX2;
+ else
+#endif
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_SCALAR;
RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
"the table has %u entries, %u buckets\n",
@@ -166,15 +177,26 @@ rte_member_lookup_ht(const struct rte_member_setsum *ss,
member_sig_t tmp_sig;
struct member_ht_bucket *buckets = ss->table;
-
*set_id = RTE_MEMBER_NO_MATCH;
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- if (search_bucket_single(prim_bucket, tmp_sig, buckets,
- set_id) ||
- search_bucket_single(sec_bucket, tmp_sig,
- buckets, set_id))
- return 1;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single_avx(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ }
return 0;
}
@@ -198,13 +220,27 @@ rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
}
for (i = 0; i < num_keys; i++) {
- if (search_bucket_single(prim_buckets[i], tmp_sig[i],
- buckets, &set_id[i]) ||
- search_bucket_single(sec_buckets[i],
- tmp_sig[i], buckets, &set_id[i]))
- ret++;
- else
- set_id[i] = RTE_MEMBER_NO_MATCH;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]) ||
+ search_bucket_single_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ ret++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
}
return ret;
}
@@ -221,12 +257,24 @@ rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
- match_per_key, set_id);
- if (ret < match_per_key)
- search_bucket_multi(sec_bucket, tmp_sig,
- buckets, &ret, match_per_key, set_id);
- return ret;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_bucket, tmp_sig, buckets,
+ &ret, match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi_avx(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+#endif
+ default:
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &ret,
+ match_per_key, set_id);
+ if (ret < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &ret, match_per_key, set_id);
+ return ret;
+ }
}
uint32_t
@@ -252,16 +300,34 @@ rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
for (i = 0; i < num_keys; i++) {
match_cnt_tmp = 0;
- search_bucket_multi(prim_buckets[i], tmp_sig[i],
- buckets, &match_cnt_tmp, match_per_key,
- &set_ids[i*match_per_key]);
- if (match_cnt_tmp < match_per_key)
- search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_buckets[i], tmp_sig[i],
buckets, &match_cnt_tmp, match_per_key,
&set_ids[i*match_per_key]);
- match_count[i] = match_cnt_tmp;
- if (match_cnt_tmp != 0)
- ret++;
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &match_cnt_tmp,
+ match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ ret++;
+ break;
+#endif
+ default:
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ ret++;
+ }
}
return ret;
}
@@ -271,6 +337,7 @@ try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
member_sig_t sig, member_set_t set_id)
{
int i;
+
/* If not full then insert into one slot*/
for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
@@ -292,12 +359,24 @@ try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
static inline int
try_overwrite(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
- member_sig_t sig, member_set_t set_id)
+ member_sig_t sig, member_set_t set_id,
+ enum rte_member_sig_compare_function cmp_fn)
{
- if (insert_overwrite_search(prim, sig, buckets, set_id) ||
- insert_overwrite_search(sec, sig, buckets,
- set_id))
- return 0;
+ switch (cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (insert_overwrite_search_avx(prim, sig, buckets, set_id) ||
+ insert_overwrite_search_avx(sec, sig, buckets,
+ set_id))
+ return 0;
+ break;
+#endif
+ default:
+ if (insert_overwrite_search(prim, sig, buckets, set_id) ||
+ insert_overwrite_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ }
return -1;
}
@@ -398,7 +477,7 @@ rte_member_add_ht(const struct rte_member_setsum *ss,
/* if it is cache based filter, we try overwriting existing entry */
if (ss->cache) {
ret = try_overwrite(buckets, prim_bucket, sec_bucket, tmp_sig,
- set_id);
+ set_id, ss->sig_cmp_fn);
if (ret != -1)
return ret;
}
diff --git a/lib/librte_member/rte_member_x86.h b/lib/librte_member/rte_member_x86.h
new file mode 100644
index 0000000..16d75e8
--- /dev/null
+++ b/lib/librte_member/rte_member_x86.h
@@ -0,0 +1,107 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_X86_H_
+#define _RTE_MEMBER_X86_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <x86intrin.h>
+
+#if defined(RTE_MACHINE_CPUFLAG_AVX2)
+
+static inline int
+insert_overwrite_search_avx(uint32_t bucket, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ if (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ buckets[bucket].sets[hit_idx] = set_id;
+ return 1;
+ }
+ return 0;
+}
+
+static inline int
+search_bucket_single_avx(uint32_t bucket, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ if (buckets[bucket].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket].sets[hit_idx];
+ return 1;
+ }
+ hitmask &= ~(3U << ((hit_idx) << 1));
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi_avx(uint32_t bucket, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ if (buckets[bucket].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket].sets[hit_idx];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ hitmask &= ~(3U << ((hit_idx) << 1));
+ }
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_X86_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v4 5/7] member: enable the library
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
` (3 preceding siblings ...)
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 4/7] member: add AVX for HT mode Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-10-02 15:47 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 6/7] test/member: add functional and perf tests Yipeng Wang
` (2 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
This patch enables the Membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
MAINTAINERS | 8 +++++++-
config/common_base | 5 +++++
lib/librte_member/Makefile | 2 ++
mk/rte.app.mk | 2 ++
4 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index a0cd75e..adb8e2c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -726,6 +726,13 @@ F: test/test/test_lpm*
F: test/test/test_func_reentrancy.c
F: test/test/test_xmmt_ops.h
+Membership - EXPERIMENTAL
+M: Yipeng Wang <yipeng1.wang@intel.com>
+M: Sameh Gobriel <sameh.gobriel@intel.com>
+F: lib/librte_member/
+F: doc/guides/prog_guide/member_lib.rst
+F: test/test/test_member*
+
Traffic metering
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
F: lib/librte_meter/
@@ -734,7 +741,6 @@ F: test/test/test_meter.c
F: examples/qos_meter/
F: doc/guides/sample_app_ug/qos_metering.rst
-
Other libraries
---------------
diff --git a/config/common_base b/config/common_base
index 5e97a08..5e31ced 100644
--- a/config/common_base
+++ b/config/common_base
@@ -595,6 +595,11 @@ CONFIG_RTE_LIBRTE_HASH_DEBUG=n
CONFIG_RTE_LIBRTE_EFD=y
#
+# Compile librte_member
+#
+CONFIG_RTE_LIBRTE_MEMBER=y
+
+#
# Compile librte_jobstats
#
CONFIG_RTE_LIBRTE_JOBSTATS=y
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 50275ed..3bac1d0 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -37,6 +37,8 @@ LIB = librte_member.a
CFLAGS := -I$(SRCDIR) $(CFLAGS)
CFLAGS += $(WERROR_FLAGS) -O3
+LDLIBS += -lm
+
EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index c25fdd9..c79acf0 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -86,6 +86,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile
_LDLIBS-y += --whole-archive
_LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost
_LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs
_LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf
@@ -196,6 +197,7 @@ endif
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lm
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrt
_LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lm
ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lnuma
endif
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v4 5/7] member: enable the library
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 5/7] member: enable the library Yipeng Wang
@ 2017-10-02 15:47 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-02 15:47 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Wednesday, September 27, 2017 6:41 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v4 5/7] member: enable the library
>
> This patch enables the Membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> ---
> MAINTAINERS | 8 +++++++-
> config/common_base | 5 +++++
> lib/librte_member/Makefile | 2 ++
> mk/rte.app.mk | 2 ++
> 4 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a0cd75e..adb8e2c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -726,6 +726,13 @@ F: test/test/test_lpm*
> F: test/test/test_func_reentrancy.c
> F: test/test/test_xmmt_ops.h
>
> +Membership - EXPERIMENTAL
> +M: Yipeng Wang <yipeng1.wang@intel.com>
> +M: Sameh Gobriel <sameh.gobriel@intel.com>
> +F: lib/librte_member/
> +F: doc/guides/prog_guide/member_lib.rst
> +F: test/test/test_member*
> +
Add these last two items in patches 6 and 7, where you are adding the files.
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v4 6/7] test/member: add functional and perf tests
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
` (4 preceding siblings ...)
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 5/7] member: enable the library Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-10-02 16:20 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 7/7] doc: add membership documentation Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
This patch adds functional and performance tests for membership
library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
test/test/Makefile | 3 +
test/test/test_member.c | 676 +++++++++++++++++++++++++++++++++++++++++++
test/test/test_member_perf.c | 630 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 1309 insertions(+)
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
diff --git a/test/test/Makefile b/test/test/Makefile
index 42d9a49..b61dde3 100644
--- a/test/test/Makefile
+++ b/test/test/Makefile
@@ -123,6 +123,9 @@ SRCS-y += test_logs.c
SRCS-y += test_memcpy.c
SRCS-y += test_memcpy_perf.c
+
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd_perf.c
diff --git a/test/test/test_member.c b/test/test/test_member.c
new file mode 100644
index 0000000..46be9bc
--- /dev/null
+++ b/test/test/test_member.c
@@ -0,0 +1,676 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* This test is for membership library's simple feature test */
+
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_member.h>
+#include <rte_byteorder.h>
+#include <rte_random.h>
+#include <rte_debug.h>
+#include <rte_ip.h>
+
+#include "test.h"
+
+struct rte_member_setsum *setsum_ht;
+struct rte_member_setsum *setsum_cache;
+struct rte_member_setsum *setsum_vbf;
+
+/* 5-tuple key type */
+struct flow_key {
+ uint32_t ip_src;
+ uint32_t ip_dst;
+ uint16_t port_src;
+ uint16_t port_dst;
+ uint8_t proto;
+} __attribute__((packed));
+
+/* Keys used by unit test functions */
+static struct flow_key keys[5] = {
+ {
+ .ip_src = IPv4(0x03, 0x02, 0x01, 0x00),
+ .ip_dst = IPv4(0x07, 0x06, 0x05, 0x04),
+ .port_src = 0x0908,
+ .port_dst = 0x0b0a,
+ .proto = 0x0c,
+ },
+ {
+ .ip_src = IPv4(0x13, 0x12, 0x11, 0x10),
+ .ip_dst = IPv4(0x17, 0x16, 0x15, 0x14),
+ .port_src = 0x1918,
+ .port_dst = 0x1b1a,
+ .proto = 0x1c,
+ },
+ {
+ .ip_src = IPv4(0x23, 0x22, 0x21, 0x20),
+ .ip_dst = IPv4(0x27, 0x26, 0x25, 0x24),
+ .port_src = 0x2928,
+ .port_dst = 0x2b2a,
+ .proto = 0x2c,
+ },
+ {
+ .ip_src = IPv4(0x33, 0x32, 0x31, 0x30),
+ .ip_dst = IPv4(0x37, 0x36, 0x35, 0x34),
+ .port_src = 0x3938,
+ .port_dst = 0x3b3a,
+ .proto = 0x3c,
+ },
+ {
+ .ip_src = IPv4(0x43, 0x42, 0x41, 0x40),
+ .ip_dst = IPv4(0x47, 0x46, 0x45, 0x44),
+ .port_src = 0x4948,
+ .port_dst = 0x4b4a,
+ .proto = 0x4c,
+ }
+};
+
+uint32_t test_set[5] = {1, 2, 3, 4, 5};
+
+#define ITERATIONS 3
+#define KEY_SIZE 4
+
+#define MAX_ENTRIES (1 << 16)
+uint8_t gened_keys[MAX_ENTRIES][KEY_SIZE];
+
+static struct rte_member_parameters params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+
+ /*num_set and false_positive_rate only relevant to vBF setsum*/
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+};
+
+/*
+ * Sequence of operations for find existing setsummary
+ *
+ * - create setsum
+ * - find existing setsum: hit
+ * - find non-existing setsum: miss
+ *
+ */
+static int
+test_member_find_existing(void)
+{
+ struct rte_member_setsum *tmp_setsum = NULL, *result = NULL;
+ struct rte_member_parameters tmp_params = {
+ .name = "member_find_existing",
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ /* Create */
+ tmp_setsum = rte_member_create(&tmp_params);
+ TEST_ASSERT(tmp_setsum != NULL, "setsum creation failed");
+
+ /* Try to find existing hash table */
+ result = rte_member_find_existing("member_find_existing");
+ TEST_ASSERT(result == tmp_setsum, "could not find existing setsum");
+
+ /* Try to find non-existing hash table */
+ result = rte_member_find_existing("member_find_non_existing");
+ TEST_ASSERT(result == NULL, "found setsum that shouldn't exist");
+
+ /* Cleanup. */
+ rte_member_free(tmp_setsum);
+
+ return 0;
+}
+
+/*
+ * Test for bad creating parameters
+ */
+static int
+test_member_create_bad_param(void)
+{
+ struct rte_member_setsum *bad_setsum = NULL;
+ struct rte_member_parameters bad_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ printf("Expected error section begin...\n");
+ bad_params.name = "bad_param1";
+ bad_params.num_set = 0;
+ bad_params.type = RTE_MEMBER_TYPE_VBF;
+ /* test with 0 set for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "number of set for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param2";
+ bad_params.false_positive_rate = 0;
+ bad_params.num_set = 32;
+ /* test with 0 false positive for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "false positive rate for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param3";
+ bad_params.false_positive_rate = 0.03;
+ bad_params.num_keys = 0;
+ /* test with 0 key per BF for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "num_keys for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param4";
+ bad_params.type = RTE_MEMBER_TYPE_HT;
+ bad_params.num_keys = RTE_MEMBER_BUCKET_ENTRIES / 2;
+ /* test with less than 1 bucket for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with too few "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ bad_params.num_keys = RTE_MEMBER_ENTRIES_MAX + 1;
+ /* test with more than maximum entries for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with to many "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ /* test with same name should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with existed "
+ "name\n");
+ return -1;
+ }
+ printf("Expected error section end...\n");
+ rte_member_free(bad_setsum);
+ return 0;
+}
+
+/* Create test setsummaries. */
+static int test_member_create(void)
+{
+ params.key_len = sizeof(struct flow_key);
+
+ params.name = "test_member_ht";
+ params.is_cache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.is_cache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+ params.name = "test_member_vbf";
+ params.type = RTE_MEMBER_TYPE_VBF;
+ setsum_vbf = rte_member_create(¶ms);
+
+ if (setsum_ht == NULL || setsum_cache == NULL || setsum_vbf == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ printf("Creation of setsums success\n");
+ return 0;
+}
+
+static int test_member_insert(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_add(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[i], test_set[i]);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "insert error");
+ }
+ printf("insert key success\n");
+ return 0;
+}
+
+static int test_member_lookup(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ member_set_t set_ids_ht[5] = {0};
+ member_set_t set_ids_cache[5] = {0};
+ member_set_t set_ids_vbf[5] = {0};
+
+ uint32_t num_key_ht = 5;
+ uint32_t num_key_cache = 5;
+ uint32_t num_key_vbf = 5;
+
+ const void *key_array[5];
+
+ /* single lookup test */
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "single lookup function error");
+
+ TEST_ASSERT(set_ht == test_set[i] &&
+ set_cache == test_set[i] &&
+ set_vbf == test_set[i],
+ "single lookup set value error");
+ }
+ printf("lookup single key success\n");
+
+ /* bulk lookup test */
+ for (i = 0; i < 5; i++)
+ key_array[i] = &keys[i];
+
+ ret_ht = rte_member_lookup_bulk(setsum_ht, &key_array[0],
+ num_key_ht, set_ids_ht);
+
+ ret_cache = rte_member_lookup_bulk(setsum_cache, &key_array[0],
+ num_key_cache, set_ids_cache);
+
+ ret_vbf = rte_member_lookup_bulk(setsum_vbf, &key_array[0],
+ num_key_vbf, set_ids_vbf);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "bulk lookup function error");
+
+ for (i = 0; i < 5; i++) {
+ TEST_ASSERT((set_ids_ht[i] == test_set[i]) &&
+ (set_ids_cache[i] == test_set[i]) &&
+ (set_ids_vbf[i] == test_set[i]),
+ "bulk lookup result error");
+ }
+
+ return 0;
+}
+
+static int test_member_delete(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_delete(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_delete(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_delete(setsum_vbf, &keys[i], test_set[i]);
+ /* VBF does not support delete yet, so return error code */
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key deletion function error");
+ TEST_ASSERT(ret_vbf < 0,
+ "vbf does not support deletion, error");
+ }
+
+ for (i = 0; i < 5; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key lookup function error");
+ TEST_ASSERT(set_ht == RTE_MEMBER_NO_MATCH &&
+ ret_cache == RTE_MEMBER_NO_MATCH,
+ "key deletion failed");
+ }
+ printf("delete success\n");
+ return 0;
+}
+
+static int test_member_multimatch(void)
+{
+ int ret_ht, ret_vbf, ret_cache;
+ member_set_t set_ids_ht[32] = {0};
+ member_set_t set_ids_vbf[32] = {0};
+ member_set_t set_ids_cache[32] = {0};
+
+ member_set_t set_ids_ht_m[5][32] = {{0} };
+ member_set_t set_ids_vbf_m[5][32] = {{0} };
+ member_set_t set_ids_cache_m[5][32] = {{0} };
+
+ uint32_t match_count_ht[5];
+ uint32_t match_count_vbf[5];
+ uint32_t match_count_cache[5];
+
+ uint32_t num_key_ht = 5;
+ uint32_t num_key_vbf = 5;
+ uint32_t num_key_cache = 5;
+
+ const void *key_array[5];
+
+ uint32_t i, j;
+
+ /* same key at most inserted 2*entry_per_bucket times for HT mode */
+ for (i = 1; i < 33; i++) {
+ for (j = 0; j < 5; j++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[j], i);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[j], i);
+ ret_cache = rte_member_add(setsum_cache, &keys[j], i);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_vbf >= 0 &&
+ ret_cache >= 0,
+ "insert function error");
+ }
+ }
+
+ /* single multimatch test */
+ for (i = 0; i < 5; i++) {
+ ret_vbf = rte_member_lookup_multi(setsum_vbf, &keys[i], 32,
+ set_ids_vbf);
+ ret_ht = rte_member_lookup_multi(setsum_ht, &keys[i], 32,
+ set_ids_ht);
+ ret_cache = rte_member_lookup_multi(setsum_cache, &keys[i], 32,
+ set_ids_cache);
+ /*
+ * for cache mode, it does not support multimatch
+ * the mutimatch should work like single match
+ */
+ TEST_ASSERT(ret_ht == 32 && ret_vbf == 32 && ret_cache == 1,
+ "single lookup_multi error");
+ TEST_ASSERT(set_ids_cache[0] == 32,
+ "single lookup_multi cache error");
+
+ for (j = 1; j < 33; j++) {
+ TEST_ASSERT(set_ids_ht[j-1] == j &&
+ set_ids_vbf[j-1] == j,
+ "single multimatch lookup error");
+ }
+ }
+ printf("lookup single key for multimatch success\n");
+
+ /* bulk multimatch test */
+
+ for (i = 0; i < 5; i++)
+ key_array[i] = &keys[i];
+ ret_vbf = rte_member_lookup_multi_bulk(setsum_vbf,
+ &key_array[0], num_key_ht, 32, match_count_vbf,
+ (member_set_t *)set_ids_vbf_m);
+
+ ret_ht = rte_member_lookup_multi_bulk(setsum_ht,
+ &key_array[0], num_key_vbf, 32, match_count_ht,
+ (member_set_t *)set_ids_ht_m);
+
+ ret_cache = rte_member_lookup_multi_bulk(setsum_cache,
+ &key_array[0], num_key_cache, 32, match_count_cache,
+ (member_set_t *)set_ids_cache_m);
+
+
+ for (j = 0; j < 5; j++) {
+ TEST_ASSERT(match_count_ht[j] == 32,
+ "bulk multimatch lookup HT match count error");
+ TEST_ASSERT(match_count_vbf[j] == 32,
+ "bulk multimatch lookup vBF match count error");
+ TEST_ASSERT(match_count_cache[j] == 1,
+ "bulk multimatch lookup CACHE match count error");
+ TEST_ASSERT(set_ids_cache_m[j][0] == 32,
+ "bulk multimatch lookup CACHE set value error");
+
+ for (i = 1; i < 33; i++) {
+ TEST_ASSERT(set_ids_ht_m[j][i-1] == i,
+ "bulk multimatch lookup HT set value error");
+ TEST_ASSERT(set_ids_vbf_m[j][i-1] == i,
+ "bulk multimatch lookup vBF set value error");
+ }
+ }
+
+ printf("lookup for bulk multimatch success\n");
+
+ return 0;
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, KEY_SIZE);
+}
+
+static void
+setup_keys_and_data(void)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ /* Reset all arrays */
+ for (i = 0; i < KEY_SIZE; i++)
+ gened_keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < MAX_ENTRIES; i++) {
+ for (j = 0; j < KEY_SIZE; j++)
+ gened_keys[i][j] = rte_rand() & 0xFF;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(gened_keys, MAX_ENTRIES, KEY_SIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < MAX_ENTRIES - 1; i++) {
+ if (memcmp(gened_keys[i], gened_keys[i + 1],
+ KEY_SIZE) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < KEY_SIZE; j++)
+ gened_keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+}
+
+static inline int
+add_gened_keys(struct rte_member_setsum *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret >= 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &gened_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static inline int
+add_gened_keys_cache(struct rte_member_setsum *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret == 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &gened_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static int
+test_member_loadfactor(void)
+{
+ unsigned int j;
+ unsigned int added_keys, average_keys_added = 0;
+ int ret;
+
+ setup_keys_and_data();
+
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+
+ params.key_len = KEY_SIZE;
+ params.name = "test_member_ht";
+ params.is_cache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.is_cache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+
+ if (setsum_ht == NULL || setsum_cache == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ /* test HT mode */
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_gened_keys(setsum_ht, &added_keys);
+ if (ret != -ENOSPC) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_ht);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when no space(non-cache) = %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+
+ /* test cache mode */
+ added_keys = average_keys_added = 0;
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_gened_keys_cache(setsum_cache, &added_keys);
+ if (ret != 1) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_cache);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when eviction happens(cache)= %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+ return 0;
+}
+
+static void
+perform_free(void)
+{
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+}
+
+static int
+test_member(void)
+{
+ if (test_member_create_bad_param() < 0)
+ return -1;
+
+ if (test_member_find_existing() < 0)
+ return -1;
+
+ if (test_member_create() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_insert() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_lookup() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_delete() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_multimatch() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_loadfactor() < 0) {
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ return -1;
+ }
+
+ perform_free();
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_autotest, test_member);
diff --git a/test/test/test_member_perf.c b/test/test/test_member_perf.c
new file mode 100644
index 0000000..55727c5
--- /dev/null
+++ b/test/test/test_member_perf.c
@@ -0,0 +1,630 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <inttypes.h>
+
+#include <rte_lcore.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+#include <rte_memcpy.h>
+#include <rte_thash.h>
+#include <rte_member.h>
+
+#include "test.h"
+
+#define NUM_KEYSIZES 10
+#define NUM_SHUFFLES 10
+#define MAX_KEYSIZE 64
+#define MAX_ENTRIES (1 << 19)
+#define KEYS_TO_ADD (MAX_ENTRIES * 75 / 100) /* 75% table utilization */
+#define NUM_LOOKUPS (KEYS_TO_ADD * 5) /* Loop among keys added, several times */
+#define VBF_SET_CNT 16
+#define BURST_SIZE 64
+#define VBF_FALSE_RATE 0.03
+
+static unsigned int test_socket_id;
+
+enum sstype {
+ HT = 0,
+ CACHE,
+ VBF,
+ NUM_TYPE
+};
+
+enum operations {
+ ADD = 0,
+ LOOKUP,
+ LOOKUP_BULK,
+ LOOKUP_MULTI,
+ LOOKUP_MULTI_BULK,
+ DELETE,
+ LOOKUP_MISS,
+ NUM_OPERATIONS
+};
+
+struct member_perf_params {
+ struct rte_member_setsum *setsum[NUM_TYPE];
+ uint32_t key_size;
+ unsigned int cycle;
+};
+
+static uint32_t hashtest_key_lens[] = {
+ /* standard key sizes */
+ 4, 8, 16, 32, 48, 64,
+ /* IPv4 SRC + DST + protocol, unpadded */
+ 9,
+ /* IPv4 5-tuple, unpadded */
+ 13,
+ /* IPv6 5-tuple, unpadded */
+ 37,
+ /* IPv6 5-tuple, padded to 8-byte boundary */
+ 40
+};
+
+/* Array to store number of cycles per operation */
+uint64_t cycles[NUM_TYPE][NUM_KEYSIZES][NUM_OPERATIONS];
+uint64_t false_data[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_bulk[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi_bulk[NUM_TYPE][NUM_KEYSIZES];
+
+uint64_t false_hit[NUM_TYPE][NUM_KEYSIZES];
+
+member_set_t data[NUM_TYPE][/* Array to store the data */KEYS_TO_ADD];
+
+/* Array to store all input keys */
+uint8_t keys[KEYS_TO_ADD][MAX_KEYSIZE];
+
+/* Shuffle the keys that have been added, so lookups will be totally random */
+static void
+shuffle_input_keys(struct member_perf_params *params)
+{
+ member_set_t temp_data;
+ unsigned int i, j;
+ uint32_t swap_idx;
+ uint8_t temp_key[MAX_KEYSIZE];
+
+ for (i = KEYS_TO_ADD - 1; i > 0; i--) {
+ swap_idx = rte_rand() % i;
+ memcpy(temp_key, keys[i], hashtest_key_lens[params->cycle]);
+ memcpy(keys[i], keys[swap_idx],
+ hashtest_key_lens[params->cycle]);
+ memcpy(keys[swap_idx], temp_key,
+ hashtest_key_lens[params->cycle]);
+ for (j = 0; j < NUM_TYPE; j++) {
+ temp_data = data[j][i];
+ data[j][i] = data[j][swap_idx];
+ data[j][swap_idx] = temp_data;
+ }
+ }
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, MAX_KEYSIZE);
+}
+
+struct rte_member_parameters member_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = 4, /* Length of hash key. */
+
+ /* num_set and false_positive_rate only relevant to vBF setsum*/
+ .num_set = VBF_SET_CNT,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 0,
+ .sec_hash_seed = 1,
+ .socket_id = 0, /* NUMA Socket ID for memory. */
+ };
+
+static int
+setup_keys_and_data(struct member_perf_params *params, unsigned int cycle,
+ int miss)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ params->key_size = hashtest_key_lens[cycle];
+ params->cycle = cycle;
+
+ /* Reset all arrays */
+ for (i = 0; i < params->key_size; i++)
+ keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+
+ data[HT][i] = data[CACHE][i] = (rte_rand() & 0x7FFE) + 1;
+ data[VBF][i] = rte_rand() % VBF_SET_CNT + 1;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(keys, KEYS_TO_ADD, MAX_KEYSIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < KEYS_TO_ADD - 1; i++) {
+ if (memcmp(keys[i], keys[i + 1],
+ params->key_size) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+
+ /* Shuffle the random values again */
+ shuffle_input_keys(params);
+
+ /* For testing miss lookup, we insert half and lookup the other half */
+ unsigned int entry_cnt, bf_key_cnt;
+ if (!miss) {
+ entry_cnt = MAX_ENTRIES;
+ bf_key_cnt = KEYS_TO_ADD;
+ } else {
+ entry_cnt = MAX_ENTRIES / 2;
+ bf_key_cnt = KEYS_TO_ADD / 2;
+ }
+ member_params.false_positive_rate = VBF_FALSE_RATE;
+ member_params.key_len = params->key_size;
+ member_params.socket_id = test_socket_id;
+ member_params.num_keys = entry_cnt;
+ member_params.name = "test_member_ht";
+ member_params.is_cache = 0;
+ member_params.type = RTE_MEMBER_TYPE_HT;
+ params->setsum[HT] = rte_member_create(&member_params);
+ if (params->setsum[HT] == NULL)
+ fprintf(stderr, "ht create fail\n");
+
+ member_params.name = "test_member_cache";
+ member_params.is_cache = 1;
+ params->setsum[CACHE] = rte_member_create(&member_params);
+ if (params->setsum[CACHE] == NULL)
+ fprintf(stderr, "CACHE create fail\n");
+
+ member_params.name = "test_member_vbf";
+ member_params.type = RTE_MEMBER_TYPE_VBF;
+ member_params.num_keys = bf_key_cnt;
+ params->setsum[VBF] = rte_member_create(&member_params);
+ if (params->setsum[VBF] == NULL)
+ fprintf(stderr, "VBF create fail\n");
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] == NULL)
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+timed_adds(struct member_perf_params *params, int type)
+{
+ const uint64_t start_tsc = rte_rdtsc();
+ unsigned int i, a;
+ int32_t ret;
+
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][ADD] = time_taken / KEYS_TO_ADD;
+ return 0;
+}
+
+static int
+timed_lookups(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+
+ false_data[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+ member_set_t result;
+ int ret;
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != data[type][j])
+ false_data[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ member_set_t result[BURST_SIZE] = {0};
+ const void *keys_burst[BURST_SIZE];
+ int ret;
+
+ false_data_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_bulk(params->setsum[type],
+ &keys_burst[0],
+ BURST_SIZE,
+ result);
+ if (ret <= 0) {
+ printf("lookup bulk has wrong return value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k] != data[type][data_idx])
+ false_data_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_BULK] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_multimatch(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ member_set_t result[RTE_MEMBER_BUCKET_ENTRIES] = {0};
+ int ret;
+ false_data_multi[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup_multi(params->setsum[type],
+ &keys[j], RTE_MEMBER_BUCKET_ENTRIES, result);
+ if (type != CACHE && ret <= 0) {
+ printf("lookup multi has wrong return value %d,"
+ "type %d\n", ret, type);
+ }
+ if (result[0] != data[type][j])
+ false_data_multi[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_multimatch_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ member_set_t result[BURST_SIZE][RTE_MEMBER_BUCKET_ENTRIES] = {{0} };
+ const void *keys_burst[BURST_SIZE];
+ uint32_t match_count[BURST_SIZE];
+ int ret;
+
+ false_data_multi_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_multi_bulk(
+ params->setsum[type],
+ &keys_burst[0], BURST_SIZE,
+ RTE_MEMBER_BUCKET_ENTRIES, match_count,
+ (member_set_t *)result);
+ if (ret < 0) {
+ printf("lookup multimatch bulk has wrong return"
+ " value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ if (type != CACHE && match_count[k] == 0) {
+ printf("lookup multimatch bulk get "
+ "wrong match count\n");
+ return -1;
+ }
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k][0] != data[type][data_idx])
+ false_data_multi_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI_BULK] = time_taken /
+ NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_deletes(struct member_perf_params *params, int type)
+{
+ unsigned int i;
+ int32_t ret;
+
+ if (type == VBF)
+ return 0;
+ const uint64_t start_tsc = rte_rdtsc();
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_delete(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (type != CACHE && ret < 0) {
+ printf("delete error\n");
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][DELETE] = time_taken / KEYS_TO_ADD;
+
+ return 0;
+}
+
+static int
+timed_miss_lookup(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ int ret;
+
+ false_hit[type][params->cycle] = 0;
+
+ for (i = 0; i < KEYS_TO_ADD / 2; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ unsigned int a;
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t start_tsc = rte_rdtsc();
+ member_set_t result;
+
+ for (i = 0; i < 2 * NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = KEYS_TO_ADD / 2; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != RTE_MEMBER_NO_MATCH)
+ false_hit[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MISS] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static void
+perform_frees(struct member_perf_params *params)
+{
+ int i;
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] != NULL) {
+ rte_member_free(params->setsum[i]);
+ params->setsum[i] = NULL;
+ }
+ }
+}
+
+static int
+exit_with_fail(const char *testname, struct member_perf_params *params,
+ unsigned int i, unsigned int j)
+{
+ printf("<<<<<Test %s failed at keysize %d iteration %d type %d>>>>>\n",
+ testname, hashtest_key_lens[params->cycle], i, j);
+ perform_frees(params);
+ return -1;
+}
+
+static int
+run_all_tbl_perf_tests(void)
+{
+ unsigned int i, j, k;
+ struct member_perf_params params;
+
+ printf("Measuring performance, please wait\n");
+ fflush(stdout);
+
+ test_socket_id = rte_socket_id();
+
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 0) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+
+ if (timed_adds(¶ms, j) < 0)
+ return exit_with_fail("timed_adds", ¶ms,
+ i, j);
+
+ for (k = 0; k < NUM_SHUFFLES; k++)
+ shuffle_input_keys(¶ms);
+
+ if (timed_lookups(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups", ¶ms,
+ i, j);
+
+ if (timed_lookups_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_bulk",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi_bulk",
+ ¶ms, i, j);
+
+ if (timed_deletes(¶ms, j) < 0)
+ return exit_with_fail("timed_deletes", ¶ms,
+ i, j);
+
+ /* Print a dot to show progress on operations */
+ }
+ printf(".");
+ fflush(stdout);
+
+ perform_frees(¶ms);
+ }
+
+ /* test false postivie rate using un-inserted keys */
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 1) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+ if (timed_miss_lookup(¶ms, j) < 0)
+ return exit_with_fail("timed_miss_lookup",
+ ¶ms, i, j);
+ }
+ perform_frees(¶ms);
+ }
+
+ printf("\nResults (in CPU cycles/operation)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "Add", "Lookup", "Lookup_bulk",
+ "lookup_multi", "lookup_multi_bulk", "Delete",
+ "miss_lookup");
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ for (k = 0; k < NUM_OPERATIONS; k++)
+ printf("%-18"PRIu64, cycles[j][i][k]);
+ printf("\n");
+ }
+ }
+
+ printf("\nFalse results rate (and false positive rate)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "fr_single", "fr_bulk", "fr_multi",
+ "fr_multi_bulk", "false_positive_rate");
+ /* key size not influence False rate so just print out one key size */
+ for (i = 0; i < 1; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ printf("%-18f", (float)false_data[j][i] / NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_hit[j][i] /
+ NUM_LOOKUPS);
+ printf("\n");
+ }
+ }
+ return 0;
+}
+
+static int
+test_member_perf(void)
+{
+
+ if (run_all_tbl_perf_tests() < 0)
+ return -1;
+
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v4 6/7] test/member: add functional and perf tests
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 6/7] test/member: add functional and perf tests Yipeng Wang
@ 2017-10-02 16:20 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-02 16:20 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Wednesday, September 27, 2017 6:41 PM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v4 6/7] test/member: add functional and perf tests
>
> This patch adds functional and performance tests for membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
...
> +++ b/test/test/test_member.c
...
> +
> +#define MAX_ENTRIES (1 << 16)
> +uint8_t gened_keys[MAX_ENTRIES][KEY_SIZE];
Gened? Is this "generated"? Maybe it is worth using the full word.
> +
> +static struct rte_member_parameters params = {
> + .num_keys = MAX_ENTRIES, /* Total hash table entries.
> */
> + .key_len = KEY_SIZE, /* Length of hash key. */
Align comments with tabs.
> +
> + /*num_set and false_positive_rate only relevant to vBF
> setsum*/
Add whitespaces around comment.
> + .num_set = 32,
> + .false_positive_rate = 0.03,
> + .prim_hash_seed = 1,
> + .sec_hash_seed = 11,
> + .socket_id = 0 /* NUMA Socket ID for memory. */
> +};
> +static int test_member_insert(void)
> +{
> + int ret_ht, ret_cache, ret_vbf, i;
> +
> + for (i = 0; i < 5; i++) {
Use macro for the value 5, used here and other functions.
> + ret_ht = rte_member_add(setsum_ht, &keys[i], test_set[i]);
> + ret_cache = rte_member_add(setsum_cache, &keys[i],
> + test_set[i]);
> + ret_vbf = rte_member_add(setsum_vbf, &keys[i],
> test_set[i]);
> + TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >=
> 0,
> + "insert error");
> + }
> + printf("insert key success\n");
> + return 0;
> +}
...
> +static int test_member_multimatch(void) {
> + int ret_ht, ret_vbf, ret_cache;
> + member_set_t set_ids_ht[32] = {0};
Same comment about the value 5 applies here, for the value 32.
> + member_set_t set_ids_vbf[32] = {0};
> + member_set_t set_ids_cache[32] = {0};
> +
> + member_set_t set_ids_ht_m[5][32] = {{0} };
> + member_set_t set_ids_vbf_m[5][32] = {{0} };
> + member_set_t set_ids_cache_m[5][32] = {{0} };
> +
> + uint32_t match_count_ht[5];
> + uint32_t match_count_vbf[5];
> + uint32_t match_count_cache[5];
> +
> + uint32_t num_key_ht = 5;
> + uint32_t num_key_vbf = 5;
> + uint32_t num_key_cache = 5;
> +
> + const void *key_array[5];
> +
> + uint32_t i, j;
> +
> + /* same key at most inserted 2*entry_per_bucket times for HT
> mode */
> + for (i = 1; i < 33; i++) {
This 33 can be expressed as 32 + 1 (using macro for 32).
Also, add a comment explaining why you are skipping value 0.
...
> --- /dev/null
> +++ b/test/test/test_member_perf.c
> @@ -0,0 +1,630 @@
...
> +static int
> +timed_lookups_bulk(struct member_perf_params *params, int type) {
> + unsigned int i, j, k;
> + member_set_t result[BURST_SIZE] = {0};
> + const void *keys_burst[BURST_SIZE];
> + int ret;
> +
> + false_data_bulk[type][params->cycle] = 0;
> +
> + const uint64_t start_tsc = rte_rdtsc();
> +
> + for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
> + for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
> + for (k = 0; k < BURST_SIZE; k++)
> + keys_burst[k] = keys[j * BURST_SIZE + k];
> +
> + ret = rte_member_lookup_bulk(params-
> >setsum[type],
> + &keys_burst[0],
Using keys_burst directly is equivalent to this, right?
...
> +static int
> +timed_lookups_multimatch(struct member_perf_params *params, int
> type) {
> + unsigned int i, j;
> + member_set_t result[RTE_MEMBER_BUCKET_ENTRIES] = {0};
> + int ret;
> + false_data_multi[type][params->cycle] = 0;
> +
> + const uint64_t start_tsc = rte_rdtsc();
> +
> + for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
> + for (j = 0; j < KEYS_TO_ADD; j++) {
> + ret = rte_member_lookup_multi(params-
> >setsum[type],
> + &keys[j], RTE_MEMBER_BUCKET_ENTRIES,
> result);
> + if (type != CACHE && ret <= 0) {
> + printf("lookup multi has wrong return value
> %d,"
> + "type %d\n", ret, type);
> + }
> + if (result[0] != data[type][j])
Why using always result[0]? A comment would be good.
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v4 7/7] doc: add membership documentation
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
` (5 preceding siblings ...)
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 6/7] test/member: add functional and perf tests Yipeng Wang
@ 2017-09-27 17:40 ` Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-09-27 17:40 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
This patch adds the documentation for membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: John McNamara <john.mcnamara@intel.com>
---
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 +++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 ++++
doc/guides/prog_guide/img/member_i6.svg | 332 +++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 420 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
12 files changed, 3595 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 19e0d4f..fe87e09 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -105,7 +105,8 @@ The public API headers are grouped by topics:
[LPM IPv4 route] (@ref rte_lpm.h),
[LPM IPv6 route] (@ref rte_lpm6.h),
[ACL] (@ref rte_acl.h),
- [EFD] (@ref rte_efd.h)
+ [EFD] (@ref rte_efd.h),
+ [member] (@ref rte_member.h)
- **QoS**:
[metering] (@ref rte_meter.h),
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index 823554f..b792d6d 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -58,6 +58,7 @@ INPUT = doc/api/doxy-api-index.md \
lib/librte_mempool \
lib/librte_meter \
lib/librte_metrics \
+ lib/librte_member \
lib/librte_net \
lib/librte_pdump \
lib/librte_pipeline \
diff --git a/doc/guides/prog_guide/img/member_i1.svg b/doc/guides/prog_guide/img/member_i1.svg
new file mode 100644
index 0000000..fc5f56a
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i1.svg
@@ -0,0 +1,1613 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i1.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.18709in" height="4.75757in"
+ viewBox="0 0 517.471 342.545" xml:space="preserve" color-interpolation-filters="sRGB" class="st61">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false">
+ <v:userDefs>
+ <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/>
+ <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/>
+ </v:userDefs>
+ </v:documentProperties>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:3}
+ .st3 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em;opacity:0.219608}
+ .st4 {font-size:1em}
+ .st5 {fill:none;stroke:#41719c;stroke-width:3}
+ .st6 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em}
+ .st7 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em;opacity:0.219608}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st9 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st11 {fill:none;stroke:none;stroke-width:0.25}
+ .st12 {fill:#ffffff;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:0.75em;font-weight:bold}
+ .st14 {marker-end:url(#mrkr5-63);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st15 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st16 {fill:#5b9bd5;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st17 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st18 {fill:#feffff;font-family:Calibri;font-size:0.499992em}
+ .st19 {fill:#deebf6;stroke:#c8c8c8;stroke-width:0.25}
+ .st20 {fill:#000000;font-family:Calibri;font-size:0.499992em}
+ .st21 {marker-end:url(#mrkr5-178);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:#ff0000;font-family:Calibri;font-size:0.666664em}
+ .st24 {fill:#5b9bd5;fill-opacity:0.22}
+ .st25 {stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st26 {fill:#ffffff}
+ .st27 {stroke:#0070c0;stroke-width:0.25}
+ .st28 {fill:#5b9bd5;stroke:#0070c0;stroke-width:0.25}
+ .st29 {fill:#5b9bd5;stroke:#ffffff;stroke-width:0.25}
+ .st30 {fill:#5b9bd5}
+ .st31 {stroke:#c8c8c8;stroke-width:0.25}
+ .st32 {fill:#acccea;stroke:#c8c8c8;stroke-width:0.25}
+ .st33 {fill:#5b9bd5;fill-opacity:0.22;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st34 {fill:#000000;fill-opacity:0;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st35 {fill:url(#grad30-309);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st36 {fill:url(#grad25-313);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st37 {fill:url(#grad35-317);stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st38 {fill:url(#grad36-325);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st39 {fill:url(#grad40-335);stroke:#000000;stroke-linecap:butt;stroke-width:0.130208}
+ .st40 {fill:url(#grad39-342);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st41 {fill:url(#grad40-355);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st42 {fill:none}
+ .st43 {stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st44 {stroke:#ffffff;stroke-linecap:butt;stroke-width:0.130208}
+ .st45 {fill:url(#grad30-383);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st46 {fill:url(#grad36-396);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st47 {fill:none;stroke:#c8c8c8;stroke-width:0.75}
+ .st48 {fill:#9a9a9a;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st49 {fill:url(#grad40-415);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st50 {fill:url(#grad40-419);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st51 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25}
+ .st52 {fill:url(#grad35-430);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st53 {stroke:#c8c8c8;stroke-width:0.75}
+ .st54 {stroke:#4f88bb;stroke-width:0.75}
+ .st55 {fill:#feffff;font-family:Calibri;font-size:0.416656em}
+ .st56 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st57 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st58 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:2.25}
+ .st59 {fill:none;stroke:#0070c0;stroke-width:2.25}
+ .st60 {fill:#595959;font-family:Arial;font-size:0.666664em}
+ .st61 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Patterns_And_Gradients">
+ <linearGradient id="grad30-309" v:fillPattern="30" v:foreground="#97c2e6" v:background="#4274a2" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#4274a2;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-313" v:fillPattern="25" v:foreground="#5491d3" v:background="#246ba6" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#5491d3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </linearGradient>
+ <pattern id="grad35-317" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-318)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-319)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-320)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-321)"/>
+ </pattern>
+ <linearGradient id="grad27-318" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-319" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-320" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-321" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-325" v:fillPattern="36" v:foreground="#c0dff1" v:background="#246ba6" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#c0dff1;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-335" v:fillPattern="40" v:foreground="#c8e5c8" v:background="#19bf19" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#c8e5c8;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#19bf19;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad39-342" v:fillPattern="39" v:foreground="#5599d7" v:background="#b9daf2" cx="1" cy="1" r="1">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-355" v:fillPattern="40" v:foreground="#5599d7" v:background="#214383" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#214383;stop-opacity:1"/>
+ </radialGradient>
+ <linearGradient id="grad30-383" v:fillPattern="30" v:foreground="#97c2e6" v:background="#6ba4dc" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#6ba4dc;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-396" v:fillPattern="36" v:foreground="#89bee9" v:background="#b9daf2" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#89bee9;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-415" v:fillPattern="40" v:foreground="#000000" v:background="#ffffff" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#000000;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffffff;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-419" v:fillPattern="40" v:foreground="#ffffff" v:background="#9a9a9a" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#9a9a9a;stop-opacity:1"/>
+ </radialGradient>
+ <pattern id="grad35-430" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-431)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-432)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-433)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-434)"/>
+ </pattern>
+ <linearGradient id="grad27-431" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-432" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-433" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-434" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ </defs>
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-63" class="st15" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-178" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/>
+ </v:userDefs>
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <v:layer v:name="Flowchart" v:index="0"/>
+ <g id="group165-1" transform="translate(21.7794,-24.0978)" v:mID="165" v:groupContext="group">
+ <title>Sheet.165</title>
+ <g id="group1-2" transform="translate(308.647,-25.7109)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-3" v:mID="2" v:groupContext="shape" transform="translate(11.5732,-58.1913)">
+ <title>Circle</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow2-4" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="shape3-13" v:mID="3" v:groupContext="shape" transform="translate(58.9839,-58.9839)">
+ <title>Circle.23</title>
+ <desc>List 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow3-14" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="17.73" y="318.02" class="st7" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="17.73" y="318.02" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <g id="shape4-19" v:mID="4" v:groupContext="shape">
+ <title>Circle.24</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow4-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="group5-29" transform="translate(50.7413,-4.53722)" v:mID="5" v:groupContext="group">
+ <title>Sheet.5</title>
+ <g id="shape6-30" v:mID="6" v:groupContext="shape" transform="translate(344.2,300.5) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-31" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st9"/>
+ </g>
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape7-34" v:mID="7" v:groupContext="shape" transform="translate(-0.884982,-14.7157)">
+ <title>Sheet.7</title>
+ <desc>setsum</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="12.9268" cy="336.238" width="25.86" height="12.6135"/>
+ <rect x="0" y="329.932" width="25.8535" height="12.6135" class="st11"/>
+ <text x="6.37" y="334.44" class="st12" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsu<tspan
+ x="10.49" dy="1.2em" class="st4">m</tspan></text> </g>
+ </g>
+ <g id="shape8-38" v:mID="8" v:groupContext="shape" transform="translate(72.5955,0)">
+ <title>Circle.29</title>
+ <desc>List 2 matching Criteria 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow8-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ </g>
+ <g id="group9-48" transform="translate(31.6515,-49.9094)" v:mID="9" v:groupContext="group">
+ <title>Sheet.9</title>
+ <g id="group10-49" transform="translate(99.5691,0)" v:mID="10" v:groupContext="group">
+ <title>Sheet.10</title>
+ <g id="shape11-50" v:mID="11" v:groupContext="shape" transform="translate(346.175,275.999) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st9"/>
+ </g>
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape12-54" v:mID="12" v:groupContext="shape" transform="translate(355.063,285.074) rotate(90)">
+ <title>Sheet.12</title>
+ <desc>Set Summary</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1985" cy="332.563" width="48.4" height="19.9638"/>
+ <rect x="0" y="322.581" width="48.397" height="19.9638" class="st11"/>
+ <text x="18.25" y="329.86" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Set <tspan
+ x="6.38" dy="1.2em" class="st4">Summary</tspan></text> </g>
+ </g>
+ <g id="shape13-58" v:mID="13" v:groupContext="shape" transform="translate(57.5835,-54.4467)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.9 342.55" class="st14"/>
+ </g>
+ <g id="shape14-64" v:mID="14" v:groupContext="shape" transform="translate(20.2363,-51.8439)">
+ <title>Sheet.14</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="25.3328" cy="333.471" width="50.67" height="18.1489"/>
+ <rect x="0" y="324.396" width="50.6656" height="18.1489" class="st11"/>
+ <text x="14.12" y="335.27" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape15-67" v:mID="15" v:groupContext="shape" transform="translate(5.02911,1.60865) rotate(-26.0815)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L39.25 342.55" class="st14"/>
+ </g>
+ <g id="shape16-72" v:mID="16" v:groupContext="shape" transform="translate(155.629,-33.273)">
+ <title>Sheet.16</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.34 342.55" class="st14"/>
+ </g>
+ <g id="shape17-77" v:mID="17" v:groupContext="shape" transform="translate(304.141,0.595416) rotate(25.6934)">
+ <title>Sheet.17</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L42.68 342.55" class="st14"/>
+ </g>
+ <g id="shape18-82" v:mID="18" v:groupContext="shape" transform="translate(102.642,654.842) rotate(180)">
+ <title>Sheet.18</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape19-87" v:mID="19" v:groupContext="shape" transform="translate(-15.1809,-33.9928)">
+ <title>Sheet.19</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.75" cy="338.045" width="85.5" height="9"/>
+ <rect x="0" y="333.545" width="85.5" height="9" class="st11"/>
+ <text x="5.06" y="339.85" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape20-90" v:mID="20" v:groupContext="shape" transform="translate(102.844,679.041) rotate(180)">
+ <title>Sheet.20</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape21-95" v:mID="21" v:groupContext="shape" transform="translate(-35.4309,-11.4928)">
+ <title>Sheet.21</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="337.971" width="108" height="9.14889"/>
+ <rect x="0" y="333.396" width="108" height="9.14889" class="st11"/>
+ <text x="6.36" y="339.77" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape22-98" v:mID="22" v:groupContext="shape" transform="translate(541.496,275.999) rotate(90)">
+ <title>Sheet.22</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape23-101" v:mID="23" v:groupContext="shape" transform="translate(541.496,300.198) rotate(90)">
+ <title>Sheet.23</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape24-104" v:mID="24" v:groupContext="shape" transform="translate(541.496,324.396) rotate(90)">
+ <title>Sheet.24</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ </g>
+ <g id="group25-107" transform="translate(285.961,-178.628)" v:mID="25" v:groupContext="group">
+ <title>Sheet.25</title>
+ <g id="shape26-108" v:mID="26" v:groupContext="shape" transform="translate(51.2583,-51.2583)">
+ <title>Circle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-109" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape27-112" v:mID="27" v:groupContext="shape" transform="translate(107.177,-55.9182)">
+ <title>Circle.156</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-113" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape28-116" v:mID="28" v:groupContext="shape" transform="translate(79.2174,-83.8773)">
+ <title>Circle.157</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow28-117" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape29-120" v:mID="29" v:groupContext="shape" transform="translate(153.775,-51.2583)">
+ <title>Circle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow29-121" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape30-124" v:mID="30" v:groupContext="shape" transform="translate(93.197,-18.6394)">
+ <title>Circle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow30-125" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape31-128" v:mID="31" v:groupContext="shape" transform="translate(27.4102,-57.9329) rotate(-7.12502)">
+ <title>Sheet.31</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L31.41 342.55" class="st14"/>
+ </g>
+ <g id="shape32-133" v:mID="32" v:groupContext="shape" transform="translate(182.13,-60.5772) rotate(9.46232)">
+ <title>Sheet.32</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L22.18 342.55" class="st14"/>
+ </g>
+ <g id="shape33-138" v:mID="33" v:groupContext="shape" transform="translate(47.8843,595.237) rotate(-160.346)">
+ <title>Sheet.33</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L63.11 342.55" class="st14"/>
+ </g>
+ <g id="shape34-143" v:mID="34" v:groupContext="shape" transform="translate(292.945,525.785) rotate(141.977)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L20.97 342.55" class="st14"/>
+ </g>
+ <g id="shape35-148" v:mID="35" v:groupContext="shape" transform="translate(-95.8971,591.793) rotate(-145.945)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L28.55 342.55" class="st14"/>
+ </g>
+ <g id="shape36-153" v:mID="36" v:groupContext="shape" transform="translate(37.2788,2.27374E-013)">
+ <title>Rectangle.167</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.8652" cy="335.555" width="21.74" height="13.9795"/>
+ <g id="shadow36-154" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st10"/>
+ <text x="5" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape37-158" v:mID="37" v:groupContext="shape" transform="translate(55.9182,2.27374E-013)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow37-159" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape38-163" v:mID="38" v:groupContext="shape" transform="translate(-1.65867E-013,-32.6189)">
+ <title>Rectangle.169</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.3796" cy="335.555" width="20.76" height="13.9795"/>
+ <g id="shadow38-164" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st10"/>
+ <text x="4.51" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape39-168" v:mID="39" v:groupContext="shape" transform="translate(18.6394,-32.6189)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow39-169" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape40-173" v:mID="40" v:groupContext="shape" transform="translate(197.019,626.053) rotate(161.565)">
+ <title>Sheet.40</title>
+ <path d="M0 328.31 A55.7483 27.2427 -124.2 0 0 42.37 334.19 L42.47 333.85" class="st21"/>
+ </g>
+ <g id="shape41-179" v:mID="41" v:groupContext="shape" transform="translate(154.607,584.177) rotate(161.121)">
+ <title>Sheet.41</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 319.39 A80.5593 29.9756 -101.99 0 0 41.7 325.37 L41.79 325.02" class="st21"/>
+ </g>
+ <g id="shape42-184" v:mID="42" v:groupContext="shape" transform="translate(3.02481,-66.7025)">
+ <title>Sheet.42</title>
+ <desc>Encode ID</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="19.4569" cy="335.555" width="38.92" height="13.9795"/>
+ <rect x="0" y="328.566" width="38.9138" height="13.9795" class="st11"/>
+ <text x="7.51" y="333.16" class="st23" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Encode <tspan
+ x="15.99" dy="1.2em" class="st4">ID</tspan></text> </g>
+ </g>
+ <g id="group43-188" transform="translate(12.0993,-165.858)" v:mID="43" v:groupContext="group">
+ <title>Sheet.43</title>
+ <g id="group44-189" transform="translate(7.21495,-75.757)" v:mID="44" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User</title>
+ <g id="shape45-190" v:mID="45" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.45</title>
+ <g id="shadow45-191" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape46-206" v:mID="46" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.46</title>
+ <g id="shadow46-207" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape47-210" v:mID="47" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.47</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape48-212" v:mID="48" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.48</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group49-215" transform="translate(7.21495,-47.1858)" v:mID="49" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.7</title>
+ <g id="shape50-216" v:mID="50" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.50</title>
+ <g id="shadow50-217" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape51-232" v:mID="51" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.51</title>
+ <g id="shadow51-233" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape52-236" v:mID="52" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.52</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape53-238" v:mID="53" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group54-241" transform="translate(7.21495,-18.6146)" v:mID="54" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.12</title>
+ <g id="shape55-242" v:mID="55" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.55</title>
+ <g id="shadow55-243" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape56-258" v:mID="56" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.56</title>
+ <g id="shadow56-259" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape57-262" v:mID="57" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape58-264" v:mID="58" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group59-267" transform="translate(171.161,-45.6707)" v:mID="59" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>Data Center</title>
+ <g id="shape60-268" v:mID="60" v:groupContext="shape" v:layerMember="0">
+ <title>Sheet.60</title>
+ <g id="shadow60-269" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st9"/>
+ </g>
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st10"/>
+ </g>
+ <g id="shape61-272" v:mID="61" v:groupContext="shape" v:layerMember="0" transform="translate(6.86487,-7.30475)">
+ <title>Sheet.61</title>
+ <g id="shadow61-273" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39
+ 332.44 L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69
+ L29.57 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64
+ 342.55 L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75
+ 342.55 L18.75 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69
+ L10.82 311.79 L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st9"/>
+ </g>
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39 332.44
+ L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69 L29.57
+ 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64 342.55
+ L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75 342.55 L18.75
+ 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69 L10.82 311.79
+ L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st10"/>
+ </g>
+ <g id="shape62-276" v:mID="62" v:groupContext="shape" v:layerMember="0" transform="translate(20.0835,-20.5174)">
+ <title>Sheet.62</title>
+ <g id="shadow62-277" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36
+ ZM23.46 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36
+ ZM2.27 341.36 A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z"
+ class="st24"/>
+ </g>
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36 ZM23.46
+ 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36 ZM2.27 341.36
+ A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z" class="st30"/>
+ </g>
+ <g id="shape63-282" v:mID="63" v:groupContext="shape" v:layerMember="0" transform="translate(14.2717,-12.5134)">
+ <title>Sheet.63</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <g id="shadow63-283" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17
+ 340.12 L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89
+ L43.09 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2
+ 342.55 ZM21.2 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69
+ L29.27 337.69 L29.27 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74
+ L-0 341.74 L-0 342.55 ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69
+ L8.08 337.69 L8.08 336.89 L-0 336.89 L-0 337.69 Z" class="st24"/>
+ </g>
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17 340.12
+ L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89 L43.09
+ 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2 342.55 ZM21.2
+ 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69 L29.27 337.69 L29.27
+ 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74 L-0 341.74 L-0 342.55
+ ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69 L8.08 337.69 L8.08 336.89
+ L-0 336.89 L-0 337.69 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group64-288" transform="translate(59.5234,-47.1858)" v:mID="64" v:groupContext="group">
+ <v:custProps>
+ <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false"
+ v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/>
+ <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true"
+ v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Device)"/>
+ <v:cp v:nameU="SubShapeType" v:lbl="SubShapeType" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Load balancer)"/>
+ </v:custProps>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/>
+ <v:ud v:nameU="SolSH" v:prompt="" v:val="VT14({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/>
+ <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <title>Load balancer</title>
+ <g id="shape65-289" v:mID="65" v:groupContext="shape" transform="translate(0,-1.653)">
+ <title>Sheet.65</title>
+ <g id="shadow65-290" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st24"/>
+ <path d="M0 332.02 L16.23 332.02" class="st25"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st25"/>
+ </g>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st30"/>
+ <path d="M0 332.02 L16.23 332.02" class="st31"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st31"/>
+ </g>
+ <g id="shape66-297" v:mID="66" v:groupContext="shape" transform="translate(1.81062,-2.91583)">
+ <title>Sheet.66</title>
+ <g id="shadow66-298" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44
+ L8.34 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45
+ 338.78 L10.78 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22
+ ZM10.48 335.2 L8.29 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46
+ 334.9 L10.21 334.58 L9.55 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19
+ 338.43 C4.19 339.56 5.11 340.48 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29
+ 7.38 336.37 6.25 336.37 ZM6.25 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02
+ 339.83 6.25 339.83 C5.47 339.83 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02
+ ZM2.62 338.14 L0 338.14 L0 338.71 L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73
+ 337.47 L1.95 337.47 L2.62 338.14 Z" class="st9"/>
+ </g>
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44 L8.34
+ 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45 338.78 L10.78
+ 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22 ZM10.48 335.2 L8.29
+ 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46 334.9 L10.21 334.58 L9.55
+ 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19 338.43 C4.19 339.56 5.11 340.48
+ 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29 7.38 336.37 6.25 336.37 ZM6.25
+ 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02 339.83 6.25 339.83 C5.47 339.83
+ 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02 ZM2.62 338.14 L0 338.14 L0 338.71
+ L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73 337.47 L1.95 337.47 L2.62 338.14 Z"
+ class="st32"/>
+ </g>
+ </g>
+ <g id="group67-301" transform="translate(104.617,-86.5795)" v:mID="67" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server</title>
+ <g id="shape68-302" v:mID="68" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.68</title>
+ <g id="shadow68-303" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape69-306" v:mID="69" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.69</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape70-310" v:mID="70" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.70</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape71-314" v:mID="71" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.71</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape72-322" v:mID="72" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.72</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape73-326" v:mID="73" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.73</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape74-329" v:mID="74" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.74</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape75-332" v:mID="75" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.75</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape76-336" v:mID="76" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.76</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape77-339" v:mID="77" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.77</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape78-343" v:mID="78" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.78</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape79-346" v:mID="79" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.79</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape80-349" v:mID="80" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.80</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape81-352" v:mID="81" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.81</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape82-356" v:mID="82" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.82</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape83-359" v:mID="83" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.83</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape84-362" v:mID="84" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.84</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape85-365" v:mID="85" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.85</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape86-368" v:mID="86" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.86</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape87-371" v:mID="87" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.87</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape88-374" v:mID="88" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.88</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape89-377" v:mID="89" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.89</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape90-380" v:mID="90" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.90</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape91-384" v:mID="91" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.91</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape92-387" v:mID="92" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.92</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape93-390" v:mID="93" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.93</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape94-393" v:mID="94" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.94</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape95-397" v:mID="95" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.95</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape96-400" v:mID="96" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.96</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape97-402" v:mID="97" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.97</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape98-405" v:mID="98" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.98</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape99-408" v:mID="99" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.99</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape100-410" v:mID="100" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.100</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape101-412" v:mID="101" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.101</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape102-416" v:mID="102" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.102</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape103-420" v:mID="103" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.103</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape104-427" v:mID="104" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.104</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape105-435" v:mID="105" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.105</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape106-440" v:mID="106" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.106</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="group107-443" transform="translate(104.617,-33.8201)" v:mID="107" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server.104</title>
+ <g id="shape108-444" v:mID="108" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.108</title>
+ <g id="shadow108-445" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape109-448" v:mID="109" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.109</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape110-451" v:mID="110" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.110</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape111-454" v:mID="111" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.111</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape112-457" v:mID="112" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.112</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape113-460" v:mID="113" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.113</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape114-463" v:mID="114" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.114</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape115-466" v:mID="115" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.115</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape116-469" v:mID="116" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.116</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape117-472" v:mID="117" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.117</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape118-475" v:mID="118" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.118</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape119-478" v:mID="119" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.119</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape120-481" v:mID="120" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.120</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape121-484" v:mID="121" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.121</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape122-487" v:mID="122" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.122</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape123-490" v:mID="123" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.123</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape124-493" v:mID="124" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.124</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape125-496" v:mID="125" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.125</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape126-499" v:mID="126" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.126</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape127-502" v:mID="127" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.127</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape128-505" v:mID="128" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.128</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape129-508" v:mID="129" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.129</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape130-511" v:mID="130" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.130</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape131-514" v:mID="131" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.131</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape132-517" v:mID="132" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.132</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape133-520" v:mID="133" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.133</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape134-523" v:mID="134" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.134</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape135-526" v:mID="135" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.135</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape136-529" v:mID="136" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.136</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape137-531" v:mID="137" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.137</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape138-534" v:mID="138" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.138</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape139-537" v:mID="139" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.139</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape140-539" v:mID="140" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.140</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape141-541" v:mID="141" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.141</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape142-544" v:mID="142" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.142</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape143-547" v:mID="143" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.143</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape144-554" v:mID="144" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.144</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape145-557" v:mID="145" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.145</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape146-562" v:mID="146" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.146</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="shape147-565" v:mID="147" v:groupContext="shape" transform="translate(427.321,214.49) rotate(90)">
+ <title>Cloud</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54 Z" class="st42"/>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54" class="st54"/>
+ <path d="M11.05 312.14 A8.59237 10.0375 0 0 1 5.37 311.54" class="st54"/>
+ <path d="M40.54 332.09 A8.62978 10.0812 -180 0 0 42.1 335.11" class="st54"/>
+ <path d="M73.92 326.65 A6.96633 8.13801 0 0 1 73.14 330.27" class="st54"/>
+ <path d="M89.7 308.52 A7.30994 8.5394 0 0 1 95.9 318.19" class="st54"/>
+ <path d="M103.15 297.64 A6.67364 7.79609 -180 0 0 106.25 294.02" class="st54"/>
+ <path d="M37.96 278.3 A10.2914 12.0219 -180 0 0 34.86 275.89" class="st54"/>
+ </g>
+ <g id="shape148-574" v:mID="148" v:groupContext="shape" transform="translate(110.222,-64.9346)">
+ <title>Triangle</title>
+ <desc>setsum</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="15.6995" cy="336.449" width="31.4" height="12.1933"/>
+ <g id="shadow148-575" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st9"/>
+ </g>
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st10"/>
+ <text x="8.35" y="337.95" class="st55" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsum</text> </g>
+ <g id="shape149-579" v:mID="149" v:groupContext="shape" transform="translate(292.639,20.8827) rotate(45)">
+ <title>Sheet.149</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L14.71 342.55" class="st14"/>
+ </g>
+ <g id="shape150-584" v:mID="150" v:groupContext="shape" transform="translate(43.2897,-54.1122)">
+ <title>Sheet.150</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L10.07 342.55" class="st14"/>
+ </g>
+ <g id="shape151-589" v:mID="151" v:groupContext="shape" transform="translate(-112.261,8.34531) rotate(-28.1394)">
+ <title>Sheet.151</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L18 342.55" class="st14"/>
+ </g>
+ <g id="shape152-594" v:mID="152" v:groupContext="shape">
+ <title>Sheet.152</title>
+ <desc>Clients</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="21.5" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Clients</text> </g>
+ <g id="shape153-597" v:mID="153" v:groupContext="shape" transform="translate(83.578,-9.58078)">
+ <title>Sheet.153</title>
+ <desc>Distributed Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.0677" cy="337.134" width="84.14" height="10.8224"/>
+ <rect x="0" y="331.723" width="84.1355" height="10.8224" class="st11"/>
+ <text x="13.1" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Distributed Cache</text> </g>
+ <g id="shape154-600" v:mID="154" v:groupContext="shape" transform="translate(181.983,-18.6146)">
+ <title>Sheet.154</title>
+ <desc>Web Servers</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="11.93" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Web Servers</text> </g>
+ <g id="shape155-603" v:mID="155" v:groupContext="shape" transform="translate(96.6068,630.978) rotate(180)">
+ <title>Simple Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow155-604" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ <g id="shape156-607" v:mID="156" v:groupContext="shape" transform="translate(173.159,625.567) rotate(180)">
+ <title>Simple Arrow.153</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow156-608" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ </g>
+ <g id="shape157-611" v:mID="157" v:groupContext="shape" transform="translate(0,-149.475)">
+ <title>Rectangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow157-612" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape158-615" v:mID="158" v:groupContext="shape" transform="translate(271.116,-149.475)">
+ <title>Rectangle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow158-616" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape159-619" v:mID="159" v:groupContext="shape">
+ <title>Rectangle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow159-620" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape160-623" v:mID="160" v:groupContext="shape" transform="translate(271.116,0)">
+ <title>Rectangle.160</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow160-624" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape161-627" v:mID="161" v:groupContext="shape" transform="translate(83.578,-151.241)">
+ <title>Sheet.161</title>
+ <desc>(a) Distributed Web Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow161-628" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(a) Distributed Web Cache</text> </g>
+ <g id="shape162-632" v:mID="162" v:groupContext="shape" transform="translate(319.513,-151.241)">
+ <title>Sheet.162</title>
+ <desc>(b) Detecting Routing Loops</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow162-633" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(b) Detecting Routing Loops</text> </g>
+ <g id="shape163-637" v:mID="163" v:groupContext="shape" transform="translate(77.5283,-3.35965)">
+ <title>Sheet.163</title>
+ <desc>(c) In-order Workload Scheduler</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="63.5211" cy="333.806" width="127.05" height="17.4792"/>
+ <g id="shadow163-638" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(c) In-order Workload Scheduler</text> </g>
+ <g id="shape164-642" v:mID="164" v:groupContext="shape" transform="translate(307.414,-3.35965)">
+ <title>Sheet.164</title>
+ <desc>(d) Database Semi-join Operations</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="66.2253" cy="333.806" width="132.46" height="17.4792"/>
+ <g id="shadow164-643" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(d) Database Semi-join Operations</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i2.svg b/doc/guides/prog_guide/img/member_i2.svg
new file mode 100644
index 0000000..759c654
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i2.svg
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i2.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="4.38194in" height="1.25694in"
+ viewBox="0 0 315.5 90.5" xml:space="preserve" color-interpolation-filters="sRGB" class="st6">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:none;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st3 {baseline-shift:32.4943%;font-size:0.649886em}
+ .st4 {font-size:1em}
+ .st5 {font-family:Cambria Math;font-size:1em}
+ .st6 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(0.25,-0.25)">
+ <title>Sheet.3</title>
+ <desc>False Positive Probability = (1-(1-1/m)kn)k ≃ (1-ekn/m)k</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="157.5" cy="45.5" width="315" height="90"/>
+ <rect x="0" y="0.5" width="315" height="90" class="st1"/>
+ <text x="8.28" y="49.82" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>False Positive Probability = (1-(1-1/m)<tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan dy="0.152em" class="st4">)</tspan><tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan><tspan dy="0.152em" class="st4"> </tspan><tspan
+ class="st5">≃</tspan> (1-e<tspan dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan class="st3"
+ v:baseFontSize="14">/</tspan><tspan class="st3" v:baseFontSize="14">m</tspan><tspan dy="0.152em"
+ class="st4">)</tspan><tspan dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i3.svg b/doc/guides/prog_guide/img/member_i3.svg
new file mode 100644
index 0000000..41e92cb
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i3.svg
@@ -0,0 +1,148 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i3.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ width="4.71875in" height="2.84375in" viewBox="0 0 339.75 204.75" xml:space="preserve" color-interpolation-filters="sRGB"
+ class="st14">
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22}
+ .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25}
+ .st4 {marker-end:url(#mrkr5-32);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st5 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st6 {fill:#feffff;font-family:Calibri;font-size:0.833336em}
+ .st7 {font-size:1em}
+ .st8 {fill:#deebf6;stroke:#c7c8c8;stroke-width:0.25}
+ .st9 {fill:#000000;font-family:Calibri;font-size:0.833336em}
+ .st10 {marker-end:url(#mrkr5-84);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st12 {fill:none;stroke:none;stroke-width:0.25}
+ .st13 {fill:#ff0000;font-family:Calibri;font-size:1.00001em}
+ .st14 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-32" class="st5" refX="-6.16" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-84" class="st11" refX="-5.8" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <defs id="Filters">
+ <filter id="filter_2">
+ <feGaussianBlur stdDeviation="2"/>
+ </filter>
+ </defs>
+ <g>
+ <title>Page-1</title>
+ <g id="group174-1" transform="translate(3.0294,-5.3478)">
+ <title>Sheet.174</title>
+ <g id="shape155-2" transform="translate(99,-99)">
+ <title>Circle</title>
+ <g id="shadow155-3" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape156-7" transform="translate(207,-108)">
+ <title>Circle.156</title>
+ <g id="shadow156-8" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape157-12" transform="translate(153,-162)">
+ <title>Circle.157</title>
+ <g id="shadow157-13" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape158-17" transform="translate(297,-99)">
+ <title>Circle.158</title>
+ <g id="shadow158-18" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape159-22" transform="translate(180,-36)">
+ <title>Circle.159</title>
+ <g id="shadow159-23" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape160-27" transform="translate(109.604,-115.419) rotate(-7.12502)">
+ <title>Sheet.160</title>
+ <path d="M0 204.75 L66.4 204.75" class="st4"/>
+ </g>
+ <g id="shape161-33" transform="translate(276.661,-123.214) rotate(9.46232)">
+ <title>Sheet.161</title>
+ <path d="M0 204.75 L48.58 204.75" class="st4"/>
+ </g>
+ <g id="shape162-38" transform="translate(246.135,262.572) rotate(-160.346)">
+ <title>Sheet.162</title>
+ <path d="M0 204.75 L127.63 204.75" class="st4"/>
+ </g>
+ <g id="shape163-43" transform="translate(284.391,198.775) rotate(141.977)">
+ <title>Sheet.163</title>
+ <path d="M0 204.75 L46.23 204.75" class="st4"/>
+ </g>
+ <g id="shape164-48" transform="translate(70.6118,307.655) rotate(-145.945)">
+ <title>Sheet.164</title>
+ <path d="M0 204.75 L60.88 204.75" class="st4"/>
+ </g>
+ <g id="shape167-53" transform="translate(72,0)">
+ <title>Rectangle.167</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow167-54" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape168-60" transform="translate(108,0)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <g id="shadow168-61" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape169-66" transform="translate(0,-63)">
+ <title>Rectangle.169</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow169-67" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape170-73" transform="translate(36,-63)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <g id="shadow170-74" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape171-79" transform="translate(240.248,331.493) rotate(161.565)">
+ <title>Sheet.171</title>
+ <path d="M-0 190.52 A81.3416 36.0611 -153.48 0 0 82.31 195.86 L82.49 195.55" class="st10"/>
+ </g>
+ <g id="shape172-85" transform="translate(156.426,260.029) rotate(161.565)">
+ <title>Sheet.172</title>
+ <path d="M-0 181.6 A88.1422 54.1439 -124.1 0 0 82.68 187.13 L82.83 186.81" class="st10"/>
+ </g>
+ <g id="shape173-90" transform="translate(18,-121.5)">
+ <title>Sheet.173</title>
+ <desc>Encode ID</desc>
+ <rect x="0" y="177.75" width="63" height="27" class="st12"/>
+ <text x="7.02" y="194.85" class="st13">Encode ID</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i4.svg b/doc/guides/prog_guide/img/member_i4.svg
new file mode 100644
index 0000000..a2b6f2f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i4.svg
@@ -0,0 +1,450 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i4.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.625in" height="3.125in" viewBox="0 0 549 225"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st18">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st3 {marker-end:url(#mrkr5-10);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st4 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st5 {visibility:visible}
+ .st6 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st7 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st8 {fill:none;stroke:none;stroke-width:0.25}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st10 {stroke:#5b9bd5;stroke-dasharray:1.5,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5}
+ .st11 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st12 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st14 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st15 {font-size:1em}
+ .st16 {marker-end:url(#mrkr5-162);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st17 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st18 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-10" class="st4" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-162" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group47-1" transform="translate(3.0294,-0.25)" v:mID="47" v:groupContext="group">
+ <title>Sheet.47</title>
+ <g id="shape1-2" v:mID="1" v:groupContext="shape" transform="translate(177.75,-191.922)">
+ <title>Sheet.1</title>
+ <desc>Element</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="216" width="108" height="18"/>
+ <rect x="0" y="207" width="108" height="18" class="st1"/>
+ <text x="33.77" y="219.6" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Element</text> </g>
+ <g id="shape2-5" v:mID="2" v:groupContext="shape" transform="translate(456.75,33.0781) rotate(90)">
+ <title>Sheet.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L18.65 225" class="st3"/>
+ </g>
+ <g id="shape3-11" v:mID="3" v:groupContext="shape" transform="translate(0,-67.0469)">
+ <title>Rectangle.54</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow3-12" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape4-15" v:mID="4" v:groupContext="shape" transform="translate(27,-67.0469)">
+ <title>Rectangle.55</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow4-16" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape5-19" v:mID="5" v:groupContext="shape" transform="translate(54,-67.0469)">
+ <title>Rectangle.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow5-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape6-23" v:mID="6" v:groupContext="shape" transform="translate(0,-53.5469)">
+ <title>Rectangle.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-24" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape7-27" v:mID="7" v:groupContext="shape" transform="translate(27,-53.5469)">
+ <title>Rectangle.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow7-28" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape8-31" v:mID="8" v:groupContext="shape" transform="translate(54,-53.5469)">
+ <title>Rectangle.59</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow8-32" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape9-35" v:mID="9" v:groupContext="shape" transform="translate(5.625,-72.6719)">
+ <title>Sheet.9</title>
+ <desc>BF-1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-1</text> </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(128.25,-65.0781)">
+ <title>Rectangle.74</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow10-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape11-42" v:mID="11" v:groupContext="shape" transform="translate(155.25,-65.0781)">
+ <title>Rectangle.75</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-43" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(182.25,-65.0781)">
+ <title>Rectangle.76</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow12-47" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape13-50" v:mID="13" v:groupContext="shape" transform="translate(128.25,-51.5781)">
+ <title>Rectangle.77</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow13-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape14-54" v:mID="14" v:groupContext="shape" transform="translate(155.25,-51.5781)">
+ <title>Rectangle.78</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow14-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape15-58" v:mID="15" v:groupContext="shape" transform="translate(182.25,-51.5781)">
+ <title>Rectangle.79</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow15-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape16-62" v:mID="16" v:groupContext="shape" transform="translate(301.5,-65.0781)">
+ <title>Rectangle.81</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow16-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape17-66" v:mID="17" v:groupContext="shape" transform="translate(328.5,-65.0781)">
+ <title>Rectangle.82</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow17-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape18-70" v:mID="18" v:groupContext="shape" transform="translate(355.5,-65.0781)">
+ <title>Rectangle.83</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow18-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape19-74" v:mID="19" v:groupContext="shape" transform="translate(301.5,-51.5781)">
+ <title>Rectangle.84</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow19-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape20-78" v:mID="20" v:groupContext="shape" transform="translate(328.5,-51.5781)">
+ <title>Rectangle.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow20-79" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape21-82" v:mID="21" v:groupContext="shape" transform="translate(355.5,-51.5781)">
+ <title>Rectangle.86</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow21-83" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape22-86" v:mID="22" v:groupContext="shape" transform="translate(447.75,-65.6406)">
+ <title>Rectangle.88</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow22-87" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape23-90" v:mID="23" v:groupContext="shape" transform="translate(474.75,-65.6406)">
+ <title>Rectangle.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow23-91" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape24-94" v:mID="24" v:groupContext="shape" transform="translate(501.75,-65.6406)">
+ <title>Rectangle.90</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow24-95" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape25-98" v:mID="25" v:groupContext="shape" transform="translate(447.75,-52.1406)">
+ <title>Rectangle.91</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow25-99" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape26-102" v:mID="26" v:groupContext="shape" transform="translate(474.75,-52.1406)">
+ <title>Rectangle.92</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-103" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape27-106" v:mID="27" v:groupContext="shape" transform="translate(501.75,-52.1406)">
+ <title>Rectangle.93</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-107" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape28-110" v:mID="28" v:groupContext="shape" transform="translate(213.75,-63.9531)">
+ <title>Sheet.28</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L83.25 225" class="st10"/>
+ </g>
+ <g id="shape29-113" v:mID="29" v:groupContext="shape" transform="translate(387,-63.9531)">
+ <title>Sheet.29</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L54 225" class="st10"/>
+ </g>
+ <g id="group31-116" transform="translate(184.5,-113.172)" v:mID="31" v:groupContext="group">
+ <title>Sheet.31</title>
+ <g id="shape32-117" v:mID="32" v:groupContext="shape" transform="translate(225,173.25) rotate(90)">
+ <title>Block Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow32-118" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st5">
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st11"/>
+ </g>
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st12"/>
+ </g>
+ <g id="shape33-121" v:mID="33" v:groupContext="shape" transform="translate(2.25,-24.3529)">
+ <title>Sheet.33</title>
+ <desc>h1, h2 .. hk</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="215.868" width="90" height="18.2647"/>
+ <rect x="0" y="206.735" width="90" height="18.2647" class="st8"/>
+ <text x="17.56" y="219.47" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>h1, h2 .. hk</text> </g>
+ </g>
+ <g id="shape34-124" v:mID="34" v:groupContext="shape" transform="translate(307.011,286.73) rotate(152.323)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L128.85 225" class="st3"/>
+ </g>
+ <g id="shape35-129" v:mID="35" v:groupContext="shape" transform="translate(433.272,125.452) rotate(99.7172)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L58.31 225" class="st3"/>
+ </g>
+ <g id="shape36-134" v:mID="36" v:groupContext="shape" transform="translate(407.724,-64.1459) rotate(45)">
+ <title>Sheet.36</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L79.16 225" class="st3"/>
+ </g>
+ <g id="shape37-139" v:mID="37" v:groupContext="shape" transform="translate(320.441,-127.12) rotate(15.6155)">
+ <title>Sheet.37</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L200.75 225" class="st3"/>
+ </g>
+ <g id="shape38-144" v:mID="38" v:groupContext="shape" transform="translate(132.75,-75.2588)">
+ <title>Sheet.38</title>
+ <desc>BF-2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-2</text> </g>
+ <g id="shape39-147" v:mID="39" v:groupContext="shape" transform="translate(303.75,-70.7588)">
+ <title>Sheet.39</title>
+ <desc>BF-X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.2" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-X</text> </g>
+ <g id="shape40-150" v:mID="40" v:groupContext="shape" transform="translate(447.75,-75.2588)">
+ <title>Sheet.40</title>
+ <desc>BF-L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.89" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-L</text> </g>
+ <g id="shape41-153" v:mID="41" v:groupContext="shape" transform="translate(300.375,-117)">
+ <title>Sheet.41</title>
+ <desc>Hashing for lookup/Insertion into a vector of BFs happens once</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="90" cy="202.5" width="180" height="45"/>
+ <rect x="0" y="180" width="180" height="45" class="st8"/>
+ <text x="4.6" y="198.9" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Hashing for lookup/Insertion into a <tspan
+ x="23.06" dy="1.2em" class="st15">vector of BFs happens once</tspan></text> </g>
+ <g id="shape44-157" v:mID="44" v:groupContext="shape" transform="translate(249.698,-151.505) rotate(-3.74012)">
+ <title>Sheet.44</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A93.4958 45.6256 42.23 0 1 79.38 221.66 L79.68 221.85" class="st16"/>
+ </g>
+ <g id="shape45-163" v:mID="45" v:groupContext="shape" transform="translate(30.375,0.25)">
+ <title>Sheet.45</title>
+ <desc>Lookup/Insertion is done in the series of BFs, one by one or ...</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="233.048" cy="202.5" width="466.1" height="45"/>
+ <rect x="0" y="180" width="466.096" height="45" class="st8"/>
+ <text x="4.34" y="206.1" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Lookup/Insertion is done in the series of BFs, one by one or can be optimized to do in parallel. </text> </g>
+ <g id="shape46-166" v:mID="46" v:groupContext="shape" transform="translate(123.252,-43.6868) rotate(17.0249)">
+ <title>Sheet.46</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A88.2185 43.0621 47.63 0 1 70.31 221.39 L70.6 221.6" class="st16"/>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i5.svg b/doc/guides/prog_guide/img/member_i5.svg
new file mode 100644
index 0000000..c1728cf
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i5.svg
@@ -0,0 +1,163 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i5.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="5.30481in" height="1.96146in"
+ viewBox="0 0 381.946 141.225" xml:space="preserve" color-interpolation-filters="sRGB" class="st15">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:none;stroke:none;stroke-width:0.25}
+ .st5 {fill:#ffffff;font-family:Calibri;font-size:1.16666em;font-weight:bold}
+ .st6 {marker-end:url(#mrkr5-14);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st7 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st10 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {marker-end:url(#mrkr5-63);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st12 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.22935779816514}
+ .st13 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st14 {font-size:1em}
+ .st15 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-14" class="st7" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-63" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="7.15" refX="-7.15" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-4.36,-4.36) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group1-1" transform="translate(191.995,-19.4751)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-2" v:mID="2" v:groupContext="shape" transform="translate(146.944,42.2251) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow2-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st2"/>
+ </g>
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st3"/>
+ </g>
+ <g id="shape3-6" v:mID="3" v:groupContext="shape" transform="translate(0,-34.65)">
+ <title>Sheet.3</title>
+ <desc>vBF</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="38.1251" cy="126.375" width="76.26" height="29.7"/>
+ <rect x="0" y="111.525" width="76.2502" height="29.7" class="st4"/>
+ <text x="27.68" y="130.58" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>vBF </text> </g>
+ </g>
+ <g id="shape4-9" v:mID="4" v:groupContext="shape" transform="translate(126.724,-100.475)">
+ <title>Sheet.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape5-15" v:mID="5" v:groupContext="shape" transform="translate(103.5,-101.775)">
+ <title>Sheet.5</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="39.9122" cy="135.863" width="79.83" height="10.7251"/>
+ <rect x="0" y="130.5" width="79.8244" height="10.7251" class="st4"/>
+ <text x="21.78" y="138.86" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape6-18" v:mID="6" v:groupContext="shape" transform="translate(221.726,-56.2468) rotate(-24.5123)">
+ <title>Sheet.6</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L65.42 141.23" class="st6"/>
+ </g>
+ <g id="shape7-23" v:mID="7" v:groupContext="shape" transform="translate(280.318,-68.9751)">
+ <title>Sheet.7</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape8-28" v:mID="8" v:groupContext="shape" transform="translate(338.125,-56.6022) rotate(24.1625)">
+ <title>Sheet.8</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L70.8 141.23" class="st6"/>
+ </g>
+ <g id="shape9-33" v:mID="9" v:groupContext="shape" transform="translate(197.714,217.975) rotate(180)">
+ <title>Sheet.9</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(18,-67.5)">
+ <title>Sheet.10</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="80.4201" cy="134.475" width="160.85" height="13.5"/>
+ <rect x="0" y="127.725" width="160.84" height="13.5" class="st4"/>
+ <text x="25.11" y="137.18" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape11-41" v:mID="11" v:groupContext="shape" transform="translate(198.032,253.975) rotate(180)">
+ <title>Sheet.11</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Sheet.12</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="81" cy="136.725" width="162.01" height="9"/>
+ <rect x="0" y="132.225" width="162" height="9" class="st4"/>
+ <text x="11.04" y="139.43" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape13-49" v:mID="13" v:groupContext="shape" transform="translate(494.552,22.75) rotate(90)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape14-52" v:mID="14" v:groupContext="shape" transform="translate(494.552,58.75) rotate(90)">
+ <title>Sheet.14</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape15-55" v:mID="15" v:groupContext="shape" transform="translate(494.552,94.75) rotate(90)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape17-58" v:mID="17" v:groupContext="shape" transform="translate(348.769,-25.0593) rotate(44.5185)">
+ <title>Sheet.17</title>
+ <path d="M-0 141.23 A35.1884 19.2595 167.75 0 1 42.43 138.27 L42.74 138.46" class="st11"/>
+ </g>
+ <g id="shape18-64" v:mID="18" v:groupContext="shape" transform="translate(222.188,-5.40005)">
+ <title>Sheet.18</title>
+ <desc>A BF corresponding to each worker thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="59.0625" cy="127.725" width="118.13" height="27"/>
+ <rect x="0" y="114.225" width="118.125" height="27" class="st4"/>
+ <text x="5.14" y="124.13" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>A BF corresponding to <tspan
+ x="11.19" dy="1.2em" class="st14">each worker thread</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i6.svg b/doc/guides/prog_guide/img/member_i6.svg
new file mode 100644
index 0000000..265179f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i6.svg
@@ -0,0 +1,332 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i6.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8in" height="3.625in" viewBox="0 0 576 261"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st16">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.666664em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.00001em}
+ .st9 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {marker-end:url(#mrkr5-29);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st12 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st13 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st14 {fill:#92d050;stroke:#c8c8c8;stroke-width:0.25}
+ .st15 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st16 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-29" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-9.8478)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(396.989,-54.9268)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st2"/>
+ </g>
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(248.261,-12.1936)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st2"/>
+ </g>
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(6.07514E-013,-29.0155)">
+ <title>Rectangle.10</title>
+ <desc>Signatures for target 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="225.767" width="99.49" height="70.4662"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st2"/>
+ </g>
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st3"/>
+ <text x="26.54" y="223.37" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(239.971,-20.4837)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(353.649,-19.9346)">
+ <title>Sheet.54</title>
+ <desc>Match 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 1</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(216.989,-210.652)">
+ <title>Sheet.55</title>
+ <desc>Packet Payload</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="252.71" width="99.49" height="16.5803"/>
+ <rect x="0" y="244.42" width="99.4817" height="16.5803" class="st9"/>
+ <text x="19.04" y="255.71" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet Payload</text> </g>
+ <g id="shape56-24" v:mID="56" v:groupContext="shape" transform="translate(526.665,52.2365) rotate(90)">
+ <title>Sheet.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L16.52 261" class="st11"/>
+ </g>
+ <g id="shape96-30" v:mID="96" v:groupContext="shape" transform="translate(-3.0294,-95.7818)">
+ <title>Sheet.96</title>
+ <desc>Attack Signature Length 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.75" cy="248.565" width="103.5" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.5" height="24.8704" class="st7"/>
+ <text x="4.79" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 1</text> </g>
+ <g id="group114-33" transform="translate(228.359,-134.152)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-34" transform="translate(0,-24.8704)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-35" v:mID="100" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-36" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape101-39" v:mID="101" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-40" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape102-43" v:mID="102" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-44" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape103-47" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-48" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape104-51" v:mID="104" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-52" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape105-55" v:mID="105" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-56" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-59" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-60" v:mID="108" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-61" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape109-64" v:mID="109" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-65" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape110-68" v:mID="110" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow110-69" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape111-72" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-73" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ <g id="shape112-76" v:mID="112" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-77" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape113-80" v:mID="113" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-81" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-84" v:mID="89" v:groupContext="shape" transform="translate(398.644,-116.927) rotate(24.4696)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L143.75 261" class="st11"/>
+ </g>
+ <g id="shape115-89" v:mID="115" v:groupContext="shape" transform="translate(116.062,-1.19371E-012)">
+ <title>Rectangle.115</title>
+ <desc>Signatures for target 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="211.259" width="99.49" height="99.4817"/>
+ <g id="shadow115-90" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st2"/>
+ </g>
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st3"/>
+ <text x="26.54" y="208.86" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>2</text> </g>
+ <g id="shape116-95" v:mID="116" v:groupContext="shape" transform="translate(117.989,-95.7818)">
+ <title>Sheet.116</title>
+ <desc>Attack Signature Length 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.9909" cy="248.565" width="103.99" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.982" height="24.8704" class="st7"/>
+ <text x="5.03" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 2</text> </g>
+ <g id="shape118-98" v:mID="118" v:groupContext="shape" transform="translate(392.971,-90.217)">
+ <title>Sheet.118</title>
+ <desc>Attack Signature Length L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="248.565" width="108" height="24.8704"/>
+ <rect x="0" y="236.13" width="108" height="24.8704" class="st7"/>
+ <text x="7.43" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length L</text> </g>
+ <g id="shape119-101" v:mID="119" v:groupContext="shape" transform="translate(384.909,-64.9346)">
+ <title>Sheet.119</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape120-103" v:mID="120" v:groupContext="shape" transform="translate(491.971,-64.9346)">
+ <title>Sheet.120</title>
+ <desc>Match 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 2</text> </g>
+ <g id="shape85-106" v:mID="85" v:groupContext="shape" transform="translate(478.538,12.9307) rotate(65.6291)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L109.61 261" class="st11"/>
+ </g>
+ <g id="shape117-111" v:mID="117" v:groupContext="shape" transform="translate(247.054,-91.2818)">
+ <title>Sheet.117</title>
+ <desc>Attack Signature Length X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="52.7082" cy="248.565" width="105.42" height="24.8704"/>
+ <rect x="0" y="236.13" width="105.416" height="24.8704" class="st7"/>
+ <text x="5.7" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length X</text> </g>
+ </g>
+ <g id="shape122-114" v:mID="122" v:groupContext="shape" transform="translate(315.114,-164.13)">
+ <title>Sheet.122</title>
+ <desc>HTSS</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="26.943" cy="248.565" width="53.89" height="24.8704"/>
+ <rect x="0" y="236.13" width="53.8859" height="24.8704" class="st7"/>
+ <text x="14.52" y="252.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i7.svg b/doc/guides/prog_guide/img/member_i7.svg
new file mode 100644
index 0000000..e23ae26
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i7.svg
@@ -0,0 +1,399 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i7.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8.5in" height="4.5in" viewBox="0 0 612 324"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st23">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.16666em}
+ .st9 {fill:none;stroke:#00b050;stroke-width:2.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st12 {fill:#a8d08d;stroke:#c8c8c8;stroke-width:0.25}
+ .st13 {marker-end:url(#mrkr5-83);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st14 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st15 {marker-end:url(#mrkr5-95);stroke:#92d050;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st16 {fill:#92d050;fill-opacity:1;stroke:#92d050;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st17 {fill:#00b050;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st18 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st19 {fill:none;stroke:#ff0000;stroke-width:2.25}
+ .st20 {fill:#ff0000;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st21 {marker-end:url(#mrkr5-123);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-83" class="st14" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-95" class="st16" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-123" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-32.2733)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(460.471,-62.2267)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="279" width="108" height="45" class="st2"/>
+ </g>
+ <rect x="0" y="279" width="108" height="45" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(320.452,-18.123)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="234" width="108" height="90" class="st2"/>
+ </g>
+ <rect x="0" y="234" width="108" height="90" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Rectangle.10</title>
+ <desc>Flow Keys Matching Mask 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="285.75" width="108" height="76.5"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="247.5" width="108" height="76.5" class="st2"/>
+ </g>
+ <rect x="0" y="247.5" width="108" height="76.5" class="st3"/>
+ <text x="12.56" y="282.75" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(311.452,-27.123)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="310.5" width="126" height="13.5" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(424.471,-26.2267)">
+ <title>Sheet.54</title>
+ <desc>Match</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="317.25" width="72" height="13.5"/>
+ <rect x="0" y="310.5" width="72" height="13.5" class="st7"/>
+ <text x="17.68" y="321.45" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(261,-247.163)">
+ <title>Sheet.55</title>
+ <desc>Flow ID1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st9"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID1</text> </g>
+ <g id="shape96-24" v:mID="96" v:groupContext="shape" transform="translate(0,-109.783)">
+ <title>Sheet.96</title>
+ <desc>Flow Mask 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 1</text> </g>
+ <g id="group114-27" transform="translate(247.5,-163.783)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-28" transform="translate(0,-27)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-29" v:mID="100" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-30" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape101-33" v:mID="101" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-34" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape102-37" v:mID="102" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-38" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape103-41" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-42" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape104-45" v:mID="104" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-46" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape105-49" v:mID="105" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-50" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-53" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-54" v:mID="108" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape109-58" v:mID="109" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape110-62" v:mID="110" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow110-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st12"/>
+ </g>
+ <g id="shape111-66" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape112-70" v:mID="112" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape113-74" v:mID="113" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-78" v:mID="89" v:groupContext="shape" transform="translate(413.723,393.802) rotate(146.31)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L153.9 324" class="st13"/>
+ </g>
+ <g id="shape115-84" v:mID="115" v:groupContext="shape" transform="translate(126,0)">
+ <title>Rectangle.115</title>
+ <desc>Flow Keys Matching Mask 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="270" width="108" height="108"/>
+ <g id="shadow115-85" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="216" width="108" height="108" class="st2"/>
+ </g>
+ <rect x="0" y="216" width="108" height="108" class="st3"/>
+ <text x="12.56" y="267" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>2</text> </g>
+ <g id="shape85-90" v:mID="85" v:groupContext="shape" transform="translate(635.321,91.2793) rotate(81.3573)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L143.93 324" class="st15"/>
+ </g>
+ <g id="shape56-96" v:mID="56" v:groupContext="shape" transform="translate(579.175,-64.556) rotate(64.1257)">
+ <title>Sheet.56</title>
+ <path d="M0 324 L54.31 324" class="st15"/>
+ </g>
+ </g>
+ <g id="shape122-101" v:mID="122" v:groupContext="shape" transform="translate(351,-213.444)">
+ <title>Sheet.122</title>
+ <desc>HTSS with False Negative (Cache)</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="304.722" width="90" height="38.556"/>
+ <rect x="0" y="285.444" width="90" height="38.556" class="st7"/>
+ <text x="13.29" y="301.72" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS with False <tspan
+ x="10.52" dy="1.2em" class="st5">Negative </tspan>(Cache)</text> </g>
+ <g id="shape123-105" v:mID="123" v:groupContext="shape" transform="translate(287.654,-290.556)">
+ <title>Sheet.123</title>
+ <desc>Active</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1875" cy="310.5" width="48.38" height="27"/>
+ <rect x="0" y="297" width="48.375" height="27" class="st7"/>
+ <text x="8.63" y="314.1" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Active</text> </g>
+ <g id="shape124-108" v:mID="124" v:groupContext="shape" transform="translate(278.827,-153)">
+ <title>Sheet.124</title>
+ <desc>Target for Flow ID 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36.0864" cy="310.5" width="72.18" height="27"/>
+ <rect x="0" y="297" width="72.1728" height="27" class="st9"/>
+ <text x="11.93" y="306.9" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Target for <tspan
+ x="13.54" dy="1.2em" class="st5">Flow ID </tspan>1</text> </g>
+ <g id="shape125-112" v:mID="125" v:groupContext="shape" transform="translate(155.857,-254.556)">
+ <title>Sheet.125</title>
+ <desc>Flow ID2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st19"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID2</text> </g>
+ <g id="shape126-115" v:mID="126" v:groupContext="shape" transform="translate(153,-270)">
+ <title>Sheet.126</title>
+ <desc>New/Inactive</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="40.5" cy="310.5" width="81" height="27"/>
+ <rect x="0" y="297" width="81" height="27" class="st7"/>
+ <text x="6.77" y="314.1" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New/Inactive</text> </g>
+ <g id="shape127-118" v:mID="127" v:groupContext="shape" transform="translate(251.739,-239.709) rotate(14.0795)">
+ <title>Sheet.127</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 318.73 A39.2404 18 -180 0 0 49.73 320.91 L50.07 320.78" class="st21"/>
+ </g>
+ <g id="shape128-124" v:mID="128" v:groupContext="shape" transform="translate(219.24,-229.5)">
+ <title>Sheet.128</title>
+ <desc>Miss</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="20.88" cy="310.5" width="41.76" height="27"/>
+ <rect x="0" y="297" width="41.76" height="27" class="st7"/>
+ <text x="7.81" y="314.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Miss</text> </g>
+ <g id="shape129-127" v:mID="129" v:groupContext="shape" transform="translate(147.029,-142.056)">
+ <title>Sheet.129</title>
+ <desc>Flow Mask 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 2</text> </g>
+ <g id="shape130-130" v:mID="130" v:groupContext="shape" transform="translate(166.845,-18.5004) rotate(18.2325)">
+ <title>Sheet.130</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A71.1913 104.269 -180 0 0 97.04 298.43 L97.25 298.14" class="st21"/>
+ </g>
+ <g id="shape131-135" v:mID="131" v:groupContext="shape" transform="translate(184.406,-3.04505) rotate(-3.24734)">
+ <title>Sheet.131</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A112.345 104.269 -180 0 0 154.25 297.52 L154.52 297.28" class="st21"/>
+ </g>
+ <g id="shape132-140" v:mID="132" v:groupContext="shape" transform="translate(301.368,16.888) rotate(-25.868)">
+ <title>Sheet.132</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A83.375 104.269 -180 0 0 113.91 298.14 L114.14 297.87" class="st21"/>
+ </g>
+ <g id="shape133-145" v:mID="133" v:groupContext="shape" transform="translate(345.029,-142.056)">
+ <title>Sheet.133</title>
+ <desc>Flow Mask X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.43" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask X</text> </g>
+ <g id="shape134-148" v:mID="134" v:groupContext="shape" transform="translate(481.5,-139.5)">
+ <title>Sheet.134</title>
+ <desc>Flow Mask L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="19.12" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask L</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 40f04a1..7ff5144 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -50,6 +50,7 @@ Programmer's Guide
timer_lib
hash_lib
efd_lib
+ member_lib
lpm_lib
lpm6_lib
packet_distrib_lib
@@ -191,6 +192,19 @@ Programmer's Guide
:numref:`figure_efd11` :ref:`figure_efd11`
+:numref:`figure_membership1` :ref:`figure_membership1`
+
+:numref:`figure_membership2` :ref:`figure_membership2`
+
+:numref:`figure_membership3` :ref:`figure_membership3`
+
+:numref:`figure_membership4` :ref:`figure_membership4`
+
+:numref:`figure_membership5` :ref:`figure_membership5`
+
+:numref:`figure_membership6` :ref:`figure_membership6`
+
+:numref:`figure_membership7` :ref:`figure_membership7`
**Tables**
diff --git a/doc/guides/prog_guide/member_lib.rst b/doc/guides/prog_guide/member_lib.rst
new file mode 100644
index 0000000..caca8dc
--- /dev/null
+++ b/doc/guides/prog_guide/member_lib.rst
@@ -0,0 +1,420 @@
+.. BSD LICENSE
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+.. _member_library:
+
+Membership Library
+==================
+
+Introduction
+------------
+
+The DPDK Membership Library provides an API for DPDK applications to insert a
+new member, delete an existing member, or query the existence of a member in a
+given set, or a group of sets. For the case of a group of sets, the library
+will return not only whether the element has been inserted before in one of
+the sets but also which set it belongs to. The Membership Library is an
+extension and generalization of a traditional filter structure (for example
+Bloom Filter [Member-bloom]) that has multiple usages in a wide variety of
+workloads and applications. In general, the Membership Library is a data
+structure that provides a "set-summary" on whether a member belongs to a set,
+and as discussed in detail later, there are two advantages of using such a
+set-summary rather than operating on a "full-blown" complete list of elements:
+first, it has a much smaller storage requirement than storing the whole list of
+elements themselves, and secondly checking an element membership (or other
+operations) in this set-summary is much faster than checking it for the
+original full-blown complete list of elements.
+
+We use the term "Set-Summary" in this guide to refer to the space-efficient,
+probabilistic membership data structure that is provided by the library. A
+membership test for an element will return the set this element belongs to or
+that the element is "not-found" with very high probability of accuracy. Set-summary
+is a fundamental data aggregation component that can be used in many network
+(and other) applications. It is a crucial structure to address performance and
+scalability issues of diverse network applications including overlay networks,
+data-centric networks, flow table summaries, network statistics and
+traffic monitoring. A set-summary is useful for applications who need to
+include a list of elements while a complete list requires too much space
+and/or too much processing cost. In these situations, the set-summary works as
+a lossy hash-based representation of a set of members. It can dramatically
+reduce space requirement and significantly improve the performance of set
+membership queries at the cost of introducing a very small membership test error
+probability.
+
+.. _figure_membership1:
+.. figure:: img/member_i1.*
+
+ Example Usages of Membership Library
+
+There are various usages for a Membership Library in a very
+large set of applications and workloads. Interested readers can refer to
+[Member-survey] for a survey of possible networking usages. The above figure
+provide a small set of examples of using the Membership Library:
+
+* Sub-figure (a)
+ depicts a distributed web cache architecture where a collection of proxies
+ attempt to share their web caches (cached from a set of back-end web servers) to
+ provide faster responses to clients, and the proxies use the Membership
+ Library to share summaries of what web pages/objects they are caching. With the
+ Membership Library, a proxy receiving an http request will inquire the
+ set-summary to find its location and quickly determine whether to retrieve the
+ requested web page from a nearby proxy or from a back-end web server.
+
+* Sub-figure (b) depicts another example for using the Membership Library to
+ prevent routing loops which is typically done using slow TTL countdown and
+ dropping packets when TTL expires. As shown in Sub-figure (b), an embedded
+ set-summary in the packet header itself can be used to summarize the set of
+ nodes a packet has gone through, and each node upon receiving a packet can check
+ whether its id is a member of the set of visited nodes, and if it is, then a
+ routing loop is detected.
+
+* Sub-Figure (c) presents another usage of the Membership
+ Library to load-balance flows to worker threads with in-order guarantee where a
+ set-summary is used to query if a packet belongs to an existing flow or a new
+ flow. Packets belonging to a new flow are forwarded to the current least loaded
+ worker thread, while those belonging to an existing flow are forwarded to the
+ pre-assigned thread to guarantee in-order processing.
+
+* Sub-figure (d) highlights
+ yet another usage example in the database domain where a set-summary is used to
+ determine joins between sets instead of creating a join by comparing each
+ element of a set against the other elements in a different set, a join is done
+ on the summaries since they can efficiently encode members of a given set.
+
+Membership Library is a configurable library that is optimized to cover set
+membership functionality for both a single set and multi-set scenarios. Two set-summary
+schemes are presented including (a) vector of Bloom Filters and (b) Hash-Table based
+set-summary schemes with and without false negative probability.
+This guide first briefly describes these different types of set-summaries, usage examples for each,
+and then it highlights the Membership Library API.
+
+Vector of Bloom Filters
+-----------------------
+
+Bloom Filter (BF) [Member-bloom] is a well-known space-efficient
+probabilistic data structure that answers set membership queries (test whether
+an element is a member of a set) with some probability of false positives and
+zero false negatives; a query for an element returns either it is "possibly in
+a set" (with very high probability) or "definitely not in a set".
+
+The BF is a method for representing a set of ``n`` elements (for example flow keys
+in network applications domain) to support membership queries. The idea of BF is
+to allocate a bit-vector ``v`` with ``m`` bits, which are initially all set to 0. Then
+it chooses ``k`` independent hash functions ``h1``, ``h2``, ... ``hk`` with hash values range from
+``0`` to ``m-1`` to perform hashing calculations on each element to be inserted. Every time when an
+element ``X`` being inserted into the set, the bits at positions ``h1(X)``, ``h2(X)``, ...
+``hk(X)`` in ``v`` are set to 1 (any particular bit might be set to 1 multiple times
+for multiple different inserted elements). Given a query for any element ``Y``, the
+bits at positions ``h1(Y)``, ``h2(Y)``, ... ``hk(Y)`` are checked. If any of them is 0,
+then Y is definitely not in the set. Otherwise there is a high probability that
+Y is a member of the set with certain false positive probability. As shown in
+the next equation, the false positive probability can be made arbitrarily small
+by changing the number of hash functions (``k``) and the vector length (``m``).
+
+.. _figure_membership2:
+.. figure:: img/member_i2.*
+
+ Bloom Filter False Positive Probability
+
+Without BF, an accurate membership testing could involve a costly hash table
+lookup and full element comparison. The advantage of using a BF is to simplify
+the membership test into a series of hash calculations and memory accesses for a
+small bit-vector, which can be easily optimized. Hence the lookup throughput
+(set membership test) can be significantly faster than a normal hash table
+lookup with element comparison.
+
+.. _figure_membership3:
+.. figure:: img/member_i3.*
+
+ Detecting Routing Loops Using BF
+
+BF is used for applications that need only one set, and the
+membership of elements is checked against the BF. The example discussed
+in the above figure is one example of potential applications that uses only one
+set to capture the node IDs that have been visited so far by the packet. Each
+node will then check this embedded BF in the packet header for its own id, and
+if the BF indicates that the current node is definitely not in the set then a
+loop-free route is guaranteed.
+
+
+.. _figure_membership4:
+.. figure:: img/member_i4.*
+
+ Vector Bloom Filter (vBF) Overview
+
+To support membership test for both multiple sets and a single set,
+the library implements a Vector Bloom Filter (vBF) scheme.
+vBF basically composes multiple bloom filters into a vector of bloom filers.
+The membership test is conducted on all of the
+bloom filters concurrently to determine which set(s) it belongs to or none of
+them. The basic idea of vBF is shown in the above figure where an element is
+used to address multiple bloom filters concurrently and the bloom filter
+index(es) with a hit is returned.
+
+.. _figure_membership5:
+.. figure:: img/member_i5.*
+
+ vBF for Flow Scheduling to Worker Thread
+
+As previously mentioned, there are many usages of such structures. vBF is used
+for applications that need to check membership against multiple sets
+simultaneously. The example shown in the above figure uses a set to capture
+all flows being assigned for processing at a given worker thread. Upon receiving
+a packet the vBF is used to quickly figure out if this packet belongs to a new flow
+so as to be forwarded to the current least loaded worker thread, or otherwise it
+should be queued for an existing thread to guarantee in-order processing (i.e.
+the property of vBF to indicate right away that a given flow is a new one or
+not is critical to minimize response time latency).
+
+It should be noted that vBF can be implemented using a set of single bloom
+filters with sequential lookup of each BF. However, being able to concurrently
+search all set-summaries is a big throughput advantage. In the library, certain
+parallelism is realized by the implementation of checking all bloom filters
+together.
+
+
+Hash-Table based Set-Summaries
+------------------------------
+
+Hash-table based set-summary (HTSS) is another scheme in the membership library.
+Cuckoo filter [Member-cfilter] is an example of HTSS.
+HTSS supports multi-set membership testing like
+vBF does. However, while vBF is better for a small number of targets, HTSS is more suitable
+and can easily outperform vBF when the number of sets is
+large, since HTSS uses a single hash table for membership testing while vBF
+requires testing a series of Bloom Filters each corresponding to one set.
+As a result, generally speaking vBF is more adequate for the case of a small limited number of sets
+while HTSS should be used with a larger number of sets.
+
+.. _figure_membership6:
+.. figure:: img/member_i6.*
+
+ Using HTSS for Attack Signature Matching
+
+As shown in the above figure, attack signature matching where each set
+represents a certain signature length (for correctness of this example, an
+attack signature should not be a subset of another one) in the payload is a good
+example for using HTSS with 0% false negative (i.e., when an element returns not
+found, it has a 100% certainty that it is not a member of any set). The packet
+inspection application benefits from knowing right away that the current payload
+does not match any attack signatures in the database to establish its
+legitimacy, otherwise a deep inspection of the packet is needed.
+
+HTSS employs a similar but simpler data structure to a traditional hash table,
+and the major difference is that HTSS stores only the signatures but not the
+full keys/elements which can significantly reduce the footprint of the table.
+Along with the signature, HTSS also stores a value to indicate the target set.
+When looking up an element, the element is hashed and the HTSS is addressed
+to retrieve the signature stored. If the signature matches then the value is
+retrieved corresponding to the index of the target set which the element belongs
+to. Because signatures can collide, HTSS can still has false positive
+probability. Furthermore, if elements are allowed to be
+overwritten or evicted when the hash table becomes full, it will also have a
+false negative probability. We discuss this case in the next section.
+
+Set-Summaries with False Negative Probability
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As previously mentioned, traditional set-summaries (e.g. Bloom Filters) do not
+have a false negative probability, i.e., it is 100% certain when an element
+returns "not to be present" for a given set. However, the Membership Library
+also supports a set-summary probabilistic data structure based on HTSS which
+allows for false negative probability.
+
+In HTSS, when the hash table becomes full, keys/elements will fail to be added
+into the table and the hash table has to be resized to accommodate for these new
+elements, which can be expensive. However, if we allow new elements to overwrite
+or evict existing elements (as a cache typically does), then the resulting
+set-summary will begin to have false negative probability. This is because the
+element that was evicted from the set-summary may still be present in the target
+set. For subsequent inquiries the set-summary will falsely report the element
+not being in the set, hence having a false negative probability.
+
+The major usage of HTSS with false negative is to use it as a cache for
+distributing elements to different target sets. By allowing HTSS to evict old
+elements, the set-summary can keep track of the most recent elements
+(i.e. active) as a cache typically does. Old inactive elements (infrequently
+used elements) will automatically and eventually get evicted from the
+set-summary. It is worth noting that the set-summary still has false positive
+probability, which means the application either can tolerate certain false positive
+or it has fall-back path when false positive happens.
+
+.. _figure_membership7:
+.. figure:: img/member_i7.*
+
+ Using HTSS with False Negatives for Wild Card Classification
+
+HTSS with false negative (i.e. a cache) also has its wide set of applications.
+For example wild card flow classification (e.g. ACL rules) highlighted in the
+above figure is an example of such application. In that case each target set
+represents a sub-table with rules defined by a certain flow mask. The flow masks
+are non-overlapping, and for flows matching more than one rule only the highest
+priority one is inserted in the corresponding sub-table (interested readers can
+refer to the Open vSwitch (OvS) design of Mega Flow Cache (MFC) [Member-OvS]
+for further details). Typically the rules will have a large number of distinct
+unique masks and hence, a large number of target sets each corresponding to one
+mask. Because the active set of flows varies widely based on the network
+traffic, HTSS with false negative will act as a cache for <flowid, target ACL
+sub-table> pair for the current active set of flows. When a miss occurs (as
+shown in red in the above figure) the sub-tables will be searched sequentially
+one by one for a possible match, and when found the flow key and target
+sub-table will be inserted into the set-summary (i.e. cache insertion) so
+subsequent packets from the same flow don’t incur the overhead of the
+sequential search of sub-tables.
+
+Library API Overview
+--------------------
+
+The design goal of the Membership Library API is to be as generic as possible to
+support all the different types of set-summaries we discussed in previous
+sections and beyond. Fundamentally, the APIs need to include creation,
+insertion, deletion, and lookup.
+
+
+Set-summary Create
+~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_create()`` function is used to create a set-summary structure, the input parameter
+is a struct to pass in parameters that needed to initialize the set-summary, while the function returns the
+pointer to the created set-summary or ``NULL`` if the creation failed.
+
+The general input arguments used when creating the set-summary should include ``name``
+which is the name of the created set-summary, *type* which is one of the types
+supported by the library (e.g. ``RTE_MEMBER_TYPE_HT`` for HTSS or ``RTE_MEMBER_TYPE_VBF`` for vBF), and ``key_len``
+which is the length of the element/key. There are other parameters
+are only used for certain type of set-summary, or which have a slightly different meaning for different types of set-summary.
+For example, ``num_keys`` parameter means the maximum number of entries for Hash table based set-summary.
+However, for bloom filter, this value means the expected number of keys that could be
+inserted into the bloom filter(s). The value is used to calculate the size of each
+bloom filter.
+
+We also pass two seeds: ``prim_hash_seed`` and
+``sec_hash_seed`` for the primary and secondary hash functions to calculate two independent hash values.
+``socket_id`` parameter is the NUMA socket ID for the memory used to create the
+set-summary. For HTSS, another parameter ``is_cache`` is used to indicate
+if this set-summary is a cache (i.e. with false negative probability) or not.
+For vBF, extra parameters are needed. For example, ``num_set`` is the number of
+sets needed to initialize the vector bloom filters. This number is equal to the
+number of bloom filters will be created.
+``false_pos_rate`` is the false positive rate. num_keys and false_pos_rate will be used to determine
+the number of hash functions and the bloom filter size.
+
+
+Set-summary Element Insertion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_add()`` function is used to insert an element/key into a set-summary structure. If it fails an
+error is returned. For success the returned value is dependent on the
+set-summary mode to provide extra information for the users. For vBF
+mode, a return value of 0 means a successful insert. For HTSS mode without false negative, the insert
+could fail with ``-ENOSPC`` if the table is full. With false negative (i.e. cache mode),
+for insert that does not cause any eviction (i.e. no overwriting happens to an
+existing entry) the return value is 0. For insertion that causes eviction, the return
+value is 1 to indicate such situation, but it is not an error.
+
+The input arguments for the function should include the ``key`` which is a pointer to the element/key that needs to
+be added to the set-summary, and ``set_id`` which is the set id associated
+with the key that needs to be added.
+
+
+Set-summary Element Lookup
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_lookup()`` function looks up a single key/element in the set-summary structure. It
+returns as soon as the first match is found. The return value is 1 if a
+match is found and 0 otherwise. The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, and ``set_id`` which is used to return the
+first target set id where the key has matched, if any.
+
+The ``rte_member_lookup_bulk()`` function is used to look up a bulk of keys/elements in the
+set-summary structure for their first match. Each key lookup returns as soon as the first match is found. The
+return value is the number of keys that find a match. The arguments of the function include ``keys``
+which is a pointer to a bulk of keys that are to be looked up,
+``num_keys`` is the number
+of keys that will be looked up, and ``set_ids`` are the return target set
+ids for the first match found for each of the input keys. ``set_ids`` is an array
+needs to be sized according to the ``num_keys``. If there is no match, the set id
+for that key will be set to RTE_MEMBER_NO_MATCH.
+
+The ``rte_member_lookup_multi()`` function looks up a single key/element in the
+set-summary structure for multiple matches. It
+returns ALL the matches (possibly more than one) found for this key when it
+is matched against all target sets (it is worth noting that for cache mode HTSS,
+the current implementation matches at most one target set). The return value is
+the number of matches
+that was found for this key (for cache mode HTSS the return value
+should be at most 1). The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, ``max_match_per_key`` which is to indicate the maximum number of matches
+the user expects to find for each key, and ``set_id`` which is used to return all
+target set ids where the key has matched, if any. The ``set_id`` array should be sized
+according to ``max_match_per_key``. For vBF, the maximum number of matches per key is equal
+to the number of sets. For HTSS, the maximum number of matches per key is equal to two time
+entry count per bucket. ``max_match_per_key`` should be equal or smaller than the maximum number of
+possible matches.
+
+The ``rte_membership_lookup_multi_bulk()`` function looks up a bulk of keys/elements in the
+set-summary structure for multiple matches, each key lookup returns ALL the matches (possibly more
+than one) found for this key when it is matched against all target sets (cache mode HTSS
+matches at most one target set). The
+return value is the number of keys that find one or more matches in the
+set-summary structure. The arguments of the
+function include ``keys`` which is
+a pointer to a bulk of keys that are to be looked up, ``num_keys`` is the number
+of keys that will be looked up, ``max_match_per_key`` is the possible
+maximum number of matches for each key, ``match_count`` which is the returned number
+of matches for each key, and ``set_ids`` are the returned target set
+ids for all matches found for each keys. ``set_ids`` is 2-D array
+containing a 1-D array for each key (the size of 1-D array per key should be set by the user according to ``max_match_per_key``).
+``max_match_per_key`` should be equal or smaller than the maximum number of
+possible matches, similar to ``rte_member_lookup_multi``.
+
+
+Set-summary Element Delete
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_membership_delete()`` function deletes an element/key from a set-summary structure, if it fails
+an error is returned. The input arguments should include ``key`` which is a pointer to the
+element/key that needs to be deleted from the set-summary, and ``set_id``
+which is the set id associated with the key to delete. It is worth noting that current
+implementation of vBF does not support deletion [1]_. An error code ``-EINVAL`` will be returned.
+
+.. [1] Traditional bloom filter does not support proactive deletion. Supporting proactive deletion require additional implementation and performance overhead.
+
+References
+-----------
+
+[Member-bloom] B H Bloom, "Space/Time Trade-offs in Hash Coding with Allowable Errors," Communications of the ACM, 1970.
+
+[Member-survey] A Broder and M Mitzenmacher, "Network Applications of Bloom Filters: A Survey," in Internet Mathematics, 2005.
+
+[Member-cfilter] B Fan, D G Andersen and M Kaminsky, "Cuckoo Filter: Practically Better Than Bloom," in Conference on emerging Networking Experiments and Technologies, 2014.
+
+[Member-OvS] B Pfaff, "The Design and Implementation of Open vSwitch," in NSDI, 2015.
diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst
index 8bf91bd..ba068ec 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -41,6 +41,23 @@ New Features
Also, make sure to start the actual text at the margin.
=========================================================
+* **Added Membership library (rte_member).**
+
+ Added membership library. It provides an API for DPDK applications to insert a
+ new member, delete an existing member, or query the existence of a member in a
+ given set, or a group of sets. For the case of a group of sets the library
+ will return not only whether the element has been inserted before in one of
+ the sets but also which set it belongs to.
+
+ The Membership Library is an extension and generalization of a traditional
+ filter (for example Bloom Filter) structure that has multiple usages in a wide
+ variety of workloads and applications. In general, the Membership Library is a
+ data structure that provides a “set-summary” and responds to set-membership
+ queries whether a certain member belongs to a set(s).
+
+ See the :ref:`Membership Library <Member_Library>` documentation in
+ the Programmers Guide document, for more information.
+
Resolved Issues
---------------
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v5 0/7] Add Membership Library
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
` (6 preceding siblings ...)
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 7/7] doc: add membership documentation Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 1/7] member: implement main API Yipeng Wang
` (7 more replies)
7 siblings, 8 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
DPDK Membership Library provides an API that can be used by many DPDK
applications to conduct one or more set-membership tests (we mention some
possible use cases below, but interested readers can refer to
[1] for a wider survey of use cases).
The basic functionalities of the Membership Library include
inserting a new member, deleting an existing member, and querying the existence
of a member. The query result would indicate with high accuracy which specific
set this member belongs to among a group of sets.
The Membership Library is an extension and generalization of traditional filter
data structures [2,3], which maintain a space-efficient “set-summary”.
There are two advantages of using such a set-summary rather than operating on a
“full-blown” complete list of elements: firstly it has a much smaller storage
requirement than storing the whole list of elements, and secondly set membership
tests (or other operations) is much more efficient than searching through the
complete list of elements.
A membership test for an element will return the set this element belongs to if
found (or return "not-found") with high accuracy. If needed, the accuracy of the
membership tests could be further increased with larger storage space.
Set-summary is a fundamental data aggregation component that can be used in many
network applications. It is a crucial structure to address performance and
scalability issues of diverse applications including overlay networks, wild card
flow classification, web-caches, load balancing, connection tracking,
data-centric networks, flow table summaries, network statistics and traffic
monitoring. Our Proof of Concept (PoC) using set-summary to optimize flow lookup
in Open vSwitch (OvS) shows a speedup of about 2-3X.
This patch set implements two types of set-summaries, i.e., hash-table based
set-summary (HTSS) and Vector Bloom Filter (vBF). HTSS supports both the
non-cache and cache modes. The non-cache mode can incur a small chance of
false-positives which is the case when the set-summary indicates a key belongs
to a given set while actually it is not. The cache mode can also have
false-negatives in addition to false-positives. False-negatives means the case
when the set-summary indicates a key does not belong to a given set while
actually it does. This happens because cache mode allows new key to evict
existing keys. vBF only has false-positives similar to the non-cache HTSS.
However, one can set the false-positive rate arbitrarily. HTSS's
false-positive rate is determined by the hash-table size and the signature size.
[1] A Broder and M Mitzenmacher, “Network Applications of Bloom Filters: A
Survey,” in Internet Mathematics, 2005.
[2] B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable Errors,”
Communications of the ACM, 1970.
[3] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically Better Than
Bloom,” in Conference on emerging Networking Experiments and Technologies, 2014.
v5:
- member lib: incorperate Pablo's review comments, including: multiple places
with coding style issues, change variable/function names to be more clear,
reorder code blocks, add descriptions, use macros for constant numbers,
fix bugs in log message, etc.
- test: modify the deletion function test to cover more cases.
- test: add check in performance test to check false negatvie for HT non-cache
mode.
- MAINTAINERS: complete in multiple commits instead of one as Pablo suggested.
v4:
- member lib: change two hash function macros to be one. crc or jhash will be
chosen depending on the architecture.
- member lib: use dynamic log type instead of static one by Thomas' comment.
- member lib: code cleanups (remove unnecessary code, change variable name,
coding style, etc).
- member lib: reorder members in setsummary struct to be more cache friendly.
- member lib: setsummary struct member "name" should be char array rather than
char pointer. Fixed.
- member lib: add check for two hash seeds to make sure they are not equal.
- member lib: vBF lookup speed optimization.
- member lib: revise comments in various places as Pablo suggested.
- member lib: change "void *" to "struct rte_member_setsum *" in public API as
Pablo suggested. test case is modified accordingly to directly use the struct.
- member lib: change order in rte_member_version.map as Pablo suggested.
- MAINTAINERS: change the maintainer block order according to Thomas' comment.
- test: add printf to show which section expects error messages. Change set
count to 16 from 32 for performance test.
- documentation: revise several places according to Pablo's and John's comments.
v3:
- documentation: update documentation to incorporate John's review.
v2:
- test: add bad parameter test functions to test the creation fail case.
- test: add find existing test.
- member lib: Add num_entries check in rte_member_create_ht.
- member lib: Add multiple checks for rte_member_create_vbf to avoid divide-by-0.
- member lib: remove unnecessary ret from rte_member.c according to Stephen's
comment.
- member lib: change the vBF creation function to be not too conservative.
Previous algorithm is too conservative on false positive.
- member lib: fix uninitialized issue fail gcc-4.8 in rte_member_find_existing.
- member lib: add rte_member_find_existing to version map.
- makefile: lib/librte_member/Makefile: change include order according to
Luca's comment.
- documentation: update the programmer guide for membership library.
Yipeng Wang (7):
member: implement main API
member: implement HT mode
member: implement vBF mode
member: add AVX for HT mode
member: enable the library
test/member: add functional and perf tests
doc: add membership documentation
MAINTAINERS | 8 +-
config/common_base | 5 +
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 ++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 +++
doc/guides/prog_guide/img/member_i6.svg | 332 ++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 420 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
lib/Makefile | 2 +
lib/librte_member/Makefile | 51 +
lib/librte_member/rte_member.c | 337 +++++++
lib/librte_member/rte_member.h | 513 ++++++++++
lib/librte_member/rte_member_ht.c | 586 +++++++++++
lib/librte_member/rte_member_ht.h | 94 ++
lib/librte_member/rte_member_vbf.c | 351 +++++++
lib/librte_member/rte_member_vbf.h | 82 ++
lib/librte_member/rte_member_version.map | 16 +
lib/librte_member/rte_member_x86.h | 107 ++
mk/rte.app.mk | 2 +
test/test/Makefile | 3 +
test/test/test_member.c | 744 ++++++++++++++
test/test/test_member_perf.c | 654 ++++++++++++
28 files changed, 7149 insertions(+), 2 deletions(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
create mode 100644 lib/librte_member/rte_member_version.map
create mode 100644 lib/librte_member/rte_member_x86.h
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v5 1/7] member: implement main API
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 8:42 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 2/7] member: implement HT mode Yipeng Wang
` (6 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
Membership library is an extension and generalization of a traditional
filter (for example Bloom Filter and cuckoo filter) structure.
In general, the Membership library is a data structure that provides a
"set-summary" and responds to set-membership queries of whether a
certain element belongs to a set(s). A membership test for an element
will return the set this element belongs to or not-found if the
element is never inserted into the set-summary.
The results of the membership test are not 100% accurate. Certain
false positive or false negative probability could exist. However,
comparing to a "full-blown" complete list of elements, a "set-summary"
is memory efficient and fast on lookup.
This patch adds the main API definition.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/Makefile | 2 +
lib/librte_member/Makefile | 49 +++
lib/librte_member/rte_member.c | 337 ++++++++++++++++++++
lib/librte_member/rte_member.h | 513 +++++++++++++++++++++++++++++++
lib/librte_member/rte_member_version.map | 16 +
5 files changed, 917 insertions(+)
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_version.map
diff --git a/lib/Makefile b/lib/Makefile
index 86caba1..c82033a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -108,6 +108,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder
DEPDIRS-librte_reorder := librte_eal librte_mempool librte_mbuf
DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump
DEPDIRS-librte_pdump := librte_eal librte_mempool librte_mbuf librte_ether
+DIRS-$(CONFIG_RTE_LIBRTE_MEMBER) += librte_member
+DEPDIRS-librte_member := librte_eal librte_hash
ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
new file mode 100644
index 0000000..1a79eaa
--- /dev/null
+++ b/lib/librte_member/Makefile
@@ -0,0 +1,49 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_member.a
+
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
+
+EXPORT_MAP := rte_member_version.map
+
+LIBABIVER := 1
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+# install includes
+SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_member/rte_member.c b/lib/librte_member/rte_member.c
new file mode 100644
index 0000000..1a78f71
--- /dev/null
+++ b/lib/librte_member/rte_member.c
@@ -0,0 +1,337 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+
+#include <rte_eal.h>
+#include <rte_eal_memconfig.h>
+#include <rte_memory.h>
+#include <rte_memzone.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+#include "rte_member_vbf.h"
+
+int librte_member_logtype;
+
+TAILQ_HEAD(rte_member_list, rte_tailq_entry);
+static struct rte_tailq_elem rte_member_tailq = {
+ .name = "RTE_MEMBER",
+};
+EAL_REGISTER_TAILQ(rte_member_tailq)
+
+struct rte_member_setsum *
+rte_member_find_existing(const char *name)
+{
+ struct rte_member_setsum *setsum = NULL;
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(name, setsum->name, RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ if (te == NULL) {
+ rte_errno = ENOENT;
+ return NULL;
+ }
+ return setsum;
+}
+
+void
+rte_member_free(struct rte_member_setsum *setsum)
+{
+ struct rte_member_list *member_list;
+ struct rte_tailq_entry *te;
+
+ if (setsum == NULL)
+ return;
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ if (te->data == (void *)setsum)
+ break;
+ }
+ if (te == NULL) {
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return;
+ }
+ TAILQ_REMOVE(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_free_ht(setsum);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_free_vbf(setsum);
+ break;
+ default:
+ break;
+ }
+ rte_free(setsum);
+ rte_free(te);
+}
+
+struct rte_member_setsum *
+rte_member_create(const struct rte_member_parameters *params)
+{
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+ struct rte_member_setsum *setsum;
+ int ret;
+
+ if (params == NULL) {
+ rte_errno = EINVAL;
+ return NULL;
+ }
+
+ if (params->key_len == 0 ||
+ params->prim_hash_seed == params->sec_hash_seed) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "rte_member_create has "
+ "invalid parameters\n");
+ return NULL;
+ }
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(params->name, setsum->name,
+ RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ setsum = NULL;
+ if (te != NULL) {
+ rte_errno = EEXIST;
+ te = NULL;
+ goto error_unlock_exit;
+ }
+ te = rte_zmalloc("MEMBER_TAILQ_ENTRY", sizeof(*te), 0);
+ if (te == NULL) {
+ RTE_MEMBER_LOG(ERR, "tailq entry allocation failed\n");
+ goto error_unlock_exit;
+ }
+
+ /* Create a new setsum structure */
+ setsum = (struct rte_member_setsum *) rte_zmalloc_socket(params->name,
+ sizeof(struct rte_member_setsum), RTE_CACHE_LINE_SIZE,
+ params->socket_id);
+ if (setsum == NULL) {
+ RTE_MEMBER_LOG(ERR, "Create setsummary failed\n");
+ goto error_unlock_exit;
+ }
+ snprintf(setsum->name, sizeof(setsum->name), "%s", params->name);
+ setsum->type = params->type;
+ setsum->socket_id = params->socket_id;
+ setsum->key_len = params->key_len;
+ setsum->num_set = params->num_set;
+ setsum->prim_hash_seed = params->prim_hash_seed;
+ setsum->sec_hash_seed = params->sec_hash_seed;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ ret = rte_member_create_ht(setsum, params);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ ret = rte_member_create_vbf(setsum, params);
+ break;
+ default:
+ goto error_unlock_exit;
+ }
+ if (ret < 0)
+ goto error_unlock_exit;
+
+ RTE_MEMBER_LOG(DEBUG, "Creating a setsummary table with "
+ "mode %u\n", setsum->type);
+
+ te->data = (void *)setsum;
+ TAILQ_INSERT_TAIL(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return setsum;
+
+error_unlock_exit:
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ rte_member_free(setsum);
+ return NULL;
+}
+
+int
+rte_member_add(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id)
+{
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_add_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_add_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t *set_id)
+{
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids)
+{
+ if (setsum == NULL || keys == NULL || set_ids == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_bulk_ht(setsum, keys, num_keys,
+ set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_bulk_vbf(setsum, keys, num_keys,
+ set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi(const struct rte_member_setsum *setsum, const void *key,
+ uint32_t match_per_key, member_set_t *set_id)
+{
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_ht(setsum, key, match_per_key,
+ set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_vbf(setsum, key, match_per_key,
+ set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key, uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ if (setsum == NULL || keys == NULL || set_ids == NULL ||
+ match_count == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_bulk_ht(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_bulk_vbf(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_delete(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id)
+{
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_delete_ht(setsum, key, set_id);
+ /* current vBF implementation does not support delete function */
+ case RTE_MEMBER_TYPE_VBF:
+ default:
+ return -EINVAL;
+ }
+}
+
+void
+rte_member_reset(const struct rte_member_setsum *setsum)
+{
+ if (setsum == NULL)
+ return;
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_reset_ht(setsum);
+ return;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_reset_vbf(setsum);
+ return;
+ default:
+ return;
+ }
+}
+
+RTE_INIT(librte_member_init_log);
+
+static void
+librte_member_init_log(void)
+{
+ librte_member_logtype = rte_log_register("librte.member");
+ if (librte_member_logtype >= 0)
+ rte_log_set_level(librte_member_logtype, RTE_LOG_DEBUG);
+}
diff --git a/lib/librte_member/rte_member.h b/lib/librte_member/rte_member.h
new file mode 100644
index 0000000..9b0c8f9
--- /dev/null
+++ b/lib/librte_member/rte_member.h
@@ -0,0 +1,513 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/**
+ * @file
+ *
+ * RTE Membership Library
+ *
+ * The Membership Library is an extension and generalization of a traditional
+ * filter (for example Bloom Filter and cuckoo filter) structure that has
+ * multiple usages in a variety of workloads and applications. The library is
+ * used to test if a key belongs to certain sets. Two types of such
+ * "set-summary" structures are implemented: hash-table based (HT) and vector
+ * bloom filter (vBF). For HT setsummary, two subtypes or modes are available,
+ * cache and non-cache modes. The table below summarize some properties of
+ * the different implementations.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ */
+
+/**
+ * <!--
+ * +==========+=====================+================+=========================+
+ * | type | vbf | HT-cache | HT-non-cache |
+ * +==========+=====================+==========================================+
+ * |structure | bloom-filter array | hash-table like without storing key |
+ * +----------+---------------------+------------------------------------------+
+ * |set id | limited by bf count | [1, 0x7fff] |
+ * | | up to 32. | |
+ * +----------+---------------------+------------------------------------------+
+ * |usages & | small set range, | can delete, | cache most recent keys, |
+ * |properties| user-specified | big set range, | have both false-positive|
+ * | | false-positive rate,| small false | and false-negative |
+ * | | no deletion support.| positive depend| depend on table size, |
+ * | | | on table size, | automatic overwritten. |
+ * | | | new key does | |
+ * | | | not overwrite | |
+ * | | | existing key. | |
+ * +----------+---------------------+----------------+-------------------------+
+ * -->
+ */
+
+#ifndef _RTE_MEMBER_H_
+#define _RTE_MEMBER_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+
+/** The set ID type that stored internally in hash table based set summary. */
+typedef uint16_t member_set_t;
+/** Invalid set ID used to mean no match found. */
+#define RTE_MEMBER_NO_MATCH 0
+/** Maximum size of hash table that can be created. */
+#define RTE_MEMBER_ENTRIES_MAX (1 << 30)
+/** Maximum number of keys that can be searched as a bulk */
+#define RTE_MEMBER_LOOKUP_BULK_MAX 64
+/** Entry count per bucket in hash table based mode. */
+#define RTE_MEMBER_BUCKET_ENTRIES 16
+/** Maximum number of characters in setsum name. */
+#define RTE_MEMBER_NAMESIZE 32
+
+/** @internal Hash function used by membership library. */
+#if defined(RTE_ARCH_X86) || defined(RTE_MACHINE_CPUFLAG_CRC32)
+#include <rte_hash_crc.h>
+#define MEMBER_HASH_FUNC rte_hash_crc
+#else
+#include <rte_jhash.h>
+#define MEMBER_HASH_FUNC rte_jhash
+#endif
+
+extern int librte_member_logtype;
+
+#define RTE_MEMBER_LOG(level, fmt, args...) \
+rte_log(RTE_LOG_ ## level, librte_member_logtype, "%s(): " fmt, \
+ __func__, ## args)
+
+/** @internal setsummary structure. */
+struct rte_member_setsum;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameter struct used to create set summary
+ */
+struct rte_member_parameters;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Define different set summary types
+ */
+enum rte_member_setsum_type {
+ RTE_MEMBER_TYPE_HT = 0, /**< Hash table based set summary. */
+ RTE_MEMBER_TYPE_VBF, /**< Vector of bloom filters. */
+ RTE_MEMBER_NUM_TYPE
+};
+
+/** @internal compare function for different arch. */
+enum rte_member_sig_compare_function {
+ RTE_MEMBER_COMPARE_SCALAR = 0,
+ RTE_MEMBER_COMPARE_AVX2,
+ RTE_MEMBER_COMPARE_NUM
+};
+
+/** @internal setsummary structure. */
+struct rte_member_setsum {
+ enum rte_member_setsum_type type; /* Type of the set summary. */
+ uint32_t key_len; /* Length of key. */
+ uint32_t prim_hash_seed; /* Primary hash function seed. */
+ uint32_t sec_hash_seed; /* Secondary hash function seed. */
+
+ /* Hash table based. */
+ uint32_t bucket_cnt; /* Number of buckets. */
+ uint32_t bucket_mask; /* Bit mask to get bucket index. */
+ /* For runtime selecting AVX, scalar, etc for signature comparison. */
+ enum rte_member_sig_compare_function sig_cmp_fn;
+ uint8_t cache; /* If it is cache mode for ht based. */
+
+ /* Vector bloom filter. */
+ uint32_t num_set; /* Number of set (bf) in vbf. */
+ uint32_t bits; /* Number of bits in each bf. */
+ uint32_t bit_mask; /* Bit mask to get bit location in bf. */
+ uint32_t num_hashes; /* Number of hash values to index bf. */
+
+ uint32_t mul_shift; /* vbf internal variable used during bit test. */
+ uint32_t div_shift; /* vbf internal variable used during bit test. */
+
+ void *table; /* This is the handler of hash table or vBF array. */
+
+
+ /* Second cache line should start here. */
+ uint32_t socket_id; /* NUMA Socket ID for memory. */
+ char name[RTE_MEMBER_NAMESIZE]; /* Name of this set summary. */
+} __rte_cache_aligned;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameters used when create the set summary table. Currently user can
+ * specify two types of setsummary: HT based and vBF. For HT based, user can
+ * specify cache or non-cache mode. Here is a table to describe some differences
+ *
+ */
+struct rte_member_parameters {
+ const char *name; /**< Name of the hash. */
+
+ /**
+ * User to specify the type of the setsummary from one of
+ * rte_member_setsum_type.
+ *
+ * HT based setsummary is implemented like a hash table. User should use
+ * this type when there are many sets.
+ *
+ * vBF setsummary is a vector of bloom filters. It is used when number
+ * of sets is not big (less than 32 for current implementation).
+ */
+ enum rte_member_setsum_type type;
+
+ /**
+ * is_cache is only used for HT based setsummary.
+ *
+ * If it is HT based setsummary, user to specify the subtype or mode
+ * of the setsummary. It could be cache, or non-cache mode.
+ * Set is_cache to be 1 if to use as cache mode.
+ *
+ * For cache mode, keys can be evicted out of the HT setsummary. Keys
+ * with the same signature and map to the same bucket
+ * will overwrite each other in the setsummary table.
+ * This mode is useful for the case that the set-summary only
+ * needs to keep record of the recently inserted keys. Both
+ * false-negative and false-positive could happen.
+ *
+ * For non-cache mode, keys cannot be evicted out of the cache. So for
+ * this mode the setsummary will become full eventually. Keys with the
+ * same signature but map to the same bucket will still occupy multiple
+ * entries. This mode does not give false-negative result.
+ */
+ uint8_t is_cache;
+
+ /**
+ * For HT setsummary, num_keys equals to the number of entries of the
+ * table. When the number of keys inserted in the HT setsummary
+ * approaches this number, eviction could happen. For cache mode,
+ * keys could be evicted out of the table. For non-cache mode, keys will
+ * be evicted to other buckets like cuckoo hash. The table will also
+ * likely to become full before the number of inserted keys equal to the
+ * total number of entries.
+ *
+ * For vBF, num_keys equal to the expected number of keys that will
+ * be inserted into the vBF. The implementation assumes the keys are
+ * evenly distributed to each BF in vBF. This is used to calculate the
+ * number of bits we need for each BF. User does not specify the size of
+ * each BF directly because the optimal size depends on the num_keys
+ * and false positive rate.
+ */
+ uint32_t num_keys;
+
+ /**
+ * The length of key is used for hash calculation. Since key is not
+ * stored in set-summary, large key does not require more memory space.
+ */
+ uint32_t key_len;
+
+ /**
+ * num_set is only used for vBF, but not used for HT setsummary.
+ *
+ * num_set is equal to the number of BFs in vBF. For current
+ * implementation, it only supports 1,2,4,8,16,32 BFs in one vBF set
+ * summary. If other number of sets are needed, for example 5, the user
+ * should allocate the minimum available value that larger than 5,
+ * which is 8.
+ */
+ uint32_t num_set;
+
+ /**
+ * false_positive_rate is only used for vBF, but not used for HT
+ * setsummary.
+ *
+ * For vBF, false_positive_rate is the user-defined false positive rate
+ * given expected number of inserted keys (num_keys). It is used to
+ * calculate the total number of bits for each BF, and the number of
+ * hash values used during lookup and insertion. For details please
+ * refer to vBF implementation and membership library documentation.
+ *
+ * For HT, This parameter is not directly set by users.
+ * HT setsummary's false positive rate is in the order of:
+ * false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit signature.
+ * This is because two keys needs to map to same bucket and same
+ * signature to have a collision (false positive). bucket_count is equal
+ * to number of entries (num_keys) divided by entry count per bucket
+ * (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_positive_rate is not
+ * directly set by users for HT mode.
+ */
+ float false_positive_rate;
+
+ /**
+ * We use two seeds to calculate two independent hashes for each key.
+ *
+ * For HT type, one hash is used as signature, and the other is used
+ * for bucket location.
+ * For vBF type, these two hashes and their combinations are used as
+ * hash locations to index the bit array.
+ */
+ uint32_t prim_hash_seed;
+
+ /**
+ * The secondary seed should be a different value from the primary seed.
+ */
+ uint32_t sec_hash_seed;
+
+ int socket_id; /**< NUMA Socket ID for memory. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Find an existing set-summary and return a pointer to it.
+ *
+ * @param name
+ * Name of the set-summary.
+ * @return
+ * Pointer to the set-summary or NULL if object not found
+ * with rte_errno set appropriately. Possible rte_errno values include:
+ * - ENOENT - value not available for return
+ */
+struct rte_member_setsum *
+rte_member_find_existing(const char *name);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Create set-summary (SS).
+ *
+ * @param params
+ * Parameters to initialize the setsummary.
+ * @return
+ * Return the pointer to the setsummary.
+ * Return value is NULL if the creation failed.
+ */
+struct rte_member_setsum *
+rte_member_create(const struct rte_member_parameters *params);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup key in set-summary (SS).
+ * Single key lookup and return as soon as the first match found
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param key
+ * Pointer of the key to be looked up.
+ * @param set_id
+ * Output the set id matches the key.
+ * @return
+ * Return 1 for found a match and 0 for not found a match.
+ */
+int
+rte_member_lookup(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t *set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup bulk of keys in set-summary (SS).
+ * Each key lookup returns as soon as the first match found
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param keys
+ * Pointer of the bulk of keys to be looked up.
+ * @param num_keys
+ * Number of keys that will be lookup.
+ * @param set_ids
+ * Output set ids for all the keys to this array.
+ * User should preallocate array that can contain all results, which size is
+ * the num_keys.
+ * @return
+ * The number of keys that found a match.
+ */
+int
+rte_member_lookup_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a key in set-summary (SS) for multiple matches.
+ * The key lookup will find all matched entries (multiple match).
+ * Note that for cache mode of HT, each key can have at most one match. This is
+ * because keys with same signature that maps to same bucket will overwrite
+ * each other. So multi-match lookup should be used for vBF and non-cache HT.
+ *
+ * @param setsum
+ * Pointer of a set-summary.
+ * @param key
+ * Pointer of the key that to be looked up.
+ * @param max_match_per_key
+ * User specified maximum number of matches for each key. The function returns
+ * as soon as this number of matches found for the key.
+ * @param set_id
+ * Output set ids for all the matches of the key. User needs to preallocate
+ * the array that can contain max_match_per_key number of results.
+ * @return
+ * The number of matches that found for the key.
+ * For cache mode HT set-summary, the number should be at most 1.
+ */
+int
+rte_member_lookup_multi(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t max_match_per_key,
+ member_set_t *set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a bulk of keys in set-summary (SS) for multiple matches each key.
+ * Each key lookup will find all matched entries (multiple match).
+ * Note that for cache mode HT, each key can have at most one match. So
+ * multi-match function is mainly used for vBF and non-cache mode HT.
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param keys
+ * Pointer of the keys to be looked up.
+ * @param num_keys
+ * The number of keys that will be lookup.
+ * @param max_match_per_key
+ * The possible maximum number of matches for each key.
+ * @param match_count
+ * Output the number of matches for each key in an array.
+ * @param set_ids
+ * Return set ids for all the matches of all keys. Users pass in a
+ * preallocated 2D array with first dimension as key index and second
+ * dimension as match index. For example set_ids[bulk_size][max_match_per_key]
+ * @return
+ * The number of keys that found one or more matches in the set-summary.
+ */
+int
+rte_member_lookup_multi_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Insert key into set-summary (SS).
+ *
+ * @param setsum
+ * Pointer of a set-summary.
+ * @param key
+ * Pointer of the key to be added.
+ * @param set_id
+ * The set id associated with the key that needs to be added. Different mode
+ * supports different set_id ranges. 0 cannot be used as set_id since
+ * RTE_MEMBER_NO_MATCH by default is set as 0.
+ * For HT mode, the set_id has range as [1, 0x7FFF], MSB is reserved.
+ * For vBF mode the set id is limited by the num_set parameter when create
+ * the set-summary.
+ * @return
+ * HT (cache mode) and vBF should never fail unless the set_id is not in the
+ * valid range. In such case -EINVAL is returned.
+ * For HT (non-cache mode) it could fail with -ENOSPC error code when table is
+ * full.
+ * For success it returns different values for different modes to provide
+ * extra information for users.
+ * Return 0 for HT (cache mode) if the add does not cause
+ * eviction, return 1 otherwise. Return 0 for non-cache mode if success,
+ * -ENOSPC for full, and 1 if cuckoo eviction happens.
+ * Always returns 0 for vBF mode.
+ */
+int
+rte_member_add(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * De-allocate memory used by set-summary.
+ *
+ * @param setsum
+ * Pointer to the set summary.
+ */
+void
+rte_member_free(struct rte_member_setsum *setsum);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Reset the set-summary tables. E.g. reset bits to be 0 in BF,
+ * reset set_id in each entry to be RTE_MEMBER_NO_MATCH in HT based SS.
+ *
+ * @param setsum
+ * Pointer to the set-summary.
+ */
+void
+rte_member_reset(const struct rte_member_setsum *setsum);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Delete items from the set-summary. Note that vBF does not support deletion
+ * in current implementation. For vBF, error code of -EINVAL will be returned.
+ *
+ * @param setsum
+ * Pointer to the set-summary.
+ * @param key
+ * Pointer of the key to be deleted.
+ * @param set_id
+ * For HT mode, we need both key and its corresponding set_id to
+ * properly delete the key. Without set_id, we may delete other keys with the
+ * same signature.
+ * @return
+ * If no entry found to delete, an error code of -ENOENT could be returned.
+ */
+int
+rte_member_delete(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_H_ */
diff --git a/lib/librte_member/rte_member_version.map b/lib/librte_member/rte_member_version.map
new file mode 100644
index 0000000..019e4cd
--- /dev/null
+++ b/lib/librte_member/rte_member_version.map
@@ -0,0 +1,16 @@
+DPDK_17.11 {
+ global:
+
+ rte_member_add;
+ rte_member_create;
+ rte_member_delete;
+ rte_member_find_existing;
+ rte_member_free;
+ rte_member_lookup;
+ rte_member_lookup_bulk;
+ rte_member_lookup_multi;
+ rte_member_lookup_multi_bulk;
+ rte_member_reset;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v5 1/7] member: implement main API
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 1/7] member: implement main API Yipeng Wang
@ 2017-10-03 8:42 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-03 8:42 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Tuesday, October 3, 2017 5:32 AM
> To: dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v5 1/7] member: implement main API
>
> Membership library is an extension and generalization of a traditional filter
> (for example Bloom Filter and cuckoo filter) structure.
> In general, the Membership library is a data structure that provides a "set-
> summary" and responds to set-membership queries of whether a certain
> element belongs to a set(s). A membership test for an element will return
> the set this element belongs to or not-found if the element is never inserted
> into the set-summary.
>
> The results of the membership test are not 100% accurate. Certain false
> positive or false negative probability could exist. However, comparing to a
> "full-blown" complete list of elements, a "set-summary"
> is memory efficient and fast on lookup.
>
> This patch adds the main API definition.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v5 2/7] member: implement HT mode
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 1/7] member: implement main API Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 8:47 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 3/7] member: implement vBF mode Yipeng Wang
` (5 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
One of the set-summary structures is hash-table based
set-summary (HTSS). One example is cuckoo filter [1].
Comparing to a traditional hash table, HTSS has a much more
compact structure. For each element, only one signature and
its corresponding set ID is stored. No key comparison is required
during lookup. For the table structure, there are multiple entries
in each bucket, and the table is composed of many buckets.
Two modes are supported for HTSS, "cache" and "none-cache" modes.
The non-cache mode is similar to the cuckoo filter [1].
When a bucket is full, one entry will be evicted to its
alternative bucket to make space for the new key. The table could
be full and then no more keys could be inserted. This mode has
false-positive rate but no false-negative. Multiple entries
with same signature could stay in the same bucket.
The "cache" mode does not evict key to its alternative bucket
when a bucket is full, an existing key will be evicted out of
the table like a cache. Thus, the table will never reject keys when
it is full. Another property is in each bucket, there cannot be
multiple entries with same signature. The mode could have both
false-positive and false-negative probability.
This patch adds the implementation of HTSS.
[1] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically
Better Than Bloom,” in Conference on emerging Networking
Experiments and Technologies, 2014.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_ht.c | 506 ++++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_ht.h | 94 +++++++
3 files changed, 601 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 1a79eaa..ad26548 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
new file mode 100644
index 0000000..08cce02
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.c
@@ -0,0 +1,506 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_errno.h>
+#include <rte_malloc.h>
+#include <rte_prefetch.h>
+#include <rte_random.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+
+/* Search bucket for entry with tmp_sig and update set_id */
+static inline int
+update_entry_search(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t set_id)
+{
+ int32_t i;
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[bucket_id].sigs[i] == tmp_sig) {
+ buckets[bucket_id].sets[i] = set_id;
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static inline int
+search_bucket_single(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t *set_id)
+{
+ int32_t iter;
+
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket_id].sigs[iter] &&
+ buckets[bucket_id].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket_id].sets[iter];
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t matches_per_key,
+ member_set_t *set_id)
+{
+ int32_t iter;
+
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket_id].sigs[iter] &&
+ buckets[bucket_id].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket_id].sets[iter];
+ (*counter)++;
+ if (*counter >= matches_per_key)
+ return;
+ }
+ }
+}
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+ uint32_t i, j;
+ uint32_t size_bucket_t;
+ uint32_t num_entries = rte_align32pow2(params->num_keys);
+
+ if ((num_entries > RTE_MEMBER_ENTRIES_MAX) ||
+ !rte_is_power_of_2(RTE_MEMBER_BUCKET_ENTRIES) ||
+ num_entries < RTE_MEMBER_BUCKET_ENTRIES) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR,
+ "Membership HT create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ uint32_t num_buckets = num_entries / RTE_MEMBER_BUCKET_ENTRIES;
+
+ size_bucket_t = sizeof(struct member_ht_bucket);
+
+ struct member_ht_bucket *buckets = rte_zmalloc_socket(NULL,
+ num_buckets * size_bucket_t,
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ if (buckets == NULL) {
+ RTE_MEMBER_LOG(ERR, "memory allocation failed for HT "
+ "setsummary\n");
+ return -ENOMEM;
+ }
+
+ ss->table = buckets;
+ ss->bucket_cnt = num_buckets;
+ ss->bucket_mask = num_buckets - 1;
+ ss->cache = params->is_cache;
+
+ for (i = 0; i < num_buckets; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+
+ RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
+ "the table has %u entries, %u buckets\n",
+ num_entries, num_buckets);
+ return 0;
+}
+
+static inline void
+get_buckets_index(const struct rte_member_setsum *ss, const void *key,
+ uint32_t *prim_bkt, uint32_t *sec_bkt, member_sig_t *sig)
+{
+ uint32_t first_hash = MEMBER_HASH_FUNC(key, ss->key_len,
+ ss->prim_hash_seed);
+ uint32_t sec_hash = MEMBER_HASH_FUNC(&first_hash, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ /*
+ * We use the first hash value for the signature, and the second hash
+ * value to derive the primary and secondary bucket locations.
+ *
+ * For non-cache mode, we use the lower bits for the primary bucket
+ * location. Then we xor primary bucket location and the signature
+ * to get the secondary bucket location. This is called "partial-key
+ * cuckoo hashing" proposed by B. Fan, et al's paper
+ * "Cuckoo Filter: Practically Better Than Bloom". The benefit to use
+ * xor is that one could derive the alternative bucket location
+ * by only using the current bucket location and the signature. This is
+ * generally required by non-cache mode's eviction and deletion
+ * process without the need to store alternative hash value nor the full
+ * key.
+ *
+ * For cache mode, we use the lower bits for the primary bucket
+ * location and the higher bits for the secondary bucket location. In
+ * cache mode, keys are simply overwritten if bucket is full. We do not
+ * use xor since lower/higher bits are more independent hash values thus
+ * should provide slightly better table load.
+ */
+ *sig = first_hash;
+ if (ss->cache) {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (sec_hash >> 16) & ss->bucket_mask;
+ } else {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (*prim_bkt ^ *sig) & ss->bucket_mask;
+ }
+}
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *ss,
+ const void *key, member_set_t *set_id)
+{
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+
+ return 0;
+}
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, member_set_t *set_id)
+{
+ uint32_t i;
+ uint32_t num_matches = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+
+ for (i = 0; i < num_keys; i++) {
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ num_matches++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t num_matches = 0;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &num_matches,
+ match_per_key, set_id);
+ if (num_matches < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &num_matches, match_per_key, set_id);
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ uint32_t i;
+ uint32_t num_matches = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t match_cnt_tmp;
+ member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_tmp = 0;
+
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ num_matches++;
+ }
+ return num_matches;
+}
+
+static inline int
+try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ member_sig_t sig, member_set_t set_id)
+{
+ int i;
+ /* If not full then insert into one slot */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[prim].sigs[i] = sig;
+ buckets[prim].sets[i] = set_id;
+ return 0;
+ }
+ }
+ /* If prim failed, we need to access second bucket */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[sec].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[sec].sigs[i] = sig;
+ buckets[sec].sets[i] = set_id;
+ return 0;
+ }
+ }
+ return -1;
+}
+
+static inline int
+try_update(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ member_sig_t sig, member_set_t set_id)
+{
+ if (update_entry_search(prim, sig, buckets, set_id) ||
+ update_entry_search(sec, sig, buckets, set_id))
+ return 0;
+ return -1;
+}
+
+static inline int
+evict_from_bucket(void)
+{
+ /* For now, we randomly pick one entry to evict */
+ return rte_rand() & (RTE_MEMBER_BUCKET_ENTRIES - 1);
+}
+
+/*
+ * This function is similar to the cuckoo hash make_space function in hash
+ * library
+ */
+static inline int
+make_space_bucket(const struct rte_member_setsum *ss, uint32_t bkt_idx,
+ unsigned int *nr_pushes)
+{
+ unsigned int i, j;
+ int ret;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t next_bucket_idx;
+ struct member_ht_bucket *next_bkt[RTE_MEMBER_BUCKET_ENTRIES];
+ struct member_ht_bucket *bkt = &buckets[bkt_idx];
+ /* MSB is set to indicate if an entry has been already pushed */
+ member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
+
+ /*
+ * Push existing item (search for bucket with space in
+ * alternative locations) to its alternative location
+ */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ /* Search for space in alternative locations */
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_idx) & ss->bucket_mask;
+ next_bkt[i] = &buckets[next_bucket_idx];
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++) {
+ if (next_bkt[i]->sets[j] == RTE_MEMBER_NO_MATCH)
+ break;
+ }
+
+ if (j != RTE_MEMBER_BUCKET_ENTRIES)
+ break;
+ }
+
+ /* Alternative location has spare room (end of recursive function) */
+ if (i != RTE_MEMBER_BUCKET_ENTRIES) {
+ next_bkt[i]->sigs[j] = bkt->sigs[i];
+ next_bkt[i]->sets[j] = bkt->sets[i];
+ return i;
+ }
+
+ /* Pick entry that has not been pushed yet */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++)
+ if ((bkt->sets[i] & flag_mask) == 0)
+ break;
+
+ /* All entries have been pushed, so entry cannot be added */
+ if (i == RTE_MEMBER_BUCKET_ENTRIES ||
+ ++(*nr_pushes) > RTE_MEMBER_MAX_PUSHES)
+ return -ENOSPC;
+
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_idx) & ss->bucket_mask;
+ /* Set flag to indicate that this entry is going to be pushed */
+ bkt->sets[i] |= flag_mask;
+
+ /* Need room in alternative bucket to insert the pushed entry */
+ ret = make_space_bucket(ss, next_bucket_idx, nr_pushes);
+ /*
+ * After recursive function.
+ * Clear flags and insert the pushed entry
+ * in its alternative location if successful,
+ * or return error
+ */
+ bkt->sets[i] &= ~flag_mask;
+ if (ret >= 0) {
+ next_bkt[i]->sigs[ret] = bkt->sigs[i];
+ next_bkt[i]->sets[ret] = bkt->sets[i];
+ return i;
+ } else
+ return ret;
+}
+
+int
+rte_member_add_ht(const struct rte_member_setsum *ss,
+ const void *key, member_set_t set_id)
+{
+ int ret;
+ unsigned int nr_pushes = 0;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+ member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
+
+ if (set_id == RTE_MEMBER_NO_MATCH || (set_id & flag_mask) != 0)
+ return -EINVAL;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ /*
+ * If it is cache based setsummary, we try overwriting (updating)
+ * existing entry with the same signature first. In cache mode, we allow
+ * false negatives and only cache the most recent keys.
+ *
+ * For non-cache mode, we do not update existing entry with the same
+ * signature. This is because if two keys with same signature update
+ * each other, false negative may happen, which is not the expected
+ * behavior for non-cache setsummary.
+ */
+ if (ss->cache) {
+ ret = try_update(buckets, prim_bucket, sec_bucket, tmp_sig,
+ set_id);
+ if (ret != -1)
+ return ret;
+ }
+ /* If not full then insert into one slot*/
+ ret = try_insert(buckets, prim_bucket, sec_bucket, tmp_sig, set_id);
+ if (ret != -1)
+ return ret;
+
+ /* Random pick prim or sec for recursive displacement */
+ uint32_t select_bucket = (tmp_sig && 1U) ? prim_bucket : sec_bucket;
+ if (ss->cache) {
+ ret = evict_from_bucket();
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ return 1;
+ }
+
+ ret = make_space_bucket(ss, select_bucket, &nr_pushes);
+ if (ret >= 0) {
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ ret = 1;
+ }
+
+ return ret;
+}
+
+void
+rte_member_free_ht(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+int
+rte_member_delete_ht(const struct rte_member_setsum *ss, const void *key,
+ member_set_t set_id)
+{
+ int i;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[prim_bucket].sigs[i] &&
+ set_id == buckets[prim_bucket].sets[i]) {
+ buckets[prim_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[sec_bucket].sigs[i] &&
+ set_id == buckets[sec_bucket].sets[i]) {
+ buckets[sec_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+ return -ENOENT;
+}
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *ss)
+{
+ uint32_t i, j;
+ struct member_ht_bucket *buckets = ss->table;
+
+ for (i = 0; i < ss->bucket_cnt; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+}
diff --git a/lib/librte_member/rte_member_ht.h b/lib/librte_member/rte_member_ht.h
new file mode 100644
index 0000000..3148a49
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.h
@@ -0,0 +1,94 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_HT_H_
+#define _RTE_MEMBER_HT_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Maximum number of pushes for cuckoo path in HT mode. */
+#define RTE_MEMBER_MAX_PUSHES 50
+
+typedef uint16_t member_sig_t; /* signature size is 16 bit */
+
+/* The bucket struct for ht setsum */
+struct member_ht_bucket {
+ member_sig_t sigs[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte signature */
+ member_set_t sets[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte set */
+} __rte_cache_aligned;
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+int
+rte_member_add_ht(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t set_id);
+
+void
+rte_member_free_ht(struct rte_member_setsum *setsum);
+
+int
+rte_member_delete_ht(const struct rte_member_setsum *ss, const void *key,
+ member_set_t set_id);
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *setsum);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_HT_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v5 2/7] member: implement HT mode
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 2/7] member: implement HT mode Yipeng Wang
@ 2017-10-03 8:47 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-03 8:47 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
Hi Yipeng,
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Tuesday, October 3, 2017 5:32 AM
> To: dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v5 2/7] member: implement HT mode
>
> One of the set-summary structures is hash-table based set-summary
> (HTSS). One example is cuckoo filter [1].
>
> Comparing to a traditional hash table, HTSS has a much more compact
> structure. For each element, only one signature and its corresponding set ID
> is stored. No key comparison is required during lookup. For the table
> structure, there are multiple entries in each bucket, and the table is
> composed of many buckets.
>
> Two modes are supported for HTSS, "cache" and "none-cache" modes.
> The non-cache mode is similar to the cuckoo filter [1].
> When a bucket is full, one entry will be evicted to its alternative bucket to
> make space for the new key. The table could be full and then no more keys
> could be inserted. This mode has false-positive rate but no false-negative.
> Multiple entries with same signature could stay in the same bucket.
>
> The "cache" mode does not evict key to its alternative bucket when a
> bucket is full, an existing key will be evicted out of the table like a cache.
> Thus, the table will never reject keys when it is full. Another property is in
> each bucket, there cannot be multiple entries with same signature. The
> mode could have both false-positive and false-negative probability.
>
> This patch adds the implementation of HTSS.
>
> [1] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically Better
> Than Bloom,” in Conference on emerging Networking Experiments and
> Technologies, 2014.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
...
Just a comment below. The rest looks fine to me (and thanks for the explanations
in the v4 patch, all clear), so keep my "Reviewed-by" in next version.
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
> +++ b/lib/librte_member/rte_member_ht.c
...
> +/* Search bucket for entry with tmp_sig and update set_id */ static
> +inline int update_entry_search(uint32_t bucket_id, member_sig_t
> +tmp_sig,
> + struct member_ht_bucket *buckets,
> + member_set_t set_id)
> +{
> + int32_t i;
"i" cannot take negative values, so use uint32_t
(same in "search_bucket_single" and "search_bucket_multi").
> +
> + for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
> + if (buckets[bucket_id].sigs[i] == tmp_sig) {
> + buckets[bucket_id].sets[i] = set_id;
> + return 1;
> + }
> + }
> + return 0;
> +}
> +
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v5 3/7] member: implement vBF mode
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 1/7] member: implement main API Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 2/7] member: implement HT mode Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 8:50 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 4/7] member: add AVX for HT mode Yipeng Wang
` (4 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
Bloom Filter (BF) [1] is a well-known space-efficient
probabilistic data structure that answers set membership queries.
Vector of Bloom Filters (vBF) is an extension to traditional BF
that supports multi-set membership testing. Traditional BF will
return found or not-found for each key. vBF will also return
which set the key belongs to if it is found.
Since each set requires a BF, vBF should be used when set count
is small. vBF's false positive rate could be set appropriately so
that its memory requirement and lookup speed is better in certain
cases comparing to HT based set-summary.
This patch adds the vBF implementation.
[1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
Errors,” Communications of the ACM, 1970.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_vbf.c | 351 +++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_vbf.h | 82 +++++++++
3 files changed, 434 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index ad26548..50275ed 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c rte_member_vbf.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_vbf.c b/lib/librte_member/rte_member_vbf.c
new file mode 100644
index 0000000..125fdd6
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.c
@@ -0,0 +1,351 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <math.h>
+#include <string.h>
+
+#include <rte_malloc.h>
+#include <rte_memory.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_vbf.h"
+
+/*
+ * vBF currently implemented as a big array.
+ * The BFs have a vertical layout. Bits in same location of all bfs will stay
+ * in the same cache line.
+ * For example, if we have 32 bloom filters, we use a uint32_t array to
+ * represent all of them. array[0] represent the first location of all the
+ * bloom filters, array[1] represents the second location of all the
+ * bloom filters, etc. The advantage of this layout is to minimize the average
+ * number of memory accesses to test all bloom filters.
+ *
+ * Currently the implementation supports vBF containing 1,2,4,8,16,32 BFs.
+ */
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+
+ if (params->num_set > RTE_MEMBER_MAX_BF ||
+ !rte_is_power_of_2(params->num_set) ||
+ params->num_keys == 0 ||
+ params->false_positive_rate == 0 ||
+ params->false_positive_rate > 1) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "vBF create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ /* We assume expected keys evenly distribute to all BFs */
+ uint32_t num_keys_per_bf = 1 + (params->num_keys - 1) / ss->num_set;
+
+ /*
+ * Note that the false positive rate is for all BFs in the vBF
+ * such that the single BF's false positive rate needs to be
+ * calculated.
+ * Assume each BF's False positive rate is fp_one_bf. The total false
+ * positive rate is fp = 1-(1-fp_one_bf)^n.
+ * => fp_one_bf = 1 - (1-fp)^(1/n)
+ */
+
+ float fp_one_bf = 1 - pow((1 - params->false_positive_rate),
+ 1.0 / ss->num_set);
+
+ if (fp_one_bf == 0) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "BF false positive rate is too small\n");
+ return -EINVAL;
+ }
+
+ uint32_t bits = ceil((num_keys_per_bf *
+ log(fp_one_bf)) /
+ log(1.0 / (pow(2.0, log(2.0)))));
+
+ /* We round to power of 2 for performance during lookup */
+ ss->bits = rte_align32pow2(bits);
+
+ ss->num_hashes = (uint32_t)(log(2.0) * bits / num_keys_per_bf);
+ ss->bit_mask = ss->bits - 1;
+
+
+ /*
+ * Since we round the bits to power of 2, the final false positive
+ * rate will probably not be same as the user specified. We log the
+ * new value as debug message.
+ */
+ float new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ ss->num_hashes)), ss->num_hashes);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ /*
+ * Reduce hash function count, until we approach the user specified
+ * false-positive rate. Otherwise it is too conservative
+ */
+ int tmp_num_hash = ss->num_hashes;
+
+ while (tmp_num_hash > 1) {
+ float tmp_fp = new_fp;
+
+ tmp_num_hash--;
+ new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ tmp_num_hash)), tmp_num_hash);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ if (new_fp > params->false_positive_rate) {
+ new_fp = tmp_fp;
+ tmp_num_hash++;
+ break;
+ }
+ }
+
+ ss->num_hashes = tmp_num_hash;
+
+ /*
+ * To avoid multiplication and division:
+ * mul_shift is used for multiplication shift during bit test
+ * div_shift is used for division shift, to be divided by number of bits
+ * represented by a uint32_t variable
+ */
+ ss->mul_shift = __builtin_ctzl(ss->num_set);
+ ss->div_shift = __builtin_ctzl(32 >> ss->mul_shift);
+
+ RTE_MEMBER_LOG(DEBUG, "vector bloom filter created, "
+ "each bloom filter expects %u keys, needs %u bits, %u hashes, "
+ "with false positive rate set as %.5f, "
+ "The new calculated vBF false positive rate is %.5f\n",
+ num_keys_per_bf, ss->bits, ss->num_hashes, fp_one_bf, new_fp);
+
+ ss->table = rte_zmalloc_socket(NULL, ss->num_set * (ss->bits >> 3),
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+ if (ss->table == NULL)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static inline uint32_t
+test_bit(uint32_t bit_loc, const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t n = ss->num_set;
+ uint32_t div_shift = ss->div_shift;
+ uint32_t mul_shift = ss->mul_shift;
+ /*
+ * a is how many bits in one BF are represented by one 32bit
+ * variable.
+ */
+ uint32_t a = 32 >> mul_shift;
+ /*
+ * x>>b is the divide, x & (a-1) is the mod, & (1<<n-1) to mask out bits
+ * we do not need
+ */
+ return (vbf[bit_loc >> div_shift] >>
+ ((bit_loc & (a - 1)) << mul_shift)) & ((1ULL << n) - 1);
+}
+
+static inline void
+set_bit(uint32_t bit_loc, const struct rte_member_setsum *ss, int32_t set)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t div_shift = ss->div_shift;
+ uint32_t mul_shift = ss->mul_shift;
+ uint32_t a = 32 >> mul_shift;
+
+ vbf[bit_loc >> div_shift] |=
+ 1UL << (((bit_loc & (a - 1)) << mul_shift) + set - 1);
+}
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *ss, const void *key,
+ member_set_t *set_id)
+{
+ uint32_t j;
+ uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ uint32_t mask = ~0;
+ uint32_t bit_loc;
+
+ for (j = 0; j < ss->num_hashes; j++) {
+ bit_loc = (h1 + j * h2) & ss->bit_mask;
+ mask &= test_bit(bit_loc, ss);
+ }
+
+ if (mask) {
+ *set_id = __builtin_ctzl(mask) + 1;
+ return 1;
+ }
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ return 0;
+}
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, member_set_t *set_ids)
+{
+ uint32_t i, k;
+ uint32_t num_matches = 0;
+ uint32_t mask[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t h1[RTE_MEMBER_LOOKUP_BULK_MAX], h2[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t bit_loc;
+
+ for (i = 0; i < num_keys; i++)
+ h1[i] = MEMBER_HASH_FUNC(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ for (i = 0; i < num_keys; i++)
+ h2[i] = MEMBER_HASH_FUNC(&h1[i], sizeof(uint32_t),
+ ss->sec_hash_seed);
+ for (i = 0; i < num_keys; i++) {
+ mask[i] = ~0;
+ for (k = 0; k < ss->num_hashes; k++) {
+ bit_loc = (h1[i] + k * h2[i]) & ss->bit_mask;
+ mask[i] &= test_bit(bit_loc, ss);
+ }
+ }
+ for (i = 0; i < num_keys; i++) {
+ if (mask[i]) {
+ set_ids[i] = __builtin_ctzl(mask[i]) + 1;
+ num_matches++;
+ } else
+ set_ids[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t num_matches = 0;
+ uint32_t j;
+ uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ uint32_t mask = ~0;
+ uint32_t bit_loc;
+
+ for (j = 0; j < ss->num_hashes; j++) {
+ bit_loc = (h1 + j * h2) & ss->bit_mask;
+ mask &= test_bit(bit_loc, ss);
+ }
+ while (mask) {
+ uint32_t loc = __builtin_ctzl(mask);
+ set_id[num_matches] = loc + 1;
+ num_matches++;
+ if (num_matches >= match_per_key)
+ return num_matches;
+ mask &= ~(1UL << loc);
+ }
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ uint32_t i, k;
+ uint32_t num_matches = 0;
+ uint32_t match_cnt_t;
+ uint32_t mask[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t h1[RTE_MEMBER_LOOKUP_BULK_MAX], h2[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t bit_loc;
+
+ for (i = 0; i < num_keys; i++)
+ h1[i] = MEMBER_HASH_FUNC(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ for (i = 0; i < num_keys; i++)
+ h2[i] = MEMBER_HASH_FUNC(&h1[i], sizeof(uint32_t),
+ ss->sec_hash_seed);
+ for (i = 0; i < num_keys; i++) {
+ mask[i] = ~0;
+ for (k = 0; k < ss->num_hashes; k++) {
+ bit_loc = (h1[i] + k * h2[i]) & ss->bit_mask;
+ mask[i] &= test_bit(bit_loc, ss);
+ }
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_t = 0;
+ while (mask[i]) {
+ uint32_t loc = __builtin_ctzl(mask[i]);
+ set_ids[i * match_per_key + match_cnt_t] = loc + 1;
+ match_cnt_t++;
+ if (match_cnt_t >= match_per_key)
+ break;
+ mask[i] &= ~(1UL << loc);
+ }
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ num_matches++;
+ }
+ return num_matches;
+}
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *ss,
+ const void *key, member_set_t set_id)
+{
+ uint32_t i, h1, h2;
+ uint32_t bit_loc;
+
+ if (set_id > ss->num_set || set_id == RTE_MEMBER_NO_MATCH)
+ return -EINVAL;
+
+ h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t), ss->sec_hash_seed);
+
+ for (i = 0; i < ss->num_hashes; i++) {
+ bit_loc = (h1 + i * h2) & ss->bit_mask;
+ set_bit(bit_loc, ss, set_id);
+ }
+ return 0;
+}
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ memset(vbf, 0, (ss->num_set * ss->bits) >> 3);
+}
diff --git a/lib/librte_member/rte_member_vbf.h b/lib/librte_member/rte_member_vbf.h
new file mode 100644
index 0000000..5bc158b
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.h
@@ -0,0 +1,82 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_VBF_H_
+#define _RTE_MEMBER_VBF_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Currently we only support up to 32 sets in vBF */
+#define RTE_MEMBER_MAX_BF 32
+
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t set_id);
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss);
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *setsum);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_VBF_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v5 3/7] member: implement vBF mode
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 3/7] member: implement vBF mode Yipeng Wang
@ 2017-10-03 8:50 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-03 8:50 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
Hi Yipeng,
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Tuesday, October 3, 2017 5:32 AM
> To: dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v5 3/7] member: implement vBF mode
>
> Bloom Filter (BF) [1] is a well-known space-efficient probabilistic data
> structure that answers set membership queries.
> Vector of Bloom Filters (vBF) is an extension to traditional BF that supports
> multi-set membership testing. Traditional BF will return found or not-found
> for each key. vBF will also return which set the key belongs to if it is found.
>
> Since each set requires a BF, vBF should be used when set count is small.
> vBF's false positive rate could be set appropriately so that its memory
> requirement and lookup speed is better in certain cases comparing to HT
> based set-summary.
>
> This patch adds the vBF implementation.
>
> [1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
> Errors,” Communications of the ACM, 1970.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Just a comment below. The rest looks fine to me (and thanks for the explanations
in the v4 patch, all clear), so keep my "Reviewed-by" in next version.
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
...
> +++ b/lib/librte_member/rte_member_vbf.c
...
> + /*
> + * To avoid multiplication and division:
> + * mul_shift is used for multiplication shift during bit test
> + * div_shift is used for division shift, to be divided by number of bits
> + * represented by a uint32_t variable
> + */
> + ss->mul_shift = __builtin_ctzl(ss->num_set);
> + ss->div_shift = __builtin_ctzl(32 >> ss->mul_shift);
Remove the whitespace added after the tab in the two lines above.
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v5 4/7] member: add AVX for HT mode
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
` (2 preceding siblings ...)
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 3/7] member: implement vBF mode Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 9:01 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 5/7] member: enable the library Yipeng Wang
` (3 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
For key search, the signatures of all entries are compared against
the signature of the key that is being looked up. Since all
signatures are contiguously put in a bucket, they can be compared
with vector instructions (AVX2), achieving higher lookup performance.
This patch adds AVX2 implementation in a separate header file.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
lib/librte_member/rte_member_ht.c | 142 +++++++++++++++++++++++++++++--------
lib/librte_member/rte_member_x86.h | 107 ++++++++++++++++++++++++++++
2 files changed, 218 insertions(+), 31 deletions(-)
create mode 100644 lib/librte_member/rte_member_x86.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
index 08cce02..2b37bda 100644
--- a/lib/librte_member/rte_member_ht.c
+++ b/lib/librte_member/rte_member_ht.c
@@ -40,6 +40,10 @@
#include "rte_member.h"
#include "rte_member_ht.h"
+#if defined(RTE_ARCH_X86)
+#include "rte_member_x86.h"
+#endif
+
/* Search bucket for entry with tmp_sig and update set_id */
static inline int
update_entry_search(uint32_t bucket_id, member_sig_t tmp_sig,
@@ -136,6 +140,13 @@ rte_member_create_ht(struct rte_member_setsum *ss,
for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
}
+#if defined(RTE_ARCH_X86)
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) &&
+ RTE_MEMBER_BUCKET_ENTRIES == 16)
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_AVX2;
+ else
+#endif
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_SCALAR;
RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
"the table has %u entries, %u buckets\n",
@@ -193,11 +204,23 @@ rte_member_lookup_ht(const struct rte_member_setsum *ss,
*set_id = RTE_MEMBER_NO_MATCH;
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- if (search_bucket_single(prim_bucket, tmp_sig, buckets,
- set_id) ||
- search_bucket_single(sec_bucket, tmp_sig,
- buckets, set_id))
- return 1;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single_avx(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ }
return 0;
}
@@ -221,13 +244,27 @@ rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
}
for (i = 0; i < num_keys; i++) {
- if (search_bucket_single(prim_buckets[i], tmp_sig[i],
- buckets, &set_id[i]) ||
- search_bucket_single(sec_buckets[i],
- tmp_sig[i], buckets, &set_id[i]))
- num_matches++;
- else
- set_id[i] = RTE_MEMBER_NO_MATCH;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]) ||
+ search_bucket_single_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ num_matches++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ num_matches++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
}
return num_matches;
}
@@ -244,12 +281,24 @@ rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- search_bucket_multi(prim_bucket, tmp_sig, buckets, &num_matches,
- match_per_key, set_id);
- if (num_matches < match_per_key)
- search_bucket_multi(sec_bucket, tmp_sig,
- buckets, &num_matches, match_per_key, set_id);
- return num_matches;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_bucket, tmp_sig, buckets,
+ &num_matches, match_per_key, set_id);
+ if (num_matches < match_per_key)
+ search_bucket_multi_avx(sec_bucket, tmp_sig,
+ buckets, &num_matches, match_per_key, set_id);
+ return num_matches;
+#endif
+ default:
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &num_matches,
+ match_per_key, set_id);
+ if (num_matches < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &num_matches, match_per_key, set_id);
+ return num_matches;
+ }
}
uint32_t
@@ -275,16 +324,34 @@ rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
for (i = 0; i < num_keys; i++) {
match_cnt_tmp = 0;
- search_bucket_multi(prim_buckets[i], tmp_sig[i],
- buckets, &match_cnt_tmp, match_per_key,
- &set_ids[i*match_per_key]);
- if (match_cnt_tmp < match_per_key)
- search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_buckets[i], tmp_sig[i],
buckets, &match_cnt_tmp, match_per_key,
&set_ids[i*match_per_key]);
- match_count[i] = match_cnt_tmp;
- if (match_cnt_tmp != 0)
- num_matches++;
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &match_cnt_tmp,
+ match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ num_matches++;
+ break;
+#endif
+ default:
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ num_matches++;
+ }
}
return num_matches;
}
@@ -315,11 +382,24 @@ try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
static inline int
try_update(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
- member_sig_t sig, member_set_t set_id)
+ member_sig_t sig, member_set_t set_id,
+ enum rte_member_sig_compare_function cmp_fn)
{
- if (update_entry_search(prim, sig, buckets, set_id) ||
- update_entry_search(sec, sig, buckets, set_id))
- return 0;
+ switch (cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (update_entry_search_avx(prim, sig, buckets, set_id) ||
+ update_entry_search_avx(sec, sig, buckets,
+ set_id))
+ return 0;
+ break;
+#endif
+ default:
+ if (update_entry_search(prim, sig, buckets, set_id) ||
+ update_entry_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ }
return -1;
}
@@ -430,7 +510,7 @@ rte_member_add_ht(const struct rte_member_setsum *ss,
*/
if (ss->cache) {
ret = try_update(buckets, prim_bucket, sec_bucket, tmp_sig,
- set_id);
+ set_id, ss->sig_cmp_fn);
if (ret != -1)
return ret;
}
diff --git a/lib/librte_member/rte_member_x86.h b/lib/librte_member/rte_member_x86.h
new file mode 100644
index 0000000..d29dd3f
--- /dev/null
+++ b/lib/librte_member/rte_member_x86.h
@@ -0,0 +1,107 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_X86_H_
+#define _RTE_MEMBER_X86_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <x86intrin.h>
+
+#if defined(RTE_MACHINE_CPUFLAG_AVX2)
+
+static inline int
+update_entry_search_avx(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket_id].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ if (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ buckets[bucket_id].sets[hit_idx] = set_id;
+ return 1;
+ }
+ return 0;
+}
+
+static inline int
+search_bucket_single_avx(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket_id].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ if (buckets[bucket_id].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket_id].sets[hit_idx];
+ return 1;
+ }
+ hitmask &= ~(3U << ((hit_idx) << 1));
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi_avx(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket_id].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ if (buckets[bucket_id].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket_id].sets[hit_idx];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ hitmask &= ~(3U << ((hit_idx) << 1));
+ }
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_X86_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v5 4/7] member: add AVX for HT mode
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 4/7] member: add AVX for HT mode Yipeng Wang
@ 2017-10-03 9:01 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-03 9:01 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Tuesday, October 3, 2017 5:32 AM
> To: dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v5 4/7] member: add AVX for HT mode
>
> For key search, the signatures of all entries are compared against the
> signature of the key that is being looked up. Since all signatures are
> contiguously put in a bucket, they can be compared with vector instructions
> (AVX2), achieving higher lookup performance.
>
> This patch adds AVX2 implementation in a separate header file.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v5 5/7] member: enable the library
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
` (3 preceding siblings ...)
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 4/7] member: add AVX for HT mode Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 9:04 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 6/7] test/member: add functional and perf tests Yipeng Wang
` (2 subsequent siblings)
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
This patch enables the Membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
MAINTAINERS | 6 +++++-
config/common_base | 5 +++++
lib/librte_member/Makefile | 2 ++
mk/rte.app.mk | 2 ++
4 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index a0cd75e..318d892 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -726,6 +726,11 @@ F: test/test/test_lpm*
F: test/test/test_func_reentrancy.c
F: test/test/test_xmmt_ops.h
+Membership - EXPERIMENTAL
+M: Yipeng Wang <yipeng1.wang@intel.com>
+M: Sameh Gobriel <sameh.gobriel@intel.com>
+F: lib/librte_member/
+
Traffic metering
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
F: lib/librte_meter/
@@ -734,7 +739,6 @@ F: test/test/test_meter.c
F: examples/qos_meter/
F: doc/guides/sample_app_ug/qos_metering.rst
-
Other libraries
---------------
diff --git a/config/common_base b/config/common_base
index 5e97a08..5e31ced 100644
--- a/config/common_base
+++ b/config/common_base
@@ -595,6 +595,11 @@ CONFIG_RTE_LIBRTE_HASH_DEBUG=n
CONFIG_RTE_LIBRTE_EFD=y
#
+# Compile librte_member
+#
+CONFIG_RTE_LIBRTE_MEMBER=y
+
+#
# Compile librte_jobstats
#
CONFIG_RTE_LIBRTE_JOBSTATS=y
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 50275ed..3bac1d0 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -37,6 +37,8 @@ LIB = librte_member.a
CFLAGS := -I$(SRCDIR) $(CFLAGS)
CFLAGS += $(WERROR_FLAGS) -O3
+LDLIBS += -lm
+
EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index c25fdd9..c79acf0 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -86,6 +86,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile
_LDLIBS-y += --whole-archive
_LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost
_LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs
_LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf
@@ -196,6 +197,7 @@ endif
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lm
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrt
_LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lm
ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lnuma
endif
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v5 5/7] member: enable the library
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 5/7] member: enable the library Yipeng Wang
@ 2017-10-03 9:04 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-03 9:04 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Tuesday, October 3, 2017 5:32 AM
> To: dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v5 5/7] member: enable the library
>
> This patch enables the Membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v5 6/7] test/member: add functional and perf tests
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
` (4 preceding siblings ...)
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 5/7] member: enable the library Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 9:07 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 7/7] doc: add membership documentation Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
This patch adds functional and performance tests for membership
library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
---
MAINTAINERS | 1 +
test/test/Makefile | 3 +
test/test/test_member.c | 744 +++++++++++++++++++++++++++++++++++++++++++
test/test/test_member_perf.c | 654 +++++++++++++++++++++++++++++++++++++
4 files changed, 1402 insertions(+)
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 318d892..26b2374 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -730,6 +730,7 @@ Membership - EXPERIMENTAL
M: Yipeng Wang <yipeng1.wang@intel.com>
M: Sameh Gobriel <sameh.gobriel@intel.com>
F: lib/librte_member/
+F: test/test/test_member*
Traffic metering
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
diff --git a/test/test/Makefile b/test/test/Makefile
index 42d9a49..b61dde3 100644
--- a/test/test/Makefile
+++ b/test/test/Makefile
@@ -123,6 +123,9 @@ SRCS-y += test_logs.c
SRCS-y += test_memcpy.c
SRCS-y += test_memcpy_perf.c
+
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd_perf.c
diff --git a/test/test/test_member.c b/test/test/test_member.c
new file mode 100644
index 0000000..02375fd
--- /dev/null
+++ b/test/test/test_member.c
@@ -0,0 +1,744 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* This test is for membership library's simple feature test */
+
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_member.h>
+#include <rte_byteorder.h>
+#include <rte_random.h>
+#include <rte_debug.h>
+#include <rte_ip.h>
+
+#include "test.h"
+
+struct rte_member_setsum *setsum_ht;
+struct rte_member_setsum *setsum_cache;
+struct rte_member_setsum *setsum_vbf;
+
+/* 5-tuple key type */
+struct flow_key {
+ uint32_t ip_src;
+ uint32_t ip_dst;
+ uint16_t port_src;
+ uint16_t port_dst;
+ uint8_t proto;
+} __attribute__((packed));
+
+/* Set ID Macros for multimatch test usage */
+#define M_MATCH_S 1 /* Not start with 0 since by default 0 means no match */
+#define M_MATCH_E 15
+#define M_MATCH_STEP 2
+#define M_MATCH_CNT \
+ (1 + (M_MATCH_E - M_MATCH_S) / M_MATCH_STEP)
+
+
+#define NUM_SAMPLES 5
+#define MAX_MATCH 32
+
+/* Keys used by unit test functions */
+static struct flow_key keys[NUM_SAMPLES] = {
+ {
+ .ip_src = IPv4(0x03, 0x02, 0x01, 0x00),
+ .ip_dst = IPv4(0x07, 0x06, 0x05, 0x04),
+ .port_src = 0x0908,
+ .port_dst = 0x0b0a,
+ .proto = 0x0c,
+ },
+ {
+ .ip_src = IPv4(0x13, 0x12, 0x11, 0x10),
+ .ip_dst = IPv4(0x17, 0x16, 0x15, 0x14),
+ .port_src = 0x1918,
+ .port_dst = 0x1b1a,
+ .proto = 0x1c,
+ },
+ {
+ .ip_src = IPv4(0x23, 0x22, 0x21, 0x20),
+ .ip_dst = IPv4(0x27, 0x26, 0x25, 0x24),
+ .port_src = 0x2928,
+ .port_dst = 0x2b2a,
+ .proto = 0x2c,
+ },
+ {
+ .ip_src = IPv4(0x33, 0x32, 0x31, 0x30),
+ .ip_dst = IPv4(0x37, 0x36, 0x35, 0x34),
+ .port_src = 0x3938,
+ .port_dst = 0x3b3a,
+ .proto = 0x3c,
+ },
+ {
+ .ip_src = IPv4(0x43, 0x42, 0x41, 0x40),
+ .ip_dst = IPv4(0x47, 0x46, 0x45, 0x44),
+ .port_src = 0x4948,
+ .port_dst = 0x4b4a,
+ .proto = 0x4c,
+ }
+};
+
+uint32_t test_set[NUM_SAMPLES] = {1, 2, 3, 4, 5};
+
+#define ITERATIONS 3
+#define KEY_SIZE 4
+
+#define MAX_ENTRIES (1 << 16)
+uint8_t generated_keys[MAX_ENTRIES][KEY_SIZE];
+
+static struct rte_member_parameters params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+
+ /* num_set and false_positive_rate only relevant to vBF */
+ .num_set = 16,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+};
+
+/*
+ * Sequence of operations for find existing setsummary
+ *
+ * - create setsum
+ * - find existing setsum: hit
+ * - find non-existing setsum: miss
+ *
+ */
+static int
+test_member_find_existing(void)
+{
+ struct rte_member_setsum *tmp_setsum = NULL, *result = NULL;
+ struct rte_member_parameters tmp_params = {
+ .name = "member_find_existing",
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ /* Create */
+ tmp_setsum = rte_member_create(&tmp_params);
+ TEST_ASSERT(tmp_setsum != NULL, "setsum creation failed");
+
+ /* Try to find existing hash table */
+ result = rte_member_find_existing("member_find_existing");
+ TEST_ASSERT(result == tmp_setsum, "could not find existing setsum");
+
+ /* Try to find non-existing hash table */
+ result = rte_member_find_existing("member_find_non_existing");
+ TEST_ASSERT(result == NULL, "found setsum that shouldn't exist");
+
+ /* Cleanup. */
+ rte_member_free(tmp_setsum);
+
+ return 0;
+}
+
+/*
+ * Test for bad creating parameters
+ */
+static int
+test_member_create_bad_param(void)
+{
+ struct rte_member_setsum *bad_setsum = NULL;
+ struct rte_member_parameters bad_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ printf("Expected error section begin...\n");
+ bad_params.name = "bad_param1";
+ bad_params.num_set = 0;
+ bad_params.type = RTE_MEMBER_TYPE_VBF;
+ /* Test with 0 set for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "number of set for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param2";
+ bad_params.false_positive_rate = 0;
+ bad_params.num_set = 32;
+ /* Test with 0 false positive for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "false positive rate for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param3";
+ bad_params.false_positive_rate = 0.03;
+ bad_params.num_keys = 0;
+ /* Test with 0 key per BF for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "num_keys for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param4";
+ bad_params.type = RTE_MEMBER_TYPE_HT;
+ bad_params.num_keys = RTE_MEMBER_BUCKET_ENTRIES / 2;
+ /* Test with less than 1 bucket for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with too few "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ bad_params.num_keys = RTE_MEMBER_ENTRIES_MAX + 1;
+ /* Test with more than maximum entries for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with to many "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ /* Test with same name should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with existed "
+ "name\n");
+ return -1;
+ }
+ printf("Expected error section end...\n");
+ rte_member_free(bad_setsum);
+ return 0;
+}
+
+/* Create test setsummaries. */
+static int test_member_create(void)
+{
+ params.key_len = sizeof(struct flow_key);
+
+ params.name = "test_member_ht";
+ params.is_cache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.is_cache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+ params.name = "test_member_vbf";
+ params.type = RTE_MEMBER_TYPE_VBF;
+ setsum_vbf = rte_member_create(¶ms);
+
+ if (setsum_ht == NULL || setsum_cache == NULL || setsum_vbf == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ printf("Creation of setsums success\n");
+ return 0;
+}
+
+static int test_member_insert(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_add(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[i], test_set[i]);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "insert error");
+ }
+ printf("insert key success\n");
+ return 0;
+}
+
+static int test_member_lookup(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ member_set_t set_ids_ht[NUM_SAMPLES] = {0};
+ member_set_t set_ids_cache[NUM_SAMPLES] = {0};
+ member_set_t set_ids_vbf[NUM_SAMPLES] = {0};
+
+ uint32_t num_key_ht = NUM_SAMPLES;
+ uint32_t num_key_cache = NUM_SAMPLES;
+ uint32_t num_key_vbf = NUM_SAMPLES;
+
+ const void *key_array[NUM_SAMPLES];
+
+ /* Single lookup test */
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "single lookup function error");
+
+ TEST_ASSERT(set_ht == test_set[i] &&
+ set_cache == test_set[i] &&
+ set_vbf == test_set[i],
+ "single lookup set value error");
+ }
+ printf("lookup single key success\n");
+
+ /* Bulk lookup test */
+ for (i = 0; i < NUM_SAMPLES; i++)
+ key_array[i] = &keys[i];
+
+ ret_ht = rte_member_lookup_bulk(setsum_ht, key_array,
+ num_key_ht, set_ids_ht);
+
+ ret_cache = rte_member_lookup_bulk(setsum_cache, key_array,
+ num_key_cache, set_ids_cache);
+
+ ret_vbf = rte_member_lookup_bulk(setsum_vbf, key_array,
+ num_key_vbf, set_ids_vbf);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "bulk lookup function error");
+
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ TEST_ASSERT((set_ids_ht[i] == test_set[i]) &&
+ (set_ids_cache[i] == test_set[i]) &&
+ (set_ids_vbf[i] == test_set[i]),
+ "bulk lookup result error");
+ }
+
+ return 0;
+}
+
+static int test_member_delete(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ const void *key_array[NUM_SAMPLES];
+ member_set_t set_ids_ht[NUM_SAMPLES] = {0};
+ member_set_t set_ids_cache[NUM_SAMPLES] = {0};
+ member_set_t set_ids_vbf[NUM_SAMPLES] = {0};
+ uint32_t num_key_ht = NUM_SAMPLES;
+ uint32_t num_key_cache = NUM_SAMPLES;
+ uint32_t num_key_vbf = NUM_SAMPLES;
+
+ /* Delete part of all inserted keys */
+ for (i = 0; i < NUM_SAMPLES / 2; i++) {
+ ret_ht = rte_member_delete(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_delete(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_delete(setsum_vbf, &keys[i], test_set[i]);
+ /* VBF does not support delete yet, so return error code */
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key deletion function error");
+ TEST_ASSERT(ret_vbf < 0,
+ "vbf does not support deletion, error");
+ }
+
+ for (i = 0; i < NUM_SAMPLES; i++)
+ key_array[i] = &keys[i];
+
+ ret_ht = rte_member_lookup_bulk(setsum_ht, key_array,
+ num_key_ht, set_ids_ht);
+
+ ret_cache = rte_member_lookup_bulk(setsum_cache, key_array,
+ num_key_cache, set_ids_cache);
+
+ ret_vbf = rte_member_lookup_bulk(setsum_vbf, key_array,
+ num_key_vbf, set_ids_vbf);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "bulk lookup function error");
+
+ for (i = 0; i < NUM_SAMPLES / 2; i++) {
+ TEST_ASSERT((set_ids_ht[i] == RTE_MEMBER_NO_MATCH) &&
+ (set_ids_cache[i] == RTE_MEMBER_NO_MATCH),
+ "bulk lookup result error");
+ }
+
+ for (i = NUM_SAMPLES / 2; i < NUM_SAMPLES; i++) {
+ TEST_ASSERT((set_ids_ht[i] == test_set[i]) &&
+ (set_ids_cache[i] == test_set[i]) &&
+ (set_ids_vbf[i] == test_set[i]),
+ "bulk lookup result error");
+ }
+
+ /* Delete the left of inserted keys */
+ for (i = NUM_SAMPLES / 2; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_delete(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_delete(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_delete(setsum_vbf, &keys[i], test_set[i]);
+ /* VBF does not support delete yet, so return error code */
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key deletion function error");
+ TEST_ASSERT(ret_vbf < 0,
+ "vbf does not support deletion, error");
+ }
+
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key lookup function error");
+ TEST_ASSERT(set_ht == RTE_MEMBER_NO_MATCH &&
+ ret_cache == RTE_MEMBER_NO_MATCH,
+ "key deletion failed");
+ }
+ /* Reset vbf for other following tests */
+ rte_member_reset(setsum_vbf);
+
+ printf("delete success\n");
+ return 0;
+}
+
+static int test_member_multimatch(void)
+{
+ int ret_ht, ret_vbf, ret_cache;
+ member_set_t set_ids_ht[MAX_MATCH] = {0};
+ member_set_t set_ids_vbf[MAX_MATCH] = {0};
+ member_set_t set_ids_cache[MAX_MATCH] = {0};
+
+ member_set_t set_ids_ht_m[NUM_SAMPLES][MAX_MATCH] = {{0} };
+ member_set_t set_ids_vbf_m[NUM_SAMPLES][MAX_MATCH] = {{0} };
+ member_set_t set_ids_cache_m[NUM_SAMPLES][MAX_MATCH] = {{0} };
+
+ uint32_t match_count_ht[NUM_SAMPLES];
+ uint32_t match_count_vbf[NUM_SAMPLES];
+ uint32_t match_count_cache[NUM_SAMPLES];
+
+ uint32_t num_key_ht = NUM_SAMPLES;
+ uint32_t num_key_vbf = NUM_SAMPLES;
+ uint32_t num_key_cache = NUM_SAMPLES;
+
+ const void *key_array[NUM_SAMPLES];
+
+ uint32_t i, j;
+
+ /* Same key at most inserted 2*entry_per_bucket times for HT mode */
+ for (i = M_MATCH_S; i <= M_MATCH_E; i += M_MATCH_STEP) {
+ for (j = 0; j < NUM_SAMPLES; j++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[j], i);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[j], i);
+ ret_cache = rte_member_add(setsum_cache, &keys[j], i);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_vbf >= 0 &&
+ ret_cache >= 0,
+ "insert function error");
+ }
+ }
+
+ /* Single multimatch test */
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_vbf = rte_member_lookup_multi(setsum_vbf, &keys[i],
+ MAX_MATCH, set_ids_vbf);
+ ret_ht = rte_member_lookup_multi(setsum_ht, &keys[i],
+ MAX_MATCH, set_ids_ht);
+ ret_cache = rte_member_lookup_multi(setsum_cache, &keys[i],
+ MAX_MATCH, set_ids_cache);
+ /*
+ * For cache mode, keys overwrite when signature same.
+ * the mutimatch should work like single match.
+ */
+ TEST_ASSERT(ret_ht == M_MATCH_CNT && ret_vbf == M_MATCH_CNT &&
+ ret_cache == 1,
+ "single lookup_multi error");
+ TEST_ASSERT(set_ids_cache[0] == M_MATCH_E,
+ "single lookup_multi cache error");
+
+ for (j = 1; j <= M_MATCH_CNT; j++) {
+ TEST_ASSERT(set_ids_ht[j-1] == j * M_MATCH_STEP - 1 &&
+ set_ids_vbf[j-1] ==
+ j * M_MATCH_STEP - 1,
+ "single multimatch lookup error");
+ }
+ }
+ printf("lookup single key for multimatch success\n");
+
+ /* Bulk multimatch test */
+ for (i = 0; i < NUM_SAMPLES; i++)
+ key_array[i] = &keys[i];
+ ret_vbf = rte_member_lookup_multi_bulk(setsum_vbf,
+ &key_array[0], num_key_ht, MAX_MATCH, match_count_vbf,
+ (member_set_t *)set_ids_vbf_m);
+
+ ret_ht = rte_member_lookup_multi_bulk(setsum_ht,
+ &key_array[0], num_key_vbf, MAX_MATCH, match_count_ht,
+ (member_set_t *)set_ids_ht_m);
+
+ ret_cache = rte_member_lookup_multi_bulk(setsum_cache,
+ &key_array[0], num_key_cache, MAX_MATCH,
+ match_count_cache, (member_set_t *)set_ids_cache_m);
+
+
+ for (j = 0; j < NUM_SAMPLES; j++) {
+ TEST_ASSERT(match_count_ht[j] == M_MATCH_CNT,
+ "bulk multimatch lookup HT match count error");
+ TEST_ASSERT(match_count_vbf[j] == M_MATCH_CNT,
+ "bulk multimatch lookup vBF match count error");
+ TEST_ASSERT(match_count_cache[j] == 1,
+ "bulk multimatch lookup CACHE match count error");
+ TEST_ASSERT(set_ids_cache_m[j][0] == M_MATCH_E,
+ "bulk multimatch lookup CACHE set value error");
+
+ for (i = 1; i <= M_MATCH_CNT; i++) {
+ TEST_ASSERT(set_ids_ht_m[j][i-1] ==
+ i * M_MATCH_STEP - 1,
+ "bulk multimatch lookup HT set value error");
+ TEST_ASSERT(set_ids_vbf_m[j][i-1] ==
+ i * M_MATCH_STEP - 1,
+ "bulk multimatch lookup vBF set value error");
+ }
+ }
+
+ printf("lookup for bulk multimatch success\n");
+
+ return 0;
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, KEY_SIZE);
+}
+
+static void
+setup_keys_and_data(void)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ /* Reset all arrays */
+ for (i = 0; i < KEY_SIZE; i++)
+ generated_keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < MAX_ENTRIES; i++) {
+ for (j = 0; j < KEY_SIZE; j++)
+ generated_keys[i][j] = rte_rand() & 0xFF;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(generated_keys, MAX_ENTRIES, KEY_SIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < MAX_ENTRIES - 1; i++) {
+ if (memcmp(generated_keys[i], generated_keys[i + 1],
+ KEY_SIZE) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < KEY_SIZE; j++)
+ generated_keys[i][j] =
+ rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+}
+
+static inline int
+add_generated_keys(struct rte_member_setsum *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret >= 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &generated_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static inline int
+add_generated_keys_cache(struct rte_member_setsum *setsum,
+ unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret == 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &generated_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static int
+test_member_loadfactor(void)
+{
+ unsigned int j;
+ unsigned int added_keys, average_keys_added = 0;
+ int ret;
+
+ setup_keys_and_data();
+
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+
+ params.key_len = KEY_SIZE;
+ params.name = "test_member_ht";
+ params.is_cache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.is_cache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+
+ if (setsum_ht == NULL || setsum_cache == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ /* Test HT non-cache mode */
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_generated_keys(setsum_ht, &added_keys);
+ if (ret != -ENOSPC) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_ht);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when no space(non-cache) = %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+
+ /* Test cache mode */
+ added_keys = average_keys_added = 0;
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_generated_keys_cache(setsum_cache, &added_keys);
+ if (ret != 1) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_cache);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when eviction happens(cache)= %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+ return 0;
+}
+
+static void
+perform_free(void)
+{
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+}
+
+static int
+test_member(void)
+{
+ if (test_member_create_bad_param() < 0)
+ return -1;
+
+ if (test_member_find_existing() < 0)
+ return -1;
+
+ if (test_member_create() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_insert() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_lookup() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_delete() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_multimatch() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_loadfactor() < 0) {
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ return -1;
+ }
+
+ perform_free();
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_autotest, test_member);
diff --git a/test/test/test_member_perf.c b/test/test/test_member_perf.c
new file mode 100644
index 0000000..e13066f
--- /dev/null
+++ b/test/test/test_member_perf.c
@@ -0,0 +1,654 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <inttypes.h>
+
+#include <rte_lcore.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+#include <rte_memcpy.h>
+#include <rte_thash.h>
+#include <rte_member.h>
+
+#include "test.h"
+
+#define NUM_KEYSIZES 10
+#define NUM_SHUFFLES 10
+#define MAX_KEYSIZE 64
+#define MAX_ENTRIES (1 << 19)
+#define KEYS_TO_ADD (MAX_ENTRIES * 75 / 100) /* 75% table utilization */
+#define NUM_LOOKUPS (KEYS_TO_ADD * 5) /* Loop among keys added, several times */
+#define VBF_SET_CNT 16
+#define BURST_SIZE 64
+#define VBF_FALSE_RATE 0.03
+
+static unsigned int test_socket_id;
+
+enum sstype {
+ HT = 0,
+ CACHE,
+ VBF,
+ NUM_TYPE
+};
+
+enum operations {
+ ADD = 0,
+ LOOKUP,
+ LOOKUP_BULK,
+ LOOKUP_MULTI,
+ LOOKUP_MULTI_BULK,
+ DELETE,
+ LOOKUP_MISS,
+ NUM_OPERATIONS
+};
+
+struct member_perf_params {
+ struct rte_member_setsum *setsum[NUM_TYPE];
+ uint32_t key_size;
+ unsigned int cycle;
+};
+
+static uint32_t hashtest_key_lens[] = {
+ /* standard key sizes */
+ 4, 8, 16, 32, 48, 64,
+ /* IPv4 SRC + DST + protocol, unpadded */
+ 9,
+ /* IPv4 5-tuple, unpadded */
+ 13,
+ /* IPv6 5-tuple, unpadded */
+ 37,
+ /* IPv6 5-tuple, padded to 8-byte boundary */
+ 40
+};
+
+/* Array to store number of cycles per operation */
+uint64_t cycles[NUM_TYPE][NUM_KEYSIZES][NUM_OPERATIONS];
+uint64_t false_data[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_bulk[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi_bulk[NUM_TYPE][NUM_KEYSIZES];
+
+uint64_t false_hit[NUM_TYPE][NUM_KEYSIZES];
+
+member_set_t data[NUM_TYPE][/* Array to store the data */KEYS_TO_ADD];
+
+/* Array to store all input keys */
+uint8_t keys[KEYS_TO_ADD][MAX_KEYSIZE];
+
+/* Shuffle the keys that have been added, so lookups will be totally random */
+static void
+shuffle_input_keys(struct member_perf_params *params)
+{
+ member_set_t temp_data;
+ unsigned int i, j;
+ uint32_t swap_idx;
+ uint8_t temp_key[MAX_KEYSIZE];
+
+ for (i = KEYS_TO_ADD - 1; i > 0; i--) {
+ swap_idx = rte_rand() % i;
+ memcpy(temp_key, keys[i], hashtest_key_lens[params->cycle]);
+ memcpy(keys[i], keys[swap_idx],
+ hashtest_key_lens[params->cycle]);
+ memcpy(keys[swap_idx], temp_key,
+ hashtest_key_lens[params->cycle]);
+ for (j = 0; j < NUM_TYPE; j++) {
+ temp_data = data[j][i];
+ data[j][i] = data[j][swap_idx];
+ data[j][swap_idx] = temp_data;
+ }
+ }
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, MAX_KEYSIZE);
+}
+
+struct rte_member_parameters member_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = 4, /* Length of hash key. */
+
+ /* num_set and false_positive_rate only relevant to vBF */
+ .num_set = VBF_SET_CNT,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 0,
+ .sec_hash_seed = 1,
+ .socket_id = 0, /* NUMA Socket ID for memory. */
+ };
+
+static int
+setup_keys_and_data(struct member_perf_params *params, unsigned int cycle,
+ int miss)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ params->key_size = hashtest_key_lens[cycle];
+ params->cycle = cycle;
+
+ /* Reset all arrays */
+ for (i = 0; i < params->key_size; i++)
+ keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+
+ data[HT][i] = data[CACHE][i] = (rte_rand() & 0x7FFE) + 1;
+ data[VBF][i] = rte_rand() % VBF_SET_CNT + 1;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(keys, KEYS_TO_ADD, MAX_KEYSIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < KEYS_TO_ADD - 1; i++) {
+ if (memcmp(keys[i], keys[i + 1],
+ params->key_size) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+
+ /* Shuffle the random values again */
+ shuffle_input_keys(params);
+
+ /* For testing miss lookup, we insert half and lookup the other half */
+ unsigned int entry_cnt, bf_key_cnt;
+ if (!miss) {
+ entry_cnt = MAX_ENTRIES;
+ bf_key_cnt = KEYS_TO_ADD;
+ } else {
+ entry_cnt = MAX_ENTRIES / 2;
+ bf_key_cnt = KEYS_TO_ADD / 2;
+ }
+ member_params.false_positive_rate = VBF_FALSE_RATE;
+ member_params.key_len = params->key_size;
+ member_params.socket_id = test_socket_id;
+ member_params.num_keys = entry_cnt;
+ member_params.name = "test_member_ht";
+ member_params.is_cache = 0;
+ member_params.type = RTE_MEMBER_TYPE_HT;
+ params->setsum[HT] = rte_member_create(&member_params);
+ if (params->setsum[HT] == NULL)
+ fprintf(stderr, "ht create fail\n");
+
+ member_params.name = "test_member_cache";
+ member_params.is_cache = 1;
+ params->setsum[CACHE] = rte_member_create(&member_params);
+ if (params->setsum[CACHE] == NULL)
+ fprintf(stderr, "CACHE create fail\n");
+
+ member_params.name = "test_member_vbf";
+ member_params.type = RTE_MEMBER_TYPE_VBF;
+ member_params.num_keys = bf_key_cnt;
+ params->setsum[VBF] = rte_member_create(&member_params);
+ if (params->setsum[VBF] == NULL)
+ fprintf(stderr, "VBF create fail\n");
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] == NULL)
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+timed_adds(struct member_perf_params *params, int type)
+{
+ const uint64_t start_tsc = rte_rdtsc();
+ unsigned int i, a;
+ int32_t ret;
+
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][ADD] = time_taken / KEYS_TO_ADD;
+ return 0;
+}
+
+static int
+timed_lookups(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+
+ false_data[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+ member_set_t result;
+ int ret;
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (type == HT && result == RTE_MEMBER_NO_MATCH) {
+ printf("HT mode shouldn't have false negative");
+ return -1;
+ }
+ if (result != data[type][j])
+ false_data[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ member_set_t result[BURST_SIZE] = {0};
+ const void *keys_burst[BURST_SIZE];
+ int ret;
+
+ false_data_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_bulk(params->setsum[type],
+ keys_burst,
+ BURST_SIZE,
+ result);
+ if (ret <= 0) {
+ printf("lookup bulk has wrong return value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (type == HT && result[k] ==
+ RTE_MEMBER_NO_MATCH) {
+ printf("HT mode shouldn't have "
+ "false negative");
+ return -1;
+ }
+ if (result[k] != data[type][data_idx])
+ false_data_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_BULK] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_multimatch(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ member_set_t result[RTE_MEMBER_BUCKET_ENTRIES] = {0};
+ int ret;
+ false_data_multi[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup_multi(params->setsum[type],
+ &keys[j], RTE_MEMBER_BUCKET_ENTRIES, result);
+ if (type != CACHE && ret <= 0) {
+ printf("lookup multi has wrong return value %d,"
+ "type %d\n", ret, type);
+ }
+ if (type == HT && ret == 0) {
+ printf("HT mode shouldn't have false negative");
+ return -1;
+ }
+ /*
+ * For performance test purpose, we do not iterate all
+ * results here. We assume most likely each key can only
+ * find one match which is result[0].
+ */
+ if (result[0] != data[type][j])
+ false_data_multi[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_multimatch_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ member_set_t result[BURST_SIZE][RTE_MEMBER_BUCKET_ENTRIES] = {{0} };
+ const void *keys_burst[BURST_SIZE];
+ uint32_t match_count[BURST_SIZE];
+ int ret;
+
+ false_data_multi_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_multi_bulk(
+ params->setsum[type],
+ keys_burst, BURST_SIZE,
+ RTE_MEMBER_BUCKET_ENTRIES, match_count,
+ (member_set_t *)result);
+ if (ret < 0) {
+ printf("lookup multimatch bulk has wrong return"
+ " value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ if (type != CACHE && match_count[k] == 0) {
+ printf("lookup multimatch bulk get "
+ "wrong match count\n");
+ return -1;
+ }
+ if (type == HT && match_count[k] == 0) {
+ printf("HT mode shouldn't have "
+ "false negative");
+ return -1;
+ }
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k][0] != data[type][data_idx])
+ false_data_multi_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI_BULK] = time_taken /
+ NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_deletes(struct member_perf_params *params, int type)
+{
+ unsigned int i;
+ int32_t ret;
+
+ if (type == VBF)
+ return 0;
+ const uint64_t start_tsc = rte_rdtsc();
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_delete(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (type != CACHE && ret < 0) {
+ printf("delete error\n");
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][DELETE] = time_taken / KEYS_TO_ADD;
+
+ return 0;
+}
+
+static int
+timed_miss_lookup(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ int ret;
+
+ false_hit[type][params->cycle] = 0;
+
+ for (i = 0; i < KEYS_TO_ADD / 2; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ unsigned int a;
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t start_tsc = rte_rdtsc();
+ member_set_t result;
+
+ for (i = 0; i < 2 * NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = KEYS_TO_ADD / 2; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != RTE_MEMBER_NO_MATCH)
+ false_hit[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MISS] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static void
+perform_frees(struct member_perf_params *params)
+{
+ int i;
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] != NULL) {
+ rte_member_free(params->setsum[i]);
+ params->setsum[i] = NULL;
+ }
+ }
+}
+
+static int
+exit_with_fail(const char *testname, struct member_perf_params *params,
+ unsigned int i, unsigned int j)
+{
+ printf("<<<<<Test %s failed at keysize %d iteration %d type %d>>>>>\n",
+ testname, hashtest_key_lens[params->cycle], i, j);
+ perform_frees(params);
+ return -1;
+}
+
+static int
+run_all_tbl_perf_tests(void)
+{
+ unsigned int i, j, k;
+ struct member_perf_params params;
+
+ printf("Measuring performance, please wait\n");
+ fflush(stdout);
+
+ test_socket_id = rte_socket_id();
+
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 0) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+
+ if (timed_adds(¶ms, j) < 0)
+ return exit_with_fail("timed_adds", ¶ms,
+ i, j);
+
+ for (k = 0; k < NUM_SHUFFLES; k++)
+ shuffle_input_keys(¶ms);
+
+ if (timed_lookups(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups", ¶ms,
+ i, j);
+
+ if (timed_lookups_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_bulk",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi_bulk",
+ ¶ms, i, j);
+
+ if (timed_deletes(¶ms, j) < 0)
+ return exit_with_fail("timed_deletes", ¶ms,
+ i, j);
+
+ /* Print a dot to show progress on operations */
+ }
+ printf(".");
+ fflush(stdout);
+
+ perform_frees(¶ms);
+ }
+
+ /* Test false positive rate using un-inserted keys */
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 1) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+ if (timed_miss_lookup(¶ms, j) < 0)
+ return exit_with_fail("timed_miss_lookup",
+ ¶ms, i, j);
+ }
+ perform_frees(¶ms);
+ }
+
+ printf("\nResults (in CPU cycles/operation)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "Add", "Lookup", "Lookup_bulk",
+ "lookup_multi", "lookup_multi_bulk", "Delete",
+ "miss_lookup");
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ for (k = 0; k < NUM_OPERATIONS; k++)
+ printf("%-18"PRIu64, cycles[j][i][k]);
+ printf("\n");
+ }
+ }
+
+ printf("\nFalse results rate (and false positive rate)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "fr_single", "fr_bulk", "fr_multi",
+ "fr_multi_bulk", "false_positive_rate");
+ /* Key size not influence False rate so just print out one key size */
+ for (i = 0; i < 1; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ printf("%-18f", (float)false_data[j][i] / NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_hit[j][i] /
+ NUM_LOOKUPS);
+ printf("\n");
+ }
+ }
+ return 0;
+}
+
+static int
+test_member_perf(void)
+{
+
+ if (run_all_tbl_perf_tests() < 0)
+ return -1;
+
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v5 6/7] test/member: add functional and perf tests
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 6/7] test/member: add functional and perf tests Yipeng Wang
@ 2017-10-03 9:07 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-03 9:07 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Tuesday, October 3, 2017 5:32 AM
> To: dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v5 6/7] test/member: add functional and perf tests
>
> This patch adds functional and performance tests for membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v5 7/7] doc: add membership documentation
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
` (5 preceding siblings ...)
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 6/7] test/member: add functional and perf tests Yipeng Wang
@ 2017-10-03 4:31 ` Yipeng Wang
2017-10-03 9:08 ` De Lara Guarch, Pablo
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-03 4:31 UTC (permalink / raw)
To: dev, pablo.de.lara.guarch
Cc: thomas, charlie.tai, sameh.gobriel, john.mcnamara, Yipeng Wang
This patch adds the documentation for membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: John McNamara <john.mcnamara@intel.com>
---
MAINTAINERS | 1 +
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 +++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 ++++
doc/guides/prog_guide/img/member_i6.svg | 332 +++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 420 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
13 files changed, 3596 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 26b2374..2d57d3b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -731,6 +731,7 @@ M: Yipeng Wang <yipeng1.wang@intel.com>
M: Sameh Gobriel <sameh.gobriel@intel.com>
F: lib/librte_member/
F: test/test/test_member*
+F: doc/guides/prog_guide/member_lib.rst
Traffic metering
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 19e0d4f..fe87e09 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -105,7 +105,8 @@ The public API headers are grouped by topics:
[LPM IPv4 route] (@ref rte_lpm.h),
[LPM IPv6 route] (@ref rte_lpm6.h),
[ACL] (@ref rte_acl.h),
- [EFD] (@ref rte_efd.h)
+ [EFD] (@ref rte_efd.h),
+ [member] (@ref rte_member.h)
- **QoS**:
[metering] (@ref rte_meter.h),
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index 823554f..b792d6d 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -58,6 +58,7 @@ INPUT = doc/api/doxy-api-index.md \
lib/librte_mempool \
lib/librte_meter \
lib/librte_metrics \
+ lib/librte_member \
lib/librte_net \
lib/librte_pdump \
lib/librte_pipeline \
diff --git a/doc/guides/prog_guide/img/member_i1.svg b/doc/guides/prog_guide/img/member_i1.svg
new file mode 100644
index 0000000..fc5f56a
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i1.svg
@@ -0,0 +1,1613 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i1.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.18709in" height="4.75757in"
+ viewBox="0 0 517.471 342.545" xml:space="preserve" color-interpolation-filters="sRGB" class="st61">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false">
+ <v:userDefs>
+ <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/>
+ <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/>
+ </v:userDefs>
+ </v:documentProperties>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:3}
+ .st3 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em;opacity:0.219608}
+ .st4 {font-size:1em}
+ .st5 {fill:none;stroke:#41719c;stroke-width:3}
+ .st6 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em}
+ .st7 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em;opacity:0.219608}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st9 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st11 {fill:none;stroke:none;stroke-width:0.25}
+ .st12 {fill:#ffffff;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:0.75em;font-weight:bold}
+ .st14 {marker-end:url(#mrkr5-63);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st15 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st16 {fill:#5b9bd5;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st17 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st18 {fill:#feffff;font-family:Calibri;font-size:0.499992em}
+ .st19 {fill:#deebf6;stroke:#c8c8c8;stroke-width:0.25}
+ .st20 {fill:#000000;font-family:Calibri;font-size:0.499992em}
+ .st21 {marker-end:url(#mrkr5-178);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:#ff0000;font-family:Calibri;font-size:0.666664em}
+ .st24 {fill:#5b9bd5;fill-opacity:0.22}
+ .st25 {stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st26 {fill:#ffffff}
+ .st27 {stroke:#0070c0;stroke-width:0.25}
+ .st28 {fill:#5b9bd5;stroke:#0070c0;stroke-width:0.25}
+ .st29 {fill:#5b9bd5;stroke:#ffffff;stroke-width:0.25}
+ .st30 {fill:#5b9bd5}
+ .st31 {stroke:#c8c8c8;stroke-width:0.25}
+ .st32 {fill:#acccea;stroke:#c8c8c8;stroke-width:0.25}
+ .st33 {fill:#5b9bd5;fill-opacity:0.22;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st34 {fill:#000000;fill-opacity:0;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st35 {fill:url(#grad30-309);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st36 {fill:url(#grad25-313);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st37 {fill:url(#grad35-317);stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st38 {fill:url(#grad36-325);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st39 {fill:url(#grad40-335);stroke:#000000;stroke-linecap:butt;stroke-width:0.130208}
+ .st40 {fill:url(#grad39-342);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st41 {fill:url(#grad40-355);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st42 {fill:none}
+ .st43 {stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st44 {stroke:#ffffff;stroke-linecap:butt;stroke-width:0.130208}
+ .st45 {fill:url(#grad30-383);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st46 {fill:url(#grad36-396);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st47 {fill:none;stroke:#c8c8c8;stroke-width:0.75}
+ .st48 {fill:#9a9a9a;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st49 {fill:url(#grad40-415);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st50 {fill:url(#grad40-419);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st51 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25}
+ .st52 {fill:url(#grad35-430);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st53 {stroke:#c8c8c8;stroke-width:0.75}
+ .st54 {stroke:#4f88bb;stroke-width:0.75}
+ .st55 {fill:#feffff;font-family:Calibri;font-size:0.416656em}
+ .st56 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st57 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st58 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:2.25}
+ .st59 {fill:none;stroke:#0070c0;stroke-width:2.25}
+ .st60 {fill:#595959;font-family:Arial;font-size:0.666664em}
+ .st61 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Patterns_And_Gradients">
+ <linearGradient id="grad30-309" v:fillPattern="30" v:foreground="#97c2e6" v:background="#4274a2" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#4274a2;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-313" v:fillPattern="25" v:foreground="#5491d3" v:background="#246ba6" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#5491d3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </linearGradient>
+ <pattern id="grad35-317" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-318)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-319)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-320)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-321)"/>
+ </pattern>
+ <linearGradient id="grad27-318" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-319" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-320" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-321" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-325" v:fillPattern="36" v:foreground="#c0dff1" v:background="#246ba6" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#c0dff1;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-335" v:fillPattern="40" v:foreground="#c8e5c8" v:background="#19bf19" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#c8e5c8;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#19bf19;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad39-342" v:fillPattern="39" v:foreground="#5599d7" v:background="#b9daf2" cx="1" cy="1" r="1">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-355" v:fillPattern="40" v:foreground="#5599d7" v:background="#214383" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#214383;stop-opacity:1"/>
+ </radialGradient>
+ <linearGradient id="grad30-383" v:fillPattern="30" v:foreground="#97c2e6" v:background="#6ba4dc" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#6ba4dc;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-396" v:fillPattern="36" v:foreground="#89bee9" v:background="#b9daf2" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#89bee9;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-415" v:fillPattern="40" v:foreground="#000000" v:background="#ffffff" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#000000;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffffff;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-419" v:fillPattern="40" v:foreground="#ffffff" v:background="#9a9a9a" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#9a9a9a;stop-opacity:1"/>
+ </radialGradient>
+ <pattern id="grad35-430" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-431)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-432)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-433)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-434)"/>
+ </pattern>
+ <linearGradient id="grad27-431" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-432" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-433" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-434" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ </defs>
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-63" class="st15" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-178" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/>
+ </v:userDefs>
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <v:layer v:name="Flowchart" v:index="0"/>
+ <g id="group165-1" transform="translate(21.7794,-24.0978)" v:mID="165" v:groupContext="group">
+ <title>Sheet.165</title>
+ <g id="group1-2" transform="translate(308.647,-25.7109)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-3" v:mID="2" v:groupContext="shape" transform="translate(11.5732,-58.1913)">
+ <title>Circle</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow2-4" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="shape3-13" v:mID="3" v:groupContext="shape" transform="translate(58.9839,-58.9839)">
+ <title>Circle.23</title>
+ <desc>List 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow3-14" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="17.73" y="318.02" class="st7" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="17.73" y="318.02" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <g id="shape4-19" v:mID="4" v:groupContext="shape">
+ <title>Circle.24</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow4-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="group5-29" transform="translate(50.7413,-4.53722)" v:mID="5" v:groupContext="group">
+ <title>Sheet.5</title>
+ <g id="shape6-30" v:mID="6" v:groupContext="shape" transform="translate(344.2,300.5) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-31" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st9"/>
+ </g>
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape7-34" v:mID="7" v:groupContext="shape" transform="translate(-0.884982,-14.7157)">
+ <title>Sheet.7</title>
+ <desc>setsum</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="12.9268" cy="336.238" width="25.86" height="12.6135"/>
+ <rect x="0" y="329.932" width="25.8535" height="12.6135" class="st11"/>
+ <text x="6.37" y="334.44" class="st12" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsu<tspan
+ x="10.49" dy="1.2em" class="st4">m</tspan></text> </g>
+ </g>
+ <g id="shape8-38" v:mID="8" v:groupContext="shape" transform="translate(72.5955,0)">
+ <title>Circle.29</title>
+ <desc>List 2 matching Criteria 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow8-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ </g>
+ <g id="group9-48" transform="translate(31.6515,-49.9094)" v:mID="9" v:groupContext="group">
+ <title>Sheet.9</title>
+ <g id="group10-49" transform="translate(99.5691,0)" v:mID="10" v:groupContext="group">
+ <title>Sheet.10</title>
+ <g id="shape11-50" v:mID="11" v:groupContext="shape" transform="translate(346.175,275.999) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st9"/>
+ </g>
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape12-54" v:mID="12" v:groupContext="shape" transform="translate(355.063,285.074) rotate(90)">
+ <title>Sheet.12</title>
+ <desc>Set Summary</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1985" cy="332.563" width="48.4" height="19.9638"/>
+ <rect x="0" y="322.581" width="48.397" height="19.9638" class="st11"/>
+ <text x="18.25" y="329.86" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Set <tspan
+ x="6.38" dy="1.2em" class="st4">Summary</tspan></text> </g>
+ </g>
+ <g id="shape13-58" v:mID="13" v:groupContext="shape" transform="translate(57.5835,-54.4467)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.9 342.55" class="st14"/>
+ </g>
+ <g id="shape14-64" v:mID="14" v:groupContext="shape" transform="translate(20.2363,-51.8439)">
+ <title>Sheet.14</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="25.3328" cy="333.471" width="50.67" height="18.1489"/>
+ <rect x="0" y="324.396" width="50.6656" height="18.1489" class="st11"/>
+ <text x="14.12" y="335.27" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape15-67" v:mID="15" v:groupContext="shape" transform="translate(5.02911,1.60865) rotate(-26.0815)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L39.25 342.55" class="st14"/>
+ </g>
+ <g id="shape16-72" v:mID="16" v:groupContext="shape" transform="translate(155.629,-33.273)">
+ <title>Sheet.16</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.34 342.55" class="st14"/>
+ </g>
+ <g id="shape17-77" v:mID="17" v:groupContext="shape" transform="translate(304.141,0.595416) rotate(25.6934)">
+ <title>Sheet.17</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L42.68 342.55" class="st14"/>
+ </g>
+ <g id="shape18-82" v:mID="18" v:groupContext="shape" transform="translate(102.642,654.842) rotate(180)">
+ <title>Sheet.18</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape19-87" v:mID="19" v:groupContext="shape" transform="translate(-15.1809,-33.9928)">
+ <title>Sheet.19</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.75" cy="338.045" width="85.5" height="9"/>
+ <rect x="0" y="333.545" width="85.5" height="9" class="st11"/>
+ <text x="5.06" y="339.85" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape20-90" v:mID="20" v:groupContext="shape" transform="translate(102.844,679.041) rotate(180)">
+ <title>Sheet.20</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape21-95" v:mID="21" v:groupContext="shape" transform="translate(-35.4309,-11.4928)">
+ <title>Sheet.21</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="337.971" width="108" height="9.14889"/>
+ <rect x="0" y="333.396" width="108" height="9.14889" class="st11"/>
+ <text x="6.36" y="339.77" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape22-98" v:mID="22" v:groupContext="shape" transform="translate(541.496,275.999) rotate(90)">
+ <title>Sheet.22</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape23-101" v:mID="23" v:groupContext="shape" transform="translate(541.496,300.198) rotate(90)">
+ <title>Sheet.23</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape24-104" v:mID="24" v:groupContext="shape" transform="translate(541.496,324.396) rotate(90)">
+ <title>Sheet.24</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ </g>
+ <g id="group25-107" transform="translate(285.961,-178.628)" v:mID="25" v:groupContext="group">
+ <title>Sheet.25</title>
+ <g id="shape26-108" v:mID="26" v:groupContext="shape" transform="translate(51.2583,-51.2583)">
+ <title>Circle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-109" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape27-112" v:mID="27" v:groupContext="shape" transform="translate(107.177,-55.9182)">
+ <title>Circle.156</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-113" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape28-116" v:mID="28" v:groupContext="shape" transform="translate(79.2174,-83.8773)">
+ <title>Circle.157</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow28-117" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape29-120" v:mID="29" v:groupContext="shape" transform="translate(153.775,-51.2583)">
+ <title>Circle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow29-121" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape30-124" v:mID="30" v:groupContext="shape" transform="translate(93.197,-18.6394)">
+ <title>Circle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow30-125" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape31-128" v:mID="31" v:groupContext="shape" transform="translate(27.4102,-57.9329) rotate(-7.12502)">
+ <title>Sheet.31</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L31.41 342.55" class="st14"/>
+ </g>
+ <g id="shape32-133" v:mID="32" v:groupContext="shape" transform="translate(182.13,-60.5772) rotate(9.46232)">
+ <title>Sheet.32</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L22.18 342.55" class="st14"/>
+ </g>
+ <g id="shape33-138" v:mID="33" v:groupContext="shape" transform="translate(47.8843,595.237) rotate(-160.346)">
+ <title>Sheet.33</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L63.11 342.55" class="st14"/>
+ </g>
+ <g id="shape34-143" v:mID="34" v:groupContext="shape" transform="translate(292.945,525.785) rotate(141.977)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L20.97 342.55" class="st14"/>
+ </g>
+ <g id="shape35-148" v:mID="35" v:groupContext="shape" transform="translate(-95.8971,591.793) rotate(-145.945)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L28.55 342.55" class="st14"/>
+ </g>
+ <g id="shape36-153" v:mID="36" v:groupContext="shape" transform="translate(37.2788,2.27374E-013)">
+ <title>Rectangle.167</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.8652" cy="335.555" width="21.74" height="13.9795"/>
+ <g id="shadow36-154" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st10"/>
+ <text x="5" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape37-158" v:mID="37" v:groupContext="shape" transform="translate(55.9182,2.27374E-013)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow37-159" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape38-163" v:mID="38" v:groupContext="shape" transform="translate(-1.65867E-013,-32.6189)">
+ <title>Rectangle.169</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.3796" cy="335.555" width="20.76" height="13.9795"/>
+ <g id="shadow38-164" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st10"/>
+ <text x="4.51" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape39-168" v:mID="39" v:groupContext="shape" transform="translate(18.6394,-32.6189)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow39-169" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape40-173" v:mID="40" v:groupContext="shape" transform="translate(197.019,626.053) rotate(161.565)">
+ <title>Sheet.40</title>
+ <path d="M0 328.31 A55.7483 27.2427 -124.2 0 0 42.37 334.19 L42.47 333.85" class="st21"/>
+ </g>
+ <g id="shape41-179" v:mID="41" v:groupContext="shape" transform="translate(154.607,584.177) rotate(161.121)">
+ <title>Sheet.41</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 319.39 A80.5593 29.9756 -101.99 0 0 41.7 325.37 L41.79 325.02" class="st21"/>
+ </g>
+ <g id="shape42-184" v:mID="42" v:groupContext="shape" transform="translate(3.02481,-66.7025)">
+ <title>Sheet.42</title>
+ <desc>Encode ID</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="19.4569" cy="335.555" width="38.92" height="13.9795"/>
+ <rect x="0" y="328.566" width="38.9138" height="13.9795" class="st11"/>
+ <text x="7.51" y="333.16" class="st23" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Encode <tspan
+ x="15.99" dy="1.2em" class="st4">ID</tspan></text> </g>
+ </g>
+ <g id="group43-188" transform="translate(12.0993,-165.858)" v:mID="43" v:groupContext="group">
+ <title>Sheet.43</title>
+ <g id="group44-189" transform="translate(7.21495,-75.757)" v:mID="44" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User</title>
+ <g id="shape45-190" v:mID="45" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.45</title>
+ <g id="shadow45-191" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape46-206" v:mID="46" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.46</title>
+ <g id="shadow46-207" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape47-210" v:mID="47" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.47</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape48-212" v:mID="48" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.48</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group49-215" transform="translate(7.21495,-47.1858)" v:mID="49" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.7</title>
+ <g id="shape50-216" v:mID="50" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.50</title>
+ <g id="shadow50-217" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape51-232" v:mID="51" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.51</title>
+ <g id="shadow51-233" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape52-236" v:mID="52" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.52</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape53-238" v:mID="53" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group54-241" transform="translate(7.21495,-18.6146)" v:mID="54" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.12</title>
+ <g id="shape55-242" v:mID="55" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.55</title>
+ <g id="shadow55-243" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape56-258" v:mID="56" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.56</title>
+ <g id="shadow56-259" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape57-262" v:mID="57" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape58-264" v:mID="58" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group59-267" transform="translate(171.161,-45.6707)" v:mID="59" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>Data Center</title>
+ <g id="shape60-268" v:mID="60" v:groupContext="shape" v:layerMember="0">
+ <title>Sheet.60</title>
+ <g id="shadow60-269" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st9"/>
+ </g>
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st10"/>
+ </g>
+ <g id="shape61-272" v:mID="61" v:groupContext="shape" v:layerMember="0" transform="translate(6.86487,-7.30475)">
+ <title>Sheet.61</title>
+ <g id="shadow61-273" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39
+ 332.44 L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69
+ L29.57 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64
+ 342.55 L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75
+ 342.55 L18.75 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69
+ L10.82 311.79 L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st9"/>
+ </g>
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39 332.44
+ L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69 L29.57
+ 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64 342.55
+ L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75 342.55 L18.75
+ 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69 L10.82 311.79
+ L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st10"/>
+ </g>
+ <g id="shape62-276" v:mID="62" v:groupContext="shape" v:layerMember="0" transform="translate(20.0835,-20.5174)">
+ <title>Sheet.62</title>
+ <g id="shadow62-277" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36
+ ZM23.46 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36
+ ZM2.27 341.36 A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z"
+ class="st24"/>
+ </g>
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36 ZM23.46
+ 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36 ZM2.27 341.36
+ A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z" class="st30"/>
+ </g>
+ <g id="shape63-282" v:mID="63" v:groupContext="shape" v:layerMember="0" transform="translate(14.2717,-12.5134)">
+ <title>Sheet.63</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <g id="shadow63-283" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17
+ 340.12 L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89
+ L43.09 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2
+ 342.55 ZM21.2 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69
+ L29.27 337.69 L29.27 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74
+ L-0 341.74 L-0 342.55 ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69
+ L8.08 337.69 L8.08 336.89 L-0 336.89 L-0 337.69 Z" class="st24"/>
+ </g>
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17 340.12
+ L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89 L43.09
+ 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2 342.55 ZM21.2
+ 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69 L29.27 337.69 L29.27
+ 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74 L-0 341.74 L-0 342.55
+ ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69 L8.08 337.69 L8.08 336.89
+ L-0 336.89 L-0 337.69 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group64-288" transform="translate(59.5234,-47.1858)" v:mID="64" v:groupContext="group">
+ <v:custProps>
+ <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false"
+ v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/>
+ <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true"
+ v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Device)"/>
+ <v:cp v:nameU="SubShapeType" v:lbl="SubShapeType" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Load balancer)"/>
+ </v:custProps>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/>
+ <v:ud v:nameU="SolSH" v:prompt="" v:val="VT14({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/>
+ <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <title>Load balancer</title>
+ <g id="shape65-289" v:mID="65" v:groupContext="shape" transform="translate(0,-1.653)">
+ <title>Sheet.65</title>
+ <g id="shadow65-290" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st24"/>
+ <path d="M0 332.02 L16.23 332.02" class="st25"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st25"/>
+ </g>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st30"/>
+ <path d="M0 332.02 L16.23 332.02" class="st31"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st31"/>
+ </g>
+ <g id="shape66-297" v:mID="66" v:groupContext="shape" transform="translate(1.81062,-2.91583)">
+ <title>Sheet.66</title>
+ <g id="shadow66-298" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44
+ L8.34 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45
+ 338.78 L10.78 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22
+ ZM10.48 335.2 L8.29 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46
+ 334.9 L10.21 334.58 L9.55 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19
+ 338.43 C4.19 339.56 5.11 340.48 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29
+ 7.38 336.37 6.25 336.37 ZM6.25 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02
+ 339.83 6.25 339.83 C5.47 339.83 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02
+ ZM2.62 338.14 L0 338.14 L0 338.71 L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73
+ 337.47 L1.95 337.47 L2.62 338.14 Z" class="st9"/>
+ </g>
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44 L8.34
+ 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45 338.78 L10.78
+ 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22 ZM10.48 335.2 L8.29
+ 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46 334.9 L10.21 334.58 L9.55
+ 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19 338.43 C4.19 339.56 5.11 340.48
+ 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29 7.38 336.37 6.25 336.37 ZM6.25
+ 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02 339.83 6.25 339.83 C5.47 339.83
+ 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02 ZM2.62 338.14 L0 338.14 L0 338.71
+ L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73 337.47 L1.95 337.47 L2.62 338.14 Z"
+ class="st32"/>
+ </g>
+ </g>
+ <g id="group67-301" transform="translate(104.617,-86.5795)" v:mID="67" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server</title>
+ <g id="shape68-302" v:mID="68" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.68</title>
+ <g id="shadow68-303" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape69-306" v:mID="69" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.69</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape70-310" v:mID="70" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.70</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape71-314" v:mID="71" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.71</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape72-322" v:mID="72" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.72</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape73-326" v:mID="73" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.73</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape74-329" v:mID="74" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.74</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape75-332" v:mID="75" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.75</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape76-336" v:mID="76" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.76</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape77-339" v:mID="77" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.77</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape78-343" v:mID="78" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.78</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape79-346" v:mID="79" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.79</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape80-349" v:mID="80" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.80</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape81-352" v:mID="81" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.81</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape82-356" v:mID="82" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.82</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape83-359" v:mID="83" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.83</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape84-362" v:mID="84" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.84</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape85-365" v:mID="85" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.85</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape86-368" v:mID="86" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.86</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape87-371" v:mID="87" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.87</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape88-374" v:mID="88" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.88</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape89-377" v:mID="89" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.89</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape90-380" v:mID="90" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.90</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape91-384" v:mID="91" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.91</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape92-387" v:mID="92" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.92</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape93-390" v:mID="93" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.93</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape94-393" v:mID="94" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.94</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape95-397" v:mID="95" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.95</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape96-400" v:mID="96" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.96</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape97-402" v:mID="97" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.97</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape98-405" v:mID="98" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.98</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape99-408" v:mID="99" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.99</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape100-410" v:mID="100" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.100</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape101-412" v:mID="101" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.101</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape102-416" v:mID="102" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.102</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape103-420" v:mID="103" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.103</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape104-427" v:mID="104" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.104</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape105-435" v:mID="105" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.105</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape106-440" v:mID="106" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.106</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="group107-443" transform="translate(104.617,-33.8201)" v:mID="107" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server.104</title>
+ <g id="shape108-444" v:mID="108" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.108</title>
+ <g id="shadow108-445" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape109-448" v:mID="109" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.109</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape110-451" v:mID="110" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.110</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape111-454" v:mID="111" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.111</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape112-457" v:mID="112" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.112</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape113-460" v:mID="113" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.113</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape114-463" v:mID="114" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.114</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape115-466" v:mID="115" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.115</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape116-469" v:mID="116" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.116</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape117-472" v:mID="117" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.117</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape118-475" v:mID="118" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.118</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape119-478" v:mID="119" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.119</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape120-481" v:mID="120" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.120</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape121-484" v:mID="121" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.121</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape122-487" v:mID="122" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.122</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape123-490" v:mID="123" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.123</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape124-493" v:mID="124" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.124</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape125-496" v:mID="125" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.125</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape126-499" v:mID="126" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.126</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape127-502" v:mID="127" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.127</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape128-505" v:mID="128" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.128</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape129-508" v:mID="129" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.129</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape130-511" v:mID="130" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.130</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape131-514" v:mID="131" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.131</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape132-517" v:mID="132" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.132</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape133-520" v:mID="133" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.133</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape134-523" v:mID="134" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.134</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape135-526" v:mID="135" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.135</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape136-529" v:mID="136" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.136</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape137-531" v:mID="137" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.137</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape138-534" v:mID="138" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.138</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape139-537" v:mID="139" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.139</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape140-539" v:mID="140" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.140</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape141-541" v:mID="141" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.141</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape142-544" v:mID="142" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.142</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape143-547" v:mID="143" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.143</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape144-554" v:mID="144" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.144</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape145-557" v:mID="145" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.145</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape146-562" v:mID="146" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.146</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="shape147-565" v:mID="147" v:groupContext="shape" transform="translate(427.321,214.49) rotate(90)">
+ <title>Cloud</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54 Z" class="st42"/>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54" class="st54"/>
+ <path d="M11.05 312.14 A8.59237 10.0375 0 0 1 5.37 311.54" class="st54"/>
+ <path d="M40.54 332.09 A8.62978 10.0812 -180 0 0 42.1 335.11" class="st54"/>
+ <path d="M73.92 326.65 A6.96633 8.13801 0 0 1 73.14 330.27" class="st54"/>
+ <path d="M89.7 308.52 A7.30994 8.5394 0 0 1 95.9 318.19" class="st54"/>
+ <path d="M103.15 297.64 A6.67364 7.79609 -180 0 0 106.25 294.02" class="st54"/>
+ <path d="M37.96 278.3 A10.2914 12.0219 -180 0 0 34.86 275.89" class="st54"/>
+ </g>
+ <g id="shape148-574" v:mID="148" v:groupContext="shape" transform="translate(110.222,-64.9346)">
+ <title>Triangle</title>
+ <desc>setsum</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="15.6995" cy="336.449" width="31.4" height="12.1933"/>
+ <g id="shadow148-575" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st9"/>
+ </g>
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st10"/>
+ <text x="8.35" y="337.95" class="st55" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsum</text> </g>
+ <g id="shape149-579" v:mID="149" v:groupContext="shape" transform="translate(292.639,20.8827) rotate(45)">
+ <title>Sheet.149</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L14.71 342.55" class="st14"/>
+ </g>
+ <g id="shape150-584" v:mID="150" v:groupContext="shape" transform="translate(43.2897,-54.1122)">
+ <title>Sheet.150</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L10.07 342.55" class="st14"/>
+ </g>
+ <g id="shape151-589" v:mID="151" v:groupContext="shape" transform="translate(-112.261,8.34531) rotate(-28.1394)">
+ <title>Sheet.151</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L18 342.55" class="st14"/>
+ </g>
+ <g id="shape152-594" v:mID="152" v:groupContext="shape">
+ <title>Sheet.152</title>
+ <desc>Clients</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="21.5" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Clients</text> </g>
+ <g id="shape153-597" v:mID="153" v:groupContext="shape" transform="translate(83.578,-9.58078)">
+ <title>Sheet.153</title>
+ <desc>Distributed Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.0677" cy="337.134" width="84.14" height="10.8224"/>
+ <rect x="0" y="331.723" width="84.1355" height="10.8224" class="st11"/>
+ <text x="13.1" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Distributed Cache</text> </g>
+ <g id="shape154-600" v:mID="154" v:groupContext="shape" transform="translate(181.983,-18.6146)">
+ <title>Sheet.154</title>
+ <desc>Web Servers</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="11.93" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Web Servers</text> </g>
+ <g id="shape155-603" v:mID="155" v:groupContext="shape" transform="translate(96.6068,630.978) rotate(180)">
+ <title>Simple Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow155-604" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ <g id="shape156-607" v:mID="156" v:groupContext="shape" transform="translate(173.159,625.567) rotate(180)">
+ <title>Simple Arrow.153</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow156-608" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ </g>
+ <g id="shape157-611" v:mID="157" v:groupContext="shape" transform="translate(0,-149.475)">
+ <title>Rectangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow157-612" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape158-615" v:mID="158" v:groupContext="shape" transform="translate(271.116,-149.475)">
+ <title>Rectangle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow158-616" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape159-619" v:mID="159" v:groupContext="shape">
+ <title>Rectangle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow159-620" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape160-623" v:mID="160" v:groupContext="shape" transform="translate(271.116,0)">
+ <title>Rectangle.160</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow160-624" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape161-627" v:mID="161" v:groupContext="shape" transform="translate(83.578,-151.241)">
+ <title>Sheet.161</title>
+ <desc>(a) Distributed Web Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow161-628" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(a) Distributed Web Cache</text> </g>
+ <g id="shape162-632" v:mID="162" v:groupContext="shape" transform="translate(319.513,-151.241)">
+ <title>Sheet.162</title>
+ <desc>(b) Detecting Routing Loops</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow162-633" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(b) Detecting Routing Loops</text> </g>
+ <g id="shape163-637" v:mID="163" v:groupContext="shape" transform="translate(77.5283,-3.35965)">
+ <title>Sheet.163</title>
+ <desc>(c) In-order Workload Scheduler</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="63.5211" cy="333.806" width="127.05" height="17.4792"/>
+ <g id="shadow163-638" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(c) In-order Workload Scheduler</text> </g>
+ <g id="shape164-642" v:mID="164" v:groupContext="shape" transform="translate(307.414,-3.35965)">
+ <title>Sheet.164</title>
+ <desc>(d) Database Semi-join Operations</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="66.2253" cy="333.806" width="132.46" height="17.4792"/>
+ <g id="shadow164-643" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(d) Database Semi-join Operations</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i2.svg b/doc/guides/prog_guide/img/member_i2.svg
new file mode 100644
index 0000000..759c654
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i2.svg
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i2.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="4.38194in" height="1.25694in"
+ viewBox="0 0 315.5 90.5" xml:space="preserve" color-interpolation-filters="sRGB" class="st6">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:none;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st3 {baseline-shift:32.4943%;font-size:0.649886em}
+ .st4 {font-size:1em}
+ .st5 {font-family:Cambria Math;font-size:1em}
+ .st6 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(0.25,-0.25)">
+ <title>Sheet.3</title>
+ <desc>False Positive Probability = (1-(1-1/m)kn)k ≃ (1-ekn/m)k</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="157.5" cy="45.5" width="315" height="90"/>
+ <rect x="0" y="0.5" width="315" height="90" class="st1"/>
+ <text x="8.28" y="49.82" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>False Positive Probability = (1-(1-1/m)<tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan dy="0.152em" class="st4">)</tspan><tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan><tspan dy="0.152em" class="st4"> </tspan><tspan
+ class="st5">≃</tspan> (1-e<tspan dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan class="st3"
+ v:baseFontSize="14">/</tspan><tspan class="st3" v:baseFontSize="14">m</tspan><tspan dy="0.152em"
+ class="st4">)</tspan><tspan dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i3.svg b/doc/guides/prog_guide/img/member_i3.svg
new file mode 100644
index 0000000..41e92cb
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i3.svg
@@ -0,0 +1,148 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i3.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ width="4.71875in" height="2.84375in" viewBox="0 0 339.75 204.75" xml:space="preserve" color-interpolation-filters="sRGB"
+ class="st14">
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22}
+ .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25}
+ .st4 {marker-end:url(#mrkr5-32);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st5 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st6 {fill:#feffff;font-family:Calibri;font-size:0.833336em}
+ .st7 {font-size:1em}
+ .st8 {fill:#deebf6;stroke:#c7c8c8;stroke-width:0.25}
+ .st9 {fill:#000000;font-family:Calibri;font-size:0.833336em}
+ .st10 {marker-end:url(#mrkr5-84);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st12 {fill:none;stroke:none;stroke-width:0.25}
+ .st13 {fill:#ff0000;font-family:Calibri;font-size:1.00001em}
+ .st14 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-32" class="st5" refX="-6.16" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-84" class="st11" refX="-5.8" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <defs id="Filters">
+ <filter id="filter_2">
+ <feGaussianBlur stdDeviation="2"/>
+ </filter>
+ </defs>
+ <g>
+ <title>Page-1</title>
+ <g id="group174-1" transform="translate(3.0294,-5.3478)">
+ <title>Sheet.174</title>
+ <g id="shape155-2" transform="translate(99,-99)">
+ <title>Circle</title>
+ <g id="shadow155-3" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape156-7" transform="translate(207,-108)">
+ <title>Circle.156</title>
+ <g id="shadow156-8" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape157-12" transform="translate(153,-162)">
+ <title>Circle.157</title>
+ <g id="shadow157-13" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape158-17" transform="translate(297,-99)">
+ <title>Circle.158</title>
+ <g id="shadow158-18" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape159-22" transform="translate(180,-36)">
+ <title>Circle.159</title>
+ <g id="shadow159-23" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape160-27" transform="translate(109.604,-115.419) rotate(-7.12502)">
+ <title>Sheet.160</title>
+ <path d="M0 204.75 L66.4 204.75" class="st4"/>
+ </g>
+ <g id="shape161-33" transform="translate(276.661,-123.214) rotate(9.46232)">
+ <title>Sheet.161</title>
+ <path d="M0 204.75 L48.58 204.75" class="st4"/>
+ </g>
+ <g id="shape162-38" transform="translate(246.135,262.572) rotate(-160.346)">
+ <title>Sheet.162</title>
+ <path d="M0 204.75 L127.63 204.75" class="st4"/>
+ </g>
+ <g id="shape163-43" transform="translate(284.391,198.775) rotate(141.977)">
+ <title>Sheet.163</title>
+ <path d="M0 204.75 L46.23 204.75" class="st4"/>
+ </g>
+ <g id="shape164-48" transform="translate(70.6118,307.655) rotate(-145.945)">
+ <title>Sheet.164</title>
+ <path d="M0 204.75 L60.88 204.75" class="st4"/>
+ </g>
+ <g id="shape167-53" transform="translate(72,0)">
+ <title>Rectangle.167</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow167-54" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape168-60" transform="translate(108,0)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <g id="shadow168-61" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape169-66" transform="translate(0,-63)">
+ <title>Rectangle.169</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow169-67" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape170-73" transform="translate(36,-63)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <g id="shadow170-74" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape171-79" transform="translate(240.248,331.493) rotate(161.565)">
+ <title>Sheet.171</title>
+ <path d="M-0 190.52 A81.3416 36.0611 -153.48 0 0 82.31 195.86 L82.49 195.55" class="st10"/>
+ </g>
+ <g id="shape172-85" transform="translate(156.426,260.029) rotate(161.565)">
+ <title>Sheet.172</title>
+ <path d="M-0 181.6 A88.1422 54.1439 -124.1 0 0 82.68 187.13 L82.83 186.81" class="st10"/>
+ </g>
+ <g id="shape173-90" transform="translate(18,-121.5)">
+ <title>Sheet.173</title>
+ <desc>Encode ID</desc>
+ <rect x="0" y="177.75" width="63" height="27" class="st12"/>
+ <text x="7.02" y="194.85" class="st13">Encode ID</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i4.svg b/doc/guides/prog_guide/img/member_i4.svg
new file mode 100644
index 0000000..a2b6f2f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i4.svg
@@ -0,0 +1,450 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i4.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.625in" height="3.125in" viewBox="0 0 549 225"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st18">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st3 {marker-end:url(#mrkr5-10);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st4 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st5 {visibility:visible}
+ .st6 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st7 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st8 {fill:none;stroke:none;stroke-width:0.25}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st10 {stroke:#5b9bd5;stroke-dasharray:1.5,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5}
+ .st11 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st12 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st14 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st15 {font-size:1em}
+ .st16 {marker-end:url(#mrkr5-162);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st17 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st18 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-10" class="st4" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-162" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group47-1" transform="translate(3.0294,-0.25)" v:mID="47" v:groupContext="group">
+ <title>Sheet.47</title>
+ <g id="shape1-2" v:mID="1" v:groupContext="shape" transform="translate(177.75,-191.922)">
+ <title>Sheet.1</title>
+ <desc>Element</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="216" width="108" height="18"/>
+ <rect x="0" y="207" width="108" height="18" class="st1"/>
+ <text x="33.77" y="219.6" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Element</text> </g>
+ <g id="shape2-5" v:mID="2" v:groupContext="shape" transform="translate(456.75,33.0781) rotate(90)">
+ <title>Sheet.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L18.65 225" class="st3"/>
+ </g>
+ <g id="shape3-11" v:mID="3" v:groupContext="shape" transform="translate(0,-67.0469)">
+ <title>Rectangle.54</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow3-12" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape4-15" v:mID="4" v:groupContext="shape" transform="translate(27,-67.0469)">
+ <title>Rectangle.55</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow4-16" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape5-19" v:mID="5" v:groupContext="shape" transform="translate(54,-67.0469)">
+ <title>Rectangle.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow5-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape6-23" v:mID="6" v:groupContext="shape" transform="translate(0,-53.5469)">
+ <title>Rectangle.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-24" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape7-27" v:mID="7" v:groupContext="shape" transform="translate(27,-53.5469)">
+ <title>Rectangle.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow7-28" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape8-31" v:mID="8" v:groupContext="shape" transform="translate(54,-53.5469)">
+ <title>Rectangle.59</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow8-32" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape9-35" v:mID="9" v:groupContext="shape" transform="translate(5.625,-72.6719)">
+ <title>Sheet.9</title>
+ <desc>BF-1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-1</text> </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(128.25,-65.0781)">
+ <title>Rectangle.74</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow10-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape11-42" v:mID="11" v:groupContext="shape" transform="translate(155.25,-65.0781)">
+ <title>Rectangle.75</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-43" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(182.25,-65.0781)">
+ <title>Rectangle.76</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow12-47" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape13-50" v:mID="13" v:groupContext="shape" transform="translate(128.25,-51.5781)">
+ <title>Rectangle.77</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow13-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape14-54" v:mID="14" v:groupContext="shape" transform="translate(155.25,-51.5781)">
+ <title>Rectangle.78</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow14-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape15-58" v:mID="15" v:groupContext="shape" transform="translate(182.25,-51.5781)">
+ <title>Rectangle.79</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow15-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape16-62" v:mID="16" v:groupContext="shape" transform="translate(301.5,-65.0781)">
+ <title>Rectangle.81</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow16-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape17-66" v:mID="17" v:groupContext="shape" transform="translate(328.5,-65.0781)">
+ <title>Rectangle.82</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow17-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape18-70" v:mID="18" v:groupContext="shape" transform="translate(355.5,-65.0781)">
+ <title>Rectangle.83</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow18-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape19-74" v:mID="19" v:groupContext="shape" transform="translate(301.5,-51.5781)">
+ <title>Rectangle.84</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow19-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape20-78" v:mID="20" v:groupContext="shape" transform="translate(328.5,-51.5781)">
+ <title>Rectangle.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow20-79" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape21-82" v:mID="21" v:groupContext="shape" transform="translate(355.5,-51.5781)">
+ <title>Rectangle.86</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow21-83" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape22-86" v:mID="22" v:groupContext="shape" transform="translate(447.75,-65.6406)">
+ <title>Rectangle.88</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow22-87" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape23-90" v:mID="23" v:groupContext="shape" transform="translate(474.75,-65.6406)">
+ <title>Rectangle.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow23-91" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape24-94" v:mID="24" v:groupContext="shape" transform="translate(501.75,-65.6406)">
+ <title>Rectangle.90</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow24-95" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape25-98" v:mID="25" v:groupContext="shape" transform="translate(447.75,-52.1406)">
+ <title>Rectangle.91</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow25-99" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape26-102" v:mID="26" v:groupContext="shape" transform="translate(474.75,-52.1406)">
+ <title>Rectangle.92</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-103" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape27-106" v:mID="27" v:groupContext="shape" transform="translate(501.75,-52.1406)">
+ <title>Rectangle.93</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-107" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape28-110" v:mID="28" v:groupContext="shape" transform="translate(213.75,-63.9531)">
+ <title>Sheet.28</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L83.25 225" class="st10"/>
+ </g>
+ <g id="shape29-113" v:mID="29" v:groupContext="shape" transform="translate(387,-63.9531)">
+ <title>Sheet.29</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L54 225" class="st10"/>
+ </g>
+ <g id="group31-116" transform="translate(184.5,-113.172)" v:mID="31" v:groupContext="group">
+ <title>Sheet.31</title>
+ <g id="shape32-117" v:mID="32" v:groupContext="shape" transform="translate(225,173.25) rotate(90)">
+ <title>Block Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow32-118" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st5">
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st11"/>
+ </g>
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st12"/>
+ </g>
+ <g id="shape33-121" v:mID="33" v:groupContext="shape" transform="translate(2.25,-24.3529)">
+ <title>Sheet.33</title>
+ <desc>h1, h2 .. hk</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="215.868" width="90" height="18.2647"/>
+ <rect x="0" y="206.735" width="90" height="18.2647" class="st8"/>
+ <text x="17.56" y="219.47" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>h1, h2 .. hk</text> </g>
+ </g>
+ <g id="shape34-124" v:mID="34" v:groupContext="shape" transform="translate(307.011,286.73) rotate(152.323)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L128.85 225" class="st3"/>
+ </g>
+ <g id="shape35-129" v:mID="35" v:groupContext="shape" transform="translate(433.272,125.452) rotate(99.7172)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L58.31 225" class="st3"/>
+ </g>
+ <g id="shape36-134" v:mID="36" v:groupContext="shape" transform="translate(407.724,-64.1459) rotate(45)">
+ <title>Sheet.36</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L79.16 225" class="st3"/>
+ </g>
+ <g id="shape37-139" v:mID="37" v:groupContext="shape" transform="translate(320.441,-127.12) rotate(15.6155)">
+ <title>Sheet.37</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L200.75 225" class="st3"/>
+ </g>
+ <g id="shape38-144" v:mID="38" v:groupContext="shape" transform="translate(132.75,-75.2588)">
+ <title>Sheet.38</title>
+ <desc>BF-2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-2</text> </g>
+ <g id="shape39-147" v:mID="39" v:groupContext="shape" transform="translate(303.75,-70.7588)">
+ <title>Sheet.39</title>
+ <desc>BF-X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.2" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-X</text> </g>
+ <g id="shape40-150" v:mID="40" v:groupContext="shape" transform="translate(447.75,-75.2588)">
+ <title>Sheet.40</title>
+ <desc>BF-L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.89" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-L</text> </g>
+ <g id="shape41-153" v:mID="41" v:groupContext="shape" transform="translate(300.375,-117)">
+ <title>Sheet.41</title>
+ <desc>Hashing for lookup/Insertion into a vector of BFs happens once</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="90" cy="202.5" width="180" height="45"/>
+ <rect x="0" y="180" width="180" height="45" class="st8"/>
+ <text x="4.6" y="198.9" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Hashing for lookup/Insertion into a <tspan
+ x="23.06" dy="1.2em" class="st15">vector of BFs happens once</tspan></text> </g>
+ <g id="shape44-157" v:mID="44" v:groupContext="shape" transform="translate(249.698,-151.505) rotate(-3.74012)">
+ <title>Sheet.44</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A93.4958 45.6256 42.23 0 1 79.38 221.66 L79.68 221.85" class="st16"/>
+ </g>
+ <g id="shape45-163" v:mID="45" v:groupContext="shape" transform="translate(30.375,0.25)">
+ <title>Sheet.45</title>
+ <desc>Lookup/Insertion is done in the series of BFs, one by one or ...</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="233.048" cy="202.5" width="466.1" height="45"/>
+ <rect x="0" y="180" width="466.096" height="45" class="st8"/>
+ <text x="4.34" y="206.1" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Lookup/Insertion is done in the series of BFs, one by one or can be optimized to do in parallel. </text> </g>
+ <g id="shape46-166" v:mID="46" v:groupContext="shape" transform="translate(123.252,-43.6868) rotate(17.0249)">
+ <title>Sheet.46</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A88.2185 43.0621 47.63 0 1 70.31 221.39 L70.6 221.6" class="st16"/>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i5.svg b/doc/guides/prog_guide/img/member_i5.svg
new file mode 100644
index 0000000..c1728cf
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i5.svg
@@ -0,0 +1,163 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i5.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="5.30481in" height="1.96146in"
+ viewBox="0 0 381.946 141.225" xml:space="preserve" color-interpolation-filters="sRGB" class="st15">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:none;stroke:none;stroke-width:0.25}
+ .st5 {fill:#ffffff;font-family:Calibri;font-size:1.16666em;font-weight:bold}
+ .st6 {marker-end:url(#mrkr5-14);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st7 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st10 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {marker-end:url(#mrkr5-63);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st12 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.22935779816514}
+ .st13 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st14 {font-size:1em}
+ .st15 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-14" class="st7" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-63" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="7.15" refX="-7.15" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-4.36,-4.36) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group1-1" transform="translate(191.995,-19.4751)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-2" v:mID="2" v:groupContext="shape" transform="translate(146.944,42.2251) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow2-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st2"/>
+ </g>
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st3"/>
+ </g>
+ <g id="shape3-6" v:mID="3" v:groupContext="shape" transform="translate(0,-34.65)">
+ <title>Sheet.3</title>
+ <desc>vBF</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="38.1251" cy="126.375" width="76.26" height="29.7"/>
+ <rect x="0" y="111.525" width="76.2502" height="29.7" class="st4"/>
+ <text x="27.68" y="130.58" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>vBF </text> </g>
+ </g>
+ <g id="shape4-9" v:mID="4" v:groupContext="shape" transform="translate(126.724,-100.475)">
+ <title>Sheet.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape5-15" v:mID="5" v:groupContext="shape" transform="translate(103.5,-101.775)">
+ <title>Sheet.5</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="39.9122" cy="135.863" width="79.83" height="10.7251"/>
+ <rect x="0" y="130.5" width="79.8244" height="10.7251" class="st4"/>
+ <text x="21.78" y="138.86" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape6-18" v:mID="6" v:groupContext="shape" transform="translate(221.726,-56.2468) rotate(-24.5123)">
+ <title>Sheet.6</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L65.42 141.23" class="st6"/>
+ </g>
+ <g id="shape7-23" v:mID="7" v:groupContext="shape" transform="translate(280.318,-68.9751)">
+ <title>Sheet.7</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape8-28" v:mID="8" v:groupContext="shape" transform="translate(338.125,-56.6022) rotate(24.1625)">
+ <title>Sheet.8</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L70.8 141.23" class="st6"/>
+ </g>
+ <g id="shape9-33" v:mID="9" v:groupContext="shape" transform="translate(197.714,217.975) rotate(180)">
+ <title>Sheet.9</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(18,-67.5)">
+ <title>Sheet.10</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="80.4201" cy="134.475" width="160.85" height="13.5"/>
+ <rect x="0" y="127.725" width="160.84" height="13.5" class="st4"/>
+ <text x="25.11" y="137.18" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape11-41" v:mID="11" v:groupContext="shape" transform="translate(198.032,253.975) rotate(180)">
+ <title>Sheet.11</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Sheet.12</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="81" cy="136.725" width="162.01" height="9"/>
+ <rect x="0" y="132.225" width="162" height="9" class="st4"/>
+ <text x="11.04" y="139.43" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape13-49" v:mID="13" v:groupContext="shape" transform="translate(494.552,22.75) rotate(90)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape14-52" v:mID="14" v:groupContext="shape" transform="translate(494.552,58.75) rotate(90)">
+ <title>Sheet.14</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape15-55" v:mID="15" v:groupContext="shape" transform="translate(494.552,94.75) rotate(90)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape17-58" v:mID="17" v:groupContext="shape" transform="translate(348.769,-25.0593) rotate(44.5185)">
+ <title>Sheet.17</title>
+ <path d="M-0 141.23 A35.1884 19.2595 167.75 0 1 42.43 138.27 L42.74 138.46" class="st11"/>
+ </g>
+ <g id="shape18-64" v:mID="18" v:groupContext="shape" transform="translate(222.188,-5.40005)">
+ <title>Sheet.18</title>
+ <desc>A BF corresponding to each worker thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="59.0625" cy="127.725" width="118.13" height="27"/>
+ <rect x="0" y="114.225" width="118.125" height="27" class="st4"/>
+ <text x="5.14" y="124.13" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>A BF corresponding to <tspan
+ x="11.19" dy="1.2em" class="st14">each worker thread</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i6.svg b/doc/guides/prog_guide/img/member_i6.svg
new file mode 100644
index 0000000..265179f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i6.svg
@@ -0,0 +1,332 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i6.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8in" height="3.625in" viewBox="0 0 576 261"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st16">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.666664em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.00001em}
+ .st9 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {marker-end:url(#mrkr5-29);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st12 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st13 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st14 {fill:#92d050;stroke:#c8c8c8;stroke-width:0.25}
+ .st15 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st16 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-29" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-9.8478)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(396.989,-54.9268)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st2"/>
+ </g>
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(248.261,-12.1936)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st2"/>
+ </g>
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(6.07514E-013,-29.0155)">
+ <title>Rectangle.10</title>
+ <desc>Signatures for target 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="225.767" width="99.49" height="70.4662"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st2"/>
+ </g>
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st3"/>
+ <text x="26.54" y="223.37" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(239.971,-20.4837)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(353.649,-19.9346)">
+ <title>Sheet.54</title>
+ <desc>Match 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 1</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(216.989,-210.652)">
+ <title>Sheet.55</title>
+ <desc>Packet Payload</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="252.71" width="99.49" height="16.5803"/>
+ <rect x="0" y="244.42" width="99.4817" height="16.5803" class="st9"/>
+ <text x="19.04" y="255.71" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet Payload</text> </g>
+ <g id="shape56-24" v:mID="56" v:groupContext="shape" transform="translate(526.665,52.2365) rotate(90)">
+ <title>Sheet.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L16.52 261" class="st11"/>
+ </g>
+ <g id="shape96-30" v:mID="96" v:groupContext="shape" transform="translate(-3.0294,-95.7818)">
+ <title>Sheet.96</title>
+ <desc>Attack Signature Length 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.75" cy="248.565" width="103.5" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.5" height="24.8704" class="st7"/>
+ <text x="4.79" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 1</text> </g>
+ <g id="group114-33" transform="translate(228.359,-134.152)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-34" transform="translate(0,-24.8704)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-35" v:mID="100" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-36" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape101-39" v:mID="101" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-40" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape102-43" v:mID="102" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-44" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape103-47" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-48" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape104-51" v:mID="104" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-52" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape105-55" v:mID="105" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-56" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-59" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-60" v:mID="108" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-61" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape109-64" v:mID="109" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-65" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape110-68" v:mID="110" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow110-69" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape111-72" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-73" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ <g id="shape112-76" v:mID="112" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-77" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape113-80" v:mID="113" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-81" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-84" v:mID="89" v:groupContext="shape" transform="translate(398.644,-116.927) rotate(24.4696)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L143.75 261" class="st11"/>
+ </g>
+ <g id="shape115-89" v:mID="115" v:groupContext="shape" transform="translate(116.062,-1.19371E-012)">
+ <title>Rectangle.115</title>
+ <desc>Signatures for target 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="211.259" width="99.49" height="99.4817"/>
+ <g id="shadow115-90" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st2"/>
+ </g>
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st3"/>
+ <text x="26.54" y="208.86" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>2</text> </g>
+ <g id="shape116-95" v:mID="116" v:groupContext="shape" transform="translate(117.989,-95.7818)">
+ <title>Sheet.116</title>
+ <desc>Attack Signature Length 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.9909" cy="248.565" width="103.99" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.982" height="24.8704" class="st7"/>
+ <text x="5.03" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 2</text> </g>
+ <g id="shape118-98" v:mID="118" v:groupContext="shape" transform="translate(392.971,-90.217)">
+ <title>Sheet.118</title>
+ <desc>Attack Signature Length L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="248.565" width="108" height="24.8704"/>
+ <rect x="0" y="236.13" width="108" height="24.8704" class="st7"/>
+ <text x="7.43" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length L</text> </g>
+ <g id="shape119-101" v:mID="119" v:groupContext="shape" transform="translate(384.909,-64.9346)">
+ <title>Sheet.119</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape120-103" v:mID="120" v:groupContext="shape" transform="translate(491.971,-64.9346)">
+ <title>Sheet.120</title>
+ <desc>Match 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 2</text> </g>
+ <g id="shape85-106" v:mID="85" v:groupContext="shape" transform="translate(478.538,12.9307) rotate(65.6291)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L109.61 261" class="st11"/>
+ </g>
+ <g id="shape117-111" v:mID="117" v:groupContext="shape" transform="translate(247.054,-91.2818)">
+ <title>Sheet.117</title>
+ <desc>Attack Signature Length X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="52.7082" cy="248.565" width="105.42" height="24.8704"/>
+ <rect x="0" y="236.13" width="105.416" height="24.8704" class="st7"/>
+ <text x="5.7" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length X</text> </g>
+ </g>
+ <g id="shape122-114" v:mID="122" v:groupContext="shape" transform="translate(315.114,-164.13)">
+ <title>Sheet.122</title>
+ <desc>HTSS</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="26.943" cy="248.565" width="53.89" height="24.8704"/>
+ <rect x="0" y="236.13" width="53.8859" height="24.8704" class="st7"/>
+ <text x="14.52" y="252.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i7.svg b/doc/guides/prog_guide/img/member_i7.svg
new file mode 100644
index 0000000..e23ae26
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i7.svg
@@ -0,0 +1,399 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i7.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8.5in" height="4.5in" viewBox="0 0 612 324"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st23">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.16666em}
+ .st9 {fill:none;stroke:#00b050;stroke-width:2.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st12 {fill:#a8d08d;stroke:#c8c8c8;stroke-width:0.25}
+ .st13 {marker-end:url(#mrkr5-83);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st14 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st15 {marker-end:url(#mrkr5-95);stroke:#92d050;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st16 {fill:#92d050;fill-opacity:1;stroke:#92d050;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st17 {fill:#00b050;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st18 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st19 {fill:none;stroke:#ff0000;stroke-width:2.25}
+ .st20 {fill:#ff0000;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st21 {marker-end:url(#mrkr5-123);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-83" class="st14" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-95" class="st16" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-123" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-32.2733)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(460.471,-62.2267)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="279" width="108" height="45" class="st2"/>
+ </g>
+ <rect x="0" y="279" width="108" height="45" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(320.452,-18.123)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="234" width="108" height="90" class="st2"/>
+ </g>
+ <rect x="0" y="234" width="108" height="90" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Rectangle.10</title>
+ <desc>Flow Keys Matching Mask 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="285.75" width="108" height="76.5"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="247.5" width="108" height="76.5" class="st2"/>
+ </g>
+ <rect x="0" y="247.5" width="108" height="76.5" class="st3"/>
+ <text x="12.56" y="282.75" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(311.452,-27.123)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="310.5" width="126" height="13.5" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(424.471,-26.2267)">
+ <title>Sheet.54</title>
+ <desc>Match</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="317.25" width="72" height="13.5"/>
+ <rect x="0" y="310.5" width="72" height="13.5" class="st7"/>
+ <text x="17.68" y="321.45" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(261,-247.163)">
+ <title>Sheet.55</title>
+ <desc>Flow ID1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st9"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID1</text> </g>
+ <g id="shape96-24" v:mID="96" v:groupContext="shape" transform="translate(0,-109.783)">
+ <title>Sheet.96</title>
+ <desc>Flow Mask 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 1</text> </g>
+ <g id="group114-27" transform="translate(247.5,-163.783)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-28" transform="translate(0,-27)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-29" v:mID="100" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-30" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape101-33" v:mID="101" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-34" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape102-37" v:mID="102" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-38" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape103-41" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-42" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape104-45" v:mID="104" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-46" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape105-49" v:mID="105" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-50" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-53" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-54" v:mID="108" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape109-58" v:mID="109" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape110-62" v:mID="110" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow110-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st12"/>
+ </g>
+ <g id="shape111-66" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape112-70" v:mID="112" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape113-74" v:mID="113" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-78" v:mID="89" v:groupContext="shape" transform="translate(413.723,393.802) rotate(146.31)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L153.9 324" class="st13"/>
+ </g>
+ <g id="shape115-84" v:mID="115" v:groupContext="shape" transform="translate(126,0)">
+ <title>Rectangle.115</title>
+ <desc>Flow Keys Matching Mask 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="270" width="108" height="108"/>
+ <g id="shadow115-85" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="216" width="108" height="108" class="st2"/>
+ </g>
+ <rect x="0" y="216" width="108" height="108" class="st3"/>
+ <text x="12.56" y="267" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>2</text> </g>
+ <g id="shape85-90" v:mID="85" v:groupContext="shape" transform="translate(635.321,91.2793) rotate(81.3573)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L143.93 324" class="st15"/>
+ </g>
+ <g id="shape56-96" v:mID="56" v:groupContext="shape" transform="translate(579.175,-64.556) rotate(64.1257)">
+ <title>Sheet.56</title>
+ <path d="M0 324 L54.31 324" class="st15"/>
+ </g>
+ </g>
+ <g id="shape122-101" v:mID="122" v:groupContext="shape" transform="translate(351,-213.444)">
+ <title>Sheet.122</title>
+ <desc>HTSS with False Negative (Cache)</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="304.722" width="90" height="38.556"/>
+ <rect x="0" y="285.444" width="90" height="38.556" class="st7"/>
+ <text x="13.29" y="301.72" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS with False <tspan
+ x="10.52" dy="1.2em" class="st5">Negative </tspan>(Cache)</text> </g>
+ <g id="shape123-105" v:mID="123" v:groupContext="shape" transform="translate(287.654,-290.556)">
+ <title>Sheet.123</title>
+ <desc>Active</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1875" cy="310.5" width="48.38" height="27"/>
+ <rect x="0" y="297" width="48.375" height="27" class="st7"/>
+ <text x="8.63" y="314.1" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Active</text> </g>
+ <g id="shape124-108" v:mID="124" v:groupContext="shape" transform="translate(278.827,-153)">
+ <title>Sheet.124</title>
+ <desc>Target for Flow ID 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36.0864" cy="310.5" width="72.18" height="27"/>
+ <rect x="0" y="297" width="72.1728" height="27" class="st9"/>
+ <text x="11.93" y="306.9" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Target for <tspan
+ x="13.54" dy="1.2em" class="st5">Flow ID </tspan>1</text> </g>
+ <g id="shape125-112" v:mID="125" v:groupContext="shape" transform="translate(155.857,-254.556)">
+ <title>Sheet.125</title>
+ <desc>Flow ID2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st19"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID2</text> </g>
+ <g id="shape126-115" v:mID="126" v:groupContext="shape" transform="translate(153,-270)">
+ <title>Sheet.126</title>
+ <desc>New/Inactive</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="40.5" cy="310.5" width="81" height="27"/>
+ <rect x="0" y="297" width="81" height="27" class="st7"/>
+ <text x="6.77" y="314.1" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New/Inactive</text> </g>
+ <g id="shape127-118" v:mID="127" v:groupContext="shape" transform="translate(251.739,-239.709) rotate(14.0795)">
+ <title>Sheet.127</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 318.73 A39.2404 18 -180 0 0 49.73 320.91 L50.07 320.78" class="st21"/>
+ </g>
+ <g id="shape128-124" v:mID="128" v:groupContext="shape" transform="translate(219.24,-229.5)">
+ <title>Sheet.128</title>
+ <desc>Miss</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="20.88" cy="310.5" width="41.76" height="27"/>
+ <rect x="0" y="297" width="41.76" height="27" class="st7"/>
+ <text x="7.81" y="314.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Miss</text> </g>
+ <g id="shape129-127" v:mID="129" v:groupContext="shape" transform="translate(147.029,-142.056)">
+ <title>Sheet.129</title>
+ <desc>Flow Mask 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 2</text> </g>
+ <g id="shape130-130" v:mID="130" v:groupContext="shape" transform="translate(166.845,-18.5004) rotate(18.2325)">
+ <title>Sheet.130</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A71.1913 104.269 -180 0 0 97.04 298.43 L97.25 298.14" class="st21"/>
+ </g>
+ <g id="shape131-135" v:mID="131" v:groupContext="shape" transform="translate(184.406,-3.04505) rotate(-3.24734)">
+ <title>Sheet.131</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A112.345 104.269 -180 0 0 154.25 297.52 L154.52 297.28" class="st21"/>
+ </g>
+ <g id="shape132-140" v:mID="132" v:groupContext="shape" transform="translate(301.368,16.888) rotate(-25.868)">
+ <title>Sheet.132</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A83.375 104.269 -180 0 0 113.91 298.14 L114.14 297.87" class="st21"/>
+ </g>
+ <g id="shape133-145" v:mID="133" v:groupContext="shape" transform="translate(345.029,-142.056)">
+ <title>Sheet.133</title>
+ <desc>Flow Mask X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.43" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask X</text> </g>
+ <g id="shape134-148" v:mID="134" v:groupContext="shape" transform="translate(481.5,-139.5)">
+ <title>Sheet.134</title>
+ <desc>Flow Mask L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="19.12" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask L</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 40f04a1..7ff5144 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -50,6 +50,7 @@ Programmer's Guide
timer_lib
hash_lib
efd_lib
+ member_lib
lpm_lib
lpm6_lib
packet_distrib_lib
@@ -191,6 +192,19 @@ Programmer's Guide
:numref:`figure_efd11` :ref:`figure_efd11`
+:numref:`figure_membership1` :ref:`figure_membership1`
+
+:numref:`figure_membership2` :ref:`figure_membership2`
+
+:numref:`figure_membership3` :ref:`figure_membership3`
+
+:numref:`figure_membership4` :ref:`figure_membership4`
+
+:numref:`figure_membership5` :ref:`figure_membership5`
+
+:numref:`figure_membership6` :ref:`figure_membership6`
+
+:numref:`figure_membership7` :ref:`figure_membership7`
**Tables**
diff --git a/doc/guides/prog_guide/member_lib.rst b/doc/guides/prog_guide/member_lib.rst
new file mode 100644
index 0000000..caca8dc
--- /dev/null
+++ b/doc/guides/prog_guide/member_lib.rst
@@ -0,0 +1,420 @@
+.. BSD LICENSE
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+.. _member_library:
+
+Membership Library
+==================
+
+Introduction
+------------
+
+The DPDK Membership Library provides an API for DPDK applications to insert a
+new member, delete an existing member, or query the existence of a member in a
+given set, or a group of sets. For the case of a group of sets, the library
+will return not only whether the element has been inserted before in one of
+the sets but also which set it belongs to. The Membership Library is an
+extension and generalization of a traditional filter structure (for example
+Bloom Filter [Member-bloom]) that has multiple usages in a wide variety of
+workloads and applications. In general, the Membership Library is a data
+structure that provides a "set-summary" on whether a member belongs to a set,
+and as discussed in detail later, there are two advantages of using such a
+set-summary rather than operating on a "full-blown" complete list of elements:
+first, it has a much smaller storage requirement than storing the whole list of
+elements themselves, and secondly checking an element membership (or other
+operations) in this set-summary is much faster than checking it for the
+original full-blown complete list of elements.
+
+We use the term "Set-Summary" in this guide to refer to the space-efficient,
+probabilistic membership data structure that is provided by the library. A
+membership test for an element will return the set this element belongs to or
+that the element is "not-found" with very high probability of accuracy. Set-summary
+is a fundamental data aggregation component that can be used in many network
+(and other) applications. It is a crucial structure to address performance and
+scalability issues of diverse network applications including overlay networks,
+data-centric networks, flow table summaries, network statistics and
+traffic monitoring. A set-summary is useful for applications who need to
+include a list of elements while a complete list requires too much space
+and/or too much processing cost. In these situations, the set-summary works as
+a lossy hash-based representation of a set of members. It can dramatically
+reduce space requirement and significantly improve the performance of set
+membership queries at the cost of introducing a very small membership test error
+probability.
+
+.. _figure_membership1:
+.. figure:: img/member_i1.*
+
+ Example Usages of Membership Library
+
+There are various usages for a Membership Library in a very
+large set of applications and workloads. Interested readers can refer to
+[Member-survey] for a survey of possible networking usages. The above figure
+provide a small set of examples of using the Membership Library:
+
+* Sub-figure (a)
+ depicts a distributed web cache architecture where a collection of proxies
+ attempt to share their web caches (cached from a set of back-end web servers) to
+ provide faster responses to clients, and the proxies use the Membership
+ Library to share summaries of what web pages/objects they are caching. With the
+ Membership Library, a proxy receiving an http request will inquire the
+ set-summary to find its location and quickly determine whether to retrieve the
+ requested web page from a nearby proxy or from a back-end web server.
+
+* Sub-figure (b) depicts another example for using the Membership Library to
+ prevent routing loops which is typically done using slow TTL countdown and
+ dropping packets when TTL expires. As shown in Sub-figure (b), an embedded
+ set-summary in the packet header itself can be used to summarize the set of
+ nodes a packet has gone through, and each node upon receiving a packet can check
+ whether its id is a member of the set of visited nodes, and if it is, then a
+ routing loop is detected.
+
+* Sub-Figure (c) presents another usage of the Membership
+ Library to load-balance flows to worker threads with in-order guarantee where a
+ set-summary is used to query if a packet belongs to an existing flow or a new
+ flow. Packets belonging to a new flow are forwarded to the current least loaded
+ worker thread, while those belonging to an existing flow are forwarded to the
+ pre-assigned thread to guarantee in-order processing.
+
+* Sub-figure (d) highlights
+ yet another usage example in the database domain where a set-summary is used to
+ determine joins between sets instead of creating a join by comparing each
+ element of a set against the other elements in a different set, a join is done
+ on the summaries since they can efficiently encode members of a given set.
+
+Membership Library is a configurable library that is optimized to cover set
+membership functionality for both a single set and multi-set scenarios. Two set-summary
+schemes are presented including (a) vector of Bloom Filters and (b) Hash-Table based
+set-summary schemes with and without false negative probability.
+This guide first briefly describes these different types of set-summaries, usage examples for each,
+and then it highlights the Membership Library API.
+
+Vector of Bloom Filters
+-----------------------
+
+Bloom Filter (BF) [Member-bloom] is a well-known space-efficient
+probabilistic data structure that answers set membership queries (test whether
+an element is a member of a set) with some probability of false positives and
+zero false negatives; a query for an element returns either it is "possibly in
+a set" (with very high probability) or "definitely not in a set".
+
+The BF is a method for representing a set of ``n`` elements (for example flow keys
+in network applications domain) to support membership queries. The idea of BF is
+to allocate a bit-vector ``v`` with ``m`` bits, which are initially all set to 0. Then
+it chooses ``k`` independent hash functions ``h1``, ``h2``, ... ``hk`` with hash values range from
+``0`` to ``m-1`` to perform hashing calculations on each element to be inserted. Every time when an
+element ``X`` being inserted into the set, the bits at positions ``h1(X)``, ``h2(X)``, ...
+``hk(X)`` in ``v`` are set to 1 (any particular bit might be set to 1 multiple times
+for multiple different inserted elements). Given a query for any element ``Y``, the
+bits at positions ``h1(Y)``, ``h2(Y)``, ... ``hk(Y)`` are checked. If any of them is 0,
+then Y is definitely not in the set. Otherwise there is a high probability that
+Y is a member of the set with certain false positive probability. As shown in
+the next equation, the false positive probability can be made arbitrarily small
+by changing the number of hash functions (``k``) and the vector length (``m``).
+
+.. _figure_membership2:
+.. figure:: img/member_i2.*
+
+ Bloom Filter False Positive Probability
+
+Without BF, an accurate membership testing could involve a costly hash table
+lookup and full element comparison. The advantage of using a BF is to simplify
+the membership test into a series of hash calculations and memory accesses for a
+small bit-vector, which can be easily optimized. Hence the lookup throughput
+(set membership test) can be significantly faster than a normal hash table
+lookup with element comparison.
+
+.. _figure_membership3:
+.. figure:: img/member_i3.*
+
+ Detecting Routing Loops Using BF
+
+BF is used for applications that need only one set, and the
+membership of elements is checked against the BF. The example discussed
+in the above figure is one example of potential applications that uses only one
+set to capture the node IDs that have been visited so far by the packet. Each
+node will then check this embedded BF in the packet header for its own id, and
+if the BF indicates that the current node is definitely not in the set then a
+loop-free route is guaranteed.
+
+
+.. _figure_membership4:
+.. figure:: img/member_i4.*
+
+ Vector Bloom Filter (vBF) Overview
+
+To support membership test for both multiple sets and a single set,
+the library implements a Vector Bloom Filter (vBF) scheme.
+vBF basically composes multiple bloom filters into a vector of bloom filers.
+The membership test is conducted on all of the
+bloom filters concurrently to determine which set(s) it belongs to or none of
+them. The basic idea of vBF is shown in the above figure where an element is
+used to address multiple bloom filters concurrently and the bloom filter
+index(es) with a hit is returned.
+
+.. _figure_membership5:
+.. figure:: img/member_i5.*
+
+ vBF for Flow Scheduling to Worker Thread
+
+As previously mentioned, there are many usages of such structures. vBF is used
+for applications that need to check membership against multiple sets
+simultaneously. The example shown in the above figure uses a set to capture
+all flows being assigned for processing at a given worker thread. Upon receiving
+a packet the vBF is used to quickly figure out if this packet belongs to a new flow
+so as to be forwarded to the current least loaded worker thread, or otherwise it
+should be queued for an existing thread to guarantee in-order processing (i.e.
+the property of vBF to indicate right away that a given flow is a new one or
+not is critical to minimize response time latency).
+
+It should be noted that vBF can be implemented using a set of single bloom
+filters with sequential lookup of each BF. However, being able to concurrently
+search all set-summaries is a big throughput advantage. In the library, certain
+parallelism is realized by the implementation of checking all bloom filters
+together.
+
+
+Hash-Table based Set-Summaries
+------------------------------
+
+Hash-table based set-summary (HTSS) is another scheme in the membership library.
+Cuckoo filter [Member-cfilter] is an example of HTSS.
+HTSS supports multi-set membership testing like
+vBF does. However, while vBF is better for a small number of targets, HTSS is more suitable
+and can easily outperform vBF when the number of sets is
+large, since HTSS uses a single hash table for membership testing while vBF
+requires testing a series of Bloom Filters each corresponding to one set.
+As a result, generally speaking vBF is more adequate for the case of a small limited number of sets
+while HTSS should be used with a larger number of sets.
+
+.. _figure_membership6:
+.. figure:: img/member_i6.*
+
+ Using HTSS for Attack Signature Matching
+
+As shown in the above figure, attack signature matching where each set
+represents a certain signature length (for correctness of this example, an
+attack signature should not be a subset of another one) in the payload is a good
+example for using HTSS with 0% false negative (i.e., when an element returns not
+found, it has a 100% certainty that it is not a member of any set). The packet
+inspection application benefits from knowing right away that the current payload
+does not match any attack signatures in the database to establish its
+legitimacy, otherwise a deep inspection of the packet is needed.
+
+HTSS employs a similar but simpler data structure to a traditional hash table,
+and the major difference is that HTSS stores only the signatures but not the
+full keys/elements which can significantly reduce the footprint of the table.
+Along with the signature, HTSS also stores a value to indicate the target set.
+When looking up an element, the element is hashed and the HTSS is addressed
+to retrieve the signature stored. If the signature matches then the value is
+retrieved corresponding to the index of the target set which the element belongs
+to. Because signatures can collide, HTSS can still has false positive
+probability. Furthermore, if elements are allowed to be
+overwritten or evicted when the hash table becomes full, it will also have a
+false negative probability. We discuss this case in the next section.
+
+Set-Summaries with False Negative Probability
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As previously mentioned, traditional set-summaries (e.g. Bloom Filters) do not
+have a false negative probability, i.e., it is 100% certain when an element
+returns "not to be present" for a given set. However, the Membership Library
+also supports a set-summary probabilistic data structure based on HTSS which
+allows for false negative probability.
+
+In HTSS, when the hash table becomes full, keys/elements will fail to be added
+into the table and the hash table has to be resized to accommodate for these new
+elements, which can be expensive. However, if we allow new elements to overwrite
+or evict existing elements (as a cache typically does), then the resulting
+set-summary will begin to have false negative probability. This is because the
+element that was evicted from the set-summary may still be present in the target
+set. For subsequent inquiries the set-summary will falsely report the element
+not being in the set, hence having a false negative probability.
+
+The major usage of HTSS with false negative is to use it as a cache for
+distributing elements to different target sets. By allowing HTSS to evict old
+elements, the set-summary can keep track of the most recent elements
+(i.e. active) as a cache typically does. Old inactive elements (infrequently
+used elements) will automatically and eventually get evicted from the
+set-summary. It is worth noting that the set-summary still has false positive
+probability, which means the application either can tolerate certain false positive
+or it has fall-back path when false positive happens.
+
+.. _figure_membership7:
+.. figure:: img/member_i7.*
+
+ Using HTSS with False Negatives for Wild Card Classification
+
+HTSS with false negative (i.e. a cache) also has its wide set of applications.
+For example wild card flow classification (e.g. ACL rules) highlighted in the
+above figure is an example of such application. In that case each target set
+represents a sub-table with rules defined by a certain flow mask. The flow masks
+are non-overlapping, and for flows matching more than one rule only the highest
+priority one is inserted in the corresponding sub-table (interested readers can
+refer to the Open vSwitch (OvS) design of Mega Flow Cache (MFC) [Member-OvS]
+for further details). Typically the rules will have a large number of distinct
+unique masks and hence, a large number of target sets each corresponding to one
+mask. Because the active set of flows varies widely based on the network
+traffic, HTSS with false negative will act as a cache for <flowid, target ACL
+sub-table> pair for the current active set of flows. When a miss occurs (as
+shown in red in the above figure) the sub-tables will be searched sequentially
+one by one for a possible match, and when found the flow key and target
+sub-table will be inserted into the set-summary (i.e. cache insertion) so
+subsequent packets from the same flow don’t incur the overhead of the
+sequential search of sub-tables.
+
+Library API Overview
+--------------------
+
+The design goal of the Membership Library API is to be as generic as possible to
+support all the different types of set-summaries we discussed in previous
+sections and beyond. Fundamentally, the APIs need to include creation,
+insertion, deletion, and lookup.
+
+
+Set-summary Create
+~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_create()`` function is used to create a set-summary structure, the input parameter
+is a struct to pass in parameters that needed to initialize the set-summary, while the function returns the
+pointer to the created set-summary or ``NULL`` if the creation failed.
+
+The general input arguments used when creating the set-summary should include ``name``
+which is the name of the created set-summary, *type* which is one of the types
+supported by the library (e.g. ``RTE_MEMBER_TYPE_HT`` for HTSS or ``RTE_MEMBER_TYPE_VBF`` for vBF), and ``key_len``
+which is the length of the element/key. There are other parameters
+are only used for certain type of set-summary, or which have a slightly different meaning for different types of set-summary.
+For example, ``num_keys`` parameter means the maximum number of entries for Hash table based set-summary.
+However, for bloom filter, this value means the expected number of keys that could be
+inserted into the bloom filter(s). The value is used to calculate the size of each
+bloom filter.
+
+We also pass two seeds: ``prim_hash_seed`` and
+``sec_hash_seed`` for the primary and secondary hash functions to calculate two independent hash values.
+``socket_id`` parameter is the NUMA socket ID for the memory used to create the
+set-summary. For HTSS, another parameter ``is_cache`` is used to indicate
+if this set-summary is a cache (i.e. with false negative probability) or not.
+For vBF, extra parameters are needed. For example, ``num_set`` is the number of
+sets needed to initialize the vector bloom filters. This number is equal to the
+number of bloom filters will be created.
+``false_pos_rate`` is the false positive rate. num_keys and false_pos_rate will be used to determine
+the number of hash functions and the bloom filter size.
+
+
+Set-summary Element Insertion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_add()`` function is used to insert an element/key into a set-summary structure. If it fails an
+error is returned. For success the returned value is dependent on the
+set-summary mode to provide extra information for the users. For vBF
+mode, a return value of 0 means a successful insert. For HTSS mode without false negative, the insert
+could fail with ``-ENOSPC`` if the table is full. With false negative (i.e. cache mode),
+for insert that does not cause any eviction (i.e. no overwriting happens to an
+existing entry) the return value is 0. For insertion that causes eviction, the return
+value is 1 to indicate such situation, but it is not an error.
+
+The input arguments for the function should include the ``key`` which is a pointer to the element/key that needs to
+be added to the set-summary, and ``set_id`` which is the set id associated
+with the key that needs to be added.
+
+
+Set-summary Element Lookup
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_lookup()`` function looks up a single key/element in the set-summary structure. It
+returns as soon as the first match is found. The return value is 1 if a
+match is found and 0 otherwise. The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, and ``set_id`` which is used to return the
+first target set id where the key has matched, if any.
+
+The ``rte_member_lookup_bulk()`` function is used to look up a bulk of keys/elements in the
+set-summary structure for their first match. Each key lookup returns as soon as the first match is found. The
+return value is the number of keys that find a match. The arguments of the function include ``keys``
+which is a pointer to a bulk of keys that are to be looked up,
+``num_keys`` is the number
+of keys that will be looked up, and ``set_ids`` are the return target set
+ids for the first match found for each of the input keys. ``set_ids`` is an array
+needs to be sized according to the ``num_keys``. If there is no match, the set id
+for that key will be set to RTE_MEMBER_NO_MATCH.
+
+The ``rte_member_lookup_multi()`` function looks up a single key/element in the
+set-summary structure for multiple matches. It
+returns ALL the matches (possibly more than one) found for this key when it
+is matched against all target sets (it is worth noting that for cache mode HTSS,
+the current implementation matches at most one target set). The return value is
+the number of matches
+that was found for this key (for cache mode HTSS the return value
+should be at most 1). The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, ``max_match_per_key`` which is to indicate the maximum number of matches
+the user expects to find for each key, and ``set_id`` which is used to return all
+target set ids where the key has matched, if any. The ``set_id`` array should be sized
+according to ``max_match_per_key``. For vBF, the maximum number of matches per key is equal
+to the number of sets. For HTSS, the maximum number of matches per key is equal to two time
+entry count per bucket. ``max_match_per_key`` should be equal or smaller than the maximum number of
+possible matches.
+
+The ``rte_membership_lookup_multi_bulk()`` function looks up a bulk of keys/elements in the
+set-summary structure for multiple matches, each key lookup returns ALL the matches (possibly more
+than one) found for this key when it is matched against all target sets (cache mode HTSS
+matches at most one target set). The
+return value is the number of keys that find one or more matches in the
+set-summary structure. The arguments of the
+function include ``keys`` which is
+a pointer to a bulk of keys that are to be looked up, ``num_keys`` is the number
+of keys that will be looked up, ``max_match_per_key`` is the possible
+maximum number of matches for each key, ``match_count`` which is the returned number
+of matches for each key, and ``set_ids`` are the returned target set
+ids for all matches found for each keys. ``set_ids`` is 2-D array
+containing a 1-D array for each key (the size of 1-D array per key should be set by the user according to ``max_match_per_key``).
+``max_match_per_key`` should be equal or smaller than the maximum number of
+possible matches, similar to ``rte_member_lookup_multi``.
+
+
+Set-summary Element Delete
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_membership_delete()`` function deletes an element/key from a set-summary structure, if it fails
+an error is returned. The input arguments should include ``key`` which is a pointer to the
+element/key that needs to be deleted from the set-summary, and ``set_id``
+which is the set id associated with the key to delete. It is worth noting that current
+implementation of vBF does not support deletion [1]_. An error code ``-EINVAL`` will be returned.
+
+.. [1] Traditional bloom filter does not support proactive deletion. Supporting proactive deletion require additional implementation and performance overhead.
+
+References
+-----------
+
+[Member-bloom] B H Bloom, "Space/Time Trade-offs in Hash Coding with Allowable Errors," Communications of the ACM, 1970.
+
+[Member-survey] A Broder and M Mitzenmacher, "Network Applications of Bloom Filters: A Survey," in Internet Mathematics, 2005.
+
+[Member-cfilter] B Fan, D G Andersen and M Kaminsky, "Cuckoo Filter: Practically Better Than Bloom," in Conference on emerging Networking Experiments and Technologies, 2014.
+
+[Member-OvS] B Pfaff, "The Design and Implementation of Open vSwitch," in NSDI, 2015.
diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst
index 8bf91bd..ba068ec 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -41,6 +41,23 @@ New Features
Also, make sure to start the actual text at the margin.
=========================================================
+* **Added Membership library (rte_member).**
+
+ Added membership library. It provides an API for DPDK applications to insert a
+ new member, delete an existing member, or query the existence of a member in a
+ given set, or a group of sets. For the case of a group of sets the library
+ will return not only whether the element has been inserted before in one of
+ the sets but also which set it belongs to.
+
+ The Membership Library is an extension and generalization of a traditional
+ filter (for example Bloom Filter) structure that has multiple usages in a wide
+ variety of workloads and applications. In general, the Membership Library is a
+ data structure that provides a “set-summary” and responds to set-membership
+ queries whether a certain member belongs to a set(s).
+
+ See the :ref:`Membership Library <Member_Library>` documentation in
+ the Programmers Guide document, for more information.
+
Resolved Issues
---------------
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v5 7/7] doc: add membership documentation
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 7/7] doc: add membership documentation Yipeng Wang
@ 2017-10-03 9:08 ` De Lara Guarch, Pablo
0 siblings, 0 replies; 81+ messages in thread
From: De Lara Guarch, Pablo @ 2017-10-03 9:08 UTC (permalink / raw)
To: Wang, Yipeng1, dev; +Cc: thomas, Tai, Charlie, Gobriel, Sameh, Mcnamara, John
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Tuesday, October 3, 2017 5:32 AM
> To: dev@dpdk.org; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v5 7/7] doc: add membership documentation
>
> This patch adds the documentation for membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> Reviewed-by: John McNamara <john.mcnamara@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* [dpdk-dev] [PATCH v6 0/7] Add Membership Library
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
` (6 preceding siblings ...)
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 7/7] doc: add membership documentation Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 1/7] member: implement main API Yipeng Wang
` (7 more replies)
7 siblings, 8 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
DPDK Membership Library provides an API that can be used by many DPDK
applications to conduct one or more set-membership tests (we mention some
possible use cases below, but interested readers can refer to
[1] for a wider survey of use cases).
The basic functionalities of the Membership Library include
inserting a new member, deleting an existing member, and querying the existence
of a member. The query result would indicate with high accuracy which specific
set this member belongs to among a group of sets.
The Membership Library is an extension and generalization of traditional filter
data structures [2,3], which maintain a space-efficient “set-summary”.
There are two advantages of using such a set-summary rather than operating on a
“full-blown” complete list of elements: firstly it has a much smaller storage
requirement than storing the whole list of elements, and secondly set membership
tests (or other operations) is much more efficient than searching through the
complete list of elements.
A membership test for an element will return the set this element belongs to if
found (or return "not-found") with high accuracy. If needed, the accuracy of the
membership tests could be further increased with larger storage space.
Set-summary is a fundamental data aggregation component that can be used in many
network applications. It is a crucial structure to address performance and
scalability issues of diverse applications including overlay networks, wild card
flow classification, web-caches, load balancing, connection tracking,
data-centric networks, flow table summaries, network statistics and traffic
monitoring. Our Proof of Concept (PoC) using set-summary to optimize flow lookup
in Open vSwitch (OvS) shows a speedup of about 2-3X.
This patch set implements two types of set-summaries, i.e., hash-table based
set-summary (HTSS) and Vector Bloom Filter (vBF). HTSS supports both the
non-cache and cache modes. The non-cache mode can incur a small chance of
false-positives which is the case when the set-summary indicates a key belongs
to a given set while actually it is not. The cache mode can also have
false-negatives in addition to false-positives. False-negatives means the case
when the set-summary indicates a key does not belong to a given set while
actually it does. This happens because cache mode allows new key to evict
existing keys. vBF only has false-positives similar to the non-cache HTSS.
However, one can set the false-positive rate arbitrarily. HTSS's
false-positive rate is determined by the hash-table size and the signature size.
[1] A Broder and M Mitzenmacher, “Network Applications of Bloom Filters: A
Survey,” in Internet Mathematics, 2005.
[2] B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable Errors,”
Communications of the ACM, 1970.
[3] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically Better Than
Bloom,” in Conference on emerging Networking Experiments and Technologies, 2014.
v6:
- member lib: minor coding style changes from Pablo's comment and checkpatch
report.
- member lib: minor log message changes.
v5:
- member lib: incorperate Pablo's review comments, including: multiple places
with coding style issues, change variable/function names to be more clear,
reorder code blocks, add descriptions, use macros for constant numbers,
fix bugs in log message, etc.
- test: modify the deletion function test to cover more cases.
- test: add check in performance test to check false negatvie for HT non-cache
mode.
- MAINTAINERS: complete in multiple commits instead of one as Pablo suggested.
v4:
- member lib: change two hash function macros to be one. crc or jhash will be
chosen depending on the architecture.
- member lib: use dynamic log type instead of static one by Thomas' comment.
- member lib: code cleanups (remove unnecessary code, change variable name,
coding style, etc).
- member lib: reorder members in setsummary struct to be more cache friendly.
- member lib: setsummary struct member "name" should be char array rather than
char pointer. Fixed.
- member lib: add check for two hash seeds to make sure they are not equal.
- member lib: vBF lookup speed optimization.
- member lib: revise comments in various places as Pablo suggested.
- member lib: change "void *" to "struct rte_member_setsum *" in public API as
Pablo suggested. test case is modified accordingly to directly use the struct.
- member lib: change order in rte_member_version.map as Pablo suggested.
- MAINTAINERS: change the maintainer block order according to Thomas' comment.
- test: add printf to show which section expects error messages. Change set
count to 16 from 32 for performance test.
- documentation: revise several places according to Pablo's and John's comments.
v3:
- documentation: update documentation to incorporate John's review.
v2:
- test: add bad parameter test functions to test the creation fail case.
- test: add find existing test.
- member lib: Add num_entries check in rte_member_create_ht.
- member lib: Add multiple checks for rte_member_create_vbf to avoid divide-by-0.
- member lib: remove unnecessary ret from rte_member.c according to Stephen's
comment.
- member lib: change the vBF creation function to be not too conservative.
Previous algorithm is too conservative on false positive.
- member lib: fix uninitialized issue fail gcc-4.8 in rte_member_find_existing.
- member lib: add rte_member_find_existing to version map.
- makefile: lib/librte_member/Makefile: change include order according to
Luca's comment.
- documentation: update the programmer guide for membership library.
Yipeng Wang (7):
member: implement main API
member: implement HT mode
member: implement vBF mode
member: add AVX for HT mode
member: enable the library
test/member: add functional and perf tests
doc: add membership documentation
MAINTAINERS | 8 +-
config/common_base | 5 +
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 ++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 +++
doc/guides/prog_guide/img/member_i6.svg | 332 ++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 420 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
lib/Makefile | 2 +
lib/librte_member/Makefile | 51 +
lib/librte_member/rte_member.c | 337 +++++++
lib/librte_member/rte_member.h | 513 ++++++++++
lib/librte_member/rte_member_ht.c | 586 +++++++++++
lib/librte_member/rte_member_ht.h | 94 ++
lib/librte_member/rte_member_vbf.c | 350 +++++++
lib/librte_member/rte_member_vbf.h | 82 ++
lib/librte_member/rte_member_version.map | 16 +
lib/librte_member/rte_member_x86.h | 107 ++
mk/rte.app.mk | 2 +
test/test/Makefile | 3 +
test/test/test_member.c | 744 ++++++++++++++
test/test/test_member_perf.c | 654 ++++++++++++
28 files changed, 7148 insertions(+), 2 deletions(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
create mode 100644 lib/librte_member/rte_member_version.map
create mode 100644 lib/librte_member/rte_member_x86.h
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v6 1/7] member: implement main API
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 2/7] member: implement HT mode Yipeng Wang
` (6 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
Membership library is an extension and generalization of a traditional
filter (for example Bloom Filter and cuckoo filter) structure.
In general, the Membership library is a data structure that provides a
"set-summary" and responds to set-membership queries of whether a
certain element belongs to a set(s). A membership test for an element
will return the set this element belongs to or not-found if the
element is never inserted into the set-summary.
The results of the membership test are not 100% accurate. Certain
false positive or false negative probability could exist. However,
comparing to a "full-blown" complete list of elements, a "set-summary"
is memory efficient and fast on lookup.
This patch adds the main API definition.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
lib/Makefile | 2 +
lib/librte_member/Makefile | 49 +++
lib/librte_member/rte_member.c | 337 ++++++++++++++++++++
lib/librte_member/rte_member.h | 513 +++++++++++++++++++++++++++++++
lib/librte_member/rte_member_version.map | 16 +
5 files changed, 917 insertions(+)
create mode 100644 lib/librte_member/Makefile
create mode 100644 lib/librte_member/rte_member.c
create mode 100644 lib/librte_member/rte_member.h
create mode 100644 lib/librte_member/rte_member_version.map
diff --git a/lib/Makefile b/lib/Makefile
index 86caba1..c82033a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -108,6 +108,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder
DEPDIRS-librte_reorder := librte_eal librte_mempool librte_mbuf
DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump
DEPDIRS-librte_pdump := librte_eal librte_mempool librte_mbuf librte_ether
+DIRS-$(CONFIG_RTE_LIBRTE_MEMBER) += librte_member
+DEPDIRS-librte_member := librte_eal librte_hash
ifeq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),y)
DIRS-$(CONFIG_RTE_LIBRTE_KNI) += librte_kni
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
new file mode 100644
index 0000000..1a79eaa
--- /dev/null
+++ b/lib/librte_member/Makefile
@@ -0,0 +1,49 @@
+# BSD LICENSE
+#
+# Copyright(c) 2017 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# library name
+LIB = librte_member.a
+
+CFLAGS := -I$(SRCDIR) $(CFLAGS)
+CFLAGS += $(WERROR_FLAGS) -O3
+
+EXPORT_MAP := rte_member_version.map
+
+LIBABIVER := 1
+
+# all source are stored in SRCS-y
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+# install includes
+SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/lib/librte_member/rte_member.c b/lib/librte_member/rte_member.c
new file mode 100644
index 0000000..b9def9a
--- /dev/null
+++ b/lib/librte_member/rte_member.c
@@ -0,0 +1,337 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+
+#include <rte_eal.h>
+#include <rte_eal_memconfig.h>
+#include <rte_memory.h>
+#include <rte_memzone.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+#include "rte_member_vbf.h"
+
+int librte_member_logtype;
+
+TAILQ_HEAD(rte_member_list, rte_tailq_entry);
+static struct rte_tailq_elem rte_member_tailq = {
+ .name = "RTE_MEMBER",
+};
+EAL_REGISTER_TAILQ(rte_member_tailq)
+
+struct rte_member_setsum *
+rte_member_find_existing(const char *name)
+{
+ struct rte_member_setsum *setsum = NULL;
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(name, setsum->name, RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ if (te == NULL) {
+ rte_errno = ENOENT;
+ return NULL;
+ }
+ return setsum;
+}
+
+void
+rte_member_free(struct rte_member_setsum *setsum)
+{
+ struct rte_member_list *member_list;
+ struct rte_tailq_entry *te;
+
+ if (setsum == NULL)
+ return;
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+ TAILQ_FOREACH(te, member_list, next) {
+ if (te->data == (void *)setsum)
+ break;
+ }
+ if (te == NULL) {
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return;
+ }
+ TAILQ_REMOVE(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_free_ht(setsum);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_free_vbf(setsum);
+ break;
+ default:
+ break;
+ }
+ rte_free(setsum);
+ rte_free(te);
+}
+
+struct rte_member_setsum *
+rte_member_create(const struct rte_member_parameters *params)
+{
+ struct rte_tailq_entry *te;
+ struct rte_member_list *member_list;
+ struct rte_member_setsum *setsum;
+ int ret;
+
+ if (params == NULL) {
+ rte_errno = EINVAL;
+ return NULL;
+ }
+
+ if (params->key_len == 0 ||
+ params->prim_hash_seed == params->sec_hash_seed) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "Create setsummary with "
+ "invalid parameters\n");
+ return NULL;
+ }
+
+ member_list = RTE_TAILQ_CAST(rte_member_tailq.head, rte_member_list);
+
+ rte_rwlock_write_lock(RTE_EAL_TAILQ_RWLOCK);
+
+ TAILQ_FOREACH(te, member_list, next) {
+ setsum = (struct rte_member_setsum *) te->data;
+ if (strncmp(params->name, setsum->name,
+ RTE_MEMBER_NAMESIZE) == 0)
+ break;
+ }
+ setsum = NULL;
+ if (te != NULL) {
+ rte_errno = EEXIST;
+ te = NULL;
+ goto error_unlock_exit;
+ }
+ te = rte_zmalloc("MEMBER_TAILQ_ENTRY", sizeof(*te), 0);
+ if (te == NULL) {
+ RTE_MEMBER_LOG(ERR, "tailq entry allocation failed\n");
+ goto error_unlock_exit;
+ }
+
+ /* Create a new setsum structure */
+ setsum = (struct rte_member_setsum *) rte_zmalloc_socket(params->name,
+ sizeof(struct rte_member_setsum), RTE_CACHE_LINE_SIZE,
+ params->socket_id);
+ if (setsum == NULL) {
+ RTE_MEMBER_LOG(ERR, "Create setsummary failed\n");
+ goto error_unlock_exit;
+ }
+ snprintf(setsum->name, sizeof(setsum->name), "%s", params->name);
+ setsum->type = params->type;
+ setsum->socket_id = params->socket_id;
+ setsum->key_len = params->key_len;
+ setsum->num_set = params->num_set;
+ setsum->prim_hash_seed = params->prim_hash_seed;
+ setsum->sec_hash_seed = params->sec_hash_seed;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ ret = rte_member_create_ht(setsum, params);
+ break;
+ case RTE_MEMBER_TYPE_VBF:
+ ret = rte_member_create_vbf(setsum, params);
+ break;
+ default:
+ goto error_unlock_exit;
+ }
+ if (ret < 0)
+ goto error_unlock_exit;
+
+ RTE_MEMBER_LOG(DEBUG, "Creating a setsummary table with "
+ "mode %u\n", setsum->type);
+
+ te->data = (void *)setsum;
+ TAILQ_INSERT_TAIL(member_list, te, next);
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ return setsum;
+
+error_unlock_exit:
+ rte_rwlock_write_unlock(RTE_EAL_TAILQ_RWLOCK);
+ rte_member_free(setsum);
+ return NULL;
+}
+
+int
+rte_member_add(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id)
+{
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_add_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_add_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t *set_id)
+{
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_ht(setsum, key, set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_vbf(setsum, key, set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids)
+{
+ if (setsum == NULL || keys == NULL || set_ids == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_bulk_ht(setsum, keys, num_keys,
+ set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_bulk_vbf(setsum, keys, num_keys,
+ set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi(const struct rte_member_setsum *setsum, const void *key,
+ uint32_t match_per_key, member_set_t *set_id)
+{
+ if (setsum == NULL || key == NULL || set_id == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_ht(setsum, key, match_per_key,
+ set_id);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_vbf(setsum, key, match_per_key,
+ set_id);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_lookup_multi_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key, uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ if (setsum == NULL || keys == NULL || set_ids == NULL ||
+ match_count == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_lookup_multi_bulk_ht(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ case RTE_MEMBER_TYPE_VBF:
+ return rte_member_lookup_multi_bulk_vbf(setsum, keys, num_keys,
+ max_match_per_key, match_count, set_ids);
+ default:
+ return -EINVAL;
+ }
+}
+
+int
+rte_member_delete(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id)
+{
+ if (setsum == NULL || key == NULL)
+ return -EINVAL;
+
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ return rte_member_delete_ht(setsum, key, set_id);
+ /* current vBF implementation does not support delete function */
+ case RTE_MEMBER_TYPE_VBF:
+ default:
+ return -EINVAL;
+ }
+}
+
+void
+rte_member_reset(const struct rte_member_setsum *setsum)
+{
+ if (setsum == NULL)
+ return;
+ switch (setsum->type) {
+ case RTE_MEMBER_TYPE_HT:
+ rte_member_reset_ht(setsum);
+ return;
+ case RTE_MEMBER_TYPE_VBF:
+ rte_member_reset_vbf(setsum);
+ return;
+ default:
+ return;
+ }
+}
+
+RTE_INIT(librte_member_init_log);
+
+static void
+librte_member_init_log(void)
+{
+ librte_member_logtype = rte_log_register("librte.member");
+ if (librte_member_logtype >= 0)
+ rte_log_set_level(librte_member_logtype, RTE_LOG_DEBUG);
+}
diff --git a/lib/librte_member/rte_member.h b/lib/librte_member/rte_member.h
new file mode 100644
index 0000000..9b0c8f9
--- /dev/null
+++ b/lib/librte_member/rte_member.h
@@ -0,0 +1,513 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/**
+ * @file
+ *
+ * RTE Membership Library
+ *
+ * The Membership Library is an extension and generalization of a traditional
+ * filter (for example Bloom Filter and cuckoo filter) structure that has
+ * multiple usages in a variety of workloads and applications. The library is
+ * used to test if a key belongs to certain sets. Two types of such
+ * "set-summary" structures are implemented: hash-table based (HT) and vector
+ * bloom filter (vBF). For HT setsummary, two subtypes or modes are available,
+ * cache and non-cache modes. The table below summarize some properties of
+ * the different implementations.
+ *
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ */
+
+/**
+ * <!--
+ * +==========+=====================+================+=========================+
+ * | type | vbf | HT-cache | HT-non-cache |
+ * +==========+=====================+==========================================+
+ * |structure | bloom-filter array | hash-table like without storing key |
+ * +----------+---------------------+------------------------------------------+
+ * |set id | limited by bf count | [1, 0x7fff] |
+ * | | up to 32. | |
+ * +----------+---------------------+------------------------------------------+
+ * |usages & | small set range, | can delete, | cache most recent keys, |
+ * |properties| user-specified | big set range, | have both false-positive|
+ * | | false-positive rate,| small false | and false-negative |
+ * | | no deletion support.| positive depend| depend on table size, |
+ * | | | on table size, | automatic overwritten. |
+ * | | | new key does | |
+ * | | | not overwrite | |
+ * | | | existing key. | |
+ * +----------+---------------------+----------------+-------------------------+
+ * -->
+ */
+
+#ifndef _RTE_MEMBER_H_
+#define _RTE_MEMBER_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+
+/** The set ID type that stored internally in hash table based set summary. */
+typedef uint16_t member_set_t;
+/** Invalid set ID used to mean no match found. */
+#define RTE_MEMBER_NO_MATCH 0
+/** Maximum size of hash table that can be created. */
+#define RTE_MEMBER_ENTRIES_MAX (1 << 30)
+/** Maximum number of keys that can be searched as a bulk */
+#define RTE_MEMBER_LOOKUP_BULK_MAX 64
+/** Entry count per bucket in hash table based mode. */
+#define RTE_MEMBER_BUCKET_ENTRIES 16
+/** Maximum number of characters in setsum name. */
+#define RTE_MEMBER_NAMESIZE 32
+
+/** @internal Hash function used by membership library. */
+#if defined(RTE_ARCH_X86) || defined(RTE_MACHINE_CPUFLAG_CRC32)
+#include <rte_hash_crc.h>
+#define MEMBER_HASH_FUNC rte_hash_crc
+#else
+#include <rte_jhash.h>
+#define MEMBER_HASH_FUNC rte_jhash
+#endif
+
+extern int librte_member_logtype;
+
+#define RTE_MEMBER_LOG(level, fmt, args...) \
+rte_log(RTE_LOG_ ## level, librte_member_logtype, "%s(): " fmt, \
+ __func__, ## args)
+
+/** @internal setsummary structure. */
+struct rte_member_setsum;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameter struct used to create set summary
+ */
+struct rte_member_parameters;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Define different set summary types
+ */
+enum rte_member_setsum_type {
+ RTE_MEMBER_TYPE_HT = 0, /**< Hash table based set summary. */
+ RTE_MEMBER_TYPE_VBF, /**< Vector of bloom filters. */
+ RTE_MEMBER_NUM_TYPE
+};
+
+/** @internal compare function for different arch. */
+enum rte_member_sig_compare_function {
+ RTE_MEMBER_COMPARE_SCALAR = 0,
+ RTE_MEMBER_COMPARE_AVX2,
+ RTE_MEMBER_COMPARE_NUM
+};
+
+/** @internal setsummary structure. */
+struct rte_member_setsum {
+ enum rte_member_setsum_type type; /* Type of the set summary. */
+ uint32_t key_len; /* Length of key. */
+ uint32_t prim_hash_seed; /* Primary hash function seed. */
+ uint32_t sec_hash_seed; /* Secondary hash function seed. */
+
+ /* Hash table based. */
+ uint32_t bucket_cnt; /* Number of buckets. */
+ uint32_t bucket_mask; /* Bit mask to get bucket index. */
+ /* For runtime selecting AVX, scalar, etc for signature comparison. */
+ enum rte_member_sig_compare_function sig_cmp_fn;
+ uint8_t cache; /* If it is cache mode for ht based. */
+
+ /* Vector bloom filter. */
+ uint32_t num_set; /* Number of set (bf) in vbf. */
+ uint32_t bits; /* Number of bits in each bf. */
+ uint32_t bit_mask; /* Bit mask to get bit location in bf. */
+ uint32_t num_hashes; /* Number of hash values to index bf. */
+
+ uint32_t mul_shift; /* vbf internal variable used during bit test. */
+ uint32_t div_shift; /* vbf internal variable used during bit test. */
+
+ void *table; /* This is the handler of hash table or vBF array. */
+
+
+ /* Second cache line should start here. */
+ uint32_t socket_id; /* NUMA Socket ID for memory. */
+ char name[RTE_MEMBER_NAMESIZE]; /* Name of this set summary. */
+} __rte_cache_aligned;
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Parameters used when create the set summary table. Currently user can
+ * specify two types of setsummary: HT based and vBF. For HT based, user can
+ * specify cache or non-cache mode. Here is a table to describe some differences
+ *
+ */
+struct rte_member_parameters {
+ const char *name; /**< Name of the hash. */
+
+ /**
+ * User to specify the type of the setsummary from one of
+ * rte_member_setsum_type.
+ *
+ * HT based setsummary is implemented like a hash table. User should use
+ * this type when there are many sets.
+ *
+ * vBF setsummary is a vector of bloom filters. It is used when number
+ * of sets is not big (less than 32 for current implementation).
+ */
+ enum rte_member_setsum_type type;
+
+ /**
+ * is_cache is only used for HT based setsummary.
+ *
+ * If it is HT based setsummary, user to specify the subtype or mode
+ * of the setsummary. It could be cache, or non-cache mode.
+ * Set is_cache to be 1 if to use as cache mode.
+ *
+ * For cache mode, keys can be evicted out of the HT setsummary. Keys
+ * with the same signature and map to the same bucket
+ * will overwrite each other in the setsummary table.
+ * This mode is useful for the case that the set-summary only
+ * needs to keep record of the recently inserted keys. Both
+ * false-negative and false-positive could happen.
+ *
+ * For non-cache mode, keys cannot be evicted out of the cache. So for
+ * this mode the setsummary will become full eventually. Keys with the
+ * same signature but map to the same bucket will still occupy multiple
+ * entries. This mode does not give false-negative result.
+ */
+ uint8_t is_cache;
+
+ /**
+ * For HT setsummary, num_keys equals to the number of entries of the
+ * table. When the number of keys inserted in the HT setsummary
+ * approaches this number, eviction could happen. For cache mode,
+ * keys could be evicted out of the table. For non-cache mode, keys will
+ * be evicted to other buckets like cuckoo hash. The table will also
+ * likely to become full before the number of inserted keys equal to the
+ * total number of entries.
+ *
+ * For vBF, num_keys equal to the expected number of keys that will
+ * be inserted into the vBF. The implementation assumes the keys are
+ * evenly distributed to each BF in vBF. This is used to calculate the
+ * number of bits we need for each BF. User does not specify the size of
+ * each BF directly because the optimal size depends on the num_keys
+ * and false positive rate.
+ */
+ uint32_t num_keys;
+
+ /**
+ * The length of key is used for hash calculation. Since key is not
+ * stored in set-summary, large key does not require more memory space.
+ */
+ uint32_t key_len;
+
+ /**
+ * num_set is only used for vBF, but not used for HT setsummary.
+ *
+ * num_set is equal to the number of BFs in vBF. For current
+ * implementation, it only supports 1,2,4,8,16,32 BFs in one vBF set
+ * summary. If other number of sets are needed, for example 5, the user
+ * should allocate the minimum available value that larger than 5,
+ * which is 8.
+ */
+ uint32_t num_set;
+
+ /**
+ * false_positive_rate is only used for vBF, but not used for HT
+ * setsummary.
+ *
+ * For vBF, false_positive_rate is the user-defined false positive rate
+ * given expected number of inserted keys (num_keys). It is used to
+ * calculate the total number of bits for each BF, and the number of
+ * hash values used during lookup and insertion. For details please
+ * refer to vBF implementation and membership library documentation.
+ *
+ * For HT, This parameter is not directly set by users.
+ * HT setsummary's false positive rate is in the order of:
+ * false_pos = (1/bucket_count)*(1/2^16), since we use 16-bit signature.
+ * This is because two keys needs to map to same bucket and same
+ * signature to have a collision (false positive). bucket_count is equal
+ * to number of entries (num_keys) divided by entry count per bucket
+ * (RTE_MEMBER_BUCKET_ENTRIES). Thus, the false_positive_rate is not
+ * directly set by users for HT mode.
+ */
+ float false_positive_rate;
+
+ /**
+ * We use two seeds to calculate two independent hashes for each key.
+ *
+ * For HT type, one hash is used as signature, and the other is used
+ * for bucket location.
+ * For vBF type, these two hashes and their combinations are used as
+ * hash locations to index the bit array.
+ */
+ uint32_t prim_hash_seed;
+
+ /**
+ * The secondary seed should be a different value from the primary seed.
+ */
+ uint32_t sec_hash_seed;
+
+ int socket_id; /**< NUMA Socket ID for memory. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Find an existing set-summary and return a pointer to it.
+ *
+ * @param name
+ * Name of the set-summary.
+ * @return
+ * Pointer to the set-summary or NULL if object not found
+ * with rte_errno set appropriately. Possible rte_errno values include:
+ * - ENOENT - value not available for return
+ */
+struct rte_member_setsum *
+rte_member_find_existing(const char *name);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Create set-summary (SS).
+ *
+ * @param params
+ * Parameters to initialize the setsummary.
+ * @return
+ * Return the pointer to the setsummary.
+ * Return value is NULL if the creation failed.
+ */
+struct rte_member_setsum *
+rte_member_create(const struct rte_member_parameters *params);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup key in set-summary (SS).
+ * Single key lookup and return as soon as the first match found
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param key
+ * Pointer of the key to be looked up.
+ * @param set_id
+ * Output the set id matches the key.
+ * @return
+ * Return 1 for found a match and 0 for not found a match.
+ */
+int
+rte_member_lookup(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t *set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup bulk of keys in set-summary (SS).
+ * Each key lookup returns as soon as the first match found
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param keys
+ * Pointer of the bulk of keys to be looked up.
+ * @param num_keys
+ * Number of keys that will be lookup.
+ * @param set_ids
+ * Output set ids for all the keys to this array.
+ * User should preallocate array that can contain all results, which size is
+ * the num_keys.
+ * @return
+ * The number of keys that found a match.
+ */
+int
+rte_member_lookup_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a key in set-summary (SS) for multiple matches.
+ * The key lookup will find all matched entries (multiple match).
+ * Note that for cache mode of HT, each key can have at most one match. This is
+ * because keys with same signature that maps to same bucket will overwrite
+ * each other. So multi-match lookup should be used for vBF and non-cache HT.
+ *
+ * @param setsum
+ * Pointer of a set-summary.
+ * @param key
+ * Pointer of the key that to be looked up.
+ * @param max_match_per_key
+ * User specified maximum number of matches for each key. The function returns
+ * as soon as this number of matches found for the key.
+ * @param set_id
+ * Output set ids for all the matches of the key. User needs to preallocate
+ * the array that can contain max_match_per_key number of results.
+ * @return
+ * The number of matches that found for the key.
+ * For cache mode HT set-summary, the number should be at most 1.
+ */
+int
+rte_member_lookup_multi(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t max_match_per_key,
+ member_set_t *set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Lookup a bulk of keys in set-summary (SS) for multiple matches each key.
+ * Each key lookup will find all matched entries (multiple match).
+ * Note that for cache mode HT, each key can have at most one match. So
+ * multi-match function is mainly used for vBF and non-cache mode HT.
+ *
+ * @param setsum
+ * Pointer of a setsummary.
+ * @param keys
+ * Pointer of the keys to be looked up.
+ * @param num_keys
+ * The number of keys that will be lookup.
+ * @param max_match_per_key
+ * The possible maximum number of matches for each key.
+ * @param match_count
+ * Output the number of matches for each key in an array.
+ * @param set_ids
+ * Return set ids for all the matches of all keys. Users pass in a
+ * preallocated 2D array with first dimension as key index and second
+ * dimension as match index. For example set_ids[bulk_size][max_match_per_key]
+ * @return
+ * The number of keys that found one or more matches in the set-summary.
+ */
+int
+rte_member_lookup_multi_bulk(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ uint32_t max_match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Insert key into set-summary (SS).
+ *
+ * @param setsum
+ * Pointer of a set-summary.
+ * @param key
+ * Pointer of the key to be added.
+ * @param set_id
+ * The set id associated with the key that needs to be added. Different mode
+ * supports different set_id ranges. 0 cannot be used as set_id since
+ * RTE_MEMBER_NO_MATCH by default is set as 0.
+ * For HT mode, the set_id has range as [1, 0x7FFF], MSB is reserved.
+ * For vBF mode the set id is limited by the num_set parameter when create
+ * the set-summary.
+ * @return
+ * HT (cache mode) and vBF should never fail unless the set_id is not in the
+ * valid range. In such case -EINVAL is returned.
+ * For HT (non-cache mode) it could fail with -ENOSPC error code when table is
+ * full.
+ * For success it returns different values for different modes to provide
+ * extra information for users.
+ * Return 0 for HT (cache mode) if the add does not cause
+ * eviction, return 1 otherwise. Return 0 for non-cache mode if success,
+ * -ENOSPC for full, and 1 if cuckoo eviction happens.
+ * Always returns 0 for vBF mode.
+ */
+int
+rte_member_add(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * De-allocate memory used by set-summary.
+ *
+ * @param setsum
+ * Pointer to the set summary.
+ */
+void
+rte_member_free(struct rte_member_setsum *setsum);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Reset the set-summary tables. E.g. reset bits to be 0 in BF,
+ * reset set_id in each entry to be RTE_MEMBER_NO_MATCH in HT based SS.
+ *
+ * @param setsum
+ * Pointer to the set-summary.
+ */
+void
+rte_member_reset(const struct rte_member_setsum *setsum);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice
+ *
+ * Delete items from the set-summary. Note that vBF does not support deletion
+ * in current implementation. For vBF, error code of -EINVAL will be returned.
+ *
+ * @param setsum
+ * Pointer to the set-summary.
+ * @param key
+ * Pointer of the key to be deleted.
+ * @param set_id
+ * For HT mode, we need both key and its corresponding set_id to
+ * properly delete the key. Without set_id, we may delete other keys with the
+ * same signature.
+ * @return
+ * If no entry found to delete, an error code of -ENOENT could be returned.
+ */
+int
+rte_member_delete(const struct rte_member_setsum *setsum, const void *key,
+ member_set_t set_id);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_H_ */
diff --git a/lib/librte_member/rte_member_version.map b/lib/librte_member/rte_member_version.map
new file mode 100644
index 0000000..019e4cd
--- /dev/null
+++ b/lib/librte_member/rte_member_version.map
@@ -0,0 +1,16 @@
+DPDK_17.11 {
+ global:
+
+ rte_member_add;
+ rte_member_create;
+ rte_member_delete;
+ rte_member_find_existing;
+ rte_member_free;
+ rte_member_lookup;
+ rte_member_lookup_bulk;
+ rte_member_lookup_multi;
+ rte_member_lookup_multi_bulk;
+ rte_member_reset;
+
+ local: *;
+};
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v6 2/7] member: implement HT mode
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 1/7] member: implement main API Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 3/7] member: implement vBF mode Yipeng Wang
` (5 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
One of the set-summary structures is hash-table based
set-summary (HTSS). One example is cuckoo filter [1].
Comparing to a traditional hash table, HTSS has a much more
compact structure. For each element, only one signature and
its corresponding set ID is stored. No key comparison is required
during lookup. For the table structure, there are multiple entries
in each bucket, and the table is composed of many buckets.
Two modes are supported for HTSS, "cache" and "none-cache" modes.
The non-cache mode is similar to the cuckoo filter [1].
When a bucket is full, one entry will be evicted to its
alternative bucket to make space for the new key. The table could
be full and then no more keys could be inserted. This mode has
false-positive rate but no false-negative. Multiple entries
with same signature could stay in the same bucket.
The "cache" mode does not evict key to its alternative bucket
when a bucket is full, an existing key will be evicted out of
the table like a cache. Thus, the table will never reject keys when
it is full. Another property is in each bucket, there cannot be
multiple entries with same signature. The mode could have both
false-positive and false-negative probability.
This patch adds the implementation of HTSS.
[1] B Fan, D G Andersen and M Kaminsky, “Cuckoo Filter: Practically
Better Than Bloom,” in Conference on emerging Networking
Experiments and Technologies, 2014.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_ht.c | 506 ++++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_ht.h | 94 +++++++
3 files changed, 601 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_ht.c
create mode 100644 lib/librte_member/rte_member_ht.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 1a79eaa..ad26548 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
new file mode 100644
index 0000000..e038987
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.c
@@ -0,0 +1,506 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_errno.h>
+#include <rte_malloc.h>
+#include <rte_prefetch.h>
+#include <rte_random.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_ht.h"
+
+/* Search bucket for entry with tmp_sig and update set_id */
+static inline int
+update_entry_search(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t set_id)
+{
+ uint32_t i;
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[bucket_id].sigs[i] == tmp_sig) {
+ buckets[bucket_id].sets[i] = set_id;
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static inline int
+search_bucket_single(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t *set_id)
+{
+ uint32_t iter;
+
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket_id].sigs[iter] &&
+ buckets[bucket_id].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket_id].sets[iter];
+ return 1;
+ }
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t matches_per_key,
+ member_set_t *set_id)
+{
+ uint32_t iter;
+
+ for (iter = 0; iter < RTE_MEMBER_BUCKET_ENTRIES; iter++) {
+ if (tmp_sig == buckets[bucket_id].sigs[iter] &&
+ buckets[bucket_id].sets[iter] !=
+ RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket_id].sets[iter];
+ (*counter)++;
+ if (*counter >= matches_per_key)
+ return;
+ }
+ }
+}
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+ uint32_t i, j;
+ uint32_t size_bucket_t;
+ uint32_t num_entries = rte_align32pow2(params->num_keys);
+
+ if ((num_entries > RTE_MEMBER_ENTRIES_MAX) ||
+ !rte_is_power_of_2(RTE_MEMBER_BUCKET_ENTRIES) ||
+ num_entries < RTE_MEMBER_BUCKET_ENTRIES) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR,
+ "Membership HT create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ uint32_t num_buckets = num_entries / RTE_MEMBER_BUCKET_ENTRIES;
+
+ size_bucket_t = sizeof(struct member_ht_bucket);
+
+ struct member_ht_bucket *buckets = rte_zmalloc_socket(NULL,
+ num_buckets * size_bucket_t,
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+
+ if (buckets == NULL) {
+ RTE_MEMBER_LOG(ERR, "memory allocation failed for HT "
+ "setsummary\n");
+ return -ENOMEM;
+ }
+
+ ss->table = buckets;
+ ss->bucket_cnt = num_buckets;
+ ss->bucket_mask = num_buckets - 1;
+ ss->cache = params->is_cache;
+
+ for (i = 0; i < num_buckets; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+
+ RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
+ "the table has %u entries, %u buckets\n",
+ num_entries, num_buckets);
+ return 0;
+}
+
+static inline void
+get_buckets_index(const struct rte_member_setsum *ss, const void *key,
+ uint32_t *prim_bkt, uint32_t *sec_bkt, member_sig_t *sig)
+{
+ uint32_t first_hash = MEMBER_HASH_FUNC(key, ss->key_len,
+ ss->prim_hash_seed);
+ uint32_t sec_hash = MEMBER_HASH_FUNC(&first_hash, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ /*
+ * We use the first hash value for the signature, and the second hash
+ * value to derive the primary and secondary bucket locations.
+ *
+ * For non-cache mode, we use the lower bits for the primary bucket
+ * location. Then we xor primary bucket location and the signature
+ * to get the secondary bucket location. This is called "partial-key
+ * cuckoo hashing" proposed by B. Fan, et al's paper
+ * "Cuckoo Filter: Practically Better Than Bloom". The benefit to use
+ * xor is that one could derive the alternative bucket location
+ * by only using the current bucket location and the signature. This is
+ * generally required by non-cache mode's eviction and deletion
+ * process without the need to store alternative hash value nor the full
+ * key.
+ *
+ * For cache mode, we use the lower bits for the primary bucket
+ * location and the higher bits for the secondary bucket location. In
+ * cache mode, keys are simply overwritten if bucket is full. We do not
+ * use xor since lower/higher bits are more independent hash values thus
+ * should provide slightly better table load.
+ */
+ *sig = first_hash;
+ if (ss->cache) {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (sec_hash >> 16) & ss->bucket_mask;
+ } else {
+ *prim_bkt = sec_hash & ss->bucket_mask;
+ *sec_bkt = (*prim_bkt ^ *sig) & ss->bucket_mask;
+ }
+}
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *ss,
+ const void *key, member_set_t *set_id)
+{
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+
+ return 0;
+}
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, member_set_t *set_id)
+{
+ uint32_t i;
+ uint32_t num_matches = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+
+ for (i = 0; i < num_keys; i++) {
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ num_matches++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t num_matches = 0;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &num_matches,
+ match_per_key, set_id);
+ if (num_matches < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &num_matches, match_per_key, set_id);
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ uint32_t i;
+ uint32_t num_matches = 0;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t match_cnt_tmp;
+ member_sig_t tmp_sig[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t prim_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t sec_buckets[RTE_MEMBER_LOOKUP_BULK_MAX];
+
+ for (i = 0; i < num_keys; i++) {
+ get_buckets_index(ss, keys[i], &prim_buckets[i],
+ &sec_buckets[i], &tmp_sig[i]);
+ rte_prefetch0(&buckets[prim_buckets[i]]);
+ rte_prefetch0(&buckets[sec_buckets[i]]);
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_tmp = 0;
+
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ num_matches++;
+ }
+ return num_matches;
+}
+
+static inline int
+try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ member_sig_t sig, member_set_t set_id)
+{
+ int i;
+ /* If not full then insert into one slot */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[prim].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[prim].sigs[i] = sig;
+ buckets[prim].sets[i] = set_id;
+ return 0;
+ }
+ }
+ /* If prim failed, we need to access second bucket */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (buckets[sec].sets[i] == RTE_MEMBER_NO_MATCH) {
+ buckets[sec].sigs[i] = sig;
+ buckets[sec].sets[i] = set_id;
+ return 0;
+ }
+ }
+ return -1;
+}
+
+static inline int
+try_update(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
+ member_sig_t sig, member_set_t set_id)
+{
+ if (update_entry_search(prim, sig, buckets, set_id) ||
+ update_entry_search(sec, sig, buckets, set_id))
+ return 0;
+ return -1;
+}
+
+static inline int
+evict_from_bucket(void)
+{
+ /* For now, we randomly pick one entry to evict */
+ return rte_rand() & (RTE_MEMBER_BUCKET_ENTRIES - 1);
+}
+
+/*
+ * This function is similar to the cuckoo hash make_space function in hash
+ * library
+ */
+static inline int
+make_space_bucket(const struct rte_member_setsum *ss, uint32_t bkt_idx,
+ unsigned int *nr_pushes)
+{
+ unsigned int i, j;
+ int ret;
+ struct member_ht_bucket *buckets = ss->table;
+ uint32_t next_bucket_idx;
+ struct member_ht_bucket *next_bkt[RTE_MEMBER_BUCKET_ENTRIES];
+ struct member_ht_bucket *bkt = &buckets[bkt_idx];
+ /* MSB is set to indicate if an entry has been already pushed */
+ member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
+
+ /*
+ * Push existing item (search for bucket with space in
+ * alternative locations) to its alternative location
+ */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ /* Search for space in alternative locations */
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_idx) & ss->bucket_mask;
+ next_bkt[i] = &buckets[next_bucket_idx];
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++) {
+ if (next_bkt[i]->sets[j] == RTE_MEMBER_NO_MATCH)
+ break;
+ }
+
+ if (j != RTE_MEMBER_BUCKET_ENTRIES)
+ break;
+ }
+
+ /* Alternative location has spare room (end of recursive function) */
+ if (i != RTE_MEMBER_BUCKET_ENTRIES) {
+ next_bkt[i]->sigs[j] = bkt->sigs[i];
+ next_bkt[i]->sets[j] = bkt->sets[i];
+ return i;
+ }
+
+ /* Pick entry that has not been pushed yet */
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++)
+ if ((bkt->sets[i] & flag_mask) == 0)
+ break;
+
+ /* All entries have been pushed, so entry cannot be added */
+ if (i == RTE_MEMBER_BUCKET_ENTRIES ||
+ ++(*nr_pushes) > RTE_MEMBER_MAX_PUSHES)
+ return -ENOSPC;
+
+ next_bucket_idx = (bkt->sigs[i] ^ bkt_idx) & ss->bucket_mask;
+ /* Set flag to indicate that this entry is going to be pushed */
+ bkt->sets[i] |= flag_mask;
+
+ /* Need room in alternative bucket to insert the pushed entry */
+ ret = make_space_bucket(ss, next_bucket_idx, nr_pushes);
+ /*
+ * After recursive function.
+ * Clear flags and insert the pushed entry
+ * in its alternative location if successful,
+ * or return error
+ */
+ bkt->sets[i] &= ~flag_mask;
+ if (ret >= 0) {
+ next_bkt[i]->sigs[ret] = bkt->sigs[i];
+ next_bkt[i]->sets[ret] = bkt->sets[i];
+ return i;
+ } else
+ return ret;
+}
+
+int
+rte_member_add_ht(const struct rte_member_setsum *ss,
+ const void *key, member_set_t set_id)
+{
+ int ret;
+ unsigned int nr_pushes = 0;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+ member_set_t flag_mask = 1U << (sizeof(member_set_t) * 8 - 1);
+
+ if (set_id == RTE_MEMBER_NO_MATCH || (set_id & flag_mask) != 0)
+ return -EINVAL;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ /*
+ * If it is cache based setsummary, we try overwriting (updating)
+ * existing entry with the same signature first. In cache mode, we allow
+ * false negatives and only cache the most recent keys.
+ *
+ * For non-cache mode, we do not update existing entry with the same
+ * signature. This is because if two keys with same signature update
+ * each other, false negative may happen, which is not the expected
+ * behavior for non-cache setsummary.
+ */
+ if (ss->cache) {
+ ret = try_update(buckets, prim_bucket, sec_bucket, tmp_sig,
+ set_id);
+ if (ret != -1)
+ return ret;
+ }
+ /* If not full then insert into one slot */
+ ret = try_insert(buckets, prim_bucket, sec_bucket, tmp_sig, set_id);
+ if (ret != -1)
+ return ret;
+
+ /* Random pick prim or sec for recursive displacement */
+ uint32_t select_bucket = (tmp_sig && 1U) ? prim_bucket : sec_bucket;
+ if (ss->cache) {
+ ret = evict_from_bucket();
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ return 1;
+ }
+
+ ret = make_space_bucket(ss, select_bucket, &nr_pushes);
+ if (ret >= 0) {
+ buckets[select_bucket].sigs[ret] = tmp_sig;
+ buckets[select_bucket].sets[ret] = set_id;
+ ret = 1;
+ }
+
+ return ret;
+}
+
+void
+rte_member_free_ht(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+int
+rte_member_delete_ht(const struct rte_member_setsum *ss, const void *key,
+ member_set_t set_id)
+{
+ int i;
+ uint32_t prim_bucket, sec_bucket;
+ member_sig_t tmp_sig;
+ struct member_ht_bucket *buckets = ss->table;
+
+ get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[prim_bucket].sigs[i] &&
+ set_id == buckets[prim_bucket].sets[i]) {
+ buckets[prim_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+
+ for (i = 0; i < RTE_MEMBER_BUCKET_ENTRIES; i++) {
+ if (tmp_sig == buckets[sec_bucket].sigs[i] &&
+ set_id == buckets[sec_bucket].sets[i]) {
+ buckets[sec_bucket].sets[i] = RTE_MEMBER_NO_MATCH;
+ return 0;
+ }
+ }
+ return -ENOENT;
+}
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *ss)
+{
+ uint32_t i, j;
+ struct member_ht_bucket *buckets = ss->table;
+
+ for (i = 0; i < ss->bucket_cnt; i++) {
+ for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
+ buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
+ }
+}
diff --git a/lib/librte_member/rte_member_ht.h b/lib/librte_member/rte_member_ht.h
new file mode 100644
index 0000000..3148a49
--- /dev/null
+++ b/lib/librte_member/rte_member_ht.h
@@ -0,0 +1,94 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_HT_H_
+#define _RTE_MEMBER_HT_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Maximum number of pushes for cuckoo path in HT mode. */
+#define RTE_MEMBER_MAX_PUSHES 50
+
+typedef uint16_t member_sig_t; /* signature size is 16 bit */
+
+/* The bucket struct for ht setsum */
+struct member_ht_bucket {
+ member_sig_t sigs[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte signature */
+ member_set_t sets[RTE_MEMBER_BUCKET_ENTRIES]; /* 2-byte set */
+} __rte_cache_aligned;
+
+int
+rte_member_create_ht(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_ht(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+uint32_t
+rte_member_lookup_multi_ht(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+int
+rte_member_add_ht(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t set_id);
+
+void
+rte_member_free_ht(struct rte_member_setsum *setsum);
+
+int
+rte_member_delete_ht(const struct rte_member_setsum *ss, const void *key,
+ member_set_t set_id);
+
+void
+rte_member_reset_ht(const struct rte_member_setsum *setsum);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_HT_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v6 3/7] member: implement vBF mode
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 1/7] member: implement main API Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 2/7] member: implement HT mode Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 4/7] member: add AVX for HT mode Yipeng Wang
` (4 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
Bloom Filter (BF) [1] is a well-known space-efficient
probabilistic data structure that answers set membership queries.
Vector of Bloom Filters (vBF) is an extension to traditional BF
that supports multi-set membership testing. Traditional BF will
return found or not-found for each key. vBF will also return
which set the key belongs to if it is found.
Since each set requires a BF, vBF should be used when set count
is small. vBF's false positive rate could be set appropriately so
that its memory requirement and lookup speed is better in certain
cases comparing to HT based set-summary.
This patch adds the vBF implementation.
[1]B H Bloom, “Space/Time Trade-offs in Hash Coding with Allowable
Errors,” Communications of the ACM, 1970.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
lib/librte_member/Makefile | 2 +-
lib/librte_member/rte_member_vbf.c | 350 +++++++++++++++++++++++++++++++++++++
lib/librte_member/rte_member_vbf.h | 82 +++++++++
3 files changed, 433 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_member/rte_member_vbf.c
create mode 100644 lib/librte_member/rte_member_vbf.h
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index ad26548..50275ed 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -42,7 +42,7 @@ EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
# all source are stored in SRCS-y
-SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += rte_member.c rte_member_ht.c rte_member_vbf.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MEMBER)-include := rte_member.h
diff --git a/lib/librte_member/rte_member_vbf.c b/lib/librte_member/rte_member_vbf.c
new file mode 100644
index 0000000..1a98ac8
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.c
@@ -0,0 +1,350 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <math.h>
+#include <string.h>
+
+#include <rte_malloc.h>
+#include <rte_memory.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+
+#include "rte_member.h"
+#include "rte_member_vbf.h"
+
+/*
+ * vBF currently implemented as a big array.
+ * The BFs have a vertical layout. Bits in same location of all bfs will stay
+ * in the same cache line.
+ * For example, if we have 32 bloom filters, we use a uint32_t array to
+ * represent all of them. array[0] represent the first location of all the
+ * bloom filters, array[1] represents the second location of all the
+ * bloom filters, etc. The advantage of this layout is to minimize the average
+ * number of memory accesses to test all bloom filters.
+ *
+ * Currently the implementation supports vBF containing 1,2,4,8,16,32 BFs.
+ */
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params)
+{
+
+ if (params->num_set > RTE_MEMBER_MAX_BF ||
+ !rte_is_power_of_2(params->num_set) ||
+ params->num_keys == 0 ||
+ params->false_positive_rate == 0 ||
+ params->false_positive_rate > 1) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "Membership vBF create with invalid parameters\n");
+ return -EINVAL;
+ }
+
+ /* We assume expected keys evenly distribute to all BFs */
+ uint32_t num_keys_per_bf = 1 + (params->num_keys - 1) / ss->num_set;
+
+ /*
+ * Note that the false positive rate is for all BFs in the vBF
+ * such that the single BF's false positive rate needs to be
+ * calculated.
+ * Assume each BF's False positive rate is fp_one_bf. The total false
+ * positive rate is fp = 1-(1-fp_one_bf)^n.
+ * => fp_one_bf = 1 - (1-fp)^(1/n)
+ */
+
+ float fp_one_bf = 1 - pow((1 - params->false_positive_rate),
+ 1.0 / ss->num_set);
+
+ if (fp_one_bf == 0) {
+ rte_errno = EINVAL;
+ RTE_MEMBER_LOG(ERR, "Membership BF false positive rate is too small\n");
+ return -EINVAL;
+ }
+
+ uint32_t bits = ceil((num_keys_per_bf *
+ log(fp_one_bf)) /
+ log(1.0 / (pow(2.0, log(2.0)))));
+
+ /* We round to power of 2 for performance during lookup */
+ ss->bits = rte_align32pow2(bits);
+
+ ss->num_hashes = (uint32_t)(log(2.0) * bits / num_keys_per_bf);
+ ss->bit_mask = ss->bits - 1;
+
+ /*
+ * Since we round the bits to power of 2, the final false positive
+ * rate will probably not be same as the user specified. We log the
+ * new value as debug message.
+ */
+ float new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ ss->num_hashes)), ss->num_hashes);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ /*
+ * Reduce hash function count, until we approach the user specified
+ * false-positive rate. Otherwise it is too conservative
+ */
+ int tmp_num_hash = ss->num_hashes;
+
+ while (tmp_num_hash > 1) {
+ float tmp_fp = new_fp;
+
+ tmp_num_hash--;
+ new_fp = pow((1 - pow((1 - 1.0 / ss->bits), num_keys_per_bf *
+ tmp_num_hash)), tmp_num_hash);
+ new_fp = 1 - pow((1 - new_fp), ss->num_set);
+
+ if (new_fp > params->false_positive_rate) {
+ new_fp = tmp_fp;
+ tmp_num_hash++;
+ break;
+ }
+ }
+
+ ss->num_hashes = tmp_num_hash;
+
+ /*
+ * To avoid multiplication and division:
+ * mul_shift is used for multiplication shift during bit test
+ * div_shift is used for division shift, to be divided by number of bits
+ * represented by a uint32_t variable
+ */
+ ss->mul_shift = __builtin_ctzl(ss->num_set);
+ ss->div_shift = __builtin_ctzl(32 >> ss->mul_shift);
+
+ RTE_MEMBER_LOG(DEBUG, "vector bloom filter created, "
+ "each bloom filter expects %u keys, needs %u bits, %u hashes, "
+ "with false positive rate set as %.5f, "
+ "The new calculated vBF false positive rate is %.5f\n",
+ num_keys_per_bf, ss->bits, ss->num_hashes, fp_one_bf, new_fp);
+
+ ss->table = rte_zmalloc_socket(NULL, ss->num_set * (ss->bits >> 3),
+ RTE_CACHE_LINE_SIZE, ss->socket_id);
+ if (ss->table == NULL)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static inline uint32_t
+test_bit(uint32_t bit_loc, const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t n = ss->num_set;
+ uint32_t div_shift = ss->div_shift;
+ uint32_t mul_shift = ss->mul_shift;
+ /*
+ * a is how many bits in one BF are represented by one 32bit
+ * variable.
+ */
+ uint32_t a = 32 >> mul_shift;
+ /*
+ * x>>b is the divide, x & (a-1) is the mod, & (1<<n-1) to mask out bits
+ * we do not need
+ */
+ return (vbf[bit_loc >> div_shift] >>
+ ((bit_loc & (a - 1)) << mul_shift)) & ((1ULL << n) - 1);
+}
+
+static inline void
+set_bit(uint32_t bit_loc, const struct rte_member_setsum *ss, int32_t set)
+{
+ uint32_t *vbf = ss->table;
+ uint32_t div_shift = ss->div_shift;
+ uint32_t mul_shift = ss->mul_shift;
+ uint32_t a = 32 >> mul_shift;
+
+ vbf[bit_loc >> div_shift] |=
+ 1UL << (((bit_loc & (a - 1)) << mul_shift) + set - 1);
+}
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *ss, const void *key,
+ member_set_t *set_id)
+{
+ uint32_t j;
+ uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ uint32_t mask = ~0;
+ uint32_t bit_loc;
+
+ for (j = 0; j < ss->num_hashes; j++) {
+ bit_loc = (h1 + j * h2) & ss->bit_mask;
+ mask &= test_bit(bit_loc, ss);
+ }
+
+ if (mask) {
+ *set_id = __builtin_ctzl(mask) + 1;
+ return 1;
+ }
+
+ *set_id = RTE_MEMBER_NO_MATCH;
+ return 0;
+}
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, member_set_t *set_ids)
+{
+ uint32_t i, k;
+ uint32_t num_matches = 0;
+ uint32_t mask[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t h1[RTE_MEMBER_LOOKUP_BULK_MAX], h2[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t bit_loc;
+
+ for (i = 0; i < num_keys; i++)
+ h1[i] = MEMBER_HASH_FUNC(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ for (i = 0; i < num_keys; i++)
+ h2[i] = MEMBER_HASH_FUNC(&h1[i], sizeof(uint32_t),
+ ss->sec_hash_seed);
+ for (i = 0; i < num_keys; i++) {
+ mask[i] = ~0;
+ for (k = 0; k < ss->num_hashes; k++) {
+ bit_loc = (h1[i] + k * h2[i]) & ss->bit_mask;
+ mask[i] &= test_bit(bit_loc, ss);
+ }
+ }
+ for (i = 0; i < num_keys; i++) {
+ if (mask[i]) {
+ set_ids[i] = __builtin_ctzl(mask[i]) + 1;
+ num_matches++;
+ } else
+ set_ids[i] = RTE_MEMBER_NO_MATCH;
+ }
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *ss,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t num_matches = 0;
+ uint32_t j;
+ uint32_t h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ uint32_t h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t),
+ ss->sec_hash_seed);
+ uint32_t mask = ~0;
+ uint32_t bit_loc;
+
+ for (j = 0; j < ss->num_hashes; j++) {
+ bit_loc = (h1 + j * h2) & ss->bit_mask;
+ mask &= test_bit(bit_loc, ss);
+ }
+ while (mask) {
+ uint32_t loc = __builtin_ctzl(mask);
+ set_id[num_matches] = loc + 1;
+ num_matches++;
+ if (num_matches >= match_per_key)
+ return num_matches;
+ mask &= ~(1UL << loc);
+ }
+ return num_matches;
+}
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *ss,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids)
+{
+ uint32_t i, k;
+ uint32_t num_matches = 0;
+ uint32_t match_cnt_t;
+ uint32_t mask[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t h1[RTE_MEMBER_LOOKUP_BULK_MAX], h2[RTE_MEMBER_LOOKUP_BULK_MAX];
+ uint32_t bit_loc;
+
+ for (i = 0; i < num_keys; i++)
+ h1[i] = MEMBER_HASH_FUNC(keys[i], ss->key_len,
+ ss->prim_hash_seed);
+ for (i = 0; i < num_keys; i++)
+ h2[i] = MEMBER_HASH_FUNC(&h1[i], sizeof(uint32_t),
+ ss->sec_hash_seed);
+ for (i = 0; i < num_keys; i++) {
+ mask[i] = ~0;
+ for (k = 0; k < ss->num_hashes; k++) {
+ bit_loc = (h1[i] + k * h2[i]) & ss->bit_mask;
+ mask[i] &= test_bit(bit_loc, ss);
+ }
+ }
+ for (i = 0; i < num_keys; i++) {
+ match_cnt_t = 0;
+ while (mask[i]) {
+ uint32_t loc = __builtin_ctzl(mask[i]);
+ set_ids[i * match_per_key + match_cnt_t] = loc + 1;
+ match_cnt_t++;
+ if (match_cnt_t >= match_per_key)
+ break;
+ mask[i] &= ~(1UL << loc);
+ }
+ match_count[i] = match_cnt_t;
+ if (match_cnt_t != 0)
+ num_matches++;
+ }
+ return num_matches;
+}
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *ss,
+ const void *key, member_set_t set_id)
+{
+ uint32_t i, h1, h2;
+ uint32_t bit_loc;
+
+ if (set_id > ss->num_set || set_id == RTE_MEMBER_NO_MATCH)
+ return -EINVAL;
+
+ h1 = MEMBER_HASH_FUNC(key, ss->key_len, ss->prim_hash_seed);
+ h2 = MEMBER_HASH_FUNC(&h1, sizeof(uint32_t), ss->sec_hash_seed);
+
+ for (i = 0; i < ss->num_hashes; i++) {
+ bit_loc = (h1 + i * h2) & ss->bit_mask;
+ set_bit(bit_loc, ss, set_id);
+ }
+ return 0;
+}
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss)
+{
+ rte_free(ss->table);
+}
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *ss)
+{
+ uint32_t *vbf = ss->table;
+ memset(vbf, 0, (ss->num_set * ss->bits) >> 3);
+}
diff --git a/lib/librte_member/rte_member_vbf.h b/lib/librte_member/rte_member_vbf.h
new file mode 100644
index 0000000..5bc158b
--- /dev/null
+++ b/lib/librte_member/rte_member_vbf.h
@@ -0,0 +1,82 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_VBF_H_
+#define _RTE_MEMBER_VBF_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Currently we only support up to 32 sets in vBF */
+#define RTE_MEMBER_MAX_BF 32
+
+int
+rte_member_create_vbf(struct rte_member_setsum *ss,
+ const struct rte_member_parameters *params);
+
+int
+rte_member_lookup_vbf(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys,
+ member_set_t *set_ids);
+
+uint32_t
+rte_member_lookup_multi_vbf(const struct rte_member_setsum *setsum,
+ const void *key, uint32_t match_per_key,
+ member_set_t *set_id);
+
+uint32_t
+rte_member_lookup_multi_bulk_vbf(const struct rte_member_setsum *setsum,
+ const void **keys, uint32_t num_keys, uint32_t match_per_key,
+ uint32_t *match_count,
+ member_set_t *set_ids);
+
+int
+rte_member_add_vbf(const struct rte_member_setsum *setsum,
+ const void *key, member_set_t set_id);
+
+void
+rte_member_free_vbf(struct rte_member_setsum *ss);
+
+void
+rte_member_reset_vbf(const struct rte_member_setsum *setsum);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_VBF_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v6 4/7] member: add AVX for HT mode
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
` (2 preceding siblings ...)
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 3/7] member: implement vBF mode Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 5/7] member: enable the library Yipeng Wang
` (3 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
For key search, the signatures of all entries are compared against
the signature of the key that is being looked up. Since all
signatures are contiguously put in a bucket, they can be compared
with vector instructions (AVX2), achieving higher lookup performance.
This patch adds AVX2 implementation in a separate header file.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
lib/librte_member/rte_member_ht.c | 142 +++++++++++++++++++++++++++++--------
lib/librte_member/rte_member_x86.h | 107 ++++++++++++++++++++++++++++
2 files changed, 218 insertions(+), 31 deletions(-)
create mode 100644 lib/librte_member/rte_member_x86.h
diff --git a/lib/librte_member/rte_member_ht.c b/lib/librte_member/rte_member_ht.c
index e038987..59332d5 100644
--- a/lib/librte_member/rte_member_ht.c
+++ b/lib/librte_member/rte_member_ht.c
@@ -40,6 +40,10 @@
#include "rte_member.h"
#include "rte_member_ht.h"
+#if defined(RTE_ARCH_X86)
+#include "rte_member_x86.h"
+#endif
+
/* Search bucket for entry with tmp_sig and update set_id */
static inline int
update_entry_search(uint32_t bucket_id, member_sig_t tmp_sig,
@@ -136,6 +140,13 @@ rte_member_create_ht(struct rte_member_setsum *ss,
for (j = 0; j < RTE_MEMBER_BUCKET_ENTRIES; j++)
buckets[i].sets[j] = RTE_MEMBER_NO_MATCH;
}
+#if defined(RTE_ARCH_X86)
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2) &&
+ RTE_MEMBER_BUCKET_ENTRIES == 16)
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_AVX2;
+ else
+#endif
+ ss->sig_cmp_fn = RTE_MEMBER_COMPARE_SCALAR;
RTE_MEMBER_LOG(DEBUG, "Hash table based filter created, "
"the table has %u entries, %u buckets\n",
@@ -193,11 +204,23 @@ rte_member_lookup_ht(const struct rte_member_setsum *ss,
*set_id = RTE_MEMBER_NO_MATCH;
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- if (search_bucket_single(prim_bucket, tmp_sig, buckets,
- set_id) ||
- search_bucket_single(sec_bucket, tmp_sig,
- buckets, set_id))
- return 1;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single_avx(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_bucket, tmp_sig, buckets,
+ set_id) ||
+ search_bucket_single(sec_bucket, tmp_sig,
+ buckets, set_id))
+ return 1;
+ }
return 0;
}
@@ -221,13 +244,27 @@ rte_member_lookup_bulk_ht(const struct rte_member_setsum *ss,
}
for (i = 0; i < num_keys; i++) {
- if (search_bucket_single(prim_buckets[i], tmp_sig[i],
- buckets, &set_id[i]) ||
- search_bucket_single(sec_buckets[i],
- tmp_sig[i], buckets, &set_id[i]))
- num_matches++;
- else
- set_id[i] = RTE_MEMBER_NO_MATCH;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (search_bucket_single_avx(prim_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]) ||
+ search_bucket_single_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ num_matches++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ break;
+#endif
+ default:
+ if (search_bucket_single(prim_buckets[i], tmp_sig[i],
+ buckets, &set_id[i]) ||
+ search_bucket_single(sec_buckets[i],
+ tmp_sig[i], buckets, &set_id[i]))
+ num_matches++;
+ else
+ set_id[i] = RTE_MEMBER_NO_MATCH;
+ }
}
return num_matches;
}
@@ -244,12 +281,24 @@ rte_member_lookup_multi_ht(const struct rte_member_setsum *ss,
get_buckets_index(ss, key, &prim_bucket, &sec_bucket, &tmp_sig);
- search_bucket_multi(prim_bucket, tmp_sig, buckets, &num_matches,
- match_per_key, set_id);
- if (num_matches < match_per_key)
- search_bucket_multi(sec_bucket, tmp_sig,
- buckets, &num_matches, match_per_key, set_id);
- return num_matches;
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_bucket, tmp_sig, buckets,
+ &num_matches, match_per_key, set_id);
+ if (num_matches < match_per_key)
+ search_bucket_multi_avx(sec_bucket, tmp_sig,
+ buckets, &num_matches, match_per_key, set_id);
+ return num_matches;
+#endif
+ default:
+ search_bucket_multi(prim_bucket, tmp_sig, buckets, &num_matches,
+ match_per_key, set_id);
+ if (num_matches < match_per_key)
+ search_bucket_multi(sec_bucket, tmp_sig,
+ buckets, &num_matches, match_per_key, set_id);
+ return num_matches;
+ }
}
uint32_t
@@ -275,16 +324,34 @@ rte_member_lookup_multi_bulk_ht(const struct rte_member_setsum *ss,
for (i = 0; i < num_keys; i++) {
match_cnt_tmp = 0;
- search_bucket_multi(prim_buckets[i], tmp_sig[i],
- buckets, &match_cnt_tmp, match_per_key,
- &set_ids[i*match_per_key]);
- if (match_cnt_tmp < match_per_key)
- search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ switch (ss->sig_cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ search_bucket_multi_avx(prim_buckets[i], tmp_sig[i],
buckets, &match_cnt_tmp, match_per_key,
&set_ids[i*match_per_key]);
- match_count[i] = match_cnt_tmp;
- if (match_cnt_tmp != 0)
- num_matches++;
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi_avx(sec_buckets[i],
+ tmp_sig[i], buckets, &match_cnt_tmp,
+ match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ num_matches++;
+ break;
+#endif
+ default:
+ search_bucket_multi(prim_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ if (match_cnt_tmp < match_per_key)
+ search_bucket_multi(sec_buckets[i], tmp_sig[i],
+ buckets, &match_cnt_tmp, match_per_key,
+ &set_ids[i*match_per_key]);
+ match_count[i] = match_cnt_tmp;
+ if (match_cnt_tmp != 0)
+ num_matches++;
+ }
}
return num_matches;
}
@@ -315,11 +382,24 @@ try_insert(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
static inline int
try_update(struct member_ht_bucket *buckets, uint32_t prim, uint32_t sec,
- member_sig_t sig, member_set_t set_id)
+ member_sig_t sig, member_set_t set_id,
+ enum rte_member_sig_compare_function cmp_fn)
{
- if (update_entry_search(prim, sig, buckets, set_id) ||
- update_entry_search(sec, sig, buckets, set_id))
- return 0;
+ switch (cmp_fn) {
+#if defined(RTE_ARCH_X86) && defined(RTE_MACHINE_CPUFLAG_AVX2)
+ case RTE_MEMBER_COMPARE_AVX2:
+ if (update_entry_search_avx(prim, sig, buckets, set_id) ||
+ update_entry_search_avx(sec, sig, buckets,
+ set_id))
+ return 0;
+ break;
+#endif
+ default:
+ if (update_entry_search(prim, sig, buckets, set_id) ||
+ update_entry_search(sec, sig, buckets,
+ set_id))
+ return 0;
+ }
return -1;
}
@@ -430,7 +510,7 @@ rte_member_add_ht(const struct rte_member_setsum *ss,
*/
if (ss->cache) {
ret = try_update(buckets, prim_bucket, sec_bucket, tmp_sig,
- set_id);
+ set_id, ss->sig_cmp_fn);
if (ret != -1)
return ret;
}
diff --git a/lib/librte_member/rte_member_x86.h b/lib/librte_member/rte_member_x86.h
new file mode 100644
index 0000000..d29dd3f
--- /dev/null
+++ b/lib/librte_member/rte_member_x86.h
@@ -0,0 +1,107 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_MEMBER_X86_H_
+#define _RTE_MEMBER_X86_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <x86intrin.h>
+
+#if defined(RTE_MACHINE_CPUFLAG_AVX2)
+
+static inline int
+update_entry_search_avx(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket_id].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ if (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ buckets[bucket_id].sets[hit_idx] = set_id;
+ return 1;
+ }
+ return 0;
+}
+
+static inline int
+search_bucket_single_avx(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ member_set_t *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket_id].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ if (buckets[bucket_id].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ *set_id = buckets[bucket_id].sets[hit_idx];
+ return 1;
+ }
+ hitmask &= ~(3U << ((hit_idx) << 1));
+ }
+ return 0;
+}
+
+static inline void
+search_bucket_multi_avx(uint32_t bucket_id, member_sig_t tmp_sig,
+ struct member_ht_bucket *buckets,
+ uint32_t *counter,
+ uint32_t match_per_key,
+ member_set_t *set_id)
+{
+ uint32_t hitmask = _mm256_movemask_epi8((__m256i)_mm256_cmpeq_epi16(
+ _mm256_load_si256((__m256i const *)buckets[bucket_id].sigs),
+ _mm256_set1_epi16(tmp_sig)));
+ while (hitmask) {
+ uint32_t hit_idx = __builtin_ctzl(hitmask) >> 1;
+ if (buckets[bucket_id].sets[hit_idx] != RTE_MEMBER_NO_MATCH) {
+ set_id[*counter] = buckets[bucket_id].sets[hit_idx];
+ (*counter)++;
+ if (*counter >= match_per_key)
+ return;
+ }
+ hitmask &= ~(3U << ((hit_idx) << 1));
+ }
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* _RTE_MEMBER_X86_H_ */
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v6 5/7] member: enable the library
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
` (3 preceding siblings ...)
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 4/7] member: add AVX for HT mode Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 6/7] test/member: add functional and perf tests Yipeng Wang
` (2 subsequent siblings)
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
This patch enables the Membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
MAINTAINERS | 6 +++++-
config/common_base | 5 +++++
lib/librte_member/Makefile | 2 ++
mk/rte.app.mk | 2 ++
4 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 8df2a7f..1358e47 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -729,6 +729,11 @@ F: test/test/test_lpm*
F: test/test/test_func_reentrancy.c
F: test/test/test_xmmt_ops.h
+Membership - EXPERIMENTAL
+M: Yipeng Wang <yipeng1.wang@intel.com>
+M: Sameh Gobriel <sameh.gobriel@intel.com>
+F: lib/librte_member/
+
Traffic metering
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
F: lib/librte_meter/
@@ -737,7 +742,6 @@ F: test/test/test_meter.c
F: examples/qos_meter/
F: doc/guides/sample_app_ug/qos_metering.rst
-
Other libraries
---------------
diff --git a/config/common_base b/config/common_base
index 12f6be9..e305e9f 100644
--- a/config/common_base
+++ b/config/common_base
@@ -596,6 +596,11 @@ CONFIG_RTE_LIBRTE_HASH_DEBUG=n
CONFIG_RTE_LIBRTE_EFD=y
#
+# Compile librte_member
+#
+CONFIG_RTE_LIBRTE_MEMBER=y
+
+#
# Compile librte_jobstats
#
CONFIG_RTE_LIBRTE_JOBSTATS=y
diff --git a/lib/librte_member/Makefile b/lib/librte_member/Makefile
index 50275ed..3bac1d0 100644
--- a/lib/librte_member/Makefile
+++ b/lib/librte_member/Makefile
@@ -37,6 +37,8 @@ LIB = librte_member.a
CFLAGS := -I$(SRCDIR) $(CFLAGS)
CFLAGS += $(WERROR_FLAGS) -O3
+LDLIBS += -lm
+
EXPORT_MAP := rte_member_version.map
LIBABIVER := 1
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index c25fdd9..c79acf0 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -86,6 +86,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_CFGFILE) += -lrte_cfgfile
_LDLIBS-y += --whole-archive
_LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost
_LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs
_LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf
@@ -196,6 +197,7 @@ endif
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lm
_LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrt
_LDLIBS-$(CONFIG_RTE_LIBRTE_METER) += -lm
+_LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lm
ifeq ($(CONFIG_RTE_LIBRTE_VHOST_NUMA),y)
_LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lnuma
endif
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v6 6/7] test/member: add functional and perf tests
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
` (4 preceding siblings ...)
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 5/7] member: enable the library Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 7/7] doc: add membership documentation Yipeng Wang
2017-10-08 22:14 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Thomas Monjalon
7 siblings, 0 replies; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
This patch adds functional and performance tests for membership
library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
MAINTAINERS | 1 +
test/test/Makefile | 3 +
test/test/test_member.c | 744 +++++++++++++++++++++++++++++++++++++++++++
test/test/test_member_perf.c | 654 +++++++++++++++++++++++++++++++++++++
4 files changed, 1402 insertions(+)
create mode 100644 test/test/test_member.c
create mode 100644 test/test/test_member_perf.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 1358e47..afd36cf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -733,6 +733,7 @@ Membership - EXPERIMENTAL
M: Yipeng Wang <yipeng1.wang@intel.com>
M: Sameh Gobriel <sameh.gobriel@intel.com>
F: lib/librte_member/
+F: test/test/test_member*
Traffic metering
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
diff --git a/test/test/Makefile b/test/test/Makefile
index 42d9a49..b61dde3 100644
--- a/test/test/Makefile
+++ b/test/test/Makefile
@@ -123,6 +123,9 @@ SRCS-y += test_logs.c
SRCS-y += test_memcpy.c
SRCS-y += test_memcpy_perf.c
+
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member.c
+SRCS-$(CONFIG_RTE_LIBRTE_MEMBER) += test_member_perf.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd.c
SRCS-$(CONFIG_RTE_LIBRTE_EFD) += test_efd_perf.c
diff --git a/test/test/test_member.c b/test/test/test_member.c
new file mode 100644
index 0000000..02375fd
--- /dev/null
+++ b/test/test/test_member.c
@@ -0,0 +1,744 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* This test is for membership library's simple feature test */
+
+#include <rte_memcpy.h>
+#include <rte_malloc.h>
+#include <rte_member.h>
+#include <rte_byteorder.h>
+#include <rte_random.h>
+#include <rte_debug.h>
+#include <rte_ip.h>
+
+#include "test.h"
+
+struct rte_member_setsum *setsum_ht;
+struct rte_member_setsum *setsum_cache;
+struct rte_member_setsum *setsum_vbf;
+
+/* 5-tuple key type */
+struct flow_key {
+ uint32_t ip_src;
+ uint32_t ip_dst;
+ uint16_t port_src;
+ uint16_t port_dst;
+ uint8_t proto;
+} __attribute__((packed));
+
+/* Set ID Macros for multimatch test usage */
+#define M_MATCH_S 1 /* Not start with 0 since by default 0 means no match */
+#define M_MATCH_E 15
+#define M_MATCH_STEP 2
+#define M_MATCH_CNT \
+ (1 + (M_MATCH_E - M_MATCH_S) / M_MATCH_STEP)
+
+
+#define NUM_SAMPLES 5
+#define MAX_MATCH 32
+
+/* Keys used by unit test functions */
+static struct flow_key keys[NUM_SAMPLES] = {
+ {
+ .ip_src = IPv4(0x03, 0x02, 0x01, 0x00),
+ .ip_dst = IPv4(0x07, 0x06, 0x05, 0x04),
+ .port_src = 0x0908,
+ .port_dst = 0x0b0a,
+ .proto = 0x0c,
+ },
+ {
+ .ip_src = IPv4(0x13, 0x12, 0x11, 0x10),
+ .ip_dst = IPv4(0x17, 0x16, 0x15, 0x14),
+ .port_src = 0x1918,
+ .port_dst = 0x1b1a,
+ .proto = 0x1c,
+ },
+ {
+ .ip_src = IPv4(0x23, 0x22, 0x21, 0x20),
+ .ip_dst = IPv4(0x27, 0x26, 0x25, 0x24),
+ .port_src = 0x2928,
+ .port_dst = 0x2b2a,
+ .proto = 0x2c,
+ },
+ {
+ .ip_src = IPv4(0x33, 0x32, 0x31, 0x30),
+ .ip_dst = IPv4(0x37, 0x36, 0x35, 0x34),
+ .port_src = 0x3938,
+ .port_dst = 0x3b3a,
+ .proto = 0x3c,
+ },
+ {
+ .ip_src = IPv4(0x43, 0x42, 0x41, 0x40),
+ .ip_dst = IPv4(0x47, 0x46, 0x45, 0x44),
+ .port_src = 0x4948,
+ .port_dst = 0x4b4a,
+ .proto = 0x4c,
+ }
+};
+
+uint32_t test_set[NUM_SAMPLES] = {1, 2, 3, 4, 5};
+
+#define ITERATIONS 3
+#define KEY_SIZE 4
+
+#define MAX_ENTRIES (1 << 16)
+uint8_t generated_keys[MAX_ENTRIES][KEY_SIZE];
+
+static struct rte_member_parameters params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+
+ /* num_set and false_positive_rate only relevant to vBF */
+ .num_set = 16,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+};
+
+/*
+ * Sequence of operations for find existing setsummary
+ *
+ * - create setsum
+ * - find existing setsum: hit
+ * - find non-existing setsum: miss
+ *
+ */
+static int
+test_member_find_existing(void)
+{
+ struct rte_member_setsum *tmp_setsum = NULL, *result = NULL;
+ struct rte_member_parameters tmp_params = {
+ .name = "member_find_existing",
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ /* Create */
+ tmp_setsum = rte_member_create(&tmp_params);
+ TEST_ASSERT(tmp_setsum != NULL, "setsum creation failed");
+
+ /* Try to find existing hash table */
+ result = rte_member_find_existing("member_find_existing");
+ TEST_ASSERT(result == tmp_setsum, "could not find existing setsum");
+
+ /* Try to find non-existing hash table */
+ result = rte_member_find_existing("member_find_non_existing");
+ TEST_ASSERT(result == NULL, "found setsum that shouldn't exist");
+
+ /* Cleanup. */
+ rte_member_free(tmp_setsum);
+
+ return 0;
+}
+
+/*
+ * Test for bad creating parameters
+ */
+static int
+test_member_create_bad_param(void)
+{
+ struct rte_member_setsum *bad_setsum = NULL;
+ struct rte_member_parameters bad_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = KEY_SIZE, /* Length of hash key. */
+ .type = RTE_MEMBER_TYPE_HT,
+ .num_set = 32,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 1,
+ .sec_hash_seed = 11,
+ .socket_id = 0 /* NUMA Socket ID for memory. */
+ };
+
+ printf("Expected error section begin...\n");
+ bad_params.name = "bad_param1";
+ bad_params.num_set = 0;
+ bad_params.type = RTE_MEMBER_TYPE_VBF;
+ /* Test with 0 set for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "number of set for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param2";
+ bad_params.false_positive_rate = 0;
+ bad_params.num_set = 32;
+ /* Test with 0 false positive for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "false positive rate for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param3";
+ bad_params.false_positive_rate = 0.03;
+ bad_params.num_keys = 0;
+ /* Test with 0 key per BF for vBF should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with invalid "
+ "num_keys for vBF\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param4";
+ bad_params.type = RTE_MEMBER_TYPE_HT;
+ bad_params.num_keys = RTE_MEMBER_BUCKET_ENTRIES / 2;
+ /* Test with less than 1 bucket for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with too few "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ bad_params.num_keys = RTE_MEMBER_ENTRIES_MAX + 1;
+ /* Test with more than maximum entries for HTSS should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with to many "
+ "number of keys(entries) for HT\n");
+ return -1;
+ }
+
+ bad_params.name = "bad_param5";
+ /* Test with same name should fail */
+ bad_setsum = rte_member_create(&bad_params);
+ if (bad_setsum != NULL) {
+ rte_member_free(bad_setsum);
+ printf("Impossible creating setsum successfully with existed "
+ "name\n");
+ return -1;
+ }
+ printf("Expected error section end...\n");
+ rte_member_free(bad_setsum);
+ return 0;
+}
+
+/* Create test setsummaries. */
+static int test_member_create(void)
+{
+ params.key_len = sizeof(struct flow_key);
+
+ params.name = "test_member_ht";
+ params.is_cache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.is_cache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+ params.name = "test_member_vbf";
+ params.type = RTE_MEMBER_TYPE_VBF;
+ setsum_vbf = rte_member_create(¶ms);
+
+ if (setsum_ht == NULL || setsum_cache == NULL || setsum_vbf == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ printf("Creation of setsums success\n");
+ return 0;
+}
+
+static int test_member_insert(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_add(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[i], test_set[i]);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "insert error");
+ }
+ printf("insert key success\n");
+ return 0;
+}
+
+static int test_member_lookup(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ member_set_t set_ids_ht[NUM_SAMPLES] = {0};
+ member_set_t set_ids_cache[NUM_SAMPLES] = {0};
+ member_set_t set_ids_vbf[NUM_SAMPLES] = {0};
+
+ uint32_t num_key_ht = NUM_SAMPLES;
+ uint32_t num_key_cache = NUM_SAMPLES;
+ uint32_t num_key_vbf = NUM_SAMPLES;
+
+ const void *key_array[NUM_SAMPLES];
+
+ /* Single lookup test */
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "single lookup function error");
+
+ TEST_ASSERT(set_ht == test_set[i] &&
+ set_cache == test_set[i] &&
+ set_vbf == test_set[i],
+ "single lookup set value error");
+ }
+ printf("lookup single key success\n");
+
+ /* Bulk lookup test */
+ for (i = 0; i < NUM_SAMPLES; i++)
+ key_array[i] = &keys[i];
+
+ ret_ht = rte_member_lookup_bulk(setsum_ht, key_array,
+ num_key_ht, set_ids_ht);
+
+ ret_cache = rte_member_lookup_bulk(setsum_cache, key_array,
+ num_key_cache, set_ids_cache);
+
+ ret_vbf = rte_member_lookup_bulk(setsum_vbf, key_array,
+ num_key_vbf, set_ids_vbf);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "bulk lookup function error");
+
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ TEST_ASSERT((set_ids_ht[i] == test_set[i]) &&
+ (set_ids_cache[i] == test_set[i]) &&
+ (set_ids_vbf[i] == test_set[i]),
+ "bulk lookup result error");
+ }
+
+ return 0;
+}
+
+static int test_member_delete(void)
+{
+ int ret_ht, ret_cache, ret_vbf, i;
+ uint16_t set_ht, set_cache, set_vbf;
+ const void *key_array[NUM_SAMPLES];
+ member_set_t set_ids_ht[NUM_SAMPLES] = {0};
+ member_set_t set_ids_cache[NUM_SAMPLES] = {0};
+ member_set_t set_ids_vbf[NUM_SAMPLES] = {0};
+ uint32_t num_key_ht = NUM_SAMPLES;
+ uint32_t num_key_cache = NUM_SAMPLES;
+ uint32_t num_key_vbf = NUM_SAMPLES;
+
+ /* Delete part of all inserted keys */
+ for (i = 0; i < NUM_SAMPLES / 2; i++) {
+ ret_ht = rte_member_delete(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_delete(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_delete(setsum_vbf, &keys[i], test_set[i]);
+ /* VBF does not support delete yet, so return error code */
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key deletion function error");
+ TEST_ASSERT(ret_vbf < 0,
+ "vbf does not support deletion, error");
+ }
+
+ for (i = 0; i < NUM_SAMPLES; i++)
+ key_array[i] = &keys[i];
+
+ ret_ht = rte_member_lookup_bulk(setsum_ht, key_array,
+ num_key_ht, set_ids_ht);
+
+ ret_cache = rte_member_lookup_bulk(setsum_cache, key_array,
+ num_key_cache, set_ids_cache);
+
+ ret_vbf = rte_member_lookup_bulk(setsum_vbf, key_array,
+ num_key_vbf, set_ids_vbf);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0 && ret_vbf >= 0,
+ "bulk lookup function error");
+
+ for (i = 0; i < NUM_SAMPLES / 2; i++) {
+ TEST_ASSERT((set_ids_ht[i] == RTE_MEMBER_NO_MATCH) &&
+ (set_ids_cache[i] == RTE_MEMBER_NO_MATCH),
+ "bulk lookup result error");
+ }
+
+ for (i = NUM_SAMPLES / 2; i < NUM_SAMPLES; i++) {
+ TEST_ASSERT((set_ids_ht[i] == test_set[i]) &&
+ (set_ids_cache[i] == test_set[i]) &&
+ (set_ids_vbf[i] == test_set[i]),
+ "bulk lookup result error");
+ }
+
+ /* Delete the left of inserted keys */
+ for (i = NUM_SAMPLES / 2; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_delete(setsum_ht, &keys[i], test_set[i]);
+ ret_cache = rte_member_delete(setsum_cache, &keys[i],
+ test_set[i]);
+ ret_vbf = rte_member_delete(setsum_vbf, &keys[i], test_set[i]);
+ /* VBF does not support delete yet, so return error code */
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key deletion function error");
+ TEST_ASSERT(ret_vbf < 0,
+ "vbf does not support deletion, error");
+ }
+
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_ht = rte_member_lookup(setsum_ht, &keys[i], &set_ht);
+ ret_cache = rte_member_lookup(setsum_cache, &keys[i],
+ &set_cache);
+ ret_vbf = rte_member_lookup(setsum_vbf, &keys[i], &set_vbf);
+ TEST_ASSERT(ret_ht >= 0 && ret_cache >= 0,
+ "key lookup function error");
+ TEST_ASSERT(set_ht == RTE_MEMBER_NO_MATCH &&
+ ret_cache == RTE_MEMBER_NO_MATCH,
+ "key deletion failed");
+ }
+ /* Reset vbf for other following tests */
+ rte_member_reset(setsum_vbf);
+
+ printf("delete success\n");
+ return 0;
+}
+
+static int test_member_multimatch(void)
+{
+ int ret_ht, ret_vbf, ret_cache;
+ member_set_t set_ids_ht[MAX_MATCH] = {0};
+ member_set_t set_ids_vbf[MAX_MATCH] = {0};
+ member_set_t set_ids_cache[MAX_MATCH] = {0};
+
+ member_set_t set_ids_ht_m[NUM_SAMPLES][MAX_MATCH] = {{0} };
+ member_set_t set_ids_vbf_m[NUM_SAMPLES][MAX_MATCH] = {{0} };
+ member_set_t set_ids_cache_m[NUM_SAMPLES][MAX_MATCH] = {{0} };
+
+ uint32_t match_count_ht[NUM_SAMPLES];
+ uint32_t match_count_vbf[NUM_SAMPLES];
+ uint32_t match_count_cache[NUM_SAMPLES];
+
+ uint32_t num_key_ht = NUM_SAMPLES;
+ uint32_t num_key_vbf = NUM_SAMPLES;
+ uint32_t num_key_cache = NUM_SAMPLES;
+
+ const void *key_array[NUM_SAMPLES];
+
+ uint32_t i, j;
+
+ /* Same key at most inserted 2*entry_per_bucket times for HT mode */
+ for (i = M_MATCH_S; i <= M_MATCH_E; i += M_MATCH_STEP) {
+ for (j = 0; j < NUM_SAMPLES; j++) {
+ ret_ht = rte_member_add(setsum_ht, &keys[j], i);
+ ret_vbf = rte_member_add(setsum_vbf, &keys[j], i);
+ ret_cache = rte_member_add(setsum_cache, &keys[j], i);
+
+ TEST_ASSERT(ret_ht >= 0 && ret_vbf >= 0 &&
+ ret_cache >= 0,
+ "insert function error");
+ }
+ }
+
+ /* Single multimatch test */
+ for (i = 0; i < NUM_SAMPLES; i++) {
+ ret_vbf = rte_member_lookup_multi(setsum_vbf, &keys[i],
+ MAX_MATCH, set_ids_vbf);
+ ret_ht = rte_member_lookup_multi(setsum_ht, &keys[i],
+ MAX_MATCH, set_ids_ht);
+ ret_cache = rte_member_lookup_multi(setsum_cache, &keys[i],
+ MAX_MATCH, set_ids_cache);
+ /*
+ * For cache mode, keys overwrite when signature same.
+ * the mutimatch should work like single match.
+ */
+ TEST_ASSERT(ret_ht == M_MATCH_CNT && ret_vbf == M_MATCH_CNT &&
+ ret_cache == 1,
+ "single lookup_multi error");
+ TEST_ASSERT(set_ids_cache[0] == M_MATCH_E,
+ "single lookup_multi cache error");
+
+ for (j = 1; j <= M_MATCH_CNT; j++) {
+ TEST_ASSERT(set_ids_ht[j-1] == j * M_MATCH_STEP - 1 &&
+ set_ids_vbf[j-1] ==
+ j * M_MATCH_STEP - 1,
+ "single multimatch lookup error");
+ }
+ }
+ printf("lookup single key for multimatch success\n");
+
+ /* Bulk multimatch test */
+ for (i = 0; i < NUM_SAMPLES; i++)
+ key_array[i] = &keys[i];
+ ret_vbf = rte_member_lookup_multi_bulk(setsum_vbf,
+ &key_array[0], num_key_ht, MAX_MATCH, match_count_vbf,
+ (member_set_t *)set_ids_vbf_m);
+
+ ret_ht = rte_member_lookup_multi_bulk(setsum_ht,
+ &key_array[0], num_key_vbf, MAX_MATCH, match_count_ht,
+ (member_set_t *)set_ids_ht_m);
+
+ ret_cache = rte_member_lookup_multi_bulk(setsum_cache,
+ &key_array[0], num_key_cache, MAX_MATCH,
+ match_count_cache, (member_set_t *)set_ids_cache_m);
+
+
+ for (j = 0; j < NUM_SAMPLES; j++) {
+ TEST_ASSERT(match_count_ht[j] == M_MATCH_CNT,
+ "bulk multimatch lookup HT match count error");
+ TEST_ASSERT(match_count_vbf[j] == M_MATCH_CNT,
+ "bulk multimatch lookup vBF match count error");
+ TEST_ASSERT(match_count_cache[j] == 1,
+ "bulk multimatch lookup CACHE match count error");
+ TEST_ASSERT(set_ids_cache_m[j][0] == M_MATCH_E,
+ "bulk multimatch lookup CACHE set value error");
+
+ for (i = 1; i <= M_MATCH_CNT; i++) {
+ TEST_ASSERT(set_ids_ht_m[j][i-1] ==
+ i * M_MATCH_STEP - 1,
+ "bulk multimatch lookup HT set value error");
+ TEST_ASSERT(set_ids_vbf_m[j][i-1] ==
+ i * M_MATCH_STEP - 1,
+ "bulk multimatch lookup vBF set value error");
+ }
+ }
+
+ printf("lookup for bulk multimatch success\n");
+
+ return 0;
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, KEY_SIZE);
+}
+
+static void
+setup_keys_and_data(void)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ /* Reset all arrays */
+ for (i = 0; i < KEY_SIZE; i++)
+ generated_keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < MAX_ENTRIES; i++) {
+ for (j = 0; j < KEY_SIZE; j++)
+ generated_keys[i][j] = rte_rand() & 0xFF;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(generated_keys, MAX_ENTRIES, KEY_SIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < MAX_ENTRIES - 1; i++) {
+ if (memcmp(generated_keys[i], generated_keys[i + 1],
+ KEY_SIZE) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < KEY_SIZE; j++)
+ generated_keys[i][j] =
+ rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+}
+
+static inline int
+add_generated_keys(struct rte_member_setsum *setsum, unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret >= 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &generated_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static inline int
+add_generated_keys_cache(struct rte_member_setsum *setsum,
+ unsigned int *added_keys)
+{
+ int ret = 0;
+
+ for (*added_keys = 0; ret == 0 && *added_keys < MAX_ENTRIES;
+ (*added_keys)++) {
+ uint16_t set = (rte_rand() & 0xf) + 1;
+ ret = rte_member_add(setsum, &generated_keys[*added_keys], set);
+ }
+ return ret;
+}
+
+static int
+test_member_loadfactor(void)
+{
+ unsigned int j;
+ unsigned int added_keys, average_keys_added = 0;
+ int ret;
+
+ setup_keys_and_data();
+
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+
+ params.key_len = KEY_SIZE;
+ params.name = "test_member_ht";
+ params.is_cache = 0;
+ params.type = RTE_MEMBER_TYPE_HT;
+ setsum_ht = rte_member_create(¶ms);
+
+ params.name = "test_member_cache";
+ params.is_cache = 1;
+ setsum_cache = rte_member_create(¶ms);
+
+
+ if (setsum_ht == NULL || setsum_cache == NULL) {
+ printf("Creation of setsums fail\n");
+ return -1;
+ }
+ /* Test HT non-cache mode */
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_generated_keys(setsum_ht, &added_keys);
+ if (ret != -ENOSPC) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_ht);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when no space(non-cache) = %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+
+ /* Test cache mode */
+ added_keys = average_keys_added = 0;
+ for (j = 0; j < ITERATIONS; j++) {
+ /* Add random entries until key cannot be added */
+ ret = add_generated_keys_cache(setsum_cache, &added_keys);
+ if (ret != 1) {
+ printf("Unexpected error when adding keys\n");
+ return -1;
+ }
+ average_keys_added += added_keys;
+
+ /* Reset the table */
+ rte_member_reset(setsum_cache);
+
+ /* Print a dot to show progress on operations */
+ printf(".");
+ fflush(stdout);
+ }
+
+ average_keys_added /= ITERATIONS;
+
+ printf("\nKeys inserted when eviction happens(cache)= %.2f%% (%u/%u)\n",
+ ((double) average_keys_added / params.num_keys * 100),
+ average_keys_added, params.num_keys);
+ return 0;
+}
+
+static void
+perform_free(void)
+{
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ rte_member_free(setsum_vbf);
+}
+
+static int
+test_member(void)
+{
+ if (test_member_create_bad_param() < 0)
+ return -1;
+
+ if (test_member_find_existing() < 0)
+ return -1;
+
+ if (test_member_create() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_insert() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_lookup() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_delete() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_multimatch() < 0) {
+ perform_free();
+ return -1;
+ }
+ if (test_member_loadfactor() < 0) {
+ rte_member_free(setsum_ht);
+ rte_member_free(setsum_cache);
+ return -1;
+ }
+
+ perform_free();
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_autotest, test_member);
diff --git a/test/test/test_member_perf.c b/test/test/test_member_perf.c
new file mode 100644
index 0000000..e13066f
--- /dev/null
+++ b/test/test/test_member_perf.c
@@ -0,0 +1,654 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <stdio.h>
+#include <inttypes.h>
+
+#include <rte_lcore.h>
+#include <rte_cycles.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+#include <rte_memcpy.h>
+#include <rte_thash.h>
+#include <rte_member.h>
+
+#include "test.h"
+
+#define NUM_KEYSIZES 10
+#define NUM_SHUFFLES 10
+#define MAX_KEYSIZE 64
+#define MAX_ENTRIES (1 << 19)
+#define KEYS_TO_ADD (MAX_ENTRIES * 75 / 100) /* 75% table utilization */
+#define NUM_LOOKUPS (KEYS_TO_ADD * 5) /* Loop among keys added, several times */
+#define VBF_SET_CNT 16
+#define BURST_SIZE 64
+#define VBF_FALSE_RATE 0.03
+
+static unsigned int test_socket_id;
+
+enum sstype {
+ HT = 0,
+ CACHE,
+ VBF,
+ NUM_TYPE
+};
+
+enum operations {
+ ADD = 0,
+ LOOKUP,
+ LOOKUP_BULK,
+ LOOKUP_MULTI,
+ LOOKUP_MULTI_BULK,
+ DELETE,
+ LOOKUP_MISS,
+ NUM_OPERATIONS
+};
+
+struct member_perf_params {
+ struct rte_member_setsum *setsum[NUM_TYPE];
+ uint32_t key_size;
+ unsigned int cycle;
+};
+
+static uint32_t hashtest_key_lens[] = {
+ /* standard key sizes */
+ 4, 8, 16, 32, 48, 64,
+ /* IPv4 SRC + DST + protocol, unpadded */
+ 9,
+ /* IPv4 5-tuple, unpadded */
+ 13,
+ /* IPv6 5-tuple, unpadded */
+ 37,
+ /* IPv6 5-tuple, padded to 8-byte boundary */
+ 40
+};
+
+/* Array to store number of cycles per operation */
+uint64_t cycles[NUM_TYPE][NUM_KEYSIZES][NUM_OPERATIONS];
+uint64_t false_data[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_bulk[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi[NUM_TYPE][NUM_KEYSIZES];
+uint64_t false_data_multi_bulk[NUM_TYPE][NUM_KEYSIZES];
+
+uint64_t false_hit[NUM_TYPE][NUM_KEYSIZES];
+
+member_set_t data[NUM_TYPE][/* Array to store the data */KEYS_TO_ADD];
+
+/* Array to store all input keys */
+uint8_t keys[KEYS_TO_ADD][MAX_KEYSIZE];
+
+/* Shuffle the keys that have been added, so lookups will be totally random */
+static void
+shuffle_input_keys(struct member_perf_params *params)
+{
+ member_set_t temp_data;
+ unsigned int i, j;
+ uint32_t swap_idx;
+ uint8_t temp_key[MAX_KEYSIZE];
+
+ for (i = KEYS_TO_ADD - 1; i > 0; i--) {
+ swap_idx = rte_rand() % i;
+ memcpy(temp_key, keys[i], hashtest_key_lens[params->cycle]);
+ memcpy(keys[i], keys[swap_idx],
+ hashtest_key_lens[params->cycle]);
+ memcpy(keys[swap_idx], temp_key,
+ hashtest_key_lens[params->cycle]);
+ for (j = 0; j < NUM_TYPE; j++) {
+ temp_data = data[j][i];
+ data[j][i] = data[j][swap_idx];
+ data[j][swap_idx] = temp_data;
+ }
+ }
+}
+
+static int key_compare(const void *key1, const void *key2)
+{
+ return memcmp(key1, key2, MAX_KEYSIZE);
+}
+
+struct rte_member_parameters member_params = {
+ .num_keys = MAX_ENTRIES, /* Total hash table entries. */
+ .key_len = 4, /* Length of hash key. */
+
+ /* num_set and false_positive_rate only relevant to vBF */
+ .num_set = VBF_SET_CNT,
+ .false_positive_rate = 0.03,
+ .prim_hash_seed = 0,
+ .sec_hash_seed = 1,
+ .socket_id = 0, /* NUMA Socket ID for memory. */
+ };
+
+static int
+setup_keys_and_data(struct member_perf_params *params, unsigned int cycle,
+ int miss)
+{
+ unsigned int i, j;
+ int num_duplicates;
+
+ params->key_size = hashtest_key_lens[cycle];
+ params->cycle = cycle;
+
+ /* Reset all arrays */
+ for (i = 0; i < params->key_size; i++)
+ keys[0][i] = 0;
+
+ /* Generate a list of keys, some of which may be duplicates */
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+
+ data[HT][i] = data[CACHE][i] = (rte_rand() & 0x7FFE) + 1;
+ data[VBF][i] = rte_rand() % VBF_SET_CNT + 1;
+ }
+
+ /* Remove duplicates from the keys array */
+ do {
+ num_duplicates = 0;
+
+ /* Sort the list of keys to make it easier to find duplicates */
+ qsort(keys, KEYS_TO_ADD, MAX_KEYSIZE, key_compare);
+
+ /* Sift through the list of keys and look for duplicates */
+ int num_duplicates = 0;
+ for (i = 0; i < KEYS_TO_ADD - 1; i++) {
+ if (memcmp(keys[i], keys[i + 1],
+ params->key_size) == 0) {
+ /* This key already exists, try again */
+ num_duplicates++;
+ for (j = 0; j < params->key_size; j++)
+ keys[i][j] = rte_rand() & 0xFF;
+ }
+ }
+ } while (num_duplicates != 0);
+
+ /* Shuffle the random values again */
+ shuffle_input_keys(params);
+
+ /* For testing miss lookup, we insert half and lookup the other half */
+ unsigned int entry_cnt, bf_key_cnt;
+ if (!miss) {
+ entry_cnt = MAX_ENTRIES;
+ bf_key_cnt = KEYS_TO_ADD;
+ } else {
+ entry_cnt = MAX_ENTRIES / 2;
+ bf_key_cnt = KEYS_TO_ADD / 2;
+ }
+ member_params.false_positive_rate = VBF_FALSE_RATE;
+ member_params.key_len = params->key_size;
+ member_params.socket_id = test_socket_id;
+ member_params.num_keys = entry_cnt;
+ member_params.name = "test_member_ht";
+ member_params.is_cache = 0;
+ member_params.type = RTE_MEMBER_TYPE_HT;
+ params->setsum[HT] = rte_member_create(&member_params);
+ if (params->setsum[HT] == NULL)
+ fprintf(stderr, "ht create fail\n");
+
+ member_params.name = "test_member_cache";
+ member_params.is_cache = 1;
+ params->setsum[CACHE] = rte_member_create(&member_params);
+ if (params->setsum[CACHE] == NULL)
+ fprintf(stderr, "CACHE create fail\n");
+
+ member_params.name = "test_member_vbf";
+ member_params.type = RTE_MEMBER_TYPE_VBF;
+ member_params.num_keys = bf_key_cnt;
+ params->setsum[VBF] = rte_member_create(&member_params);
+ if (params->setsum[VBF] == NULL)
+ fprintf(stderr, "VBF create fail\n");
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] == NULL)
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+timed_adds(struct member_perf_params *params, int type)
+{
+ const uint64_t start_tsc = rte_rdtsc();
+ unsigned int i, a;
+ int32_t ret;
+
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][ADD] = time_taken / KEYS_TO_ADD;
+ return 0;
+}
+
+static int
+timed_lookups(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+
+ false_data[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+ member_set_t result;
+ int ret;
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (type == HT && result == RTE_MEMBER_NO_MATCH) {
+ printf("HT mode shouldn't have false negative");
+ return -1;
+ }
+ if (result != data[type][j])
+ false_data[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ member_set_t result[BURST_SIZE] = {0};
+ const void *keys_burst[BURST_SIZE];
+ int ret;
+
+ false_data_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_bulk(params->setsum[type],
+ keys_burst,
+ BURST_SIZE,
+ result);
+ if (ret <= 0) {
+ printf("lookup bulk has wrong return value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (type == HT && result[k] ==
+ RTE_MEMBER_NO_MATCH) {
+ printf("HT mode shouldn't have "
+ "false negative");
+ return -1;
+ }
+ if (result[k] != data[type][data_idx])
+ false_data_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_BULK] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_multimatch(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ member_set_t result[RTE_MEMBER_BUCKET_ENTRIES] = {0};
+ int ret;
+ false_data_multi[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup_multi(params->setsum[type],
+ &keys[j], RTE_MEMBER_BUCKET_ENTRIES, result);
+ if (type != CACHE && ret <= 0) {
+ printf("lookup multi has wrong return value %d,"
+ "type %d\n", ret, type);
+ }
+ if (type == HT && ret == 0) {
+ printf("HT mode shouldn't have false negative");
+ return -1;
+ }
+ /*
+ * For performance test purpose, we do not iterate all
+ * results here. We assume most likely each key can only
+ * find one match which is result[0].
+ */
+ if (result[0] != data[type][j])
+ false_data_multi[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_lookups_multimatch_bulk(struct member_perf_params *params, int type)
+{
+ unsigned int i, j, k;
+ member_set_t result[BURST_SIZE][RTE_MEMBER_BUCKET_ENTRIES] = {{0} };
+ const void *keys_burst[BURST_SIZE];
+ uint32_t match_count[BURST_SIZE];
+ int ret;
+
+ false_data_multi_bulk[type][params->cycle] = 0;
+
+ const uint64_t start_tsc = rte_rdtsc();
+
+ for (i = 0; i < NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = 0; j < KEYS_TO_ADD / BURST_SIZE; j++) {
+ for (k = 0; k < BURST_SIZE; k++)
+ keys_burst[k] = keys[j * BURST_SIZE + k];
+
+ ret = rte_member_lookup_multi_bulk(
+ params->setsum[type],
+ keys_burst, BURST_SIZE,
+ RTE_MEMBER_BUCKET_ENTRIES, match_count,
+ (member_set_t *)result);
+ if (ret < 0) {
+ printf("lookup multimatch bulk has wrong return"
+ " value\n");
+ return -1;
+ }
+ for (k = 0; k < BURST_SIZE; k++) {
+ if (type != CACHE && match_count[k] == 0) {
+ printf("lookup multimatch bulk get "
+ "wrong match count\n");
+ return -1;
+ }
+ if (type == HT && match_count[k] == 0) {
+ printf("HT mode shouldn't have "
+ "false negative");
+ return -1;
+ }
+ uint32_t data_idx = j * BURST_SIZE + k;
+ if (result[k][0] != data[type][data_idx])
+ false_data_multi_bulk[type][params->cycle]++;
+ }
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MULTI_BULK] = time_taken /
+ NUM_LOOKUPS;
+
+ return 0;
+}
+
+static int
+timed_deletes(struct member_perf_params *params, int type)
+{
+ unsigned int i;
+ int32_t ret;
+
+ if (type == VBF)
+ return 0;
+ const uint64_t start_tsc = rte_rdtsc();
+ for (i = 0; i < KEYS_TO_ADD; i++) {
+ ret = rte_member_delete(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (type != CACHE && ret < 0) {
+ printf("delete error\n");
+ return -1;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][DELETE] = time_taken / KEYS_TO_ADD;
+
+ return 0;
+}
+
+static int
+timed_miss_lookup(struct member_perf_params *params, int type)
+{
+ unsigned int i, j;
+ int ret;
+
+ false_hit[type][params->cycle] = 0;
+
+ for (i = 0; i < KEYS_TO_ADD / 2; i++) {
+ ret = rte_member_add(params->setsum[type], &keys[i],
+ data[type][i]);
+ if (ret < 0) {
+ unsigned int a;
+ printf("Error %d in rte_member_add - key=0x", ret);
+ for (a = 0; a < params->key_size; a++)
+ printf("%02x", keys[i][a]);
+ printf(" value=%d, type: %d\n", data[type][i], type);
+
+ return -1;
+ }
+ }
+
+ const uint64_t start_tsc = rte_rdtsc();
+ member_set_t result;
+
+ for (i = 0; i < 2 * NUM_LOOKUPS / KEYS_TO_ADD; i++) {
+ for (j = KEYS_TO_ADD / 2; j < KEYS_TO_ADD; j++) {
+ ret = rte_member_lookup(params->setsum[type], &keys[j],
+ &result);
+ if (ret < 0) {
+ printf("lookup wrong internally");
+ return -1;
+ }
+ if (result != RTE_MEMBER_NO_MATCH)
+ false_hit[type][params->cycle]++;
+ }
+ }
+
+ const uint64_t end_tsc = rte_rdtsc();
+ const uint64_t time_taken = end_tsc - start_tsc;
+
+ cycles[type][params->cycle][LOOKUP_MISS] = time_taken / NUM_LOOKUPS;
+
+ return 0;
+}
+
+static void
+perform_frees(struct member_perf_params *params)
+{
+ int i;
+ for (i = 0; i < NUM_TYPE; i++) {
+ if (params->setsum[i] != NULL) {
+ rte_member_free(params->setsum[i]);
+ params->setsum[i] = NULL;
+ }
+ }
+}
+
+static int
+exit_with_fail(const char *testname, struct member_perf_params *params,
+ unsigned int i, unsigned int j)
+{
+ printf("<<<<<Test %s failed at keysize %d iteration %d type %d>>>>>\n",
+ testname, hashtest_key_lens[params->cycle], i, j);
+ perform_frees(params);
+ return -1;
+}
+
+static int
+run_all_tbl_perf_tests(void)
+{
+ unsigned int i, j, k;
+ struct member_perf_params params;
+
+ printf("Measuring performance, please wait\n");
+ fflush(stdout);
+
+ test_socket_id = rte_socket_id();
+
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 0) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+
+ if (timed_adds(¶ms, j) < 0)
+ return exit_with_fail("timed_adds", ¶ms,
+ i, j);
+
+ for (k = 0; k < NUM_SHUFFLES; k++)
+ shuffle_input_keys(¶ms);
+
+ if (timed_lookups(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups", ¶ms,
+ i, j);
+
+ if (timed_lookups_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_bulk",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi",
+ ¶ms, i, j);
+
+ if (timed_lookups_multimatch_bulk(¶ms, j) < 0)
+ return exit_with_fail("timed_lookups_multi_bulk",
+ ¶ms, i, j);
+
+ if (timed_deletes(¶ms, j) < 0)
+ return exit_with_fail("timed_deletes", ¶ms,
+ i, j);
+
+ /* Print a dot to show progress on operations */
+ }
+ printf(".");
+ fflush(stdout);
+
+ perform_frees(¶ms);
+ }
+
+ /* Test false positive rate using un-inserted keys */
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ if (setup_keys_and_data(¶ms, i, 1) < 0) {
+ printf("Could not create keys/data/table\n");
+ return -1;
+ }
+ for (j = 0; j < NUM_TYPE; j++) {
+ if (timed_miss_lookup(¶ms, j) < 0)
+ return exit_with_fail("timed_miss_lookup",
+ ¶ms, i, j);
+ }
+ perform_frees(¶ms);
+ }
+
+ printf("\nResults (in CPU cycles/operation)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "Add", "Lookup", "Lookup_bulk",
+ "lookup_multi", "lookup_multi_bulk", "Delete",
+ "miss_lookup");
+ for (i = 0; i < NUM_KEYSIZES; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ for (k = 0; k < NUM_OPERATIONS; k++)
+ printf("%-18"PRIu64, cycles[j][i][k]);
+ printf("\n");
+ }
+ }
+
+ printf("\nFalse results rate (and false positive rate)\n");
+ printf("-----------------------------------\n");
+ printf("\n%-18s%-18s%-18s%-18s%-18s%-18s%-18s\n",
+ "Keysize", "type", "fr_single", "fr_bulk", "fr_multi",
+ "fr_multi_bulk", "false_positive_rate");
+ /* Key size not influence False rate so just print out one key size */
+ for (i = 0; i < 1; i++) {
+ for (j = 0; j < NUM_TYPE; j++) {
+ printf("%-18d", hashtest_key_lens[i]);
+ printf("%-18d", j);
+ printf("%-18f", (float)false_data[j][i] / NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_data_multi_bulk[j][i] /
+ NUM_LOOKUPS);
+ printf("%-18f", (float)false_hit[j][i] /
+ NUM_LOOKUPS);
+ printf("\n");
+ }
+ }
+ return 0;
+}
+
+static int
+test_member_perf(void)
+{
+
+ if (run_all_tbl_perf_tests() < 0)
+ return -1;
+
+ return 0;
+}
+
+REGISTER_TEST_COMMAND(member_perf_autotest, test_member_perf);
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* [dpdk-dev] [PATCH v6 7/7] doc: add membership documentation
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
` (5 preceding siblings ...)
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 6/7] test/member: add functional and perf tests Yipeng Wang
@ 2017-10-04 3:12 ` Yipeng Wang
2017-10-04 13:44 ` Mcnamara, John
2017-10-08 22:14 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Thomas Monjalon
7 siblings, 1 reply; 81+ messages in thread
From: Yipeng Wang @ 2017-10-04 3:12 UTC (permalink / raw)
To: dev
Cc: thomas, charlie.tai, sameh.gobriel, pablo.de.lara.guarch,
john.mcnamara, Yipeng Wang
This patch adds the documentation for membership library.
Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
Reviewed-by: John McNamara <john.mcnamara@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
MAINTAINERS | 1 +
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf | 1 +
doc/guides/prog_guide/img/member_i1.svg | 1613 +++++++++++++++++++++++++++++++
doc/guides/prog_guide/img/member_i2.svg | 36 +
doc/guides/prog_guide/img/member_i3.svg | 148 +++
doc/guides/prog_guide/img/member_i4.svg | 450 +++++++++
doc/guides/prog_guide/img/member_i5.svg | 163 ++++
doc/guides/prog_guide/img/member_i6.svg | 332 +++++++
doc/guides/prog_guide/img/member_i7.svg | 399 ++++++++
doc/guides/prog_guide/index.rst | 14 +
doc/guides/prog_guide/member_lib.rst | 420 ++++++++
doc/guides/rel_notes/release_17_11.rst | 17 +
13 files changed, 3596 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/prog_guide/img/member_i1.svg
create mode 100644 doc/guides/prog_guide/img/member_i2.svg
create mode 100644 doc/guides/prog_guide/img/member_i3.svg
create mode 100644 doc/guides/prog_guide/img/member_i4.svg
create mode 100644 doc/guides/prog_guide/img/member_i5.svg
create mode 100644 doc/guides/prog_guide/img/member_i6.svg
create mode 100644 doc/guides/prog_guide/img/member_i7.svg
create mode 100644 doc/guides/prog_guide/member_lib.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index afd36cf..51ef958 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -734,6 +734,7 @@ M: Yipeng Wang <yipeng1.wang@intel.com>
M: Sameh Gobriel <sameh.gobriel@intel.com>
F: lib/librte_member/
F: test/test/test_member*
+F: doc/guides/prog_guide/member_lib.rst
Traffic metering
M: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 19e0d4f..fe87e09 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -105,7 +105,8 @@ The public API headers are grouped by topics:
[LPM IPv4 route] (@ref rte_lpm.h),
[LPM IPv6 route] (@ref rte_lpm6.h),
[ACL] (@ref rte_acl.h),
- [EFD] (@ref rte_efd.h)
+ [EFD] (@ref rte_efd.h),
+ [member] (@ref rte_member.h)
- **QoS**:
[metering] (@ref rte_meter.h),
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index 823554f..b792d6d 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -58,6 +58,7 @@ INPUT = doc/api/doxy-api-index.md \
lib/librte_mempool \
lib/librte_meter \
lib/librte_metrics \
+ lib/librte_member \
lib/librte_net \
lib/librte_pdump \
lib/librte_pipeline \
diff --git a/doc/guides/prog_guide/img/member_i1.svg b/doc/guides/prog_guide/img/member_i1.svg
new file mode 100644
index 0000000..fc5f56a
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i1.svg
@@ -0,0 +1,1613 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i1.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.18709in" height="4.75757in"
+ viewBox="0 0 517.471 342.545" xml:space="preserve" color-interpolation-filters="sRGB" class="st61">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false">
+ <v:userDefs>
+ <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/>
+ <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/>
+ </v:userDefs>
+ </v:documentProperties>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:3}
+ .st3 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em;opacity:0.219608}
+ .st4 {font-size:1em}
+ .st5 {fill:none;stroke:#41719c;stroke-width:3}
+ .st6 {fill:#5b9bd5;font-family:Calibri;font-size:0.666664em}
+ .st7 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em;opacity:0.219608}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st9 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st11 {fill:none;stroke:none;stroke-width:0.25}
+ .st12 {fill:#ffffff;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:0.75em;font-weight:bold}
+ .st14 {marker-end:url(#mrkr5-63);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st15 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st16 {fill:#5b9bd5;font-family:Calibri;font-size:0.499992em;font-weight:bold}
+ .st17 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st18 {fill:#feffff;font-family:Calibri;font-size:0.499992em}
+ .st19 {fill:#deebf6;stroke:#c8c8c8;stroke-width:0.25}
+ .st20 {fill:#000000;font-family:Calibri;font-size:0.499992em}
+ .st21 {marker-end:url(#mrkr5-178);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:#ff0000;font-family:Calibri;font-size:0.666664em}
+ .st24 {fill:#5b9bd5;fill-opacity:0.22}
+ .st25 {stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st26 {fill:#ffffff}
+ .st27 {stroke:#0070c0;stroke-width:0.25}
+ .st28 {fill:#5b9bd5;stroke:#0070c0;stroke-width:0.25}
+ .st29 {fill:#5b9bd5;stroke:#ffffff;stroke-width:0.25}
+ .st30 {fill:#5b9bd5}
+ .st31 {stroke:#c8c8c8;stroke-width:0.25}
+ .st32 {fill:#acccea;stroke:#c8c8c8;stroke-width:0.25}
+ .st33 {fill:#5b9bd5;fill-opacity:0.22;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st34 {fill:#000000;fill-opacity:0;stroke:none;stroke-linecap:butt;stroke-width:0.75}
+ .st35 {fill:url(#grad30-309);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st36 {fill:url(#grad25-313);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st37 {fill:url(#grad35-317);stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st38 {fill:url(#grad36-325);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st39 {fill:url(#grad40-335);stroke:#000000;stroke-linecap:butt;stroke-width:0.130208}
+ .st40 {fill:url(#grad39-342);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st41 {fill:url(#grad40-355);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st42 {fill:none}
+ .st43 {stroke:#308dda;stroke-linecap:butt;stroke-width:0.130208}
+ .st44 {stroke:#ffffff;stroke-linecap:butt;stroke-width:0.130208}
+ .st45 {fill:url(#grad30-383);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st46 {fill:url(#grad36-396);stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st47 {fill:none;stroke:#c8c8c8;stroke-width:0.75}
+ .st48 {fill:#9a9a9a;stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st49 {fill:url(#grad40-415);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st50 {fill:url(#grad40-419);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st51 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.25}
+ .st52 {fill:url(#grad35-430);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.0833333}
+ .st53 {stroke:#c8c8c8;stroke-width:0.75}
+ .st54 {stroke:#4f88bb;stroke-width:0.75}
+ .st55 {fill:#feffff;font-family:Calibri;font-size:0.416656em}
+ .st56 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st57 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st58 {fill:none;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:2.25}
+ .st59 {fill:none;stroke:#0070c0;stroke-width:2.25}
+ .st60 {fill:#595959;font-family:Arial;font-size:0.666664em}
+ .st61 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Patterns_And_Gradients">
+ <linearGradient id="grad30-309" v:fillPattern="30" v:foreground="#97c2e6" v:background="#4274a2" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#4274a2;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-313" v:fillPattern="25" v:foreground="#5491d3" v:background="#246ba6" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#5491d3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </linearGradient>
+ <pattern id="grad35-317" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-318)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-319)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-320)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-321)"/>
+ </pattern>
+ <linearGradient id="grad27-318" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-319" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-320" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-321" v:fillPattern="35" v:foreground="#569bd3" v:background="#aed0ec" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#569bd3;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#aed0ec;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-325" v:fillPattern="36" v:foreground="#c0dff1" v:background="#246ba6" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#c0dff1;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#246ba6;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-335" v:fillPattern="40" v:foreground="#c8e5c8" v:background="#19bf19" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#c8e5c8;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#19bf19;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad39-342" v:fillPattern="39" v:foreground="#5599d7" v:background="#b9daf2" cx="1" cy="1" r="1">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-355" v:fillPattern="40" v:foreground="#5599d7" v:background="#214383" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#5599d7;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#214383;stop-opacity:1"/>
+ </radialGradient>
+ <linearGradient id="grad30-383" v:fillPattern="30" v:foreground="#97c2e6" v:background="#6ba4dc" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#97c2e6;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#6ba4dc;stop-opacity:1"/>
+ </linearGradient>
+ <radialGradient id="grad36-396" v:fillPattern="36" v:foreground="#89bee9" v:background="#b9daf2" cx="0" cy="0" r="1">
+ <stop offset="0" style="stop-color:#89bee9;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#b9daf2;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-415" v:fillPattern="40" v:foreground="#000000" v:background="#ffffff" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#000000;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffffff;stop-opacity:1"/>
+ </radialGradient>
+ <radialGradient id="grad40-419" v:fillPattern="40" v:foreground="#ffffff" v:background="#9a9a9a" cx="0.5" cy="0.5" r="0.5">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#9a9a9a;stop-opacity:1"/>
+ </radialGradient>
+ <pattern id="grad35-430" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x="0" y="0" width="1" height="1"
+ patternContentUnits="objectBoundingBox">
+ <path d="M 0.5 0.5 L 0 0 L 0 1 z" style="fill:url(#grad27-431)"/>
+ <path d="M 0.5 0.5 L 1 0 L 1 1 z" style="fill:url(#grad25-432)"/>
+ <path d="M 0.5 0.5 L 0 0 L 1 0 z" style="fill:url(#grad30-433)"/>
+ <path d="M 0.5 0.5 L 0 1 L 1 1 z" style="fill:url(#grad28-434)"/>
+ </pattern>
+ <linearGradient id="grad27-431" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="1" y1="0" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad25-432" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="1"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad30-433" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="1" x2="0"
+ y2="0">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ <linearGradient id="grad28-434" v:fillPattern="35" v:foreground="#ffffff" v:background="#ffcc00" x1="0" y1="0" x2="0"
+ y2="1">
+ <stop offset="0" style="stop-color:#ffffff;stop-opacity:1"/>
+ <stop offset="1" style="stop-color:#ffcc00;stop-opacity:1"/>
+ </linearGradient>
+ </defs>
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-63" class="st15" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-178" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/>
+ </v:userDefs>
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <v:layer v:name="Flowchart" v:index="0"/>
+ <g id="group165-1" transform="translate(21.7794,-24.0978)" v:mID="165" v:groupContext="group">
+ <title>Sheet.165</title>
+ <g id="group1-2" transform="translate(308.647,-25.7109)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-3" v:mID="2" v:groupContext="shape" transform="translate(11.5732,-58.1913)">
+ <title>Circle</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow2-4" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="shape3-13" v:mID="3" v:groupContext="shape" transform="translate(58.9839,-58.9839)">
+ <title>Circle.23</title>
+ <desc>List 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow3-14" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="17.73" y="318.02" class="st7" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="17.73" y="318.02" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2</text> </g>
+ <g id="shape4-19" v:mID="4" v:groupContext="shape">
+ <title>Circle.24</title>
+ <desc>List 1 matching Criteria 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow4-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 1 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>1</text> </g>
+ <g id="group5-29" transform="translate(50.7413,-4.53722)" v:mID="5" v:groupContext="group">
+ <title>Sheet.5</title>
+ <g id="shape6-30" v:mID="6" v:groupContext="shape" transform="translate(344.2,300.5) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-31" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st9"/>
+ </g>
+ <path d="M42.04 342.55 L21.02 318.64 L0 342.55 L42.04 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape7-34" v:mID="7" v:groupContext="shape" transform="translate(-0.884982,-14.7157)">
+ <title>Sheet.7</title>
+ <desc>setsum</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="12.9268" cy="336.238" width="25.86" height="12.6135"/>
+ <rect x="0" y="329.932" width="25.8535" height="12.6135" class="st11"/>
+ <text x="6.37" y="334.44" class="st12" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsu<tspan
+ x="10.49" dy="1.2em" class="st4">m</tspan></text> </g>
+ </g>
+ <g id="shape8-38" v:mID="8" v:groupContext="shape" transform="translate(72.5955,0)">
+ <title>Circle.29</title>
+ <desc>List 2 matching Criteria 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.2233" cy="315.322" width="47.65" height="40.835"/>
+ <g id="shadow8-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st2"/>
+ <text x="18.79" y="308.12" class="st3" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ <path d="M0 315.32 A27.2233 27.2233 0 1 1 54.45 315.32 A27.2233 27.2233 0 1 1 0 315.32 Z" class="st5"/>
+ <text x="18.79" y="308.12" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>List 2 <tspan
+ x="12.08" dy="1.2em" class="st4">matching </tspan><tspan x="12.29" dy="1.2em" class="st4">Criteria </tspan>2</text> </g>
+ </g>
+ <g id="group9-48" transform="translate(31.6515,-49.9094)" v:mID="9" v:groupContext="group">
+ <title>Sheet.9</title>
+ <g id="group10-49" transform="translate(99.5691,0)" v:mID="10" v:groupContext="group">
+ <title>Sheet.10</title>
+ <g id="shape11-50" v:mID="11" v:groupContext="shape" transform="translate(346.175,275.999) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st9"/>
+ </g>
+ <path d="M66.55 342.55 L33.27 290.12 L0 342.55 L66.55 342.55 Z" class="st10"/>
+ </g>
+ <g id="shape12-54" v:mID="12" v:groupContext="shape" transform="translate(355.063,285.074) rotate(90)">
+ <title>Sheet.12</title>
+ <desc>Set Summary</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1985" cy="332.563" width="48.4" height="19.9638"/>
+ <rect x="0" y="322.581" width="48.397" height="19.9638" class="st11"/>
+ <text x="18.25" y="329.86" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Set <tspan
+ x="6.38" dy="1.2em" class="st4">Summary</tspan></text> </g>
+ </g>
+ <g id="shape13-58" v:mID="13" v:groupContext="shape" transform="translate(57.5835,-54.4467)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.9 342.55" class="st14"/>
+ </g>
+ <g id="shape14-64" v:mID="14" v:groupContext="shape" transform="translate(20.2363,-51.8439)">
+ <title>Sheet.14</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="25.3328" cy="333.471" width="50.67" height="18.1489"/>
+ <rect x="0" y="324.396" width="50.6656" height="18.1489" class="st11"/>
+ <text x="14.12" y="335.27" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape15-67" v:mID="15" v:groupContext="shape" transform="translate(5.02911,1.60865) rotate(-26.0815)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L39.25 342.55" class="st14"/>
+ </g>
+ <g id="shape16-72" v:mID="16" v:groupContext="shape" transform="translate(155.629,-33.273)">
+ <title>Sheet.16</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L38.34 342.55" class="st14"/>
+ </g>
+ <g id="shape17-77" v:mID="17" v:groupContext="shape" transform="translate(304.141,0.595416) rotate(25.6934)">
+ <title>Sheet.17</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L42.68 342.55" class="st14"/>
+ </g>
+ <g id="shape18-82" v:mID="18" v:groupContext="shape" transform="translate(102.642,654.842) rotate(180)">
+ <title>Sheet.18</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape19-87" v:mID="19" v:groupContext="shape" transform="translate(-15.1809,-33.9928)">
+ <title>Sheet.19</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.75" cy="338.045" width="85.5" height="9"/>
+ <rect x="0" y="333.545" width="85.5" height="9" class="st11"/>
+ <text x="5.06" y="339.85" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape20-90" v:mID="20" v:groupContext="shape" transform="translate(102.844,679.041) rotate(180)">
+ <title>Sheet.20</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L30.14 342.55" class="st14"/>
+ </g>
+ <g id="shape21-95" v:mID="21" v:groupContext="shape" transform="translate(-35.4309,-11.4928)">
+ <title>Sheet.21</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="337.971" width="108" height="9.14889"/>
+ <rect x="0" y="333.396" width="108" height="9.14889" class="st11"/>
+ <text x="6.36" y="339.77" class="st16" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape22-98" v:mID="22" v:groupContext="shape" transform="translate(541.496,275.999) rotate(90)">
+ <title>Sheet.22</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape23-101" v:mID="23" v:groupContext="shape" transform="translate(541.496,300.198) rotate(90)">
+ <title>Sheet.23</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ <g id="shape24-104" v:mID="24" v:groupContext="shape" transform="translate(541.496,324.396) rotate(90)">
+ <title>Sheet.24</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 335.81 C2.14 344.21 5.09 343.6 7.56 340.31 C10.62 336.25 12.94 328.1 18.15 335.81" class="st17"/>
+ </g>
+ </g>
+ <g id="group25-107" transform="translate(285.961,-178.628)" v:mID="25" v:groupContext="group">
+ <title>Sheet.25</title>
+ <g id="shape26-108" v:mID="26" v:groupContext="shape" transform="translate(51.2583,-51.2583)">
+ <title>Circle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-109" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape27-112" v:mID="27" v:groupContext="shape" transform="translate(107.177,-55.9182)">
+ <title>Circle.156</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-113" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape28-116" v:mID="28" v:groupContext="shape" transform="translate(79.2174,-83.8773)">
+ <title>Circle.157</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow28-117" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape29-120" v:mID="29" v:groupContext="shape" transform="translate(153.775,-51.2583)">
+ <title>Circle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow29-121" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape30-124" v:mID="30" v:groupContext="shape" transform="translate(93.197,-18.6394)">
+ <title>Circle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow30-125" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st9"/>
+ </g>
+ <path d="M0 333.23 A9.3197 9.3197 0 0 1 18.64 333.23 A9.3197 9.3197 0 0 1 0 333.23 Z" class="st10"/>
+ </g>
+ <g id="shape31-128" v:mID="31" v:groupContext="shape" transform="translate(27.4102,-57.9329) rotate(-7.12502)">
+ <title>Sheet.31</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L31.41 342.55" class="st14"/>
+ </g>
+ <g id="shape32-133" v:mID="32" v:groupContext="shape" transform="translate(182.13,-60.5772) rotate(9.46232)">
+ <title>Sheet.32</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L22.18 342.55" class="st14"/>
+ </g>
+ <g id="shape33-138" v:mID="33" v:groupContext="shape" transform="translate(47.8843,595.237) rotate(-160.346)">
+ <title>Sheet.33</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L63.11 342.55" class="st14"/>
+ </g>
+ <g id="shape34-143" v:mID="34" v:groupContext="shape" transform="translate(292.945,525.785) rotate(141.977)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L20.97 342.55" class="st14"/>
+ </g>
+ <g id="shape35-148" v:mID="35" v:groupContext="shape" transform="translate(-95.8971,591.793) rotate(-145.945)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L28.55 342.55" class="st14"/>
+ </g>
+ <g id="shape36-153" v:mID="36" v:groupContext="shape" transform="translate(37.2788,2.27374E-013)">
+ <title>Rectangle.167</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.8652" cy="335.555" width="21.74" height="13.9795"/>
+ <g id="shadow36-154" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="21.7305" height="13.9795" class="st10"/>
+ <text x="5" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape37-158" v:mID="37" v:groupContext="shape" transform="translate(55.9182,2.27374E-013)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow37-159" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape38-163" v:mID="38" v:groupContext="shape" transform="translate(-1.65867E-013,-32.6189)">
+ <title>Rectangle.169</title>
+ <desc>SUM</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="10.3796" cy="335.555" width="20.76" height="13.9795"/>
+ <g id="shadow38-164" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="20.7593" height="13.9795" class="st10"/>
+ <text x="4.51" y="337.36" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>SUM</text> </g>
+ <g id="shape39-168" v:mID="39" v:groupContext="shape" transform="translate(18.6394,-32.6189)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="23.2992" cy="335.555" width="46.6" height="13.9795"/>
+ <g id="shadow39-169" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st9"/>
+ </g>
+ <rect x="0" y="328.566" width="46.5985" height="13.9795" class="st19"/>
+ <text x="15.18" y="337.36" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet</text> </g>
+ <g id="shape40-173" v:mID="40" v:groupContext="shape" transform="translate(197.019,626.053) rotate(161.565)">
+ <title>Sheet.40</title>
+ <path d="M0 328.31 A55.7483 27.2427 -124.2 0 0 42.37 334.19 L42.47 333.85" class="st21"/>
+ </g>
+ <g id="shape41-179" v:mID="41" v:groupContext="shape" transform="translate(154.607,584.177) rotate(161.121)">
+ <title>Sheet.41</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 319.39 A80.5593 29.9756 -101.99 0 0 41.7 325.37 L41.79 325.02" class="st21"/>
+ </g>
+ <g id="shape42-184" v:mID="42" v:groupContext="shape" transform="translate(3.02481,-66.7025)">
+ <title>Sheet.42</title>
+ <desc>Encode ID</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="19.4569" cy="335.555" width="38.92" height="13.9795"/>
+ <rect x="0" y="328.566" width="38.9138" height="13.9795" class="st11"/>
+ <text x="7.51" y="333.16" class="st23" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Encode <tspan
+ x="15.99" dy="1.2em" class="st4">ID</tspan></text> </g>
+ </g>
+ <g id="group43-188" transform="translate(12.0993,-165.858)" v:mID="43" v:groupContext="group">
+ <title>Sheet.43</title>
+ <g id="group44-189" transform="translate(7.21495,-75.757)" v:mID="44" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User</title>
+ <g id="shape45-190" v:mID="45" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.45</title>
+ <g id="shadow45-191" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape46-206" v:mID="46" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.46</title>
+ <g id="shadow46-207" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape47-210" v:mID="47" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.47</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape48-212" v:mID="48" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.48</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group49-215" transform="translate(7.21495,-47.1858)" v:mID="49" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.7</title>
+ <g id="shape50-216" v:mID="50" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.50</title>
+ <g id="shadow50-217" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape51-232" v:mID="51" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.51</title>
+ <g id="shadow51-233" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape52-236" v:mID="52" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.52</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape53-238" v:mID="53" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group54-241" transform="translate(7.21495,-18.6146)" v:mID="54" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>User.12</title>
+ <g id="shape55-242" v:mID="55" v:groupContext="shape" v:layerMember="0"
+ transform="translate(13.3353,-1.13687E-013)">
+ <title>Sheet.55</title>
+ <g id="shadow55-243" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z"
+ class="st24"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st25"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52
+ 13.66 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55
+ L21.12 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36
+ 329.46 C22.2 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77"
+ class="st25"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st25"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st25"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st25"/>
+ <path d="M0 337.32 L13.47 337.32" class="st25"/>
+ </g>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42 ZM20.96
+ 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77 Z" class="st26"/>
+ <path d="M20.77 325.42 A2.63551 2.63601 -180 1 0 15.5 325.42 A2.63551 2.63601 -180 1 0 20.77 325.42"
+ class="st27"/>
+ <path d="M20.96 328.77 L15.25 328.77 C14.84 328.77 14.46 328.91 14.16 329.14 C13.95 329.31 13.78 329.52 13.66
+ 329.76 C13.54 330 13.47 330.27 13.47 330.55 L13.47 337.32 L15.03 337.32 L15.03 342.55 L21.12
+ 342.55 L21.12 337.32 L22.74 337.32 L22.74 330.55 C22.74 330.14 22.6 329.76 22.36 329.46 C22.2
+ 329.25 21.99 329.08 21.75 328.96 C21.51 328.84 21.24 328.77 20.96 328.77" class="st27"/>
+ <path d="M18.1 342.55 L18.1 338.37" class="st27"/>
+ <path d="M15.03 337.32 L15.03 333.71" class="st27"/>
+ <path d="M21.12 337.32 L21.12 333.71" class="st27"/>
+ <path d="M0 337.32 L13.47 337.32" class="st27"/>
+ </g>
+ <g id="shape56-258" v:mID="56" v:groupContext="shape" v:layerMember="0" transform="translate(0,-8.39743)">
+ <title>Sheet.56</title>
+ <g id="shadow56-259" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06
+ C19.97 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23
+ C2.35 324.38 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09
+ 336.04 L21.09 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06
+ 340.51 L21.18 337.33 Z" class="st9"/>
+ </g>
+ <path d="M21.09 325.52 C21.09 325.13 20.96 324.79 20.74 324.51 C20.59 324.32 20.4 324.16 20.19 324.06 C19.97
+ 323.95 19.72 323.89 19.46 323.89 L3.55 323.89 C3.16 323.89 2.82 324.02 2.54 324.23 C2.35 324.38
+ 2.19 324.57 2.09 324.79 C1.98 325.01 1.92 325.25 1.92 325.52 L1.92 336.04 L21.09 336.04 L21.09
+ 325.52 ZM21.18 337.33 L1.77 337.33 L0 340.51 L0 342.55 L23.06 342.55 L23.06 340.51 L21.18 337.33
+ Z" class="st28"/>
+ </g>
+ <g id="shape57-262" v:mID="57" v:groupContext="shape" v:layerMember="0" transform="translate(3.19243,-16.175)">
+ <title>Sheet.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M16.62 342.55 L16.62 333.29 C16.62 333.23 16.61 333.18 16.58 333.13 C16.55 333.07 16.5 333.02 16.44
+ 332.98 C16.39 332.95 16.33 332.94 16.27 332.94 L0.35 332.94 C0.29 332.94 0.24 332.95 0.19 332.98
+ C0.13 333.01 0.08 333.07 0.04 333.12 C0.02 333.17 0 333.23 0 333.29 L0 342.55 L16.62 342.55
+ Z" class="st29"/>
+ </g>
+ <g id="shape58-264" v:mID="58" v:groupContext="shape" v:layerMember="0" transform="translate(1.97942,-10.81)">
+ <title>Sheet.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <path d="M0.96 340.83 L0 342.55 L19.06 342.55 L18.1 340.83 L0.96 340.83 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group59-267" transform="translate(171.161,-45.6707)" v:mID="59" v:groupContext="group" v:layerMember="0">
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ConnGap" v:prompt="" v:val="VT0(0.083333333333333):0"/>
+ </v:userDefs>
+ <title>Data Center</title>
+ <g id="shape60-268" v:mID="60" v:groupContext="shape" v:layerMember="0">
+ <title>Sheet.60</title>
+ <g id="shadow60-269" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st9"/>
+ </g>
+ <ellipse cx="37.8785" cy="331.299" rx="37.8785" ry="11.246" class="st10"/>
+ </g>
+ <g id="shape61-272" v:mID="61" v:groupContext="shape" v:layerMember="0" transform="translate(6.86487,-7.30475)">
+ <title>Sheet.61</title>
+ <g id="shadow61-273" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39
+ 332.44 L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69
+ L29.57 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64
+ 342.55 L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75
+ 342.55 L18.75 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69
+ L10.82 311.79 L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st9"/>
+ </g>
+ <path d="M54.1 311.79 L43.28 311.79 L43.28 342.55 L62.03 342.55 L62.03 311.79 L54.1 311.79 ZM43.28 332.44
+ L43.28 311.79 L51.21 311.79 L51.21 301.69 L32.33 301.69 L32.33 311.79 L40.39 311.79 L40.39 332.44
+ L43.28 332.44 ZM40.39 301.69 L40.39 293.03 L21.64 293.03 L21.64 301.69 L29.57 301.69 L29.57
+ 311.79 L32.46 311.79 L32.46 301.69 L40.39 301.69 ZM32.46 311.79 L21.64 311.79 L21.64 342.55
+ L40.39 342.55 L40.39 311.79 L32.46 311.79 ZM10.82 311.79 L0 311.79 L0 342.55 L18.75 342.55 L18.75
+ 311.79 L10.82 311.79 ZM21.64 311.79 L29.57 311.79 L29.57 301.69 L10.82 301.69 L10.82 311.79
+ L18.75 311.79 L18.75 332.44 L21.64 332.44 L21.64 311.79 Z" class="st10"/>
+ </g>
+ <g id="shape62-276" v:mID="62" v:groupContext="shape" v:layerMember="0" transform="translate(20.0835,-20.5174)">
+ <title>Sheet.62</title>
+ <g id="shadow62-277" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36
+ ZM23.46 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36
+ ZM2.27 341.36 A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z"
+ class="st24"/>
+ </g>
+ <path d="M45.36 341.36 A1.13296 1.18615 -180 1 0 43.09 341.36 A1.13296 1.18615 -180 1 0 45.36 341.36 ZM23.46
+ 341.36 A1.13296 1.18615 -180 1 0 21.2 341.36 A1.13296 1.18615 -180 1 0 23.46 341.36 ZM2.27 341.36
+ A1.13296 1.18615 -180 1 0 0 341.36 A1.13296 1.18615 -180 1 0 2.27 341.36 Z" class="st30"/>
+ </g>
+ <g id="shape63-282" v:mID="63" v:groupContext="shape" v:layerMember="0" transform="translate(14.2717,-12.5134)">
+ <title>Sheet.63</title>
+ <v:userDefs>
+ <v:ud v:nameU="SurroundingRegionColor" v:prompt="" v:val="VT5(#5b9bd5)"/>
+ </v:userDefs>
+ <g id="shadow63-283" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17
+ 340.12 L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89
+ L43.09 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2
+ 342.55 ZM21.2 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69
+ L29.27 337.69 L29.27 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74
+ L-0 341.74 L-0 342.55 ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69
+ L8.08 337.69 L8.08 336.89 L-0 336.89 L-0 337.69 Z" class="st24"/>
+ </g>
+ <path d="M43.09 342.55 L51.17 342.55 L51.17 341.74 L43.09 341.74 L43.09 342.55 ZM43.09 340.12 L51.17 340.12
+ L51.17 339.32 L43.09 339.32 L43.09 340.12 ZM43.09 337.69 L51.17 337.69 L51.17 336.89 L43.09
+ 336.89 L43.09 337.69 ZM21.2 342.55 L29.27 342.55 L29.27 341.74 L21.2 341.74 L21.2 342.55 ZM21.2
+ 340.12 L29.27 340.12 L29.27 339.32 L21.2 339.32 L21.2 340.12 ZM21.2 337.69 L29.27 337.69 L29.27
+ 336.89 L21.2 336.89 L21.2 337.69 ZM-0 342.55 L8.08 342.55 L8.08 341.74 L-0 341.74 L-0 342.55
+ ZM-0 340.12 L8.08 340.12 L8.08 339.32 L-0 339.32 L-0 340.12 ZM-0 337.69 L8.08 337.69 L8.08 336.89
+ L-0 336.89 L-0 337.69 Z" class="st26"/>
+ </g>
+ </g>
+ <g id="group64-288" transform="translate(59.5234,-47.1858)" v:mID="64" v:groupContext="group">
+ <v:custProps>
+ <v:cp v:nameU="AssetNumber" v:lbl="Asset Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SerialNumber" v:lbl="Serial Number" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Location" v:lbl="Location" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Building" v:lbl="Building" v:prompt="" v:type="0" v:format="" v:sortKey="Asset"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Room" v:lbl="Room" v:prompt="" v:type="0" v:format="" v:sortKey="Asset" v:invis="false"
+ v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="Manufacturer" v:lbl="Manufacturer" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductNumber" v:lbl="Product Number" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="PartNumber" v:lbl="Part Number" v:prompt="" v:type="0" v:format="" v:sortKey="Equipment"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ProductDescription" v:lbl="Product Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Equipment" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkName" v:lbl="Network Name" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="IPAddress" v:lbl="IP Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="SubnetMask" v:lbl="Subnet Mask" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="AdminInterface" v:lbl="Administrative Interface" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NumberofPorts" v:lbl="Number of Ports" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="CommunityString" v:lbl="Community String" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="NetworkDescription" v:lbl="Network Description" v:prompt="" v:type="0" v:format=""
+ v:sortKey="Network" v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="MACAddress" v:lbl="MAC Address" v:prompt="" v:type="0" v:format="" v:sortKey="Network"
+ v:invis="false" v:ask="false" v:langID="1033" v:cal="0"/>
+ <v:cp v:nameU="ShapeClass" v:lbl="ShapeClass" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Equipment)"/>
+ <v:cp v:nameU="ShapeType" v:lbl="ShapeType" v:prompt="" v:type="0" v:format="" v:sortKey="" v:invis="true"
+ v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Device)"/>
+ <v:cp v:nameU="SubShapeType" v:lbl="SubShapeType" v:prompt="" v:type="0" v:format="" v:sortKey=""
+ v:invis="true" v:ask="false" v:langID="1033" v:cal="0" v:val="VT4(Load balancer)"/>
+ </v:custProps>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ShapeClass" v:prompt="" v:val="VT0(5):26"/>
+ <v:ud v:nameU="SolSH" v:prompt="" v:val="VT14({BF0433D9-CD73-4EB5-8390-8653BE590246}):41"/>
+ <v:ud v:nameU="visLegendShape" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <title>Load balancer</title>
+ <g id="shape65-289" v:mID="65" v:groupContext="shape" transform="translate(0,-1.653)">
+ <title>Sheet.65</title>
+ <g id="shadow65-290" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st24"/>
+ <path d="M0 332.02 L16.23 332.02" class="st25"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st25"/>
+ </g>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62 Z"
+ class="st30"/>
+ <path d="M0 332.02 L16.23 332.02" class="st31"/>
+ <path d="M12.18 329.62 L4.06 329.62 L0 332.02 L0 342.55 L16.23 342.55 L16.23 332.02 L12.18 329.62"
+ class="st31"/>
+ </g>
+ <g id="shape66-297" v:mID="66" v:groupContext="shape" transform="translate(1.81062,-2.91583)">
+ <title>Sheet.66</title>
+ <g id="shadow66-298" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44
+ L8.34 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45
+ 338.78 L10.78 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22
+ ZM10.48 335.2 L8.29 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46
+ 334.9 L10.21 334.58 L9.55 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19
+ 338.43 C4.19 339.56 5.11 340.48 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29
+ 7.38 336.37 6.25 336.37 ZM6.25 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02
+ 339.83 6.25 339.83 C5.47 339.83 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02
+ ZM2.62 338.14 L0 338.14 L0 338.71 L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73
+ 337.47 L1.95 337.47 L2.62 338.14 Z" class="st9"/>
+ </g>
+ <path d="M10.22 341.92 L9.29 342.12 L9.95 342.55 L11.2 342.23 L10.99 340.96 L10.33 340.52 L10.53 341.44 L8.34
+ 340.01 L8.03 340.49 L10.22 341.92 ZM11.46 338.22 L8.84 338.22 L8.84 338.78 L11.45 338.78 L10.78
+ 339.45 L11.57 339.45 L12.45 338.5 L11.57 337.55 L10.78 337.55 L11.46 338.22 ZM10.48 335.2 L8.29
+ 336.64 L8.6 337.12 L10.79 335.68 L10.59 336.61 L11.25 336.17 L11.46 334.9 L10.21 334.58 L9.55
+ 335.01 L10.48 335.2 ZM6.25 336.37 C5.11 336.37 4.19 337.29 4.19 338.43 C4.19 339.56 5.11 340.48
+ 6.25 340.48 C7.38 340.48 8.31 339.56 8.31 338.43 C8.31 337.29 7.38 336.37 6.25 336.37 ZM6.25
+ 337.02 C7.02 337.02 7.66 337.65 7.66 338.43 C7.66 339.2 7.02 339.83 6.25 339.83 C5.47 339.83
+ 4.84 339.2 4.84 338.43 C4.84 337.65 5.47 337.02 6.25 337.02 ZM2.62 338.14 L0 338.14 L0 338.71
+ L2.62 338.71 L1.94 339.38 L2.74 339.38 L3.61 338.43 L2.73 337.47 L1.95 337.47 L2.62 338.14 Z"
+ class="st32"/>
+ </g>
+ </g>
+ <g id="group67-301" transform="translate(104.617,-86.5795)" v:mID="67" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server</title>
+ <g id="shape68-302" v:mID="68" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.68</title>
+ <g id="shadow68-303" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape69-306" v:mID="69" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.69</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape70-310" v:mID="70" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.70</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape71-314" v:mID="71" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.71</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape72-322" v:mID="72" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.72</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape73-326" v:mID="73" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.73</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape74-329" v:mID="74" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.74</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape75-332" v:mID="75" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.75</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape76-336" v:mID="76" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.76</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape77-339" v:mID="77" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.77</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape78-343" v:mID="78" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.78</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape79-346" v:mID="79" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.79</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape80-349" v:mID="80" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.80</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape81-352" v:mID="81" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.81</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape82-356" v:mID="82" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.82</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape83-359" v:mID="83" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.83</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape84-362" v:mID="84" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.84</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape85-365" v:mID="85" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.85</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape86-368" v:mID="86" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.86</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape87-371" v:mID="87" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.87</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape88-374" v:mID="88" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.88</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape89-377" v:mID="89" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.89</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape90-380" v:mID="90" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.90</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape91-384" v:mID="91" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.91</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape92-387" v:mID="92" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.92</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape93-390" v:mID="93" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.93</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape94-393" v:mID="94" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.94</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape95-397" v:mID="95" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.95</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape96-400" v:mID="96" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.96</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape97-402" v:mID="97" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.97</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape98-405" v:mID="98" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.98</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape99-408" v:mID="99" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.99</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape100-410" v:mID="100" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.100</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape101-412" v:mID="101" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.101</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape102-416" v:mID="102" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.102</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape103-420" v:mID="103" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.103</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape104-427" v:mID="104" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.104</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape105-435" v:mID="105" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.105</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape106-440" v:mID="106" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.106</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="group107-443" transform="translate(104.617,-33.8201)" v:mID="107" v:groupContext="group">
+ <v:userDefs>
+ <v:ud v:nameU="SkinColor" v:prompt="" v:val="VT5(#da8c36)"/>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <title>Directory server.104</title>
+ <g id="shape108-444" v:mID="108" v:groupContext="shape" transform="translate(0,-0.451005)">
+ <title>Sheet.108</title>
+ <g id="shadow108-445" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24
+ L18.24 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16
+ 321.13 L3.16 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st33"/>
+ </g>
+ <path d="M0.47 329.86 L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24
+ 340.22 L22.24 342.55 L22.24 339.27 L36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16
+ 321.5 L0 319.68 L0 329.58 L0.47 329.86 Z" class="st34"/>
+ </g>
+ <g id="shape109-448" v:mID="109" v:groupContext="shape" transform="translate(3.1636,-11.8063)">
+ <title>Sheet.109</title>
+ <path d="M16.48 323.24 L32.91 332.66 L16.31 342.55 L0 333.26 L0 332.52 L16.48 323.24 Z" class="st35"/>
+ </g>
+ <g id="shape110-451" v:mID="110" v:groupContext="shape" transform="translate(19.06,-3.68954)">
+ <title>Sheet.110</title>
+ <path d="M17.01 324.55 L0 334.19 L3.18 342.55 L17.01 334.56 L17.01 324.55 Z" class="st36"/>
+ </g>
+ <g id="shape111-454" v:mID="111" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.111</title>
+ <path d="M22.24 342.55 L0 329.58 L0 319.68 L22.24 332.43 L22.24 342.55 Z" class="st37"/>
+ </g>
+ <g id="shape112-457" v:mID="112" v:groupContext="shape" transform="translate(0.82443,-19.8334)">
+ <title>Sheet.112</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape113-460" v:mID="113" v:groupContext="shape" transform="translate(3.62283,-15.1638)">
+ <title>Sheet.113</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape114-463" v:mID="114" v:groupContext="shape" transform="translate(3.62283,-10.4867)">
+ <title>Sheet.114</title>
+ <path d="M3.22 339.78 A1.86495 1.86495 -180 0 0 1.11 338.38 A1.8709 1.8709 -180 0 0 0.38 340.85 A1.86532
+ 1.86532 -180 0 0 2.5 342.25 A1.87264 1.87264 -180 0 0 3.22 339.78 Z" class="st38"/>
+ </g>
+ <g id="shape115-466" v:mID="115" v:groupContext="shape" transform="translate(4.52404,-16.3668)">
+ <title>Sheet.115</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935408 0.935408 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape116-469" v:mID="116" v:groupContext="shape" transform="translate(4.52404,-11.6897)">
+ <title>Sheet.116</title>
+ <path d="M1.61 341.16 a0.931952 0.931952 -180 0 0 -1.05741 -0.702645 a0.935875 0.935875 -180 0 0 -0.361118
+ 1.23585 a0.932139 0.932139 -180 0 0 1.05794 0.702645 a0.935822 0.935822 -180 0 0 0.360589 -1.23585
+ Z" class="st39"/>
+ </g>
+ <g id="shape117-472" v:mID="117" v:groupContext="shape" transform="translate(7.78787,-8.83469)">
+ <title>Sheet.117</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape118-475" v:mID="118" v:groupContext="shape" transform="translate(10.204,-7.4008)">
+ <title>Sheet.118</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape119-478" v:mID="119" v:groupContext="shape" transform="translate(12.6196,-5.96639)">
+ <title>Sheet.119</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape120-481" v:mID="120" v:groupContext="shape" transform="translate(15.0357,-4.53251)">
+ <title>Sheet.120</title>
+ <path d="M0 341.57 L0.05 333.6 L1.83 334.57 L1.78 342.55 L0 341.57 Z" class="st40"/>
+ </g>
+ <g id="shape121-484" v:mID="121" v:groupContext="shape" transform="translate(8.24006,-10.0631)">
+ <title>Sheet.121</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape122-487" v:mID="122" v:groupContext="shape" transform="translate(10.6556,-8.62924)">
+ <title>Sheet.122</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape123-490" v:mID="123" v:groupContext="shape" transform="translate(13.0717,-7.19483)">
+ <title>Sheet.123</title>
+ <path d="M0.85 342.24 a0.388199 0.388199 0 0 1 -0.425188 0.308698 a0.638045 0.638045 0 0 1 -0.424658 -0.573447
+ L0 336.5 a0.387575 0.387575 0 0 1 0.424658 -0.308698 a0.637725 0.637725 0 0 1 0.425188 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape124-493" v:mID="124" v:groupContext="shape" transform="translate(15.4873,-5.76095)">
+ <title>Sheet.124</title>
+ <path d="M0.85 342.24 a0.388502 0.388502 0 0 1 -0.425717 0.308698 a0.638367 0.638367 0 0 1 -0.424129 -0.573447
+ L0 336.5 a0.387272 0.387272 0 0 1 0.424129 -0.308698 a0.638235 0.638235 0 0 1 0.425717 0.573447
+ L0.85 342.24 Z" class="st41"/>
+ </g>
+ <g id="shape125-496" v:mID="125" v:groupContext="shape" transform="translate(7.78787,-9.81214)">
+ <title>Sheet.125</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape126-499" v:mID="126" v:groupContext="shape" transform="translate(10.204,-8.37826)">
+ <title>Sheet.126</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape127-502" v:mID="127" v:groupContext="shape" transform="translate(12.6196,-6.94385)">
+ <title>Sheet.127</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape128-505" v:mID="128" v:groupContext="shape" transform="translate(15.0357,-5.50996)">
+ <title>Sheet.128</title>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55 L0 342.55 Z" class="st42"/>
+ <path d="M0 342.55 L0.05 334.57 L1.83 335.55" class="st43"/>
+ </g>
+ <g id="shape129-508" v:mID="129" v:groupContext="shape" transform="translate(7.78787,-4.53251)">
+ <title>Sheet.129</title>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57 L9.08 334.57 ZM6.66 333.14 L6.61 341.11 L4.83 340.13 L6.66
+ 333.14 ZM4.25 331.7 L4.2 339.68 L2.42 338.7 L4.25 331.7 ZM1.83 330.27 L1.78 338.24 L0 337.27
+ L1.83 330.27 Z" class="st42"/>
+ <path d="M9.08 334.57 L9.03 342.55 L7.25 341.57M6.66 333.14 L6.61 341.11 L4.83 340.13M4.25 331.7 L4.2 339.68
+ L2.42 338.7M1.83 330.27 L1.78 338.24 L0 337.27" class="st44"/>
+ </g>
+ <g id="shape130-511" v:mID="130" v:groupContext="shape" transform="translate(2.22125,-11.8454)">
+ <title>Sheet.130</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape131-514" v:mID="131" v:groupContext="shape" transform="translate(17.1796,-3.17487)">
+ <title>Sheet.131</title>
+ <path d="M0 341.85 L0.63 341.42 L1.42 341.78 L0.03 342.55 L0 341.85 Z" class="st45"/>
+ </g>
+ <g id="shape132-517" v:mID="132" v:groupContext="shape" transform="translate(1.46036,-10.3893)">
+ <title>Sheet.132</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape133-520" v:mID="133" v:groupContext="shape" transform="translate(16.4187,-1.71875)">
+ <title>Sheet.133</title>
+ <path d="M2.12 341.35 L0 342.55 L0 333.54 L2.12 332.29 L2.12 333.41 L0.79 334.15 L0.79 341.09 L2.18 340.33
+ L2.12 341.35 Z" class="st36"/>
+ </g>
+ <g id="shape134-523" v:mID="134" v:groupContext="shape" transform="translate(0.467548,-10.3893)">
+ <title>Sheet.134</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape135-526" v:mID="135" v:groupContext="shape" transform="translate(15.4259,-1.71875)">
+ <title>Sheet.135</title>
+ <path d="M0.99 333.54 L3.11 332.29 L2.12 331.66 L0 332.91 L0 341.92 L0.99 342.55 L0.99 333.54 Z"
+ class="st46"/>
+ </g>
+ <g id="shape136-529" v:mID="136" v:groupContext="shape" transform="translate(0.467548,-1.71928)">
+ <title>Sheet.136</title>
+ <path d="M17.34 339.96 L16.75 340.37 L16.75 334.15 L18.07 333.41 L18.07 332.29 L17.08 331.66 L14.96 332.91
+ L14.96 341.92 L15.95 342.55 L18.07 341.35 L18.14 340.33 L17.34 339.96 ZM2.38 331.29 L1.79 331.7
+ L1.79 325.48 L3.11 324.74 L3.11 323.62 L2.12 322.99 L0 324.24 L0 333.25 L0.99 333.87 L3.11 332.68
+ L3.18 331.66 L2.38 331.29 Z" class="st47"/>
+ </g>
+ <g id="shape137-531" v:mID="137" v:groupContext="shape" transform="translate(19.9526,-8.71396)">
+ <title>Sheet.137</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape138-534" v:mID="138" v:groupContext="shape" transform="translate(19.9526,-2.35997)">
+ <title>Sheet.138</title>
+ <path d="M1.13 341.58 a0.653986 0.653986 -180 0 0 -0.73971 -0.492434 a0.656072 0.656072 -180 0 0 -0.253101
+ 0.865731 a0.653066 0.653066 -180 0 0 0.740769 0.491375 a0.655459 0.655459 -180 0 0 0.252042
+ -0.864672 Z" class="st38"/>
+ </g>
+ <g id="shape139-537" v:mID="139" v:groupContext="shape" transform="translate(0,-0.415652)">
+ <title>Sheet.139</title>
+ <path d="M36.07 331.28 L36.07 321.27 L19.64 311.85 L3.16 321.13 L3.16 321.52 L0 319.68 L0 329.58 L0.47 329.86
+ L0.47 331.94 L1.46 332.57 L3.33 331.52 L15.43 338.57 L15.43 340.61 L16.42 341.24 L18.24 340.22
+ L22.24 342.55 L22.24 339.27 L36.07 331.28 Z" class="st47"/>
+ </g>
+ <g id="shape140-539" v:mID="140" v:groupContext="shape" transform="translate(27.8077,-2.86477)">
+ <title>Sheet.140</title>
+ <path d="M0.29 342.55 L6.62 338.89 A1.82805 1.82805 0 0 1 6.62 336.9 L0.29 340.55 A1.82805 1.82805 -180 0
+ 0 0.29 342.55 Z" class="st48"/>
+ </g>
+ <g id="shape141-541" v:mID="141" v:groupContext="shape" transform="translate(23.5035,-4.85627)">
+ <title>Sheet.141</title>
+ <path d="M4.6 342.55 L10.92 338.89 L6.32 336.24 L0 339.89 L4.6 342.55 Z" class="st49"/>
+ </g>
+ <g id="shape142-544" v:mID="142" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.142</title>
+ <path d="M0.14 339.89 L4.74 342.55 A1.82805 1.82805 0 0 1 4.74 340.55 L0.14 337.9 A3.49826 3.49826 -180 0
+ 0 0.14 339.89 Z" class="st50"/>
+ </g>
+ <g id="shape143-547" v:mID="143" v:groupContext="shape" transform="translate(25.8933,-5.98478)">
+ <title>Sheet.143</title>
+ <path d="M2.87 342.55 L0 340.89" class="st51"/>
+ <path d="M0.94 340.34 L3.82 342" class="st51"/>
+ <path d="M1.88 339.8 L4.76 341.46" class="st51"/>
+ <path d="M2.82 339.26 L5.7 340.92" class="st51"/>
+ <path d="M3.76 338.71 L6.64 340.37" class="st51"/>
+ </g>
+ <g id="shape144-554" v:mID="144" v:groupContext="shape" transform="translate(23.5035,-7.51159)">
+ <title>Sheet.144</title>
+ <path d="M5.13 341.17 L11.45 337.52 A11.9345 11.9345 0 0 1 6.32 338.89 L0 342.55 A11.9345 11.9345 -180 0
+ 0 5.13 341.17 Z" class="st52"/>
+ </g>
+ <g id="shape145-557" v:mID="145" v:groupContext="shape" transform="translate(30.2106,-4.74563)">
+ <title>Sheet.145</title>
+ <path d="M0.98 341.98 L0 342.55" class="st51"/>
+ <path d="M1.26 341.48 L2.24 340.92" class="st51"/>
+ <path d="M2.53 340.42 L3.51 339.86" class="st51"/>
+ </g>
+ <g id="shape146-562" v:mID="146" v:groupContext="shape" transform="translate(23.3588,-2.86477)">
+ <title>Sheet.146</title>
+ <path d="M0.14 339.89 L4.74 342.55 L11.07 338.89 A1.82805 1.82805 0 0 1 11.07 336.9 L7.85 335.04 L11.6 332.87
+ A11.9345 11.9345 0 0 1 6.47 334.25 L0.14 337.9 A3.49826 3.49826 -180 0 0 0.14 339.89"
+ class="st53"/>
+ </g>
+ </g>
+ <g id="shape147-565" v:mID="147" v:groupContext="shape" transform="translate(427.321,214.49) rotate(90)">
+ <title>Cloud</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54 Z" class="st42"/>
+ <path d="M5.37 311.54 A8.61618 10.0654 0 0 1 9.5 292.2 A17.4727 20.4114 0 0 1 34.86 275.89 A20.0634 23.4379 0
+ 0 1 56.58 272.26 A12.5816 14.6977 0 0 1 75.21 271.05 A14.3244 16.7336 0 0 1 97.98 277.09 A10.2423
+ 11.9646 0 0 1 106.25 294.02 A12.6864 14.8197 0 0 1 95.9 318.19 A16.0049 18.6962 0 0 1 73.14 330.27
+ A18.8712 22.0444 0 0 1 42.1 335.11 A23.9217 27.9441 0 0 1 15.2 330.27 A9.43759 11.0249 0 0 1 5.37
+ 311.54" class="st54"/>
+ <path d="M11.05 312.14 A8.59237 10.0375 0 0 1 5.37 311.54" class="st54"/>
+ <path d="M40.54 332.09 A8.62978 10.0812 -180 0 0 42.1 335.11" class="st54"/>
+ <path d="M73.92 326.65 A6.96633 8.13801 0 0 1 73.14 330.27" class="st54"/>
+ <path d="M89.7 308.52 A7.30994 8.5394 0 0 1 95.9 318.19" class="st54"/>
+ <path d="M103.15 297.64 A6.67364 7.79609 -180 0 0 106.25 294.02" class="st54"/>
+ <path d="M37.96 278.3 A10.2914 12.0219 -180 0 0 34.86 275.89" class="st54"/>
+ </g>
+ <g id="shape148-574" v:mID="148" v:groupContext="shape" transform="translate(110.222,-64.9346)">
+ <title>Triangle</title>
+ <desc>setsum</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="15.6995" cy="336.449" width="31.4" height="12.1933"/>
+ <g id="shadow148-575" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st9"/>
+ </g>
+ <path d="M31.4 342.55 L14.67 324.26 L0 342.55 L31.4 342.55 Z" class="st10"/>
+ <text x="8.35" y="337.95" class="st55" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>setsum</text> </g>
+ <g id="shape149-579" v:mID="149" v:groupContext="shape" transform="translate(292.639,20.8827) rotate(45)">
+ <title>Sheet.149</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L14.71 342.55" class="st14"/>
+ </g>
+ <g id="shape150-584" v:mID="150" v:groupContext="shape" transform="translate(43.2897,-54.1122)">
+ <title>Sheet.150</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L10.07 342.55" class="st14"/>
+ </g>
+ <g id="shape151-589" v:mID="151" v:groupContext="shape" transform="translate(-112.261,8.34531) rotate(-28.1394)">
+ <title>Sheet.151</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 342.55 L18 342.55" class="st14"/>
+ </g>
+ <g id="shape152-594" v:mID="152" v:groupContext="shape">
+ <title>Sheet.152</title>
+ <desc>Clients</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="21.5" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Clients</text> </g>
+ <g id="shape153-597" v:mID="153" v:groupContext="shape" transform="translate(83.578,-9.58078)">
+ <title>Sheet.153</title>
+ <desc>Distributed Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="42.0677" cy="337.134" width="84.14" height="10.8224"/>
+ <rect x="0" y="331.723" width="84.1355" height="10.8224" class="st11"/>
+ <text x="13.1" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Distributed Cache</text> </g>
+ <g id="shape154-600" v:mID="154" v:groupContext="shape" transform="translate(181.983,-18.6146)">
+ <title>Sheet.154</title>
+ <desc>Web Servers</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="32.4673" cy="337.134" width="64.94" height="10.8224"/>
+ <rect x="0" y="331.723" width="64.9346" height="10.8224" class="st11"/>
+ <text x="11.93" y="339.53" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Web Servers</text> </g>
+ <g id="shape155-603" v:mID="155" v:groupContext="shape" transform="translate(96.6068,630.978) rotate(180)">
+ <title>Simple Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow155-604" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ <g id="shape156-607" v:mID="156" v:groupContext="shape" transform="translate(173.159,625.567) rotate(180)">
+ <title>Simple Arrow.153</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="ArrowType" v:prompt="" v:val="VT0(2):26"/>
+ </v:userDefs>
+ <g id="shadow156-608" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,-0.3456,-1.9728)" class="st1">
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54
+ L0 342.55 Z" class="st56"/>
+ </g>
+ <path d="M0 342.55 L12 330.55 L12 336.55 L16.23 336.55 L16.23 342.55 L16.23 348.54 L12 348.54 L12 354.54 L0 342.55
+ Z" class="st57"/>
+ </g>
+ </g>
+ <g id="shape157-611" v:mID="157" v:groupContext="shape" transform="translate(0,-149.475)">
+ <title>Rectangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow157-612" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape158-615" v:mID="158" v:groupContext="shape" transform="translate(271.116,-149.475)">
+ <title>Rectangle.158</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow158-616" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape159-619" v:mID="159" v:groupContext="shape">
+ <title>Rectangle.159</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow159-620" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="271.116" height="148.722" class="st59"/>
+ </g>
+ <g id="shape160-623" v:mID="160" v:groupContext="shape" transform="translate(271.116,0)">
+ <title>Rectangle.160</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow160-624" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st58"/>
+ </g>
+ <rect x="0" y="193.823" width="202.104" height="148.722" class="st59"/>
+ </g>
+ <g id="shape161-627" v:mID="161" v:groupContext="shape" transform="translate(83.578,-151.241)">
+ <title>Sheet.161</title>
+ <desc>(a) Distributed Web Cache</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow161-628" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(a) Distributed Web Cache</text> </g>
+ <g id="shape162-632" v:mID="162" v:groupContext="shape" transform="translate(319.513,-151.241)">
+ <title>Sheet.162</title>
+ <desc>(b) Detecting Routing Loops</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54.3546" cy="333.806" width="108.71" height="17.4792"/>
+ <g id="shadow162-633" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="108.709" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(b) Detecting Routing Loops</text> </g>
+ <g id="shape163-637" v:mID="163" v:groupContext="shape" transform="translate(77.5283,-3.35965)">
+ <title>Sheet.163</title>
+ <desc>(c) In-order Workload Scheduler</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="63.5211" cy="333.806" width="127.05" height="17.4792"/>
+ <g id="shadow163-638" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="127.042" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(c) In-order Workload Scheduler</text> </g>
+ <g id="shape164-642" v:mID="164" v:groupContext="shape" transform="translate(307.414,-3.35965)">
+ <title>Sheet.164</title>
+ <desc>(d) Database Semi-join Operations</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="66.2253" cy="333.806" width="132.46" height="17.4792"/>
+ <g id="shadow164-643" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st9"/>
+ </g>
+ <rect x="0" y="325.066" width="132.451" height="17.4792" class="st10"/>
+ <text x="4" y="336.81" class="st60" v:langID="1033"><v:paragraph v:spLine="-1.5" v:spBefore="8" v:spAfter="16"
+ v:bulletSize="0.166667"/><v:tabList/>(d) Database Semi-join Operations</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i2.svg b/doc/guides/prog_guide/img/member_i2.svg
new file mode 100644
index 0000000..759c654
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i2.svg
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i2.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="4.38194in" height="1.25694in"
+ viewBox="0 0 315.5 90.5" xml:space="preserve" color-interpolation-filters="sRGB" class="st6">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:none;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st3 {baseline-shift:32.4943%;font-size:0.649886em}
+ .st4 {font-size:1em}
+ .st5 {font-family:Cambria Math;font-size:1em}
+ .st6 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(0.25,-0.25)">
+ <title>Sheet.3</title>
+ <desc>False Positive Probability = (1-(1-1/m)kn)k ≃ (1-ekn/m)k</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="157.5" cy="45.5" width="315" height="90"/>
+ <rect x="0" y="0.5" width="315" height="90" class="st1"/>
+ <text x="8.28" y="49.82" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>False Positive Probability = (1-(1-1/m)<tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan dy="0.152em" class="st4">)</tspan><tspan
+ dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan><tspan dy="0.152em" class="st4"> </tspan><tspan
+ class="st5">≃</tspan> (1-e<tspan dy="-0.234em" class="st3" v:baseFontSize="14">kn</tspan><tspan class="st3"
+ v:baseFontSize="14">/</tspan><tspan class="st3" v:baseFontSize="14">m</tspan><tspan dy="0.152em"
+ class="st4">)</tspan><tspan dy="-0.234em" class="st3" v:baseFontSize="14">k</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i3.svg b/doc/guides/prog_guide/img/member_i3.svg
new file mode 100644
index 0000000..41e92cb
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i3.svg
@@ -0,0 +1,148 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Generated by Microsoft Visio, SVG Export memship_i3.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ width="4.71875in" height="2.84375in" viewBox="0 0 339.75 204.75" xml:space="preserve" color-interpolation-filters="sRGB"
+ class="st14">
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22}
+ .st3 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25}
+ .st4 {marker-end:url(#mrkr5-32);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st5 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st6 {fill:#feffff;font-family:Calibri;font-size:0.833336em}
+ .st7 {font-size:1em}
+ .st8 {fill:#deebf6;stroke:#c7c8c8;stroke-width:0.25}
+ .st9 {fill:#000000;font-family:Calibri;font-size:0.833336em}
+ .st10 {marker-end:url(#mrkr5-84);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st12 {fill:none;stroke:none;stroke-width:0.25}
+ .st13 {fill:#ff0000;font-family:Calibri;font-size:1.00001em}
+ .st14 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-32" class="st5" refX="-6.16" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-84" class="st11" refX="-5.8" orient="auto" markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <defs id="Filters">
+ <filter id="filter_2">
+ <feGaussianBlur stdDeviation="2"/>
+ </filter>
+ </defs>
+ <g>
+ <title>Page-1</title>
+ <g id="group174-1" transform="translate(3.0294,-5.3478)">
+ <title>Sheet.174</title>
+ <g id="shape155-2" transform="translate(99,-99)">
+ <title>Circle</title>
+ <g id="shadow155-3" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape156-7" transform="translate(207,-108)">
+ <title>Circle.156</title>
+ <g id="shadow156-8" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape157-12" transform="translate(153,-162)">
+ <title>Circle.157</title>
+ <g id="shadow157-13" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape158-17" transform="translate(297,-99)">
+ <title>Circle.158</title>
+ <g id="shadow158-18" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape159-22" transform="translate(180,-36)">
+ <title>Circle.159</title>
+ <g id="shadow159-23" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st2"/>
+ </g>
+ <path d="M0 186.75 A18 18 0 0 1 36 186.75 A18 18 0 1 1 0 186.75 Z" class="st3"/>
+ </g>
+ <g id="shape160-27" transform="translate(109.604,-115.419) rotate(-7.12502)">
+ <title>Sheet.160</title>
+ <path d="M0 204.75 L66.4 204.75" class="st4"/>
+ </g>
+ <g id="shape161-33" transform="translate(276.661,-123.214) rotate(9.46232)">
+ <title>Sheet.161</title>
+ <path d="M0 204.75 L48.58 204.75" class="st4"/>
+ </g>
+ <g id="shape162-38" transform="translate(246.135,262.572) rotate(-160.346)">
+ <title>Sheet.162</title>
+ <path d="M0 204.75 L127.63 204.75" class="st4"/>
+ </g>
+ <g id="shape163-43" transform="translate(284.391,198.775) rotate(141.977)">
+ <title>Sheet.163</title>
+ <path d="M0 204.75 L46.23 204.75" class="st4"/>
+ </g>
+ <g id="shape164-48" transform="translate(70.6118,307.655) rotate(-145.945)">
+ <title>Sheet.164</title>
+ <path d="M0 204.75 L60.88 204.75" class="st4"/>
+ </g>
+ <g id="shape167-53" transform="translate(72,0)">
+ <title>Rectangle.167</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow167-54" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape168-60" transform="translate(108,0)">
+ <title>Rectangle.168</title>
+ <desc>Packet</desc>
+ <g id="shadow168-61" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape169-66" transform="translate(0,-63)">
+ <title>Rectangle.169</title>
+ <desc>BF of IDs</desc>
+ <g id="shadow169-67" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="36" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="36" height="27" class="st3"/>
+ <text x="7.69" y="188.25" class="st6">BF of <tspan x="11.71" dy="1.2em" class="st7">IDs</tspan></text> </g>
+ <g id="shape170-73" transform="translate(36,-63)">
+ <title>Rectangle.170</title>
+ <desc>Packet</desc>
+ <g id="shadow170-74" transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1">
+ <rect x="0" y="177.75" width="90" height="27" class="st2"/>
+ </g>
+ <rect x="0" y="177.75" width="90" height="27" class="st8"/>
+ <text x="31.47" y="194.25" class="st9">Packet</text> </g>
+ <g id="shape171-79" transform="translate(240.248,331.493) rotate(161.565)">
+ <title>Sheet.171</title>
+ <path d="M-0 190.52 A81.3416 36.0611 -153.48 0 0 82.31 195.86 L82.49 195.55" class="st10"/>
+ </g>
+ <g id="shape172-85" transform="translate(156.426,260.029) rotate(161.565)">
+ <title>Sheet.172</title>
+ <path d="M-0 181.6 A88.1422 54.1439 -124.1 0 0 82.68 187.13 L82.83 186.81" class="st10"/>
+ </g>
+ <g id="shape173-90" transform="translate(18,-121.5)">
+ <title>Sheet.173</title>
+ <desc>Encode ID</desc>
+ <rect x="0" y="177.75" width="63" height="27" class="st12"/>
+ <text x="7.02" y="194.85" class="st13">Encode ID</text> </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i4.svg b/doc/guides/prog_guide/img/member_i4.svg
new file mode 100644
index 0000000..a2b6f2f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i4.svg
@@ -0,0 +1,450 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i4.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="7.625in" height="3.125in" viewBox="0 0 549 225"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st18">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st2 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st3 {marker-end:url(#mrkr5-10);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st4 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st5 {visibility:visible}
+ .st6 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st7 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st8 {fill:none;stroke:none;stroke-width:0.25}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st10 {stroke:#5b9bd5;stroke-dasharray:1.5,3;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.5}
+ .st11 {fill:#5b9bd5;fill-opacity:0.25;stroke:#5b9bd5;stroke-opacity:0.25;stroke-width:0.75}
+ .st12 {fill:#4f88bb;stroke:#41719c;stroke-width:0.75}
+ .st13 {fill:#ffffff;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st14 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st15 {font-size:1em}
+ .st16 {marker-end:url(#mrkr5-162);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st17 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st18 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-10" class="st4" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-162" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group47-1" transform="translate(3.0294,-0.25)" v:mID="47" v:groupContext="group">
+ <title>Sheet.47</title>
+ <g id="shape1-2" v:mID="1" v:groupContext="shape" transform="translate(177.75,-191.922)">
+ <title>Sheet.1</title>
+ <desc>Element</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="216" width="108" height="18"/>
+ <rect x="0" y="207" width="108" height="18" class="st1"/>
+ <text x="33.77" y="219.6" class="st2" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Element</text> </g>
+ <g id="shape2-5" v:mID="2" v:groupContext="shape" transform="translate(456.75,33.0781) rotate(90)">
+ <title>Sheet.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L18.65 225" class="st3"/>
+ </g>
+ <g id="shape3-11" v:mID="3" v:groupContext="shape" transform="translate(0,-67.0469)">
+ <title>Rectangle.54</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow3-12" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape4-15" v:mID="4" v:groupContext="shape" transform="translate(27,-67.0469)">
+ <title>Rectangle.55</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow4-16" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape5-19" v:mID="5" v:groupContext="shape" transform="translate(54,-67.0469)">
+ <title>Rectangle.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow5-20" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape6-23" v:mID="6" v:groupContext="shape" transform="translate(0,-53.5469)">
+ <title>Rectangle.57</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow6-24" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape7-27" v:mID="7" v:groupContext="shape" transform="translate(27,-53.5469)">
+ <title>Rectangle.58</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow7-28" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape8-31" v:mID="8" v:groupContext="shape" transform="translate(54,-53.5469)">
+ <title>Rectangle.59</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow8-32" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape9-35" v:mID="9" v:groupContext="shape" transform="translate(5.625,-72.6719)">
+ <title>Sheet.9</title>
+ <desc>BF-1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-1</text> </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(128.25,-65.0781)">
+ <title>Rectangle.74</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow10-39" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape11-42" v:mID="11" v:groupContext="shape" transform="translate(155.25,-65.0781)">
+ <title>Rectangle.75</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow11-43" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(182.25,-65.0781)">
+ <title>Rectangle.76</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow12-47" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape13-50" v:mID="13" v:groupContext="shape" transform="translate(128.25,-51.5781)">
+ <title>Rectangle.77</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow13-51" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape14-54" v:mID="14" v:groupContext="shape" transform="translate(155.25,-51.5781)">
+ <title>Rectangle.78</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow14-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape15-58" v:mID="15" v:groupContext="shape" transform="translate(182.25,-51.5781)">
+ <title>Rectangle.79</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow15-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape16-62" v:mID="16" v:groupContext="shape" transform="translate(301.5,-65.0781)">
+ <title>Rectangle.81</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow16-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape17-66" v:mID="17" v:groupContext="shape" transform="translate(328.5,-65.0781)">
+ <title>Rectangle.82</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow17-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape18-70" v:mID="18" v:groupContext="shape" transform="translate(355.5,-65.0781)">
+ <title>Rectangle.83</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow18-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape19-74" v:mID="19" v:groupContext="shape" transform="translate(301.5,-51.5781)">
+ <title>Rectangle.84</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow19-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape20-78" v:mID="20" v:groupContext="shape" transform="translate(328.5,-51.5781)">
+ <title>Rectangle.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow20-79" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape21-82" v:mID="21" v:groupContext="shape" transform="translate(355.5,-51.5781)">
+ <title>Rectangle.86</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow21-83" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape22-86" v:mID="22" v:groupContext="shape" transform="translate(447.75,-65.6406)">
+ <title>Rectangle.88</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow22-87" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape23-90" v:mID="23" v:groupContext="shape" transform="translate(474.75,-65.6406)">
+ <title>Rectangle.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow23-91" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape24-94" v:mID="24" v:groupContext="shape" transform="translate(501.75,-65.6406)">
+ <title>Rectangle.90</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow24-95" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape25-98" v:mID="25" v:groupContext="shape" transform="translate(447.75,-52.1406)">
+ <title>Rectangle.91</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow25-99" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape26-102" v:mID="26" v:groupContext="shape" transform="translate(474.75,-52.1406)">
+ <title>Rectangle.92</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow26-103" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape27-106" v:mID="27" v:groupContext="shape" transform="translate(501.75,-52.1406)">
+ <title>Rectangle.93</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow27-107" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st5">
+ <rect x="0" y="211.5" width="27" height="13.5" class="st6"/>
+ </g>
+ <rect x="0" y="211.5" width="27" height="13.5" class="st7"/>
+ </g>
+ <g id="shape28-110" v:mID="28" v:groupContext="shape" transform="translate(213.75,-63.9531)">
+ <title>Sheet.28</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L83.25 225" class="st10"/>
+ </g>
+ <g id="shape29-113" v:mID="29" v:groupContext="shape" transform="translate(387,-63.9531)">
+ <title>Sheet.29</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L54 225" class="st10"/>
+ </g>
+ <g id="group31-116" transform="translate(184.5,-113.172)" v:mID="31" v:groupContext="group">
+ <title>Sheet.31</title>
+ <g id="shape32-117" v:mID="32" v:groupContext="shape" transform="translate(225,173.25) rotate(90)">
+ <title>Block Arrow</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:prompt="" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow32-118" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st5">
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st11"/>
+ </g>
+ <path d="M0 225 L25.87 225 L51.75 177.75 L25.87 130.5 L0 130.5 L0 225 Z" class="st12"/>
+ </g>
+ <g id="shape33-121" v:mID="33" v:groupContext="shape" transform="translate(2.25,-24.3529)">
+ <title>Sheet.33</title>
+ <desc>h1, h2 .. hk</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="215.868" width="90" height="18.2647"/>
+ <rect x="0" y="206.735" width="90" height="18.2647" class="st8"/>
+ <text x="17.56" y="219.47" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>h1, h2 .. hk</text> </g>
+ </g>
+ <g id="shape34-124" v:mID="34" v:groupContext="shape" transform="translate(307.011,286.73) rotate(152.323)">
+ <title>Sheet.34</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L128.85 225" class="st3"/>
+ </g>
+ <g id="shape35-129" v:mID="35" v:groupContext="shape" transform="translate(433.272,125.452) rotate(99.7172)">
+ <title>Sheet.35</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L58.31 225" class="st3"/>
+ </g>
+ <g id="shape36-134" v:mID="36" v:groupContext="shape" transform="translate(407.724,-64.1459) rotate(45)">
+ <title>Sheet.36</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L79.16 225" class="st3"/>
+ </g>
+ <g id="shape37-139" v:mID="37" v:groupContext="shape" transform="translate(320.441,-127.12) rotate(15.6155)">
+ <title>Sheet.37</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 225 L200.75 225" class="st3"/>
+ </g>
+ <g id="shape38-144" v:mID="38" v:groupContext="shape" transform="translate(132.75,-75.2588)">
+ <title>Sheet.38</title>
+ <desc>BF-2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.29" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-2</text> </g>
+ <g id="shape39-147" v:mID="39" v:groupContext="shape" transform="translate(303.75,-70.7588)">
+ <title>Sheet.39</title>
+ <desc>BF-X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.2" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-X</text> </g>
+ <g id="shape40-150" v:mID="40" v:groupContext="shape" transform="translate(447.75,-75.2588)">
+ <title>Sheet.40</title>
+ <desc>BF-L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="211.5" width="72" height="27"/>
+ <rect x="0" y="198" width="72" height="27" class="st8"/>
+ <text x="23.89" y="215.7" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>BF-L</text> </g>
+ <g id="shape41-153" v:mID="41" v:groupContext="shape" transform="translate(300.375,-117)">
+ <title>Sheet.41</title>
+ <desc>Hashing for lookup/Insertion into a vector of BFs happens once</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="90" cy="202.5" width="180" height="45"/>
+ <rect x="0" y="180" width="180" height="45" class="st8"/>
+ <text x="4.6" y="198.9" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Hashing for lookup/Insertion into a <tspan
+ x="23.06" dy="1.2em" class="st15">vector of BFs happens once</tspan></text> </g>
+ <g id="shape44-157" v:mID="44" v:groupContext="shape" transform="translate(249.698,-151.505) rotate(-3.74012)">
+ <title>Sheet.44</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A93.4958 45.6256 42.23 0 1 79.38 221.66 L79.68 221.85" class="st16"/>
+ </g>
+ <g id="shape45-163" v:mID="45" v:groupContext="shape" transform="translate(30.375,0.25)">
+ <title>Sheet.45</title>
+ <desc>Lookup/Insertion is done in the series of BFs, one by one or ...</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="233.048" cy="202.5" width="466.1" height="45"/>
+ <rect x="0" y="180" width="466.096" height="45" class="st8"/>
+ <text x="4.34" y="206.1" class="st14" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Lookup/Insertion is done in the series of BFs, one by one or can be optimized to do in parallel. </text> </g>
+ <g id="shape46-166" v:mID="46" v:groupContext="shape" transform="translate(123.252,-43.6868) rotate(17.0249)">
+ <title>Sheet.46</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M-0 225 A88.2185 43.0621 47.63 0 1 70.31 221.39 L70.6 221.6" class="st16"/>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i5.svg b/doc/guides/prog_guide/img/member_i5.svg
new file mode 100644
index 0000000..c1728cf
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i5.svg
@@ -0,0 +1,163 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i5.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="5.30481in" height="1.96146in"
+ viewBox="0 0 381.946 141.225" xml:space="preserve" color-interpolation-filters="sRGB" class="st15">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:none;stroke:none;stroke-width:0.25}
+ .st5 {fill:#ffffff;font-family:Calibri;font-size:1.16666em;font-weight:bold}
+ .st6 {marker-end:url(#mrkr5-14);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st7 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st8 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st9 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st10 {stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st11 {marker-end:url(#mrkr5-63);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75}
+ .st12 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.22935779816514}
+ .st13 {fill:#000000;font-family:Calibri;font-size:1.00001em}
+ .st14 {font-size:1em}
+ .st15 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-14" class="st7" v:arrowType="5" v:arrowSize="2" v:setback="6.16" refX="-6.16" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ <marker id="mrkr5-63" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="7.15" refX="-7.15" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-4.36,-4.36) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group1-1" transform="translate(191.995,-19.4751)" v:mID="1" v:groupContext="group">
+ <title>Sheet.1</title>
+ <g id="shape2-2" v:mID="2" v:groupContext="shape" transform="translate(146.944,42.2251) rotate(90)">
+ <title>Triangle</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow2-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,1.9728,-0.3456)" class="st1">
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st2"/>
+ </g>
+ <path d="M99 141.23 L49.5 58.62 L0 141.23 L99 141.23 Z" class="st3"/>
+ </g>
+ <g id="shape3-6" v:mID="3" v:groupContext="shape" transform="translate(0,-34.65)">
+ <title>Sheet.3</title>
+ <desc>vBF</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="38.1251" cy="126.375" width="76.26" height="29.7"/>
+ <rect x="0" y="111.525" width="76.2502" height="29.7" class="st4"/>
+ <text x="27.68" y="130.58" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>vBF </text> </g>
+ </g>
+ <g id="shape4-9" v:mID="4" v:groupContext="shape" transform="translate(126.724,-100.475)">
+ <title>Sheet.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape5-15" v:mID="5" v:groupContext="shape" transform="translate(103.5,-101.775)">
+ <title>Sheet.5</title>
+ <desc>Flow Key</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="39.9122" cy="135.863" width="79.83" height="10.7251"/>
+ <rect x="0" y="130.5" width="79.8244" height="10.7251" class="st4"/>
+ <text x="21.78" y="138.86" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Key</text> </g>
+ <g id="shape6-18" v:mID="6" v:groupContext="shape" transform="translate(221.726,-56.2468) rotate(-24.5123)">
+ <title>Sheet.6</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L65.42 141.23" class="st6"/>
+ </g>
+ <g id="shape7-23" v:mID="7" v:groupContext="shape" transform="translate(280.318,-68.9751)">
+ <title>Sheet.7</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L64.83 141.23" class="st6"/>
+ </g>
+ <g id="shape8-28" v:mID="8" v:groupContext="shape" transform="translate(338.125,-56.6022) rotate(24.1625)">
+ <title>Sheet.8</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L70.8 141.23" class="st6"/>
+ </g>
+ <g id="shape9-33" v:mID="9" v:groupContext="shape" transform="translate(197.714,217.975) rotate(180)">
+ <title>Sheet.9</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape10-38" v:mID="10" v:groupContext="shape" transform="translate(18,-67.5)">
+ <title>Sheet.10</title>
+ <desc>New Flow => New Assignment</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="80.4201" cy="134.475" width="160.85" height="13.5"/>
+ <rect x="0" y="127.725" width="160.84" height="13.5" class="st4"/>
+ <text x="25.11" y="137.18" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New Flow => New Assignment</text> </g>
+ <g id="shape11-41" v:mID="11" v:groupContext="shape" transform="translate(198.032,253.975) rotate(180)">
+ <title>Sheet.11</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 141.23 L51.03 141.23" class="st6"/>
+ </g>
+ <g id="shape12-46" v:mID="12" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Sheet.12</title>
+ <desc>Old Flow => forward to specific thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="81" cy="136.725" width="162.01" height="9"/>
+ <rect x="0" y="132.225" width="162" height="9" class="st4"/>
+ <text x="11.04" y="139.43" class="st9" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Old Flow => forward to specific thread</text> </g>
+ <g id="shape13-49" v:mID="13" v:groupContext="shape" transform="translate(494.552,22.75) rotate(90)">
+ <title>Sheet.13</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape14-52" v:mID="14" v:groupContext="shape" transform="translate(494.552,58.75) rotate(90)">
+ <title>Sheet.14</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape15-55" v:mID="15" v:groupContext="shape" transform="translate(494.552,94.75) rotate(90)">
+ <title>Sheet.15</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 134.49 C3.18 142.89 7.57 142.28 11.25 138.99 C15.79 134.93 19.26 126.78 27 134.49" class="st10"/>
+ </g>
+ <g id="shape17-58" v:mID="17" v:groupContext="shape" transform="translate(348.769,-25.0593) rotate(44.5185)">
+ <title>Sheet.17</title>
+ <path d="M-0 141.23 A35.1884 19.2595 167.75 0 1 42.43 138.27 L42.74 138.46" class="st11"/>
+ </g>
+ <g id="shape18-64" v:mID="18" v:groupContext="shape" transform="translate(222.188,-5.40005)">
+ <title>Sheet.18</title>
+ <desc>A BF corresponding to each worker thread</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="59.0625" cy="127.725" width="118.13" height="27"/>
+ <rect x="0" y="114.225" width="118.125" height="27" class="st4"/>
+ <text x="5.14" y="124.13" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>A BF corresponding to <tspan
+ x="11.19" dy="1.2em" class="st14">each worker thread</tspan></text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i6.svg b/doc/guides/prog_guide/img/member_i6.svg
new file mode 100644
index 0000000..265179f
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i6.svg
@@ -0,0 +1,332 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i6.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8in" height="3.625in" viewBox="0 0 576 261"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st16">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.666664em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.00001em}
+ .st9 {fill:none;stroke:#ff0000;stroke-width:0.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {marker-end:url(#mrkr5-29);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st12 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st13 {fill:#5b9bd5;font-family:Calibri;font-size:0.75em}
+ .st14 {fill:#92d050;stroke:#c8c8c8;stroke-width:0.25}
+ .st15 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st16 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-29" class="st12" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-9.8478)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(396.989,-54.9268)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st2"/>
+ </g>
+ <rect x="0" y="219.549" width="99.4817" height="41.4507" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(248.261,-12.1936)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st2"/>
+ </g>
+ <rect x="0" y="178.099" width="99.4817" height="82.9014" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(6.07514E-013,-29.0155)">
+ <title>Rectangle.10</title>
+ <desc>Signatures for target 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="225.767" width="99.49" height="70.4662"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st2"/>
+ </g>
+ <rect x="0" y="190.534" width="99.4817" height="70.4662" class="st3"/>
+ <text x="26.54" y="223.37" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(239.971,-20.4837)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(353.649,-19.9346)">
+ <title>Sheet.54</title>
+ <desc>Match 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 1</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(216.989,-210.652)">
+ <title>Sheet.55</title>
+ <desc>Packet Payload</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="252.71" width="99.49" height="16.5803"/>
+ <rect x="0" y="244.42" width="99.4817" height="16.5803" class="st9"/>
+ <text x="19.04" y="255.71" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Packet Payload</text> </g>
+ <g id="shape56-24" v:mID="56" v:groupContext="shape" transform="translate(526.665,52.2365) rotate(90)">
+ <title>Sheet.56</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L16.52 261" class="st11"/>
+ </g>
+ <g id="shape96-30" v:mID="96" v:groupContext="shape" transform="translate(-3.0294,-95.7818)">
+ <title>Sheet.96</title>
+ <desc>Attack Signature Length 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.75" cy="248.565" width="103.5" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.5" height="24.8704" class="st7"/>
+ <text x="4.79" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 1</text> </g>
+ <g id="group114-33" transform="translate(228.359,-134.152)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-34" transform="translate(0,-24.8704)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-35" v:mID="100" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-36" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape101-39" v:mID="101" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-40" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape102-43" v:mID="102" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-44" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape103-47" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-48" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape104-51" v:mID="104" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-52" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape105-55" v:mID="105" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-56" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-59" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-60" v:mID="108" v:groupContext="shape" transform="translate(3.65707E-013,-12.4352)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-61" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape109-64" v:mID="109" v:groupContext="shape" transform="translate(24.8704,-12.4352)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-65" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape110-68" v:mID="110" v:groupContext="shape" transform="translate(49.7409,-12.4352)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow110-69" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape111-72" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-73" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ <g id="shape112-76" v:mID="112" v:groupContext="shape" transform="translate(24.8704,1.13687E-013)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-77" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st3"/>
+ </g>
+ <g id="shape113-80" v:mID="113" v:groupContext="shape" transform="translate(49.7409,1.13687E-013)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-81" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st2"/>
+ </g>
+ <rect x="0" y="248.565" width="24.8704" height="12.4352" class="st14"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-84" v:mID="89" v:groupContext="shape" transform="translate(398.644,-116.927) rotate(24.4696)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L143.75 261" class="st11"/>
+ </g>
+ <g id="shape115-89" v:mID="115" v:groupContext="shape" transform="translate(116.062,-1.19371E-012)">
+ <title>Rectangle.115</title>
+ <desc>Signatures for target 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="49.7409" cy="211.259" width="99.49" height="99.4817"/>
+ <g id="shadow115-90" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st2"/>
+ </g>
+ <rect x="0" y="161.518" width="99.4817" height="99.4817" class="st3"/>
+ <text x="26.54" y="208.86" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Signatures for<v:newlineChar/><tspan
+ x="36.73" dy="1.2em" class="st5">target </tspan>2</text> </g>
+ <g id="shape116-95" v:mID="116" v:groupContext="shape" transform="translate(117.989,-95.7818)">
+ <title>Sheet.116</title>
+ <desc>Attack Signature Length 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="51.9909" cy="248.565" width="103.99" height="24.8704"/>
+ <rect x="0" y="236.13" width="103.982" height="24.8704" class="st7"/>
+ <text x="5.03" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length 2</text> </g>
+ <g id="shape118-98" v:mID="118" v:groupContext="shape" transform="translate(392.971,-90.217)">
+ <title>Sheet.118</title>
+ <desc>Attack Signature Length L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="248.565" width="108" height="24.8704"/>
+ <rect x="0" y="236.13" width="108" height="24.8704" class="st7"/>
+ <text x="7.43" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length L</text> </g>
+ <g id="shape119-101" v:mID="119" v:groupContext="shape" transform="translate(384.909,-64.9346)">
+ <title>Sheet.119</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="248.565" width="116.062" height="12.4352" class="st6"/>
+ </g>
+ <g id="shape120-103" v:mID="120" v:groupContext="shape" transform="translate(491.971,-64.9346)">
+ <title>Sheet.120</title>
+ <desc>Match 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="33.1606" cy="254.782" width="66.33" height="12.4352"/>
+ <rect x="0" y="248.565" width="66.3211" height="12.4352" class="st7"/>
+ <text x="13.06" y="258.38" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match 2</text> </g>
+ <g id="shape85-106" v:mID="85" v:groupContext="shape" transform="translate(478.538,12.9307) rotate(65.6291)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 261 L109.61 261" class="st11"/>
+ </g>
+ <g id="shape117-111" v:mID="117" v:groupContext="shape" transform="translate(247.054,-91.2818)">
+ <title>Sheet.117</title>
+ <desc>Attack Signature Length X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="52.7082" cy="248.565" width="105.42" height="24.8704"/>
+ <rect x="0" y="236.13" width="105.416" height="24.8704" class="st7"/>
+ <text x="5.7" y="251.26" class="st13" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Attack Signature Length X</text> </g>
+ </g>
+ <g id="shape122-114" v:mID="122" v:groupContext="shape" transform="translate(315.114,-164.13)">
+ <title>Sheet.122</title>
+ <desc>HTSS</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="26.943" cy="248.565" width="53.89" height="24.8704"/>
+ <rect x="0" y="236.13" width="53.8859" height="24.8704" class="st7"/>
+ <text x="14.52" y="252.16" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/img/member_i7.svg b/doc/guides/prog_guide/img/member_i7.svg
new file mode 100644
index 0000000..e23ae26
--- /dev/null
+++ b/doc/guides/prog_guide/img/member_i7.svg
@@ -0,0 +1,399 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
+<!-- Generated by Microsoft Visio 11.0, SVG Export, v1.0 memship_i7.svg Page-1 -->
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events"
+ xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="8.5in" height="4.5in" viewBox="0 0 612 324"
+ xml:space="preserve" color-interpolation-filters="sRGB" class="st23">
+ <v:documentProperties v:langID="1033" v:viewMarkup="false"/>
+
+ <style type="text/css">
+ <![CDATA[
+ .st1 {visibility:visible}
+ .st2 {fill:#5b9bd5;fill-opacity:0.22;stroke:#5b9bd5;stroke-opacity:0.22;stroke-width:0.25}
+ .st3 {fill:#5b9bd5;stroke:#c8c8c8;stroke-width:0.25}
+ .st4 {fill:#feffff;font-family:Calibri;font-size:0.833336em;font-weight:bold}
+ .st5 {font-size:1em}
+ .st6 {fill:#70ad47;fill-opacity:0.5;stroke:#00b050;stroke-width:1.5}
+ .st7 {fill:none;stroke:none;stroke-width:0.25}
+ .st8 {fill:#00b050;font-family:Calibri;font-size:1.16666em}
+ .st9 {fill:none;stroke:#00b050;stroke-width:2.25}
+ .st10 {fill:#5b9bd5;font-family:Calibri;font-size:0.833336em}
+ .st11 {fill:#5b9bd5;font-family:Calibri;font-size:1.16666em}
+ .st12 {fill:#a8d08d;stroke:#c8c8c8;stroke-width:0.25}
+ .st13 {marker-end:url(#mrkr5-83);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st14 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st15 {marker-end:url(#mrkr5-95);stroke:#92d050;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.25}
+ .st16 {fill:#92d050;fill-opacity:1;stroke:#92d050;stroke-opacity:1;stroke-width:0.47169811320755}
+ .st17 {fill:#00b050;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st18 {fill:#5b9bd5;font-family:Calibri;font-size:1.00001em}
+ .st19 {fill:none;stroke:#ff0000;stroke-width:2.25}
+ .st20 {fill:#ff0000;font-family:Calibri;font-size:1.00001em;font-weight:bold}
+ .st21 {marker-end:url(#mrkr5-123);stroke:#ff0000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1}
+ .st22 {fill:#ff0000;fill-opacity:1;stroke:#ff0000;stroke-opacity:1;stroke-width:0.28409090909091}
+ .st23 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3}
+ ]]>
+ </style>
+
+ <defs id="Markers">
+ <g id="lend5">
+ <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/>
+ </g>
+ <marker id="mrkr5-83" class="st14" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-95" class="st16" v:arrowType="5" v:arrowSize="2" v:setback="3.71" refX="-3.71" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-2.12,-2.12) "/>
+ </marker>
+ <marker id="mrkr5-123" class="st22" v:arrowType="5" v:arrowSize="2" v:setback="5.8" refX="-5.8" orient="auto"
+ markerUnits="strokeWidth" overflow="visible">
+ <use xlink:href="#lend5" transform="scale(-3.52,-3.52) "/>
+ </marker>
+ </defs>
+ <g v:mID="0" v:index="1" v:groupContext="foregroundPage">
+ <title>Page-1</title>
+ <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/>
+ <g id="group121-1" transform="translate(21.0294,-32.2733)" v:mID="121" v:groupContext="group">
+ <title>Sheet.121</title>
+ <g id="shape49-2" v:mID="49" v:groupContext="shape" transform="translate(460.471,-62.2267)">
+ <title>Rectangle.2</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow49-3" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="279" width="108" height="45" class="st2"/>
+ </g>
+ <rect x="0" y="279" width="108" height="45" class="st3"/>
+ </g>
+ <g id="shape50-6" v:mID="50" v:groupContext="shape" transform="translate(320.452,-18.123)">
+ <title>Rectangle.4</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow50-7" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="234" width="108" height="90" class="st2"/>
+ </g>
+ <rect x="0" y="234" width="108" height="90" class="st3"/>
+ </g>
+ <g id="shape52-10" v:mID="52" v:groupContext="shape" transform="translate(0,-31.5)">
+ <title>Rectangle.10</title>
+ <desc>Flow Keys Matching Mask 1</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="285.75" width="108" height="76.5"/>
+ <g id="shadow52-11" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="247.5" width="108" height="76.5" class="st2"/>
+ </g>
+ <rect x="0" y="247.5" width="108" height="76.5" class="st3"/>
+ <text x="12.56" y="282.75" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>1</text> </g>
+ <g id="shape53-16" v:mID="53" v:groupContext="shape" transform="translate(311.452,-27.123)">
+ <title>Sheet.53</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <rect x="0" y="310.5" width="126" height="13.5" class="st6"/>
+ </g>
+ <g id="shape54-18" v:mID="54" v:groupContext="shape" transform="translate(424.471,-26.2267)">
+ <title>Sheet.54</title>
+ <desc>Match</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36" cy="317.25" width="72" height="13.5"/>
+ <rect x="0" y="310.5" width="72" height="13.5" class="st7"/>
+ <text x="17.68" y="321.45" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Match</text> </g>
+ <g id="shape55-21" v:mID="55" v:groupContext="shape" transform="translate(261,-247.163)">
+ <title>Sheet.55</title>
+ <desc>Flow ID1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st9"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID1</text> </g>
+ <g id="shape96-24" v:mID="96" v:groupContext="shape" transform="translate(0,-109.783)">
+ <title>Sheet.96</title>
+ <desc>Flow Mask 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 1</text> </g>
+ <g id="group114-27" transform="translate(247.5,-163.783)" v:mID="114" v:groupContext="group">
+ <title>Sheet.114</title>
+ <g id="group106-28" transform="translate(0,-27)" v:mID="106" v:groupContext="group">
+ <title>Sheet.106</title>
+ <g id="shape100-29" v:mID="100" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow100-30" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape101-33" v:mID="101" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow101-34" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape102-37" v:mID="102" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow102-38" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape103-41" v:mID="103" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow103-42" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape104-45" v:mID="104" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow104-46" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape105-49" v:mID="105" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow105-50" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ <g id="group107-53" v:mID="107" v:groupContext="group">
+ <title>Sheet.107</title>
+ <g id="shape108-54" v:mID="108" v:groupContext="shape" transform="translate(0,-13.5)">
+ <title>Rectangle.100</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow108-55" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape109-58" v:mID="109" v:groupContext="shape" transform="translate(27,-13.5)">
+ <title>Rectangle.101</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow109-59" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape110-62" v:mID="110" v:groupContext="shape" transform="translate(54,-13.5)">
+ <title>Rectangle.102</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow110-63" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st12"/>
+ </g>
+ <g id="shape111-66" v:mID="111" v:groupContext="shape">
+ <title>Rectangle.103</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow111-67" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape112-70" v:mID="112" v:groupContext="shape" transform="translate(27,0)">
+ <title>Rectangle.104</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <g id="shadow112-71" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ <g id="shape113-74" v:mID="113" v:groupContext="shape" transform="translate(54,0)">
+ <title>Rectangle.105</title>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <g id="shadow113-75" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728"
+ v:shadowType="1" transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="310.5" width="27" height="13.5" class="st2"/>
+ </g>
+ <rect x="0" y="310.5" width="27" height="13.5" class="st3"/>
+ </g>
+ </g>
+ </g>
+ <g id="shape89-78" v:mID="89" v:groupContext="shape" transform="translate(413.723,393.802) rotate(146.31)">
+ <title>Sheet.89</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L153.9 324" class="st13"/>
+ </g>
+ <g id="shape115-84" v:mID="115" v:groupContext="shape" transform="translate(126,0)">
+ <title>Rectangle.115</title>
+ <desc>Flow Keys Matching Mask 2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="visVersion" v:val="VT0(15):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="270" width="108" height="108"/>
+ <g id="shadow115-85" v:groupContext="shadow" v:shadowOffsetX="0.3456" v:shadowOffsetY="-1.9728" v:shadowType="1"
+ transform="matrix(1,0,0,1,0.3456,1.9728)" class="st1">
+ <rect x="0" y="216" width="108" height="108" class="st2"/>
+ </g>
+ <rect x="0" y="216" width="108" height="108" class="st3"/>
+ <text x="12.56" y="267" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Keys Matching <tspan
+ x="39.1" dy="1.2em" class="st5">Mask </tspan>2</text> </g>
+ <g id="shape85-90" v:mID="85" v:groupContext="shape" transform="translate(635.321,91.2793) rotate(81.3573)">
+ <title>Sheet.85</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 324 L143.93 324" class="st15"/>
+ </g>
+ <g id="shape56-96" v:mID="56" v:groupContext="shape" transform="translate(579.175,-64.556) rotate(64.1257)">
+ <title>Sheet.56</title>
+ <path d="M0 324 L54.31 324" class="st15"/>
+ </g>
+ </g>
+ <g id="shape122-101" v:mID="122" v:groupContext="shape" transform="translate(351,-213.444)">
+ <title>Sheet.122</title>
+ <desc>HTSS with False Negative (Cache)</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="45" cy="304.722" width="90" height="38.556"/>
+ <rect x="0" y="285.444" width="90" height="38.556" class="st7"/>
+ <text x="13.29" y="301.72" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>HTSS with False <tspan
+ x="10.52" dy="1.2em" class="st5">Negative </tspan>(Cache)</text> </g>
+ <g id="shape123-105" v:mID="123" v:groupContext="shape" transform="translate(287.654,-290.556)">
+ <title>Sheet.123</title>
+ <desc>Active</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="24.1875" cy="310.5" width="48.38" height="27"/>
+ <rect x="0" y="297" width="48.375" height="27" class="st7"/>
+ <text x="8.63" y="314.1" class="st17" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Active</text> </g>
+ <g id="shape124-108" v:mID="124" v:groupContext="shape" transform="translate(278.827,-153)">
+ <title>Sheet.124</title>
+ <desc>Target for Flow ID 1</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="36.0864" cy="310.5" width="72.18" height="27"/>
+ <rect x="0" y="297" width="72.1728" height="27" class="st9"/>
+ <text x="11.93" y="306.9" class="st18" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Target for <tspan
+ x="13.54" dy="1.2em" class="st5">Flow ID </tspan>1</text> </g>
+ <g id="shape125-112" v:mID="125" v:groupContext="shape" transform="translate(155.857,-254.556)">
+ <title>Sheet.125</title>
+ <desc>Flow ID2</desc>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="27.1728" cy="315" width="54.35" height="18"/>
+ <rect x="0" y="306" width="54.3456" height="18" class="st19"/>
+ <text x="9.52" y="318" class="st10" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow ID2</text> </g>
+ <g id="shape126-115" v:mID="126" v:groupContext="shape" transform="translate(153,-270)">
+ <title>Sheet.126</title>
+ <desc>New/Inactive</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="40.5" cy="310.5" width="81" height="27"/>
+ <rect x="0" y="297" width="81" height="27" class="st7"/>
+ <text x="6.77" y="314.1" class="st20" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>New/Inactive</text> </g>
+ <g id="shape127-118" v:mID="127" v:groupContext="shape" transform="translate(251.739,-239.709) rotate(14.0795)">
+ <title>Sheet.127</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 318.73 A39.2404 18 -180 0 0 49.73 320.91 L50.07 320.78" class="st21"/>
+ </g>
+ <g id="shape128-124" v:mID="128" v:groupContext="shape" transform="translate(219.24,-229.5)">
+ <title>Sheet.128</title>
+ <desc>Miss</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="20.88" cy="310.5" width="41.76" height="27"/>
+ <rect x="0" y="297" width="41.76" height="27" class="st7"/>
+ <text x="7.81" y="314.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Miss</text> </g>
+ <g id="shape129-127" v:mID="129" v:groupContext="shape" transform="translate(147.029,-142.056)">
+ <title>Sheet.129</title>
+ <desc>Flow Mask 2</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.51" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask 2</text> </g>
+ <g id="shape130-130" v:mID="130" v:groupContext="shape" transform="translate(166.845,-18.5004) rotate(18.2325)">
+ <title>Sheet.130</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A71.1913 104.269 -180 0 0 97.04 298.43 L97.25 298.14" class="st21"/>
+ </g>
+ <g id="shape131-135" v:mID="131" v:groupContext="shape" transform="translate(184.406,-3.04505) rotate(-3.24734)">
+ <title>Sheet.131</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A112.345 104.269 -180 0 0 154.25 297.52 L154.52 297.28" class="st21"/>
+ </g>
+ <g id="shape132-140" v:mID="132" v:groupContext="shape" transform="translate(301.368,16.888) rotate(-25.868)">
+ <title>Sheet.132</title>
+ <v:userDefs>
+ <v:ud v:nameU="msvThemeColors" v:val="VT0(254):26"/>
+ </v:userDefs>
+ <path d="M0 293.46 A83.375 104.269 -180 0 0 113.91 298.14 L114.14 297.87" class="st21"/>
+ </g>
+ <g id="shape133-145" v:mID="133" v:groupContext="shape" transform="translate(345.029,-142.056)">
+ <title>Sheet.133</title>
+ <desc>Flow Mask X</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="18.43" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask X</text> </g>
+ <g id="shape134-148" v:mID="134" v:groupContext="shape" transform="translate(481.5,-139.5)">
+ <title>Sheet.134</title>
+ <desc>Flow Mask L</desc>
+ <v:textBlock v:margins="rect(4,4,4,4)"/>
+ <v:textRect cx="54" cy="319.5" width="108" height="9"/>
+ <rect x="0" y="315" width="108" height="9" class="st7"/>
+ <text x="19.12" y="323.7" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Flow Mask L</text> </g>
+ </g>
+</svg>
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 40f04a1..7ff5144 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -50,6 +50,7 @@ Programmer's Guide
timer_lib
hash_lib
efd_lib
+ member_lib
lpm_lib
lpm6_lib
packet_distrib_lib
@@ -191,6 +192,19 @@ Programmer's Guide
:numref:`figure_efd11` :ref:`figure_efd11`
+:numref:`figure_membership1` :ref:`figure_membership1`
+
+:numref:`figure_membership2` :ref:`figure_membership2`
+
+:numref:`figure_membership3` :ref:`figure_membership3`
+
+:numref:`figure_membership4` :ref:`figure_membership4`
+
+:numref:`figure_membership5` :ref:`figure_membership5`
+
+:numref:`figure_membership6` :ref:`figure_membership6`
+
+:numref:`figure_membership7` :ref:`figure_membership7`
**Tables**
diff --git a/doc/guides/prog_guide/member_lib.rst b/doc/guides/prog_guide/member_lib.rst
new file mode 100644
index 0000000..caca8dc
--- /dev/null
+++ b/doc/guides/prog_guide/member_lib.rst
@@ -0,0 +1,420 @@
+.. BSD LICENSE
+ Copyright(c) 2017 Intel Corporation. All rights reserved.
+ All rights reserved.
+
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions
+ are met:
+
+ * Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in
+ the documentation and/or other materials provided with the
+ distribution.
+ * Neither the name of Intel Corporation nor the names of its
+ contributors may be used to endorse or promote products derived
+ from this software without specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+.. _member_library:
+
+Membership Library
+==================
+
+Introduction
+------------
+
+The DPDK Membership Library provides an API for DPDK applications to insert a
+new member, delete an existing member, or query the existence of a member in a
+given set, or a group of sets. For the case of a group of sets, the library
+will return not only whether the element has been inserted before in one of
+the sets but also which set it belongs to. The Membership Library is an
+extension and generalization of a traditional filter structure (for example
+Bloom Filter [Member-bloom]) that has multiple usages in a wide variety of
+workloads and applications. In general, the Membership Library is a data
+structure that provides a "set-summary" on whether a member belongs to a set,
+and as discussed in detail later, there are two advantages of using such a
+set-summary rather than operating on a "full-blown" complete list of elements:
+first, it has a much smaller storage requirement than storing the whole list of
+elements themselves, and secondly checking an element membership (or other
+operations) in this set-summary is much faster than checking it for the
+original full-blown complete list of elements.
+
+We use the term "Set-Summary" in this guide to refer to the space-efficient,
+probabilistic membership data structure that is provided by the library. A
+membership test for an element will return the set this element belongs to or
+that the element is "not-found" with very high probability of accuracy. Set-summary
+is a fundamental data aggregation component that can be used in many network
+(and other) applications. It is a crucial structure to address performance and
+scalability issues of diverse network applications including overlay networks,
+data-centric networks, flow table summaries, network statistics and
+traffic monitoring. A set-summary is useful for applications who need to
+include a list of elements while a complete list requires too much space
+and/or too much processing cost. In these situations, the set-summary works as
+a lossy hash-based representation of a set of members. It can dramatically
+reduce space requirement and significantly improve the performance of set
+membership queries at the cost of introducing a very small membership test error
+probability.
+
+.. _figure_membership1:
+.. figure:: img/member_i1.*
+
+ Example Usages of Membership Library
+
+There are various usages for a Membership Library in a very
+large set of applications and workloads. Interested readers can refer to
+[Member-survey] for a survey of possible networking usages. The above figure
+provide a small set of examples of using the Membership Library:
+
+* Sub-figure (a)
+ depicts a distributed web cache architecture where a collection of proxies
+ attempt to share their web caches (cached from a set of back-end web servers) to
+ provide faster responses to clients, and the proxies use the Membership
+ Library to share summaries of what web pages/objects they are caching. With the
+ Membership Library, a proxy receiving an http request will inquire the
+ set-summary to find its location and quickly determine whether to retrieve the
+ requested web page from a nearby proxy or from a back-end web server.
+
+* Sub-figure (b) depicts another example for using the Membership Library to
+ prevent routing loops which is typically done using slow TTL countdown and
+ dropping packets when TTL expires. As shown in Sub-figure (b), an embedded
+ set-summary in the packet header itself can be used to summarize the set of
+ nodes a packet has gone through, and each node upon receiving a packet can check
+ whether its id is a member of the set of visited nodes, and if it is, then a
+ routing loop is detected.
+
+* Sub-Figure (c) presents another usage of the Membership
+ Library to load-balance flows to worker threads with in-order guarantee where a
+ set-summary is used to query if a packet belongs to an existing flow or a new
+ flow. Packets belonging to a new flow are forwarded to the current least loaded
+ worker thread, while those belonging to an existing flow are forwarded to the
+ pre-assigned thread to guarantee in-order processing.
+
+* Sub-figure (d) highlights
+ yet another usage example in the database domain where a set-summary is used to
+ determine joins between sets instead of creating a join by comparing each
+ element of a set against the other elements in a different set, a join is done
+ on the summaries since they can efficiently encode members of a given set.
+
+Membership Library is a configurable library that is optimized to cover set
+membership functionality for both a single set and multi-set scenarios. Two set-summary
+schemes are presented including (a) vector of Bloom Filters and (b) Hash-Table based
+set-summary schemes with and without false negative probability.
+This guide first briefly describes these different types of set-summaries, usage examples for each,
+and then it highlights the Membership Library API.
+
+Vector of Bloom Filters
+-----------------------
+
+Bloom Filter (BF) [Member-bloom] is a well-known space-efficient
+probabilistic data structure that answers set membership queries (test whether
+an element is a member of a set) with some probability of false positives and
+zero false negatives; a query for an element returns either it is "possibly in
+a set" (with very high probability) or "definitely not in a set".
+
+The BF is a method for representing a set of ``n`` elements (for example flow keys
+in network applications domain) to support membership queries. The idea of BF is
+to allocate a bit-vector ``v`` with ``m`` bits, which are initially all set to 0. Then
+it chooses ``k`` independent hash functions ``h1``, ``h2``, ... ``hk`` with hash values range from
+``0`` to ``m-1`` to perform hashing calculations on each element to be inserted. Every time when an
+element ``X`` being inserted into the set, the bits at positions ``h1(X)``, ``h2(X)``, ...
+``hk(X)`` in ``v`` are set to 1 (any particular bit might be set to 1 multiple times
+for multiple different inserted elements). Given a query for any element ``Y``, the
+bits at positions ``h1(Y)``, ``h2(Y)``, ... ``hk(Y)`` are checked. If any of them is 0,
+then Y is definitely not in the set. Otherwise there is a high probability that
+Y is a member of the set with certain false positive probability. As shown in
+the next equation, the false positive probability can be made arbitrarily small
+by changing the number of hash functions (``k``) and the vector length (``m``).
+
+.. _figure_membership2:
+.. figure:: img/member_i2.*
+
+ Bloom Filter False Positive Probability
+
+Without BF, an accurate membership testing could involve a costly hash table
+lookup and full element comparison. The advantage of using a BF is to simplify
+the membership test into a series of hash calculations and memory accesses for a
+small bit-vector, which can be easily optimized. Hence the lookup throughput
+(set membership test) can be significantly faster than a normal hash table
+lookup with element comparison.
+
+.. _figure_membership3:
+.. figure:: img/member_i3.*
+
+ Detecting Routing Loops Using BF
+
+BF is used for applications that need only one set, and the
+membership of elements is checked against the BF. The example discussed
+in the above figure is one example of potential applications that uses only one
+set to capture the node IDs that have been visited so far by the packet. Each
+node will then check this embedded BF in the packet header for its own id, and
+if the BF indicates that the current node is definitely not in the set then a
+loop-free route is guaranteed.
+
+
+.. _figure_membership4:
+.. figure:: img/member_i4.*
+
+ Vector Bloom Filter (vBF) Overview
+
+To support membership test for both multiple sets and a single set,
+the library implements a Vector Bloom Filter (vBF) scheme.
+vBF basically composes multiple bloom filters into a vector of bloom filers.
+The membership test is conducted on all of the
+bloom filters concurrently to determine which set(s) it belongs to or none of
+them. The basic idea of vBF is shown in the above figure where an element is
+used to address multiple bloom filters concurrently and the bloom filter
+index(es) with a hit is returned.
+
+.. _figure_membership5:
+.. figure:: img/member_i5.*
+
+ vBF for Flow Scheduling to Worker Thread
+
+As previously mentioned, there are many usages of such structures. vBF is used
+for applications that need to check membership against multiple sets
+simultaneously. The example shown in the above figure uses a set to capture
+all flows being assigned for processing at a given worker thread. Upon receiving
+a packet the vBF is used to quickly figure out if this packet belongs to a new flow
+so as to be forwarded to the current least loaded worker thread, or otherwise it
+should be queued for an existing thread to guarantee in-order processing (i.e.
+the property of vBF to indicate right away that a given flow is a new one or
+not is critical to minimize response time latency).
+
+It should be noted that vBF can be implemented using a set of single bloom
+filters with sequential lookup of each BF. However, being able to concurrently
+search all set-summaries is a big throughput advantage. In the library, certain
+parallelism is realized by the implementation of checking all bloom filters
+together.
+
+
+Hash-Table based Set-Summaries
+------------------------------
+
+Hash-table based set-summary (HTSS) is another scheme in the membership library.
+Cuckoo filter [Member-cfilter] is an example of HTSS.
+HTSS supports multi-set membership testing like
+vBF does. However, while vBF is better for a small number of targets, HTSS is more suitable
+and can easily outperform vBF when the number of sets is
+large, since HTSS uses a single hash table for membership testing while vBF
+requires testing a series of Bloom Filters each corresponding to one set.
+As a result, generally speaking vBF is more adequate for the case of a small limited number of sets
+while HTSS should be used with a larger number of sets.
+
+.. _figure_membership6:
+.. figure:: img/member_i6.*
+
+ Using HTSS for Attack Signature Matching
+
+As shown in the above figure, attack signature matching where each set
+represents a certain signature length (for correctness of this example, an
+attack signature should not be a subset of another one) in the payload is a good
+example for using HTSS with 0% false negative (i.e., when an element returns not
+found, it has a 100% certainty that it is not a member of any set). The packet
+inspection application benefits from knowing right away that the current payload
+does not match any attack signatures in the database to establish its
+legitimacy, otherwise a deep inspection of the packet is needed.
+
+HTSS employs a similar but simpler data structure to a traditional hash table,
+and the major difference is that HTSS stores only the signatures but not the
+full keys/elements which can significantly reduce the footprint of the table.
+Along with the signature, HTSS also stores a value to indicate the target set.
+When looking up an element, the element is hashed and the HTSS is addressed
+to retrieve the signature stored. If the signature matches then the value is
+retrieved corresponding to the index of the target set which the element belongs
+to. Because signatures can collide, HTSS can still has false positive
+probability. Furthermore, if elements are allowed to be
+overwritten or evicted when the hash table becomes full, it will also have a
+false negative probability. We discuss this case in the next section.
+
+Set-Summaries with False Negative Probability
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As previously mentioned, traditional set-summaries (e.g. Bloom Filters) do not
+have a false negative probability, i.e., it is 100% certain when an element
+returns "not to be present" for a given set. However, the Membership Library
+also supports a set-summary probabilistic data structure based on HTSS which
+allows for false negative probability.
+
+In HTSS, when the hash table becomes full, keys/elements will fail to be added
+into the table and the hash table has to be resized to accommodate for these new
+elements, which can be expensive. However, if we allow new elements to overwrite
+or evict existing elements (as a cache typically does), then the resulting
+set-summary will begin to have false negative probability. This is because the
+element that was evicted from the set-summary may still be present in the target
+set. For subsequent inquiries the set-summary will falsely report the element
+not being in the set, hence having a false negative probability.
+
+The major usage of HTSS with false negative is to use it as a cache for
+distributing elements to different target sets. By allowing HTSS to evict old
+elements, the set-summary can keep track of the most recent elements
+(i.e. active) as a cache typically does. Old inactive elements (infrequently
+used elements) will automatically and eventually get evicted from the
+set-summary. It is worth noting that the set-summary still has false positive
+probability, which means the application either can tolerate certain false positive
+or it has fall-back path when false positive happens.
+
+.. _figure_membership7:
+.. figure:: img/member_i7.*
+
+ Using HTSS with False Negatives for Wild Card Classification
+
+HTSS with false negative (i.e. a cache) also has its wide set of applications.
+For example wild card flow classification (e.g. ACL rules) highlighted in the
+above figure is an example of such application. In that case each target set
+represents a sub-table with rules defined by a certain flow mask. The flow masks
+are non-overlapping, and for flows matching more than one rule only the highest
+priority one is inserted in the corresponding sub-table (interested readers can
+refer to the Open vSwitch (OvS) design of Mega Flow Cache (MFC) [Member-OvS]
+for further details). Typically the rules will have a large number of distinct
+unique masks and hence, a large number of target sets each corresponding to one
+mask. Because the active set of flows varies widely based on the network
+traffic, HTSS with false negative will act as a cache for <flowid, target ACL
+sub-table> pair for the current active set of flows. When a miss occurs (as
+shown in red in the above figure) the sub-tables will be searched sequentially
+one by one for a possible match, and when found the flow key and target
+sub-table will be inserted into the set-summary (i.e. cache insertion) so
+subsequent packets from the same flow don’t incur the overhead of the
+sequential search of sub-tables.
+
+Library API Overview
+--------------------
+
+The design goal of the Membership Library API is to be as generic as possible to
+support all the different types of set-summaries we discussed in previous
+sections and beyond. Fundamentally, the APIs need to include creation,
+insertion, deletion, and lookup.
+
+
+Set-summary Create
+~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_create()`` function is used to create a set-summary structure, the input parameter
+is a struct to pass in parameters that needed to initialize the set-summary, while the function returns the
+pointer to the created set-summary or ``NULL`` if the creation failed.
+
+The general input arguments used when creating the set-summary should include ``name``
+which is the name of the created set-summary, *type* which is one of the types
+supported by the library (e.g. ``RTE_MEMBER_TYPE_HT`` for HTSS or ``RTE_MEMBER_TYPE_VBF`` for vBF), and ``key_len``
+which is the length of the element/key. There are other parameters
+are only used for certain type of set-summary, or which have a slightly different meaning for different types of set-summary.
+For example, ``num_keys`` parameter means the maximum number of entries for Hash table based set-summary.
+However, for bloom filter, this value means the expected number of keys that could be
+inserted into the bloom filter(s). The value is used to calculate the size of each
+bloom filter.
+
+We also pass two seeds: ``prim_hash_seed`` and
+``sec_hash_seed`` for the primary and secondary hash functions to calculate two independent hash values.
+``socket_id`` parameter is the NUMA socket ID for the memory used to create the
+set-summary. For HTSS, another parameter ``is_cache`` is used to indicate
+if this set-summary is a cache (i.e. with false negative probability) or not.
+For vBF, extra parameters are needed. For example, ``num_set`` is the number of
+sets needed to initialize the vector bloom filters. This number is equal to the
+number of bloom filters will be created.
+``false_pos_rate`` is the false positive rate. num_keys and false_pos_rate will be used to determine
+the number of hash functions and the bloom filter size.
+
+
+Set-summary Element Insertion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_add()`` function is used to insert an element/key into a set-summary structure. If it fails an
+error is returned. For success the returned value is dependent on the
+set-summary mode to provide extra information for the users. For vBF
+mode, a return value of 0 means a successful insert. For HTSS mode without false negative, the insert
+could fail with ``-ENOSPC`` if the table is full. With false negative (i.e. cache mode),
+for insert that does not cause any eviction (i.e. no overwriting happens to an
+existing entry) the return value is 0. For insertion that causes eviction, the return
+value is 1 to indicate such situation, but it is not an error.
+
+The input arguments for the function should include the ``key`` which is a pointer to the element/key that needs to
+be added to the set-summary, and ``set_id`` which is the set id associated
+with the key that needs to be added.
+
+
+Set-summary Element Lookup
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_member_lookup()`` function looks up a single key/element in the set-summary structure. It
+returns as soon as the first match is found. The return value is 1 if a
+match is found and 0 otherwise. The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, and ``set_id`` which is used to return the
+first target set id where the key has matched, if any.
+
+The ``rte_member_lookup_bulk()`` function is used to look up a bulk of keys/elements in the
+set-summary structure for their first match. Each key lookup returns as soon as the first match is found. The
+return value is the number of keys that find a match. The arguments of the function include ``keys``
+which is a pointer to a bulk of keys that are to be looked up,
+``num_keys`` is the number
+of keys that will be looked up, and ``set_ids`` are the return target set
+ids for the first match found for each of the input keys. ``set_ids`` is an array
+needs to be sized according to the ``num_keys``. If there is no match, the set id
+for that key will be set to RTE_MEMBER_NO_MATCH.
+
+The ``rte_member_lookup_multi()`` function looks up a single key/element in the
+set-summary structure for multiple matches. It
+returns ALL the matches (possibly more than one) found for this key when it
+is matched against all target sets (it is worth noting that for cache mode HTSS,
+the current implementation matches at most one target set). The return value is
+the number of matches
+that was found for this key (for cache mode HTSS the return value
+should be at most 1). The arguments for the function include ``key`` which is a pointer to the
+element/key that needs to be looked up, ``max_match_per_key`` which is to indicate the maximum number of matches
+the user expects to find for each key, and ``set_id`` which is used to return all
+target set ids where the key has matched, if any. The ``set_id`` array should be sized
+according to ``max_match_per_key``. For vBF, the maximum number of matches per key is equal
+to the number of sets. For HTSS, the maximum number of matches per key is equal to two time
+entry count per bucket. ``max_match_per_key`` should be equal or smaller than the maximum number of
+possible matches.
+
+The ``rte_membership_lookup_multi_bulk()`` function looks up a bulk of keys/elements in the
+set-summary structure for multiple matches, each key lookup returns ALL the matches (possibly more
+than one) found for this key when it is matched against all target sets (cache mode HTSS
+matches at most one target set). The
+return value is the number of keys that find one or more matches in the
+set-summary structure. The arguments of the
+function include ``keys`` which is
+a pointer to a bulk of keys that are to be looked up, ``num_keys`` is the number
+of keys that will be looked up, ``max_match_per_key`` is the possible
+maximum number of matches for each key, ``match_count`` which is the returned number
+of matches for each key, and ``set_ids`` are the returned target set
+ids for all matches found for each keys. ``set_ids`` is 2-D array
+containing a 1-D array for each key (the size of 1-D array per key should be set by the user according to ``max_match_per_key``).
+``max_match_per_key`` should be equal or smaller than the maximum number of
+possible matches, similar to ``rte_member_lookup_multi``.
+
+
+Set-summary Element Delete
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``rte_membership_delete()`` function deletes an element/key from a set-summary structure, if it fails
+an error is returned. The input arguments should include ``key`` which is a pointer to the
+element/key that needs to be deleted from the set-summary, and ``set_id``
+which is the set id associated with the key to delete. It is worth noting that current
+implementation of vBF does not support deletion [1]_. An error code ``-EINVAL`` will be returned.
+
+.. [1] Traditional bloom filter does not support proactive deletion. Supporting proactive deletion require additional implementation and performance overhead.
+
+References
+-----------
+
+[Member-bloom] B H Bloom, "Space/Time Trade-offs in Hash Coding with Allowable Errors," Communications of the ACM, 1970.
+
+[Member-survey] A Broder and M Mitzenmacher, "Network Applications of Bloom Filters: A Survey," in Internet Mathematics, 2005.
+
+[Member-cfilter] B Fan, D G Andersen and M Kaminsky, "Cuckoo Filter: Practically Better Than Bloom," in Conference on emerging Networking Experiments and Technologies, 2014.
+
+[Member-OvS] B Pfaff, "The Design and Implementation of Open vSwitch," in NSDI, 2015.
diff --git a/doc/guides/rel_notes/release_17_11.rst b/doc/guides/rel_notes/release_17_11.rst
index 8bf91bd..ba068ec 100644
--- a/doc/guides/rel_notes/release_17_11.rst
+++ b/doc/guides/rel_notes/release_17_11.rst
@@ -41,6 +41,23 @@ New Features
Also, make sure to start the actual text at the margin.
=========================================================
+* **Added Membership library (rte_member).**
+
+ Added membership library. It provides an API for DPDK applications to insert a
+ new member, delete an existing member, or query the existence of a member in a
+ given set, or a group of sets. For the case of a group of sets the library
+ will return not only whether the element has been inserted before in one of
+ the sets but also which set it belongs to.
+
+ The Membership Library is an extension and generalization of a traditional
+ filter (for example Bloom Filter) structure that has multiple usages in a wide
+ variety of workloads and applications. In general, the Membership Library is a
+ data structure that provides a “set-summary” and responds to set-membership
+ queries whether a certain member belongs to a set(s).
+
+ See the :ref:`Membership Library <Member_Library>` documentation in
+ the Programmers Guide document, for more information.
+
Resolved Issues
---------------
--
2.7.4
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [dpdk-dev] [PATCH v6 7/7] doc: add membership documentation
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 7/7] doc: add membership documentation Yipeng Wang
@ 2017-10-04 13:44 ` Mcnamara, John
0 siblings, 0 replies; 81+ messages in thread
From: Mcnamara, John @ 2017-10-04 13:44 UTC (permalink / raw)
To: Wang, Yipeng1, dev
Cc: thomas, Tai, Charlie, Gobriel, Sameh, De Lara Guarch, Pablo
> -----Original Message-----
> From: Wang, Yipeng1
> Sent: Wednesday, October 4, 2017 4:12 AM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; Tai, Charlie <charlie.tai@intel.com>; Gobriel,
> Sameh <sameh.gobriel@intel.com>; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; Mcnamara, John
> <john.mcnamara@intel.com>; Wang, Yipeng1 <yipeng1.wang@intel.com>
> Subject: [PATCH v6 7/7] doc: add membership documentation
>
> This patch adds the documentation for membership library.
>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> Reviewed-by: John McNamara <john.mcnamara@intel.com>
> Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [dpdk-dev] [PATCH v6 0/7] Add Membership Library
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
` (6 preceding siblings ...)
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 7/7] doc: add membership documentation Yipeng Wang
@ 2017-10-08 22:14 ` Thomas Monjalon
7 siblings, 0 replies; 81+ messages in thread
From: Thomas Monjalon @ 2017-10-08 22:14 UTC (permalink / raw)
To: Yipeng Wang
Cc: dev, charlie.tai, sameh.gobriel, pablo.de.lara.guarch, john.mcnamara
> Yipeng Wang (7):
> member: implement main API
> member: implement HT mode
> member: implement vBF mode
> member: add AVX for HT mode
> member: enable the library
> test/member: add functional and perf tests
> doc: add membership documentation
It looks good.
I have fixed some minor details (alphabetical sorting, alignments),
and I have enabled the compilation from the first patch.
Applied, thanks
^ permalink raw reply [flat|nested] 81+ messages in thread
