Re: [dpdk-dev] [PATCH v6] Add toeplitz hash algorithm used by RSS

[Tony Lu <zlu@ezchip.com>]

Hi, Vladimir

When compiling thash for no-X86 arches, it fails with the following errors.
I wonder if
it is possible to make the thash library arch-independent?

== Build app/test
  CC test_thash.o
In file included from /u/zlu.bjg/git/dpdk.org/app/test/test_thash.c:40:
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:56:22:
error: rte_vect.h: No such file or directory
In file included from /u/zlu.bjg/git/dpdk.org/app/test/test_thash.c:40:
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:62:
error: expected '=', ',', ';', 'asm' or '__attribute__' before
'rte_thash_ipv6_bswap_mask'
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:129:
error: requested alignment is not a constant
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h: In
function 'rte_thash_load_v6_addrs':
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:160:
error: '__m128i' undeclared (first use in this function)
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:160:
error: (Each undeclared identifier is reported only once
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:160:
error: for each function it appears in.)
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:160:
error: expected ';' before 'ipv6'
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:161:
error: expected expression before ')' token
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:163:
error: 'ipv6' undeclared (first use in this function)
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:163:
warning: implicit declaration of function '_mm_loadu_si128'
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:163:
warning: nested extern declaration of '_mm_loadu_si128'
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:163:
error: expected ')' before '__m128i'
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:163:
warning: type defaults to 'int' in declaration of 'type name'
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:163:
warning: cast from pointer to integer of different size
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:164:
error: expected expression before ')' token
/u/zlu.bjg/git/dpdk.org/tile-tilegx-linuxapp-gcc/include/rte_thash.h:158:
warning: unused parameter 'targ'
make[3]: *** [test_thash.o] Error 1
make[2]: *** [test] Error 2
make[1]: *** [app] Error 2
make: *** [all] Error 2

Thanks
-Zhigang Lu

>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vladimir Medvedkin
>Sent: Wednesday, July 01, 2015 7:40 AM
>To: dev@dpdk.org
>Subject: [dpdk-dev] [PATCH v6] Add toeplitz hash algorithm used by RSS
>
>Software implementation of the Toeplitz hash function used by RSS.
>Can be used either for packet distribution on single queue NIC or for
simulating
>of RSS computation on specific NIC (for example after GRE header
>decapsulating).
>
>v6 changes
>- Fix compilation error
>- Rename some defines and function
>
>v5 changes
>- Fix errors reported by checkpatch.pl
>
>v4 changes
>- Fix copyright
>- rename bswap_mask constant, add rte_ prefix
>- change rte_ipv[46]_tuple struct
>- change rte_thash_load_v6_addr prototype
>
>v3 changes
>- Rework API to be more generic
>- Add sctp_tag into tuple
>
>v2 changes
>- Add ipv6 support
>- Various style fixes
>
>Signed-off-by: Vladimir Medvedkin <medvedkinv@gmail.com>
>---
> lib/librte_hash/Makefile    |   1 +
> lib/librte_hash/rte_thash.h | 231
>++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 232 insertions(+)
> create mode 100644 lib/librte_hash/rte_thash.h
>
>diff --git a/lib/librte_hash/Makefile b/lib/librte_hash/Makefile index
>3696cb1..981230b 100644
>--- a/lib/librte_hash/Makefile
>+++ b/lib/librte_hash/Makefile
>@@ -49,6 +49,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) += rte_fbk_hash.c
>SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include := rte_hash.h
>SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_hash_crc.h
>SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_jhash.h
>+SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_thash.h
> SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_fbk_hash.h
>
> # this lib needs eal
>diff --git a/lib/librte_hash/rte_thash.h b/lib/librte_hash/rte_thash.h new
file
>mode 100644 index 0000000..1808f47
>--- /dev/null
>+++ b/lib/librte_hash/rte_thash.h
>@@ -0,0 +1,231 @@
>+/*-
>+ *   BSD LICENSE
>+ *
>+ *   Copyright(c) 2015 Vladimir Medvedkin <medvedkinv@gmail.com>
>+ *   All rights reserved.
>+ *
>+ *   Redistribution and use in source and binary forms, with or without
>+ *   modification, are permitted provided that the following conditions
>+ *   are met:
>+ *
>+ *     * Redistributions of source code must retain the above copyright
>+ *       notice, this list of conditions and the following disclaimer.
>+ *     * Redistributions in binary form must reproduce the above copyright
>+ *       notice, this list of conditions and the following disclaimer in
>+ *       the documentation and/or other materials provided with the
>+ *       distribution.
>+ *     * Neither the name of Intel Corporation nor the names of its
>+ *       contributors may be used to endorse or promote products derived
>+ *       from this software without specific prior written permission.
>+ *
>+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
>CONTRIBUTORS
>+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
>NOT
>+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
>FITNESS FOR
>+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
>COPYRIGHT
>+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
>INCIDENTAL,
>+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
>NOT
>+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
>OF USE,
>+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
>AND ON ANY
>+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
>TORT
>+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
>THE USE
>+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
>DAMAGE.
>+ */
>+
>+#ifndef _RTE_THASH_H
>+#define _RTE_THASH_H
>+
>+/**
>+ * @file
>+ *
>+ * toeplitz hash functions.
>+ */
>+
>+#ifdef __cplusplus
>+extern "C" {
>+#endif
>+
>+/**
>+ * Software implementation of the Toeplitz hash function used by RSS.
>+ * Can be used either for packet distribution on single queue NIC
>+ * or for simulating of RSS computation on specific NIC (for example
>+ * after GRE header decapsulating)
>+ */
>+
>+#include <stdint.h>
>+#include <rte_byteorder.h>
>+#include <rte_vect.h>
>+#include <rte_ip.h>
>+
>+/* Byte swap mask used for converting IPv6 address
>+ * 4-byte chunks to CPU byte order
>+ */
>+static const __m128i rte_thash_ipv6_bswap_mask = {
>+		0x0405060700010203, 0x0C0D0E0F08090A0B};
>+
>+/**
>+ * length in dwords of input tuple to
>+ * calculate hash of ipv4 header only
>+ */
>+#define RTE_THASH_V4_L3_LEN	((sizeof(struct rte_ipv4_tuple) -	\
>+			sizeof(((struct rte_ipv4_tuple *)0)->sctp_tag)) / 4)
>+
>+/**
>+ * length in dwords of input tuple to
>+ * calculate hash of ipv4 header +
>+ * transport header
>+ */
>+#define RTE_THASH_V4_L4_LEN	 ((sizeof(struct rte_ipv4_tuple)) / 4)
>+
>+/**
>+ * length in dwords of input tuple to
>+ * calculate hash of ipv6 header only
>+ */
>+#define RTE_THASH_V6_L3_LEN	((sizeof(struct rte_ipv6_tuple) -       \
>+			sizeof(((struct rte_ipv6_tuple *)0)->sctp_tag)) / 4)
>+
>+/**
>+ * length in dwords of input tuple to
>+ * calculate hash of ipv6 header +
>+ * transport header
>+ */
>+#define RTE_THASH_V6_L4_LEN	((sizeof(struct rte_ipv6_tuple)) / 4)
>+
>+/**
>+ * IPv4 tuple
>+ * addreses and ports/sctp_tag have to be CPU byte order  */ struct
>+rte_ipv4_tuple {
>+	uint32_t	src_addr;
>+	uint32_t	dst_addr;
>+	union {
>+		struct {
>+			uint16_t dport;
>+			uint16_t sport;
>+		};
>+		uint32_t        sctp_tag;
>+	};
>+};
>+
>+/**
>+ * IPv6 tuple
>+ * Addresses have to be filled by rte_thash_load_v6_addr()
>+ * ports/sctp_tag have to be CPU byte order  */ struct rte_ipv6_tuple {
>+	uint8_t		src_addr[16];
>+	uint8_t		dst_addr[16];
>+	union {
>+		struct {
>+			uint16_t dport;
>+			uint16_t sport;
>+		};
>+		uint32_t        sctp_tag;
>+	};
>+};
>+
>+union rte_thash_tuple {
>+	struct rte_ipv4_tuple	v4;
>+	struct rte_ipv6_tuple	v6;
>+} __attribute__((aligned(XMM_SIZE)));
>+
>+/**
>+ * Prepare special converted key to use with rte_softrss_be()
>+ * @param orig
>+ *   pointer to original RSS key
>+ * @param targ
>+ *   pointer to target RSS key
>+ * @param len
>+ *   RSS key length
>+ */
>+static inline void
>+rte_convert_rss_key(const uint32_t *orig, uint32_t *targ, int len) {
>+	int i;
>+
>+	for (i = 0; i < (len >> 2); i++)
>+		targ[i] = rte_be_to_cpu_32(orig[i]);
>+}
>+
>+/**
>+ * Prepare and load IPv6 addresses (src and dst)
>+ * into target tuple
>+ * @param orig
>+ *   Pointer to ipv6 header of the original packet
>+ * @param targ
>+ *   Pointer to rte_ipv6_tuple structure
>+ */
>+static inline void
>+rte_thash_load_v6_addrs(const struct ipv6_hdr *orig, union
>+rte_thash_tuple *targ) {
>+	__m128i ipv6 = _mm_loadu_si128((const __m128i *)orig->src_addr);
>+	*(__m128i *)targ->v6.src_addr =
>+			_mm_shuffle_epi8(ipv6, rte_thash_ipv6_bswap_mask);
>+	ipv6 = _mm_loadu_si128((const __m128i *)orig->dst_addr);
>+	*(__m128i *)targ->v6.dst_addr =
>+			_mm_shuffle_epi8(ipv6, rte_thash_ipv6_bswap_mask); }
>+
>+/**
>+ * Generic implementation. Can be used with original rss_key
>+ * @param input_tuple
>+ *   Pointer to input tuple
>+ * @param input_len
>+ *   Length of input_tuple in 4-bytes chunks
>+ * @param rss_key
>+ *   Pointer to RSS hash key.
>+ * @return
>+ *   Calculated hash value.
>+ */
>+static inline uint32_t
>+rte_softrss(uint32_t *input_tuple, uint32_t input_len,
>+		const uint8_t *rss_key)
>+{
>+	uint32_t i, j, ret = 0;
>+
>+	for (j = 0; j < input_len; j++) {
>+		for (i = 0; i < 32; i++) {
>+			if (input_tuple[j] & (1 << (31 - i))) {
>+				ret ^= rte_cpu_to_be_32(((const uint32_t
*)rss_key)[j]) << i
>|
>+
(uint32_t)((uint64_t)(rte_cpu_to_be_32(((const
>uint32_t *)rss_key)[j + 1])) >>
>+					(32 - i));
>+			}
>+		}
>+	}
>+	return ret;
>+}
>+
>+/**
>+ * Optimized implementation.
>+ * If you want the calculated hash value matches NIC RSS value
>+ * you have to use special converted key with rte_convert_rss_key() fn.
>+ * @param input_tuple
>+ *   Pointer to input tuple
>+ * @param input_len
>+ *   Length of input_tuple in 4-bytes chunks
>+ * @param *rss_key
>+ *   Pointer to RSS hash key.
>+ * @return
>+ *   Calculated hash value.
>+ */
>+static inline uint32_t
>+rte_softrss_be(uint32_t *input_tuple, uint32_t input_len,
>+		const uint8_t *rss_key)
>+{
>+	uint32_t i, j, ret = 0;
>+
>+	for (j = 0; j < input_len; j++) {
>+		for (i = 0; i < 32; i++) {
>+			if (input_tuple[j] & (1 << (31 - i))) {
>+				ret ^= ((const uint32_t *)rss_key)[j] << i |
>+					(uint32_t)((uint64_t)(((const
uint32_t *)rss_key)[j +
>1]) >> (32 - i));
>+			}
>+		}
>+	}
>+	return ret;
>+}
>+
>+#ifdef __cplusplus
>+}
>+#endif
>+
>+#endif /* _RTE_THASH_H */
>--
>1.8.3.2