From: Harjot Singh <Harjot.Singh@arm.com>
To: Thomas Monjalon <thomas@monjalon.net>,
Yipeng Wang <yipeng1.wang@intel.com>,
Sameh Gobriel <sameh.gobriel@intel.com>,
Bruce Richardson <bruce.richardson@intel.com>,
Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Cc: dev@dpdk.org, nd@arm.com, Harjot Singh <harjot.singh@arm.com>,
Nathan Brown <nathan.brown@arm.com>,
Feifei Wang <feifei.wang2@arm.com>,
Jieqiang Wang <jieqiang.wang@arm.com>,
Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Subject: [PATCH 1/1] hash: add SVE support for bulk key lookup
Date: Thu, 17 Aug 2023 21:24:17 +0000 [thread overview]
Message-ID: <20230817212417.3637080-2-Harjot.Singh@arm.com> (raw)
In-Reply-To: <20230817212417.3637080-1-Harjot.Singh@arm.com>
From: Harjot Singh <harjot.singh@arm.com>
- Implemented Vector Length Agnostic SVE code for comparing signatures
in bulk lookup.
- Added Defines in code for SVE code support.
- New Optimised SVE code is 1-2 CPU cycle slower than NEON for N2
processor.
Performance Numbers from hash_perf_autotest :
Elements in Primary or Secondary Location
Results (in CPU cycles/operation)
-----------------------------------
Operations without data
Without pre-computed hash values
Keysize Add/Lookup/Lookup_bulk
Neon SVE
4 93/71/26 93/71/27
8 93/70/26 93/70/27
9 94/74/27 94/74/28
13 100/80/31 100/79/32
16 100/78/30 100/78/31
32 109/110/38 108/110/39
With pre-computed hash values
Keysize Add/Lookup/Lookup_bulk
Neon SVE
4 83/58/27 83/58/29
8 83/57/27 83/57/28
9 83/60/28 83/60/29
13 84/60/28 83/60/29
16 83/58/27 83/58/29
32 84/68/31 84/68/32
Signed-off-by: Harjot Singh <harjot.singh@arm.com>
Reviewed-by: Nathan Brown <nathan.brown@arm.com>
Reviewed-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Jieqiang Wang <jieqiang.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
---
.mailmap | 1 +
lib/hash/rte_cuckoo_hash.c | 37 ++++++++++++++++++++++++++++++++++++-
lib/hash/rte_cuckoo_hash.h | 1 +
3 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/.mailmap b/.mailmap
index 864d33ee46..2cce48c900 100644
--- a/.mailmap
+++ b/.mailmap
@@ -481,6 +481,7 @@ Hari Kumar Vemula <hari.kumarx.vemula@intel.com>
Harini Ramakrishnan <harini.ramakrishnan@microsoft.com>
Hariprasad Govindharajan <hariprasad.govindharajan@intel.com>
Harish Patil <harish.patil@cavium.com> <harish.patil@qlogic.com>
+Harjot Singh <harjot.singh@arm.com>
Harman Kalra <hkalra@marvell.com>
Harneet Singh <harneet.singh@intel.com>
Harold Huang <baymaxhuang@gmail.com>
diff --git a/lib/hash/rte_cuckoo_hash.c b/lib/hash/rte_cuckoo_hash.c
index d92a903bb3..fdb06eb33e 100644
--- a/lib/hash/rte_cuckoo_hash.c
+++ b/lib/hash/rte_cuckoo_hash.c
@@ -435,8 +435,11 @@ rte_hash_create(const struct rte_hash_parameters *params)
h->sig_cmp_fn = RTE_HASH_COMPARE_SSE;
else
#elif defined(RTE_ARCH_ARM64)
- if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON))
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON)) {
h->sig_cmp_fn = RTE_HASH_COMPARE_NEON;
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_SVE))
+ h->sig_cmp_fn = RTE_HASH_COMPARE_SVE;
+ }
else
#endif
h->sig_cmp_fn = RTE_HASH_COMPARE_SCALAR;
@@ -1892,6 +1895,38 @@ compare_signatures(uint32_t *prim_hash_matches, uint32_t *sec_hash_matches,
*sec_hash_matches = (uint32_t)(vaddvq_u16(x));
}
break;
+#if defined(RTE_HAS_SVE_ACLE)
+ case RTE_HASH_COMPARE_SVE: {
+ svuint16_t vsign, shift, sv_prim_matches, sv_sec_matches;
+ svbool_t pred, p_match, s_match;
+ int i = 0;
+ uint64_t vl = svcnth();
+
+ vsign = svdup_u16(sig);
+ shift = svindex_u16(0, 2);
+ do {
+ pred = svwhilelt_b16(i, RTE_HASH_BUCKET_ENTRIES);
+ /* Compare all signatures in the primary bucket */
+ p_match = svcmpeq_u16(pred, vsign, svld1_u16(pred,
+ &prim_bkt->sig_current[i]));
+ if (svptest_any(svptrue_b16(), p_match)) {
+ sv_prim_matches = svdup_u16_z(p_match, 1);
+ sv_prim_matches = svlsl_u16_z(pred, sv_prim_matches, shift);
+ *prim_hash_matches |= svorv_u16(pred, sv_prim_matches);
+ }
+ /* Compare all signatures in the secondary bucket */
+ s_match = svcmpeq_u16(pred, vsign, svld1_u16(pred,
+ &sec_bkt->sig_current[i]));
+ if (svptest_any(svptrue_b16(), s_match)) {
+ sv_sec_matches = svdup_u16_z(s_match, 1);
+ sv_sec_matches = svlsl_u16_z(pred, sv_sec_matches, shift);
+ *sec_hash_matches |= svorv_u16(pred, sv_sec_matches);
+ }
+ i += vl;
+ } while (i < RTE_HASH_BUCKET_ENTRIES);
+ }
+ break;
+#endif
#endif
default:
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
diff --git a/lib/hash/rte_cuckoo_hash.h b/lib/hash/rte_cuckoo_hash.h
index eb2644f74b..356ec2a69e 100644
--- a/lib/hash/rte_cuckoo_hash.h
+++ b/lib/hash/rte_cuckoo_hash.h
@@ -148,6 +148,7 @@ enum rte_hash_sig_compare_function {
RTE_HASH_COMPARE_SCALAR = 0,
RTE_HASH_COMPARE_SSE,
RTE_HASH_COMPARE_NEON,
+ RTE_HASH_COMPARE_SVE,
RTE_HASH_COMPARE_NUM
};
--
2.25.1
next prev parent reply other threads:[~2023-08-17 21:24 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-17 21:24 [PATCH 0/1] " Harjot Singh
2023-08-17 21:24 ` Harjot Singh [this message]
2023-09-29 15:36 ` [PATCH 1/1] " David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230817212417.3637080-2-Harjot.Singh@arm.com \
--to=harjot.singh@arm.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=feifei.wang2@arm.com \
--cc=honnappa.nagarahalli@arm.com \
--cc=jieqiang.wang@arm.com \
--cc=nathan.brown@arm.com \
--cc=nd@arm.com \
--cc=sameh.gobriel@intel.com \
--cc=thomas@monjalon.net \
--cc=vladimir.medvedkin@intel.com \
--cc=yipeng1.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).