From: "Wang, Yipeng1" <yipeng1.wang@intel.com>
To: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>,
"De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"dharmik.thakkar@arm.com" <dharmik.thakkar@arm.com>,
"gavin.hu@arm.com" <gavin.hu@arm.com>, "nd@arm.com" <nd@arm.com>,
"Gobriel, Sameh" <sameh.gobriel@intel.com>
Subject: Re: [dpdk-dev] [PATCH v3 7/7] test/hash: read-write lock-free concurrency test
Date: Sat, 13 Oct 2018 02:52:52 +0000 [thread overview]
Message-ID: <D2C4A16CA39F7F4E8E384D204491D7A6614EFBDD@FMSMSX151.amr.corp.intel.com> (raw)
In-Reply-To: <1539325918-125438-8-git-send-email-honnappa.nagarahalli@arm.com>
>-----Original Message-----
>From: Honnappa Nagarahalli [mailto:honnappa.nagarahalli@arm.com]
>Sent: Thursday, October 11, 2018 11:32 PM
>To: Richardson, Bruce <bruce.richardson@intel.com>; De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
>Cc: dev@dpdk.org; Wang, Yipeng1 <yipeng1.wang@intel.com>; honnappa.nagarahalli@arm.com; dharmik.thakkar@arm.com;
>gavin.hu@arm.com; nd@arm.com
>Subject: [PATCH v3 7/7] test/hash: read-write lock-free concurrency test
>
>From: Dharmik Thakkar <dharmik.thakkar@arm.com>
>
>Unit tests to check for hash lookup perf with lock-free enabled and
>with lock-free disabled.
>Unit tests performed with readers running in parallel with writers.
>
>Tests include:
>
>- hash lookup on existing keys with:
> - hash add causing NO key-shifts of existing keys in the table
>
>- hash lookup on existing keys likely to be on shift-path with:
> - hash add causing key-shifts of existing keys in the table
>
>- hash lookup on existing keys NOT likely to be on shift-path with:
> - hash add causing key-shifts of existing keys in the table
>
>- hash lookup on non-existing keys with:
> - hash add causing NO key-shifts of existing keys in the table
> - hash add causing key-shifts of existing keys in the table
>
>- hash lookup on keys likely to be on shift-path with:
> - multiple writers causing key-shifts of existing keys in the table
>
>Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
>Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
>Reviewed-by: Gavin Hu <gavin.hu@arm.com>
>---
> test/test/Makefile | 1 +
> test/test/meson.build | 1 +
> test/test/test_hash_readwrite_lf.c | 1084 ++++++++++++++++++++++++++++++++++++
> 3 files changed, 1086 insertions(+)
> create mode 100644 test/test/test_hash_readwrite_lf.c
>
>diff --git a/test/test/Makefile b/test/test/Makefile
>index 5d8b1dc..0e0e6c4 100644
>--- a/test/test/Makefile
>+++ b/test/test/Makefile
>@@ -116,6 +116,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_functions.c
> SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_scaling.c
> SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_multiwriter.c
> SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_readwrite.c
>+SRCS-$(CONFIG_RTE_LIBRTE_HASH) += test_hash_readwrite_lf.c
>
> SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm.c
> SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c
>diff --git a/test/test/meson.build b/test/test/meson.build
>index d696f5e..bc3350f 100644
>--- a/test/test/meson.build
>+++ b/test/test/meson.build
>@@ -46,6 +46,7 @@ test_sources = files('commands.c',
> 'test_hash_multiwriter.c',
> 'test_hash_readwrite.c',
> 'test_hash_perf.c',
>+ 'test_hash_readwrite_lf.c',
> 'test_hash_scaling.c',
> 'test_interrupts.c',
> 'test_kni.c',
>diff --git a/test/test/test_hash_readwrite_lf.c b/test/test/test_hash_readwrite_lf.c
>new file mode 100644
>index 0000000..841e989
>--- /dev/null
>+++ b/test/test/test_hash_readwrite_lf.c
>@@ -0,0 +1,1084 @@
>+/* SPDX-License-Identifier: BSD-3-Clause
>+ * Copyright(c) 2018 Arm Limited
>+ */
>+
>+#include <inttypes.h>
>+#include <locale.h>
>+
>+#include <rte_cycles.h>
>+#include <rte_hash.h>
>+#include <rte_hash_crc.h>
>+#include <rte_jhash.h>
>+#include <rte_launch.h>
>+#include <rte_malloc.h>
>+#include <rte_random.h>
>+#include <rte_spinlock.h>
>+
>+#include "test.h"
>+
>+#ifndef RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF
>+#define RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF 0
>+#endif
>+
>+#define RUN_WITH_HTM_DISABLED 0
>+
>+#if (RUN_WITH_HTM_DISABLED)
>+
>+#define TOTAL_ENTRY (5*1024)
>+#define TOTAL_INSERT (5*1024)
>+
>+#else
>+
>+#define TOTAL_ENTRY (16*1024*1024)
>+#define TOTAL_INSERT (16*1024*1024)
[Wang, Yipeng]
You can make the number smaller since a previous patch on multi-write test
suggests to use a smaller number (how about 5 million) to reduce the total testing time.
>+
>+static rte_atomic64_t gread_cycles;
>+static rte_atomic64_t greads;
>+
>+static volatile uint8_t writer_done;
>+static volatile uint8_t multi_writer_done[4];
>+uint8_t num_test;
>+uint8_t htm;
[Wang, Yipeng] It would be better to make the variables local.
>+
>+ for (i = 0; i < count_keys_no_ks; i++) {
>+ /* Identify keys in keys_no_ks with value less than 1M */
[Wang, Yipeng] TOTAL_INSERT is 16M. Other comments too.
>+ if (keys_no_ks[i] < TOTAL_INSERT)
>+ found[keys_no_ks[i]]++;
>+ }
>+
>+ for (i = 0; i < count_keys_ks; i++) {
>+ /* Identify keys in keys_ks with value less than 1M */
>+ if (keys_ks[i] < TOTAL_INSERT)
>+ found[keys_ks[i]]++;
>+ }
>+
>+ uint32_t count_keys_absent = 0;
>+ for (i = 0; i < TOTAL_INSERT; i++) {
>+ /* Identify missing keys between 0 and 1M */
>+/*
>+ * Test lookup perf:
[Wang, Yipeng] add comment: for multi-writer
>+ * Reader(s) lookup keys present in the table and likely on the shift-path while
>+ * Writers add keys causing key-shifts.
>+ */
>+static int
>+test_hash_multi_add_lookup(struct rwc_perf *rwc_perf_results, int rwc_lf)
>+{
>+ unsigned int n, m;
>+ uint64_t i;
>+static int
>+test_hash_readwrite_lf_main(void)
>+{
>+ /*
>+ * Variables used to choose different tests.
>+ * rwc_lf indicates if read-write concurrency lock-free support is
>+ * enabled.
>+ * htm indicates if Hardware transactional memory support is enabled.
>+ */
>+ int rwc_lf = 0;
>+ int use_jhash = 0;
>+ num_test = NUM_TEST;
>+ if (rte_lcore_count() == 1) {
>+ printf("More than one lcore is required "
>+ "to do read write lock-free concurrency test\n");
>+ return -1;
>+ }
>+
>+ setlocale(LC_NUMERIC, "");
>+
>+ if (rte_tm_supported())
>+ htm = 1;
>+ else
>+ htm = 0;
>+
>+ if (init_params(rwc_lf, use_jhash) != 0)
>+ return -1;
>+ if (generate_keys() != 0)
[Wang, Yipeng] Generate_keys may run for a long time, so print something to indicate
the test has started.
>+ return -1;
>+ if (get_enabled_cores_list() != 0)
>+ return -1;
>+
[Wang, Yipeng]
Please change/add multi-lookup(bulk lookup) to all the tests. Bulk lookup is usually more commonly used and people care about
as a performance test.
It would make sense to change other multi-thread performance unit test to use bulk lookup as well in future.
prev parent reply other threads:[~2018-10-13 2:52 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-12 6:31 [dpdk-dev] [PATCH v3 0/7] Address reader-writer concurrency in rte_hash Honnappa Nagarahalli
2018-10-12 6:31 ` [dpdk-dev] [PATCH v3 1/7] hash: separate multi-writer from rw-concurrency Honnappa Nagarahalli
2018-10-13 1:02 ` Wang, Yipeng1
2018-10-15 20:15 ` Honnappa Nagarahalli
2018-10-12 6:31 ` [dpdk-dev] [PATCH v3 2/7] hash: support do not recycle on delete Honnappa Nagarahalli
2018-10-13 1:17 ` Wang, Yipeng1
2018-10-16 1:25 ` Honnappa Nagarahalli
2018-10-12 6:31 ` [dpdk-dev] [PATCH v3 3/7] hash: correct key store element alignment Honnappa Nagarahalli
2018-10-13 1:20 ` Wang, Yipeng1
2018-10-16 23:26 ` Honnappa Nagarahalli
2018-10-12 6:31 ` [dpdk-dev] [PATCH v3 4/7] hash: add memory ordering to avoid race conditions Honnappa Nagarahalli
2018-10-13 1:56 ` Wang, Yipeng1
2018-10-16 23:28 ` Honnappa Nagarahalli
2018-10-12 6:31 ` [dpdk-dev] [PATCH v3 5/7] hash: fix rw concurrency while moving keys Honnappa Nagarahalli
2018-10-13 2:07 ` Wang, Yipeng1
2018-10-12 6:31 ` [dpdk-dev] [PATCH v3 6/7] hash: enable lock-free reader-writer concurrency Honnappa Nagarahalli
2018-10-13 2:32 ` Wang, Yipeng1
2018-10-17 13:54 ` Honnappa Nagarahalli
2018-10-12 6:31 ` [dpdk-dev] [PATCH v3 7/7] test/hash: read-write lock-free concurrency test Honnappa Nagarahalli
2018-10-13 2:52 ` Wang, Yipeng1 [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=D2C4A16CA39F7F4E8E384D204491D7A6614EFBDD@FMSMSX151.amr.corp.intel.com \
--to=yipeng1.wang@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=dharmik.thakkar@arm.com \
--cc=gavin.hu@arm.com \
--cc=honnappa.nagarahalli@arm.com \
--cc=nd@arm.com \
--cc=pablo.de.lara.guarch@intel.com \
--cc=sameh.gobriel@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).