From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id E27DA1B3B3 for ; Sat, 13 Oct 2018 04:52:54 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Oct 2018 19:52:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,375,1534834800"; d="scan'208";a="88001117" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by FMSMGA003.fm.intel.com with ESMTP; 12 Oct 2018 19:52:53 -0700 Received: from FMSMSX109.amr.corp.intel.com (10.18.116.9) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 12 Oct 2018 19:52:53 -0700 Received: from fmsmsx151.amr.corp.intel.com ([169.254.7.87]) by FMSMSX109.amr.corp.intel.com ([169.254.15.83]) with mapi id 14.03.0319.002; Fri, 12 Oct 2018 19:52:52 -0700 From: "Wang, Yipeng1" To: Honnappa Nagarahalli , "Richardson, Bruce" , "De Lara Guarch, Pablo" CC: "dev@dpdk.org" , "dharmik.thakkar@arm.com" , "gavin.hu@arm.com" , "nd@arm.com" , "Gobriel, Sameh" Thread-Topic: [PATCH v3 7/7] test/hash: read-write lock-free concurrency test Thread-Index: AQHUYfVTgrT2OaZCxEq37Y6bI0FSj6Ucdi2Q Date: Sat, 13 Oct 2018 02:52:52 +0000 Message-ID: References: <1539325918-125438-1-git-send-email-honnappa.nagarahalli@arm.com> <1539325918-125438-8-git-send-email-honnappa.nagarahalli@arm.com> In-Reply-To: <1539325918-125438-8-git-send-email-honnappa.nagarahalli@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNmUxMjJmMjgtYTkyYS00Y2E0LWE5MmYtZjYyNzY1MTVjZTNkIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiY1pLRThvNFdXRjhHRHdDNVJDWk4xTVM4YlhLRWt4M1lcL3gzK0pQcnB3MzlFNjExTWh1QThpOU4xcE01U0JSMGMifQ== x-originating-ip: [10.1.200.107] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v3 7/7] test/hash: read-write lock-free concurrency test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2018 02:52:55 -0000 >-----Original Message----- >From: Honnappa Nagarahalli [mailto:honnappa.nagarahalli@arm.com] >Sent: Thursday, October 11, 2018 11:32 PM >To: Richardson, Bruce ; De Lara Guarch, Pablo = >Cc: dev@dpdk.org; Wang, Yipeng1 ; honnappa.nagarah= alli@arm.com; dharmik.thakkar@arm.com; >gavin.hu@arm.com; nd@arm.com >Subject: [PATCH v3 7/7] test/hash: read-write lock-free concurrency test > >From: Dharmik Thakkar > >Unit tests to check for hash lookup perf with lock-free enabled and >with lock-free disabled. >Unit tests performed with readers running in parallel with writers. > >Tests include: > >- hash lookup on existing keys with: > - hash add causing NO key-shifts of existing keys in the table > >- hash lookup on existing keys likely to be on shift-path with: > - hash add causing key-shifts of existing keys in the table > >- hash lookup on existing keys NOT likely to be on shift-path with: > - hash add causing key-shifts of existing keys in the table > >- hash lookup on non-existing keys with: > - hash add causing NO key-shifts of existing keys in the table > - hash add causing key-shifts of existing keys in the table > >- hash lookup on keys likely to be on shift-path with: > - multiple writers causing key-shifts of existing keys in the table > >Signed-off-by: Dharmik Thakkar >Reviewed-by: Honnappa Nagarahalli >Reviewed-by: Gavin Hu >--- > test/test/Makefile | 1 + > test/test/meson.build | 1 + > test/test/test_hash_readwrite_lf.c | 1084 +++++++++++++++++++++++++++++++= +++++ > 3 files changed, 1086 insertions(+) > create mode 100644 test/test/test_hash_readwrite_lf.c > >diff --git a/test/test/Makefile b/test/test/Makefile >index 5d8b1dc..0e0e6c4 100644 >--- a/test/test/Makefile >+++ b/test/test/Makefile >@@ -116,6 +116,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_HASH) +=3D test_hash_function= s.c > SRCS-$(CONFIG_RTE_LIBRTE_HASH) +=3D test_hash_scaling.c > SRCS-$(CONFIG_RTE_LIBRTE_HASH) +=3D test_hash_multiwriter.c > SRCS-$(CONFIG_RTE_LIBRTE_HASH) +=3D test_hash_readwrite.c >+SRCS-$(CONFIG_RTE_LIBRTE_HASH) +=3D test_hash_readwrite_lf.c > > SRCS-$(CONFIG_RTE_LIBRTE_LPM) +=3D test_lpm.c > SRCS-$(CONFIG_RTE_LIBRTE_LPM) +=3D test_lpm_perf.c >diff --git a/test/test/meson.build b/test/test/meson.build >index d696f5e..bc3350f 100644 >--- a/test/test/meson.build >+++ b/test/test/meson.build >@@ -46,6 +46,7 @@ test_sources =3D files('commands.c', > 'test_hash_multiwriter.c', > 'test_hash_readwrite.c', > 'test_hash_perf.c', >+ 'test_hash_readwrite_lf.c', > 'test_hash_scaling.c', > 'test_interrupts.c', > 'test_kni.c', >diff --git a/test/test/test_hash_readwrite_lf.c b/test/test/test_hash_read= write_lf.c >new file mode 100644 >index 0000000..841e989 >--- /dev/null >+++ b/test/test/test_hash_readwrite_lf.c >@@ -0,0 +1,1084 @@ >+/* SPDX-License-Identifier: BSD-3-Clause >+ * Copyright(c) 2018 Arm Limited >+ */ >+ >+#include >+#include >+ >+#include >+#include >+#include >+#include >+#include >+#include >+#include >+#include >+ >+#include "test.h" >+ >+#ifndef RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF >+#define RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF 0 >+#endif >+ >+#define RUN_WITH_HTM_DISABLED 0 >+ >+#if (RUN_WITH_HTM_DISABLED) >+ >+#define TOTAL_ENTRY (5*1024) >+#define TOTAL_INSERT (5*1024) >+ >+#else >+ >+#define TOTAL_ENTRY (16*1024*1024) >+#define TOTAL_INSERT (16*1024*1024) [Wang, Yipeng]=20 You can make the number smaller since a previous patch on multi-write test suggests to use a smaller number (how about 5 million) to reduce the total = testing time. >+ >+static rte_atomic64_t gread_cycles; >+static rte_atomic64_t greads; >+ >+static volatile uint8_t writer_done; >+static volatile uint8_t multi_writer_done[4]; >+uint8_t num_test; >+uint8_t htm; [Wang, Yipeng] It would be better to make the variables local. >+ >+ for (i =3D 0; i < count_keys_no_ks; i++) { >+ /* Identify keys in keys_no_ks with value less than 1M */ [Wang, Yipeng] TOTAL_INSERT is 16M. Other comments too. >+ if (keys_no_ks[i] < TOTAL_INSERT) >+ found[keys_no_ks[i]]++; >+ } >+ >+ for (i =3D 0; i < count_keys_ks; i++) { >+ /* Identify keys in keys_ks with value less than 1M */ >+ if (keys_ks[i] < TOTAL_INSERT) >+ found[keys_ks[i]]++; >+ } >+ >+ uint32_t count_keys_absent =3D 0; >+ for (i =3D 0; i < TOTAL_INSERT; i++) { >+ /* Identify missing keys between 0 and 1M */ >+/* >+ * Test lookup perf: [Wang, Yipeng] add comment: for multi-writer >+ * Reader(s) lookup keys present in the table and likely on the shift-pat= h while >+ * Writers add keys causing key-shifts. >+ */ >+static int >+test_hash_multi_add_lookup(struct rwc_perf *rwc_perf_results, int rwc_lf) >+{ >+ unsigned int n, m; >+ uint64_t i; >+static int >+test_hash_readwrite_lf_main(void) >+{ >+ /* >+ * Variables used to choose different tests. >+ * rwc_lf indicates if read-write concurrency lock-free support is >+ * enabled. >+ * htm indicates if Hardware transactional memory support is enabled. >+ */ >+ int rwc_lf =3D 0; >+ int use_jhash =3D 0; >+ num_test =3D NUM_TEST; >+ if (rte_lcore_count() =3D=3D 1) { >+ printf("More than one lcore is required " >+ "to do read write lock-free concurrency test\n"); >+ return -1; >+ } >+ >+ setlocale(LC_NUMERIC, ""); >+ >+ if (rte_tm_supported()) >+ htm =3D 1; >+ else >+ htm =3D 0; >+ >+ if (init_params(rwc_lf, use_jhash) !=3D 0) >+ return -1; >+ if (generate_keys() !=3D 0) [Wang, Yipeng] Generate_keys may run for a long time, so print something to= indicate the test has started. >+ return -1; >+ if (get_enabled_cores_list() !=3D 0) >+ return -1; >+ [Wang, Yipeng] Please change/add multi-lookup(bulk lookup) to all the tests. Bulk lookup i= s usually more commonly used and people care about as a performance test. It would make sense to change other multi-thread performance unit test to u= se bulk lookup as well in future.=20