From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8BCC7A0598; Fri, 10 Apr 2020 18:59:43 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 81D111D606; Fri, 10 Apr 2020 18:59:42 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2087.outbound.protection.outlook.com [40.107.20.87]) by dpdk.org (Postfix) with ESMTP id 9DC6F1D5B7 for ; Fri, 10 Apr 2020 18:59:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ng+SS3FqzbkF4wU68ygqALKEMPfdJqbb6fzU+KNo8BI=; b=va6r1Kh0JWR45onbCZlS8kzONOg2xQY2V4K3mfNyuqV28Eg2V+ieq3dPoIwPDbjd0wB4I8CCCokX6uocvPgbTt1EoxBOQgCmI26S3puHawCH0N/k4PvtH2068M+LTPxN6cRRwHdFxiN4r1zW7pcJO2oDN7VbpIup0RD5JfQzb3I= Received: from AM6P192CA0101.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:8d::42) by AM0PR08MB3252.eurprd08.prod.outlook.com (2603:10a6:208:5e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.15; Fri, 10 Apr 2020 16:59:39 +0000 Received: from AM5EUR03FT028.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:8d:cafe::9e) by AM6P192CA0101.outlook.office365.com (2603:10a6:209:8d::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.15 via Frontend Transport; Fri, 10 Apr 2020 16:59:39 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT028.mail.protection.outlook.com (10.152.16.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.18 via Frontend Transport; Fri, 10 Apr 2020 16:59:38 +0000 Received: ("Tessian outbound 1425309d4c0b:v50"); Fri, 10 Apr 2020 16:59:38 +0000 X-CR-MTA-TID: 64aa7808 Received: from 9c9b050f44f4.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E836F06D-2933-41EA-8726-E0CAF1142067.1; Fri, 10 Apr 2020 16:59:33 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9c9b050f44f4.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 10 Apr 2020 16:59:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RuqviugYk47YgkPmlDbHk2Te5Q278azZeA2LnKYG+ERiu+fuXnSQMusg7rCeTVt6VHMASaTEL2BBA7I0olSmcpBJvpql3yXhQyfcX5fm4Wf24d9PIK65F0rm5emfpJTxLe98a0NyvDtoRWC7mbX3VpfzOIMdC4BiuYxCemLHLlnZvQDXa8zfxmUNdSH5Z2NVJO72UnHPQxuV9qrI93Z0qFrpWKhnhdUzg0hlzBtlbFf4v0Iicyo0rBMBsMt7lha4lKL2bvRFm6M6DXDqFMXX9wZz2jM7oO+T5Pzh5X73SfnJo9tbii5vpdVJ4bWjZko6PycNhqKJJX/EUFqAb5d4Rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ng+SS3FqzbkF4wU68ygqALKEMPfdJqbb6fzU+KNo8BI=; b=mQZtsKu7Z/UkMw3ytV6PgcERPn4Kjpa6GzAhfMDVHVY00tn846khpkAGaNOLzqDUHP42C9nKRa7snqsuzTVgB+CwJymLCcBOfe151T01zWe7gJI8DYXS9KgViJ6TyuhqBSLaQirzEFz+10Hwfx8qGBsZWU8E2vdK3weLo9cuOxMkkHHFC+4nRX9KD/NAwdqujIskXJV0WCxCIocbl/AcYvE6KydIpS8Vpx/vBAQRX4H08MUG+8roCfieJLYGX/f8sc9iEsa3nzQCzVzaGfh1G400RC1B+6GITSYA01lDXwYN6Dqh2YQq8bWolLO1gCzlCFc5DJWSxCahnRMsXsnxeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ng+SS3FqzbkF4wU68ygqALKEMPfdJqbb6fzU+KNo8BI=; b=va6r1Kh0JWR45onbCZlS8kzONOg2xQY2V4K3mfNyuqV28Eg2V+ieq3dPoIwPDbjd0wB4I8CCCokX6uocvPgbTt1EoxBOQgCmI26S3puHawCH0N/k4PvtH2068M+LTPxN6cRRwHdFxiN4r1zW7pcJO2oDN7VbpIup0RD5JfQzb3I= Received: from DBBPR08MB4646.eurprd08.prod.outlook.com (2603:10a6:10:f5::16) by DBBPR08MB4632.eurprd08.prod.outlook.com (2603:10a6:10:db::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.24; Fri, 10 Apr 2020 16:59:26 +0000 Received: from DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d]) by DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d%5]) with mapi id 15.20.2878.023; Fri, 10 Apr 2020 16:59:25 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "jielong.zjl@antfin.com" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v3 1/9] test/ring: add contention stress test Thread-Index: AQHWCd9R1w3FUy2800GE5i4fCZDVVqhsxRDQgAP+NICAAdTm8A== Date: Fri, 10 Apr 2020 16:59:25 +0000 Message-ID: References: <20200402220959.29885-1-konstantin.ananyev@intel.com> <20200403174235.23308-1-konstantin.ananyev@intel.com> <20200403174235.23308-2-konstantin.ananyev@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 246896b0-528d-493f-b20b-85c4b1ef00f7.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 3e75ee18-95ae-45d7-9cf1-08d7dd708e1e x-ms-traffictypediagnostic: DBBPR08MB4632:|DBBPR08MB4632:|AM0PR08MB3252: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:1923;OLM:1923; x-forefront-prvs: 0369E8196C X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBBPR08MB4646.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(396003)(346002)(39860400002)(376002)(136003)(366004)(30864003)(186003)(8676002)(54906003)(52536014)(7696005)(478600001)(8936002)(26005)(66476007)(66556008)(6506007)(5660300002)(64756008)(76116006)(81156014)(66446008)(66946007)(33656002)(9686003)(2906002)(55016002)(4326008)(71200400001)(316002)(110136005)(86362001)(579004); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 5ztTF2jsxR1MWetzKPvzJcDG9wq2KBv8QmFwVGf0xHYfvsYPsg47Z/OlM6GVY/Y9Y6NwNmP4U3t3GnP8x85GMVgq7no/3Ha12yK1bVs6qFSZtV/kib/8Rr27gG0Uaql3Qo3wtK1gXghC2EUBfJu4iDXxbeu57O4n2gTzKyOUIjdt4jNmraba4H04upMI1I7kl79iAsCmc/+tV6o5A3X1uJwKQ081aBIThQ5nRUQuxkHi9TPK/7ch8jzQHi5uIPCYNl4ofPrVE0AvIw5qmChYC/GWeANQJYfOzsLjceJU4oBlqxsXC1O8U/62/M77cdw1opPH2bMjOPSBs05CVV9qICKCWNbT0GzX62WjU2hvXD8VH7jyw3a6om/nYohA3smtXpzZW9+YJlaS4Rpy0Of5rAHZ7OV7a/VtWzCHcYIDKWEUCajDQYwMIlidmP4czmo/ x-ms-exchange-antispam-messagedata: xyzuWwFEMI9gxEpXl6W3R3BVQxsRn0eZxKQX24UJ7S+fvT8Qleh5iPK6XwKQx5X3xiOkEcG1uYunDyZoRRXTCcl2DVOTORqbj2RNEyFYYyOBrls5kP5Msy2Mmtfu8Z7DkRaNeoxnxTiXc91XjemEYg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4632 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT028.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(39860400002)(396003)(376002)(136003)(346002)(46966005)(8936002)(52536014)(2906002)(4326008)(26005)(336012)(186003)(356005)(7696005)(55016002)(9686003)(81156014)(8676002)(81166007)(6506007)(26826003)(478600001)(33656002)(47076004)(70586007)(5660300002)(36906005)(54906003)(30864003)(110136005)(316002)(70206006)(86362001)(82740400003); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 8eaa18ce-6e75-4545-078d-08d7dd708692 X-Forefront-PRVS: 0369E8196C X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tbyxsGxiBwp05M66pxReX5oIBHDhFc/Vx6sWDs3imxjd9X+FHZtEa6uIIXuLS56UgizyerPTuuFnDOiOj7tj1x2qDQWo04TB8jjtiVcY/nkr93RsEnqzpi5e0xzLDgaPnXqNGg3iMOq/cN16Ok1r9v3v4TZ3A54ORVbcKXQeVrN/NuCAaMVgtbfzWF1uzzpOzAm6pvAqKqG/ncJ+C7IUuSCPAXlkYbRcIzEyn0cJR3DK7CWujZNYOko1PxPT1PC74oO2eHkxXDDyaCK2TdguCfaFrCNMPc832Aqe9NJ4toGeKf0nc6KxDQAKp/eR2sGJB0DIqnZiK11k68MpZJ62VnszzmzRk5eCfcILacJh/pkNd+5l81yDiHIh4RMYG6InrQuqCU9HdUBAeTs0Qp4aAZaBawp8ZzqgYclNwZVtV02W3sbi0MwbFE7J2AaI49SVoRhljp7gk3xk/5CAAc1UkNDFoEXzJ0x3++P9v/LLvuJVoKndeFDv4ecgMH9z+LuHcIEstk+8n34/wESptmdC/Q== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Apr 2020 16:59:38.5950 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3e75ee18-95ae-45d7-9cf1-08d7dd708e1e X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3252 Subject: Re: [dpdk-dev] [PATCH v3 1/9] test/ring: add contention stress test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > Subject: [PATCH v3 1/9] test/ring: add contention stress test > > Minor, would 'add stress test for overcommitted use case' sound better? >=20 > I liked to point out that this test-case can be used as contention stres= s-test > (many threads do enqueue/dequeue to/from the same ring) for both over- > committed and not scenarios... > Will probably try to add few extra explanations in v4. The test cases for non-over committed case are already available in the fun= ction 'run_on_all_cores'. There we are running enqueue/dequeue in all the a= vailable cores. >=20 > > > > > > Introduce new test-case to measure ring perfomance under contention > > Minor, 'over committed' seems to the word commonly used from the > references you provided. Does it make sense to use that? > > > > > (miltiple producers/consumers). > > ^^^^^^^ multiple >=20 > ack. >=20 > > > > > Starts dequeue/enqueue loop on all available slave lcores. > > > > > > Signed-off-by: Konstantin Ananyev > > > --- > > > app/test/Makefile | 2 + > > > app/test/meson.build | 2 + > > > app/test/test_ring_mpmc_stress.c | 31 +++ > > > app/test/test_ring_stress.c | 48 ++++ > > > app/test/test_ring_stress.h | 35 +++ > > > app/test/test_ring_stress_impl.h | 444 > > > +++++++++++++++++++++++++++++++ > > Would be good to change the file names to indicate that these tests are= for > over-committed usecase/configuration. > > These are performance tests, better to have 'perf' or 'performance' in = their > names. > > > > > 6 files changed, 562 insertions(+) > > > create mode 100644 app/test/test_ring_mpmc_stress.c create mode > > > 100644 app/test/test_ring_stress.c create mode 100644 > > > app/test/test_ring_stress.h create mode 100644 > > > app/test/test_ring_stress_impl.h > > > > > > diff --git a/app/test/Makefile b/app/test/Makefile index > > > 1f080d162..4eefaa887 100644 > > > --- a/app/test/Makefile > > > +++ b/app/test/Makefile > > > @@ -77,7 +77,9 @@ SRCS-y +=3D test_external_mem.c SRCS-y +=3D > > > test_rand_perf.c > > > > > > SRCS-y +=3D test_ring.c > > > +SRCS-y +=3D test_ring_mpmc_stress.c > > > SRCS-y +=3D test_ring_perf.c > > > +SRCS-y +=3D test_ring_stress.c > > > SRCS-y +=3D test_pmd_perf.c > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_TABLE),y) diff --git > > > a/app/test/meson.build b/app/test/meson.build index > > > 351d29cb6..827b04886 100644 > > > --- a/app/test/meson.build > > > +++ b/app/test/meson.build > > > @@ -100,7 +100,9 @@ test_sources =3D files('commands.c', > > > 'test_rib.c', > > > 'test_rib6.c', > > > 'test_ring.c', > > > + 'test_ring_mpmc_stress.c', > > > 'test_ring_perf.c', > > > + 'test_ring_stress.c', > > > 'test_rwlock.c', > > > 'test_sched.c', > > > 'test_service_cores.c', > > > diff --git a/app/test/test_ring_mpmc_stress.c > > > b/app/test/test_ring_mpmc_stress.c > > > new file mode 100644 > > > index 000000000..1524b0248 > > > --- /dev/null > > > +++ b/app/test/test_ring_mpmc_stress.c > > > @@ -0,0 +1,31 @@ > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > + * Copyright(c) 2020 Intel Corporation */ > > > + > > > +#include "test_ring_stress_impl.h" > > > + > > > +static inline uint32_t > > > +_st_ring_dequeue_bulk(struct rte_ring *r, void **obj, uint32_t n, > > > + uint32_t *avail) > > > +{ > > > + return rte_ring_mc_dequeue_bulk(r, obj, n, avail); } > > > + > > > +static inline uint32_t > > > +_st_ring_enqueue_bulk(struct rte_ring *r, void * const *obj, uint32_= t n, > > > + uint32_t *free) > > > +{ > > > + return rte_ring_mp_enqueue_bulk(r, obj, n, free); } > > > + > > > +static int > > > +_st_ring_init(struct rte_ring *r, const char *name, uint32_t num) { > > > + return rte_ring_init(r, name, num, 0); } > > > + > > > +const struct test test_ring_mpmc_stress =3D { > > > + .name =3D "MP/MC", > > > + .nb_case =3D RTE_DIM(tests), > > > + .cases =3D tests, > > > +}; > > > diff --git a/app/test/test_ring_stress.c > > > b/app/test/test_ring_stress.c new file mode 100644 index > > > 000000000..60706f799 > > > --- /dev/null > > > +++ b/app/test/test_ring_stress.c > > > @@ -0,0 +1,48 @@ > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > + * Copyright(c) 2020 Intel Corporation */ > > > + > > > +#include "test_ring_stress.h" > > > + > > > +static int > > > +run_test(const struct test *test) > > > +{ > > > + int32_t rc; > > > + uint32_t i, k; > > > + > > > + for (i =3D 0, k =3D 0; i !=3D test->nb_case; i++) { > > > + > > > + printf("TEST-CASE %s %s START\n", > > > + test->name, test->cases[i].name); > > > + > > > + rc =3D test->cases[i].func(test->cases[i].wfunc); > > > + k +=3D (rc =3D=3D 0); > > > + > > > + if (rc !=3D 0) > > > + printf("TEST-CASE %s %s FAILED\n", > > > + test->name, test->cases[i].name); > > > + else > > > + printf("TEST-CASE %s %s OK\n", > > > + test->name, test->cases[i].name); > > > + } > > > + > > > + return k; > > > +} > > > + > > > +static int > > > +test_ring_stress(void) > > > +{ > > > + uint32_t n, k; > > > + > > > + n =3D 0; > > > + k =3D 0; > > > + > > > + n +=3D test_ring_mpmc_stress.nb_case; > > > + k +=3D run_test(&test_ring_mpmc_stress); > > > + > > > + printf("Number of tests:\t%u\nSuccess:\t%u\nFailed:\t%u\n", > > > + n, k, n - k); > > > + return (k !=3D n); > > > +} > > > + > > > +REGISTER_TEST_COMMAND(ring_stress_autotest, test_ring_stress); > > > diff --git a/app/test/test_ring_stress.h > > > b/app/test/test_ring_stress.h new file mode 100644 index > > > 000000000..60eac6216 > > > --- /dev/null > > > +++ b/app/test/test_ring_stress.h > > > @@ -0,0 +1,35 @@ > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > + * Copyright(c) 2020 Intel Corporation */ > > > + > > > + > > > +#include > > > +#include > > > +#include > > > +#include > > > +#include > > > +#include > > > + > > > +#include > > > +#include > > > +#include > > > +#include > > > +#include > > > +#include > > > +#include > > > + > > > +#include "test.h" > > > + > > > +struct test_case { > > > + const char *name; > > > + int (*func)(int (*)(void *)); > > > + int (*wfunc)(void *arg); > > > +}; > > > + > > > +struct test { > > > + const char *name; > > > + uint32_t nb_case; > > > + const struct test_case *cases; > > > +}; > > > + > > > +extern const struct test test_ring_mpmc_stress; > > > diff --git a/app/test/test_ring_stress_impl.h > > > b/app/test/test_ring_stress_impl.h > > > new file mode 100644 > > > index 000000000..11476d28c > > > --- /dev/null > > > +++ b/app/test/test_ring_stress_impl.h > > > @@ -0,0 +1,444 @@ > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > + * Copyright(c) 2020 Intel Corporation */ > > > + > > > +#include "test_ring_stress.h" > > > + > > > +/* > > > + * Measures performance of ring enqueue/dequeue under high > > > +contention */ > > > + > > > +#define RING_NAME "RING_STRESS" > > > +#define BULK_NUM 32 > > > +#define RING_SIZE (2 * BULK_NUM * RTE_MAX_LCORE) > > > + > > > +enum { > > > + WRK_CMD_STOP, > > > + WRK_CMD_RUN, > > > +}; > > > + > > > +static volatile uint32_t wrk_cmd __rte_cache_aligned; > > > + > > > +/* test run-time in seconds */ > > > +static const uint32_t run_time =3D 60; static const uint32_t verbose= ; > > > + > > > +struct lcore_stat { > > > + uint64_t nb_cycle; > > > + struct { > > > + uint64_t nb_call; > > > + uint64_t nb_obj; > > > + uint64_t nb_cycle; > > > + uint64_t max_cycle; > > > + uint64_t min_cycle; > > > + } op; > > > +}; > > > + > > > +struct lcore_arg { > > > + struct rte_ring *rng; > > > + struct lcore_stat stats; > > > +} __rte_cache_aligned; > > > + > > > +struct ring_elem { > > > + uint32_t cnt[RTE_CACHE_LINE_SIZE / sizeof(uint32_t)]; } > > > +__rte_cache_aligned; > > > + > > > +/* > > > + * redefinable functions > > > + */ > > > +static uint32_t > > > +_st_ring_dequeue_bulk(struct rte_ring *r, void **obj, uint32_t n, > > > + uint32_t *avail); > > > + > > > +static uint32_t > > > +_st_ring_enqueue_bulk(struct rte_ring *r, void * const *obj, uint32_= t n, > > > + uint32_t *free); > > > + > > > +static int > > > +_st_ring_init(struct rte_ring *r, const char *name, uint32_t num); > > > + > > > + > > > +static void > > > +lcore_stat_update(struct lcore_stat *ls, uint64_t call, uint64_t obj= , > > > + uint64_t tm, int32_t prcs) > > > +{ > > > + ls->op.nb_call +=3D call; > > > + ls->op.nb_obj +=3D obj; > > > + ls->op.nb_cycle +=3D tm; > > > + if (prcs) { > > > + ls->op.max_cycle =3D RTE_MAX(ls->op.max_cycle, tm); > > > + ls->op.min_cycle =3D RTE_MIN(ls->op.min_cycle, tm); > > > + } > > > +} > > > + > > > +static void > > > +lcore_op_stat_aggr(struct lcore_stat *ms, const struct lcore_stat > > > +*ls) { > > > + > > > + ms->op.nb_call +=3D ls->op.nb_call; > > > + ms->op.nb_obj +=3D ls->op.nb_obj; > > > + ms->op.nb_cycle +=3D ls->op.nb_cycle; > > > + ms->op.max_cycle =3D RTE_MAX(ms->op.max_cycle, ls->op.max_cycle); > > > + ms->op.min_cycle =3D RTE_MIN(ms->op.min_cycle, ls->op.min_cycle); } > > > + > > > +static void > > > +lcore_stat_aggr(struct lcore_stat *ms, const struct lcore_stat *ls) = { > > > + ms->nb_cycle =3D RTE_MAX(ms->nb_cycle, ls->nb_cycle); > > > + lcore_op_stat_aggr(ms, ls); > > > +} > > > + > > > +static void > > > +lcore_stat_dump(FILE *f, uint32_t lc, const struct lcore_stat *ls) { > > > + long double st; > > > + > > > + st =3D (long double)rte_get_timer_hz() / US_PER_S; > > > + > > > + if (lc =3D=3D UINT32_MAX) > > > + fprintf(f, "%s(AGGREGATE)=3D{\n", __func__); > > > + else > > > + fprintf(f, "%s(lc=3D%u)=3D{\n", __func__, lc); > > > + > > > + fprintf(f, "\tnb_cycle=3D%" PRIu64 "(%.2Lf usec),\n", > > > + ls->nb_cycle, (long double)ls->nb_cycle / st); > > > + > > > + fprintf(f, "\tDEQ+ENQ=3D{\n"); > > > + > > > + fprintf(f, "\t\tnb_call=3D%" PRIu64 ",\n", ls->op.nb_call); > > > + fprintf(f, "\t\tnb_obj=3D%" PRIu64 ",\n", ls->op.nb_obj); > > > + fprintf(f, "\t\tnb_cycle=3D%" PRIu64 ",\n", ls->op.nb_cycle); > > > + fprintf(f, "\t\tobj/call(avg): %.2Lf\n", > > > + (long double)ls->op.nb_obj / ls->op.nb_call); > > > + fprintf(f, "\t\tcycles/obj(avg): %.2Lf\n", > > > + (long double)ls->op.nb_cycle / ls->op.nb_obj); > > > + fprintf(f, "\t\tcycles/call(avg): %.2Lf\n", > > > + (long double)ls->op.nb_cycle / ls->op.nb_call); > > > + > > > + /* if min/max cycles per call stats was collected */ > > > + if (ls->op.min_cycle !=3D UINT64_MAX) { > > > + fprintf(f, "\t\tmax cycles/call=3D%" PRIu64 "(%.2Lf usec),\n", > > > + ls->op.max_cycle, > > > + (long double)ls->op.max_cycle / st); > > > + fprintf(f, "\t\tmin cycles/call=3D%" PRIu64 "(%.2Lf usec),\n", > > > + ls->op.min_cycle, > > > + (long double)ls->op.min_cycle / st); > > > + } > > > + > > > + fprintf(f, "\t},\n"); > > > + fprintf(f, "};\n"); > > > +} > > > + > > > +static void > > > +fill_ring_elm(struct ring_elem *elm, uint32_t fill) { > > > + uint32_t i; > > > + > > > + for (i =3D 0; i !=3D RTE_DIM(elm->cnt); i++) > > > + elm->cnt[i] =3D fill; > > > +} > > > + > > > +static int32_t > > > +check_updt_elem(struct ring_elem *elm[], uint32_t num, > > > + const struct ring_elem *check, const struct ring_elem *fill) { > > > + uint32_t i; > > > + > > > + static rte_spinlock_t dump_lock; > > > + > > > + for (i =3D 0; i !=3D num; i++) { > > > + if (memcmp(check, elm[i], sizeof(*check)) !=3D 0) { > > > + rte_spinlock_lock(&dump_lock); > > > + printf("%s(lc=3D%u, num=3D%u) failed at %u-th iter, " > > > + "offending object: %p\n", > > > + __func__, rte_lcore_id(), num, i, elm[i]); > > > + rte_memdump(stdout, "expected", check, > > > sizeof(*check)); > > > + rte_memdump(stdout, "result", elm[i], sizeof(elm[i])); > > > + rte_spinlock_unlock(&dump_lock); > > > + return -EINVAL; > > > + } > > > + memcpy(elm[i], fill, sizeof(*elm[i])); > > > + } > > > + > > > + return 0; > > > +} > > > + > > > +static int > > > +check_ring_op(uint32_t exp, uint32_t res, uint32_t lc, > > minor, lcore instead of lc would be better > > > > > + const char *fname, const char *opname) { > > > + if (exp !=3D res) { > > > + printf("%s(lc=3D%u) failure: %s expected: %u, returned %u\n", > > Suggest using lcore in the printf > > > > > + fname, lc, opname, exp, res); > > > + return -ENOSPC; > > > + } > > > + return 0; > > > +} > > > + > > > +static int > > > +test_worker_prcs(void *arg) > > > +{ > > > + int32_t rc; > > > + uint32_t lc, n, num; > > minor, lcore instead of lc would be better > > > > > + uint64_t cl, tm0, tm1; > > > + struct lcore_arg *la; > > > + struct ring_elem def_elm, loc_elm; > > > + struct ring_elem *obj[2 * BULK_NUM]; > > > + > > > + la =3D arg; > > > + lc =3D rte_lcore_id(); > > > + > > > + fill_ring_elm(&def_elm, UINT32_MAX); > > > + fill_ring_elm(&loc_elm, lc); > > > + > > > + while (wrk_cmd !=3D WRK_CMD_RUN) { > > > + rte_smp_rmb(); > > > + rte_pause(); > > > + } > > > + > > > + cl =3D rte_rdtsc_precise(); > > > + > > > + do { > > > + /* num in interval [7/8, 11/8] of BULK_NUM */ > > > + num =3D 7 * BULK_NUM / 8 + rte_rand() % (BULK_NUM / 2); > > > + > > > + /* reset all pointer values */ > > > + memset(obj, 0, sizeof(obj)); > > > + > > > + /* dequeue num elems */ > > > + tm0 =3D rte_rdtsc_precise(); > > > + n =3D _st_ring_dequeue_bulk(la->rng, (void **)obj, num, NULL); > > > + tm0 =3D rte_rdtsc_precise() - tm0; > > > + > > > + /* check return value and objects */ > > > + rc =3D check_ring_op(num, n, lc, __func__, > > > + RTE_STR(_st_ring_dequeue_bulk)); > > > + if (rc =3D=3D 0) > > > + rc =3D check_updt_elem(obj, num, &def_elm, > > > &loc_elm); > > > + if (rc !=3D 0) > > > + break; > > Since this seems like a performance test, should we skip validating the > objects? > > Did these tests run on Travis CI? I believe Travis CI has trouble runni= ng > stress/performance tests if they take too much time. > > The RTS and HTS tests should be added to functional tests. > > > > > + > > > + /* enqueue num elems */ > > > + rte_compiler_barrier(); > > > + rc =3D check_updt_elem(obj, num, &loc_elm, &def_elm); > > > + if (rc !=3D 0) > > > + break; > > > + > > > + tm1 =3D rte_rdtsc_precise(); > > > + n =3D _st_ring_enqueue_bulk(la->rng, (void **)obj, num, NULL); > > > + tm1 =3D rte_rdtsc_precise() - tm1; > > > + > > > + /* check return value */ > > > + rc =3D check_ring_op(num, n, lc, __func__, > > > + RTE_STR(_st_ring_enqueue_bulk)); > > > + if (rc !=3D 0) > > > + break; > > > + > > > + lcore_stat_update(&la->stats, 1, num, tm0 + tm1, 1); > > > + > > > + } while (wrk_cmd =3D=3D WRK_CMD_RUN); > > > + > > > + la->stats.nb_cycle =3D rte_rdtsc_precise() - cl; > > > + return rc; > > > +} > > > + > > > +static int > > > +test_worker_avg(void *arg) > > > +{ > > > + int32_t rc; > > > + uint32_t lc, n, num; > > > + uint64_t cl; > > > + struct lcore_arg *la; > > > + struct ring_elem def_elm, loc_elm; > > > + struct ring_elem *obj[2 * BULK_NUM]; > > > + > > > + la =3D arg; > > > + lc =3D rte_lcore_id(); > > > + > > > + fill_ring_elm(&def_elm, UINT32_MAX); > > > + fill_ring_elm(&loc_elm, lc); > > > + > > > + while (wrk_cmd !=3D WRK_CMD_RUN) { > > > + rte_smp_rmb(); > > > + rte_pause(); > > > + } > > > + > > > + cl =3D rte_rdtsc_precise(); > > > + > > > + do { > > > + /* num in interval [7/8, 11/8] of BULK_NUM */ > > > + num =3D 7 * BULK_NUM / 8 + rte_rand() % (BULK_NUM / 2); > > > + > > > + /* reset all pointer values */ > > > + memset(obj, 0, sizeof(obj)); > > > + > > > + /* dequeue num elems */ > > > + n =3D _st_ring_dequeue_bulk(la->rng, (void **)obj, num, NULL); > > > + > > > + /* check return value and objects */ > > > + rc =3D check_ring_op(num, n, lc, __func__, > > > + RTE_STR(_st_ring_dequeue_bulk)); > > > + if (rc =3D=3D 0) > > > + rc =3D check_updt_elem(obj, num, &def_elm, > > > &loc_elm); > > > + if (rc !=3D 0) > > > + break; > > > + > > > + /* enqueue num elems */ > > > + rte_compiler_barrier(); > > > + rc =3D check_updt_elem(obj, num, &loc_elm, &def_elm); > > > + if (rc !=3D 0) > > > + break; > > > + > > > + n =3D _st_ring_enqueue_bulk(la->rng, (void **)obj, num, NULL); > > > + > > > + /* check return value */ > > > + rc =3D check_ring_op(num, n, lc, __func__, > > > + RTE_STR(_st_ring_enqueue_bulk)); > > > + if (rc !=3D 0) > > > + break; > > > + > > > + lcore_stat_update(&la->stats, 1, num, 0, 0); > > > + > > > + } while (wrk_cmd =3D=3D WRK_CMD_RUN); > > > + > > > + /* final stats update */ > > > + cl =3D rte_rdtsc_precise() - cl; > > > + lcore_stat_update(&la->stats, 0, 0, cl, 0); > > > + la->stats.nb_cycle =3D cl; > > > + > > > + return rc; > > > +} > > Just wondering about the need of 2 tests which run the same functionali= ty. > The difference is the way in which numbers are collected. > > Does 'test_worker_avg' adding any value? IMO, we can remove > 'test_worker_avg'. >=20 > Yeh, they are quite similar. > I added _average_ version for two reasons: > 1. In precise I call rte_rdtsc_precise() straight before/after > enqueue/dequeue op. > At least at IA rte_rdtsc_precise() means mb(). > This extra sync point might hide some sync problems in the ring > enqueue/dequeue itself. > So having a separate test without such extra sync points > gives extra confidence that these tests would catch ring sync problem= s if > any. The functional tests should be captured separately as we need to run them o= n Travis CI (please see the comment above). Do we need the precise version? I think the average version captures the da= ta well already. Consider merging the two functions into a single one with a parameter to ca= pture precise or average cycles. Since such a parameter would be a constant= , compiler will remove the unwanted code. > 2. People usually don't do enqueue/dequeue on its own. > One of common patterns: dequeue/read-write data from the dequed > objects/enqueue. > So this test measures cycles for dequeue/enqueue plus some reads/writ= es > to the objects from the ring. Users could be doing lot of things after the ring operations, we do not kno= w what all can happen in the application. My concern here is, the numbers include cycles spent on things other than r= ing operations. For ex: we cannot compare the average numbers with precise = numbers. If we move the validation of the results to functional tests, we should be = good. >=20 > > > + > > > +static void > > > +mt1_fini(struct rte_ring *rng, void *data) { > > > + rte_free(rng); > > > + rte_free(data); > > > +} > > > + > > > +static int > > > +mt1_init(struct rte_ring **rng, void **data, uint32_t num) { > > > + int32_t rc; > > > + size_t sz; > > > + uint32_t i, nr; > > > + struct rte_ring *r; > > > + struct ring_elem *elm; > > > + void *p; > > > + > > > + *rng =3D NULL; > > > + *data =3D NULL; > > > + > > > + sz =3D num * sizeof(*elm); > > > + elm =3D rte_zmalloc(NULL, sz, __alignof__(*elm)); > > > + if (elm =3D=3D NULL) { > > > + printf("%s: alloc(%zu) for %u elems data failed", > > > + __func__, sz, num); > > > + return -ENOMEM; > > > + } > > > + > > > + *data =3D elm; > > > + > > > + /* alloc ring */ > > > + nr =3D 2 * num; > > > + sz =3D rte_ring_get_memsize(nr); > > > + r =3D rte_zmalloc(NULL, sz, __alignof__(*r)); > > > + if (r =3D=3D NULL) { > > > + printf("%s: alloc(%zu) for FIFO with %u elems failed", > > > + __func__, sz, nr); > > > + return -ENOMEM; > > > + } > > > + > > > + *rng =3D r; > > > + > > > + rc =3D _st_ring_init(r, RING_NAME, nr); > > > + if (rc !=3D 0) { > > > + printf("%s: _st_ring_init(%p, %u) failed, error: %d(%s)\n", > > > + __func__, r, nr, rc, strerror(-rc)); > > > + return rc; > > > + } > > > + > > > + for (i =3D 0; i !=3D num; i++) { > > > + fill_ring_elm(elm + i, UINT32_MAX); > > > + p =3D elm + i; > > > + if (_st_ring_enqueue_bulk(r, &p, 1, NULL) !=3D 1) > > > + break; > > > + } > > > + > > > + if (i !=3D num) { > > > + printf("%s: _st_ring_enqueue(%p, %u) returned %u\n", > > > + __func__, r, num, i); > > > + return -ENOSPC; > > > + } > > > + > > > + return 0; > > > +} > > > + > > > +static int > > > +test_mt1(int (*test)(void *)) > > > +{ > > > + int32_t rc; > > > + uint32_t lc, mc; > > > + struct rte_ring *r; > > > + void *data; > > > + struct lcore_arg arg[RTE_MAX_LCORE]; > > > + > > > + static const struct lcore_stat init_stat =3D { > > > + .op.min_cycle =3D UINT64_MAX, > > > + }; > > > + > > > + rc =3D mt1_init(&r, &data, RING_SIZE); > > > + if (rc !=3D 0) { > > > + mt1_fini(r, data); > > > + return rc; > > > + } > > > + > > > + memset(arg, 0, sizeof(arg)); > > > + > > > + /* launch on all slaves */ > > > + RTE_LCORE_FOREACH_SLAVE(lc) { > > > + arg[lc].rng =3D r; > > > + arg[lc].stats =3D init_stat; > > > + rte_eal_remote_launch(test, &arg[lc], lc); > > > + } > > > + > > > + /* signal worker to start test */ > > > + wrk_cmd =3D WRK_CMD_RUN; > > > + rte_smp_wmb(); > > > + > > > + usleep(run_time * US_PER_S); > > > + > > > + /* signal worker to start test */ > > > + wrk_cmd =3D WRK_CMD_STOP; > > > + rte_smp_wmb(); > > > + > > > + /* wait for slaves and collect stats. */ > > > + mc =3D rte_lcore_id(); > > > + arg[mc].stats =3D init_stat; > > > + > > > + rc =3D 0; > > > + RTE_LCORE_FOREACH_SLAVE(lc) { > > > + rc |=3D rte_eal_wait_lcore(lc); > > > + lcore_stat_aggr(&arg[mc].stats, &arg[lc].stats); > > > + if (verbose !=3D 0) > > > + lcore_stat_dump(stdout, lc, &arg[lc].stats); > > > + } > > > + > > > + lcore_stat_dump(stdout, UINT32_MAX, &arg[mc].stats); > > > + mt1_fini(r, data); > > > + return rc; > > > +} > > > + > > > +static const struct test_case tests[] =3D { > > > + { > > > + .name =3D "MT-WRK_ENQ_DEQ-MST_NONE-PRCS", > > > + .func =3D test_mt1, > > > + .wfunc =3D test_worker_prcs, > > > + }, > > > + { > > > + .name =3D "MT-WRK_ENQ_DEQ-MST_NONE-AVG", > > > + .func =3D test_mt1, > > > + .wfunc =3D test_worker_avg, > > > + }, > > > +}; > > > -- > > > 2.17.1