From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140074.outbound.protection.outlook.com [40.107.14.74]) by dpdk.org (Postfix) with ESMTP id 74A9D2BCE for ; Thu, 17 Jan 2019 16:42:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K4L11mAeuLm+gR2xDGhQtwYMlk3kXOj9YdWZKiHDhaE=; b=gCuaSjdHVhlIpnaDo1iAUfMRLryzvJFJV0jOcH4hkEWU+TtXdGe/uezzexI6+kYRuauyoEpqqhPRWA1nMspD4+dF9t68nKOj2sxTdWcepBjy2udGq56qCQ8eK84Nl+H8rqNc/giPsbdZV/N5/gxy8h6ln3/a+SeGXVztZH1cudQ= Received: from DB7PR08MB3163.eurprd08.prod.outlook.com (52.134.110.149) by DB7PR08MB3516.eurprd08.prod.outlook.com (20.177.120.78) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1537.26; Thu, 17 Jan 2019 15:42:15 +0000 Received: from DB7PR08MB3163.eurprd08.prod.outlook.com ([fe80::d119:44d1:4793:5724]) by DB7PR08MB3163.eurprd08.prod.outlook.com ([fe80::d119:44d1:4793:5724%5]) with mapi id 15.20.1516.019; Thu, 17 Jan 2019 15:42:15 +0000 From: "Gavin Hu (Arm Technology China)" To: "Eads, Gage" , "Richardson, Bruce" CC: "dev@dpdk.org" , "olivier.matz@6wind.com" , "arybchenko@solarflare.com" , "Ananyev, Konstantin" , Honnappa Nagarahalli , "Ruifeng Wang (Arm Technology China)" , "Phil Yang (Arm Technology China)" Thread-Topic: [dpdk-dev] [PATCH v2 2/2] mempool/nb_stack: add non-blocking stack mempool Thread-Index: AQHUrSJso0tnZem+LkyN190Zvz4vjKWzG7mggABprv+AAA+QAIAABgdg Date: Thu, 17 Jan 2019 15:42:15 +0000 Message-ID: References: <20190110205538.24435-1-gage.eads@intel.com> <20190115223232.31866-1-gage.eads@intel.com> <20190115223232.31866-3-gage.eads@intel.com> <9184057F7FC11744A2107296B6B8EB1E541C83E4@FMSMSX108.amr.corp.intel.com> <20190117142036.GA379232@bricha3-MOBL.ger.corp.intel.com> <9184057F7FC11744A2107296B6B8EB1E541C84BB@FMSMSX108.amr.corp.intel.com> In-Reply-To: <9184057F7FC11744A2107296B6B8EB1E541C84BB@FMSMSX108.amr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR08MB3516; 6:ma82Gn+fQV4ElVVByIK+HNkKNOqukK9wT9YFbJRTeONES3Q9qgfzskz4YeO5jrsSWpF4keV99TDc4bxpdPK7zEGcSpaEzu3LxMjOGHbzEvHP8ooqxbG9MIqlBzILcLq2VkXOB3ENwnEqXws/AgjWNjCRCpZ1Yl3tJiM7EnLt0EINKCNrvgJrlvcBKzG4JfRygif39JZPMnIUzil8RK00eAphwXKi+F37NayKYHSodEHowFgyGeUd77E3G0kOAtBSZB0j/+Xs2MnqS6vXsAgD7vzLVR50kgYUAtHvKk+daBIpHszC2G16AI5UVmXcwznMY6AbNeaK5fsT8qAwdA5TCetChpOwx9pgcqcNFjifzNLerkB9qAnAs888MwTSyh9yfuaB5mrSS/uV8kIGVAM+xtla115SfOY/bqsLur/UbdBKJiMTne+vKTCt7Eqhp4AfUHRTfvF6Qwa2zt/wSNkTUA==; 5:/swuTn/qOSgdsKLQ+ffdUxI/0r2kjg3wNyWoqUTDwocNYn6we1LlFZbBDXPtxqx+9T/RT2DfVsfPiTXJqsUGzVKocFeUwCN9ZlrwiGQx+rCUn7trVTDOFWP+RHSOpcyM00wsTrPZq463ao+YmYjwIb+6Xw0qfuKG6XUzj41pEsfSPB8UI/RL0G/ApQ3iuSMIpneyqdCEyV55RmcoBxMDnA==; 7:rsQU1Xr+QaLe7mSzPYFHrOBLToXePbfiQy9MlOC0jixuYJcS2y2RCxu3K/Dm/59EhEkiur3JIqpp4DuakH3Xdo9eRy0gNkuUL9jUwB9N7WxNUX3/EpNXMwApGjWER97Oz4+tjL8V5tijbN8JLt6xNw== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 68cfeea2-5ac5-42aa-bb23-08d67c925af3 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR08MB3516; x-ms-traffictypediagnostic: DB7PR08MB3516: x-microsoft-antispam-prvs: x-forefront-prvs: 0920602B08 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(396003)(376002)(346002)(39860400002)(366004)(189003)(199004)(13464003)(40434004)(81156014)(81166006)(3846002)(97736004)(74316002)(8936002)(33656002)(6436002)(53936002)(305945005)(486006)(256004)(6116002)(7736002)(5024004)(14444005)(99286004)(5660300001)(229853002)(6246003)(68736007)(4326008)(55016002)(25786009)(2906002)(105586002)(110136005)(54906003)(316002)(106356001)(9686003)(53546011)(55236004)(476003)(7696005)(6506007)(66066001)(76176011)(26005)(71200400001)(71190400001)(93886005)(446003)(14454004)(72206003)(11346002)(186003)(86362001)(478600001)(102836004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3516; H:DB7PR08MB3163.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: J2c101gPeyuI1Avw7bQZq8dPayXtKosaJ/O6Beg1WmlleKP2ZLAO6p5z2Bsqj+Ys+jhbLOAcPxGB5ICTNNViEOMH90h5LTeMiIwwmiAH2uDWh3e+4G/WZ5L8vxsWK4p8ZGOzDw41MVKnM9I81KCvUCHjDN1Gqzjlfyv+lwAriTCczW+GTysnYL9Tj6ahY7m53FkfhmEHALg1T34lGH5Vl3282D/RTzgr92s1wFrErGI6f7ajxxiKxrm2pmR770dufi+ipCPk4HNbIu9Gqk/h9KKaKdN9HHXVu5/KtIX6L+Q6C7tz/tZQZUfWLYVzzJg5bde4OUvhsShyQBoTyULcMzohQvXjcpS8NRR408SUUUILAYN+HTcWKfba2NLuI6PCL9hnP9TkFWlEWRGGehn2i199sd8TGbM54yFJ6w6/M9M= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 68cfeea2-5ac5-42aa-bb23-08d67c925af3 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jan 2019 15:42:15.1395 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3516 Subject: Re: [dpdk-dev] [PATCH v2 2/2] mempool/nb_stack: add non-blocking stack mempool X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jan 2019 15:42:16 -0000 > -----Original Message----- > From: Eads, Gage > Sent: Thursday, January 17, 2019 11:16 PM > To: Richardson, Bruce > Cc: Gavin Hu (Arm Technology China) ; > dev@dpdk.org; olivier.matz@6wind.com; arybchenko@solarflare.com; > Ananyev, Konstantin ; Honnappa > Nagarahalli ; Ruifeng Wang (Arm > Technology China) ; Phil Yang (Arm Technology > China) > Subject: RE: [dpdk-dev] [PATCH v2 2/2] mempool/nb_stack: add non- > blocking stack mempool > > > > > -----Original Message----- > > From: Richardson, Bruce > > Sent: Thursday, January 17, 2019 8:21 AM > > To: Eads, Gage > > Cc: Gavin Hu (Arm Technology China) ; > dev@dpdk.org; > > olivier.matz@6wind.com; arybchenko@solarflare.com; Ananyev, > Konstantin > > ; Honnappa Nagarahalli > > ; Ruifeng Wang (Arm Technology > China) > > ; Phil Yang (Arm Technology China) > > > > Subject: Re: [dpdk-dev] [PATCH v2 2/2] mempool/nb_stack: add non- > blocking > > stack mempool > > > > On Thu, Jan 17, 2019 at 02:11:22PM +0000, Eads, Gage wrote: > > > > > > > > > > -----Original Message----- > > > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com] > > > > Sent: Thursday, January 17, 2019 2:06 AM > > > > To: Eads, Gage ; dev@dpdk.org > > > > Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; Richardson, > > > > Bruce ; Ananyev, Konstantin > > > > ; Honnappa Nagarahalli > > > > ; Ruifeng Wang (Arm Technology > China) > > > > ; Phil Yang (Arm Technology China) > > > > > > > > Subject: RE: [dpdk-dev] [PATCH v2 2/2] mempool/nb_stack: add > > > > non-blocking stack mempool > > > > > > > > > > > > > -----Original Message----- > > > > > From: dev On Behalf Of Gage Eads > > > > > Sent: Wednesday, January 16, 2019 6:33 AM > > > > > To: dev@dpdk.org > > > > > Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; > > > > > bruce.richardson@intel.com; konstantin.ananyev@intel.com > > > > > Subject: [dpdk-dev] [PATCH v2 2/2] mempool/nb_stack: add > > > > > non-blocking stack mempool > > > > > > > > > > This commit adds support for non-blocking (linked list based) > > > > > stack mempool handler. The stack uses a 128-bit compare-and- > swap > > > > > instruction, and thus is limited to x86_64. The 128-bit CAS > > > > > atomically updates the stack top pointer and a modification > > > > > counter, which protects against the ABA problem. > > > > > > > > > > In mempool_perf_autotest the lock-based stack outperforms the > non- > > > > > blocking handler*, however: > > > > > - For applications with preemptible pthreads, a lock-based stack'= s > > > > > worst-case performance (i.e. one thread being preempted while > > > > > holding the spinlock) is much worse than the non-blocking stack= 's. > > > > > - Using per-thread mempool caches will largely mitigate the > performance > > > > > difference. > > > > > > > > > > *Test setup: x86_64 build with default config, dual-socket Xeon > > > > > E5-2699 v4, running on isolcpus cores with a tickless scheduler. > > > > > The lock-based stack's rate_persec was 1x-3.5x the non-blocking > stack's. > > > > > > > > > > Signed-off-by: Gage Eads > > > > > --- > > > > > MAINTAINERS | 4 + > > > > > config/common_base | 1 + > > > > > doc/guides/prog_guide/env_abstraction_layer.rst | 5 + > > > > > drivers/mempool/Makefile | 3 + > > > > > drivers/mempool/meson.build | 5 + > > > > > drivers/mempool/nb_stack/Makefile | 23 ++++ > > > > > drivers/mempool/nb_stack/meson.build | 4 + > > > > > drivers/mempool/nb_stack/nb_lifo.h | 147 > > > > > +++++++++++++++++++++ > > > > > drivers/mempool/nb_stack/rte_mempool_nb_stack.c | 125 > > > > > ++++++++++++++++++ > > > > > .../nb_stack/rte_mempool_nb_stack_version.map | 4 + > > > > > mk/rte.app.mk | 7 +- > > > > > 11 files changed, 326 insertions(+), 2 deletions(-) create mode > > > > > 100644 drivers/mempool/nb_stack/Makefile create mode 100644 > > > > > drivers/mempool/nb_stack/meson.build > > > > > create mode 100644 drivers/mempool/nb_stack/nb_lifo.h > > > > > create mode 100644 > > > > > drivers/mempool/nb_stack/rte_mempool_nb_stack.c > > > > > create mode 100644 > > > > > drivers/mempool/nb_stack/rte_mempool_nb_stack_version.map > > > > > > > > > > diff --git a/MAINTAINERS b/MAINTAINERS index > 470f36b9c..5519d3323 > > > > > 100644 > > > > > --- a/MAINTAINERS > > > > > +++ b/MAINTAINERS > > > > > @@ -416,6 +416,10 @@ M: Artem V. Andreev > > > > > > > > > > M: Andrew Rybchenko > > > > > F: drivers/mempool/bucket/ > > > > > > > > > > +Non-blocking stack memory pool > > > > > +M: Gage Eads > > > > > +F: drivers/mempool/nb_stack/ > > > > > + > > > > > > > > > > Bus Drivers > > > > > ----------- > > > > > diff --git a/config/common_base b/config/common_base index > > > > > 964a6956e..8a51f36b1 100644 > > > > > --- a/config/common_base > > > > > +++ b/config/common_base > > > > > @@ -726,6 +726,7 @@ CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG=3Dn > # > > > > > CONFIG_RTE_DRIVER_MEMPOOL_BUCKET=3Dy > > > > > CONFIG_RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB=3D64 > > > > > +CONFIG_RTE_DRIVER_MEMPOOL_NB_STACK=3Dy > > > > > > > > NAK, as this applies to x86_64 only, it will break arm/ppc and eve= n > > > > 32bit i386 configurations. > > > > > > > > > > Hi Gavin, > > > > > > This patch resolves that in the make and meson build files, which > ensure that > > the library is only built for x86-64 targets: Looking down to the changes with Makefile and meson.build, it will be compi= led out for arm/ppc/i386. That works at least. But having this entry in the arm/ppc/i386 configurations is very strange, s= ince they have no such implementations. Why not put it into defconfig_x86_64-native-linuxapp-icc/gcc/clang to limit= the scope? > > > > > > diff --git a/drivers/mempool/Makefile b/drivers/mempool/Makefile > index > > > 28c2e8360..895cf8a34 100644 > > > --- a/drivers/mempool/Makefile > > > +++ b/drivers/mempool/Makefile > > > @@ -10,6 +10,9 @@ endif > > > ifeq ($(CONFIG_RTE_EAL_VFIO)$(CONFIG_RTE_LIBRTE_FSLMC_BUS),yy) > > > DIRS-$(CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL) +=3D dpaa2 endif > > > +ifeq ($(CONFIG_RTE_ARCH_X86_64),y) > > > +DIRS-$(CONFIG_RTE_DRIVER_MEMPOOL_NB_STACK) +=3D nb_stack endif > > > > > > diff --git a/drivers/mempool/nb_stack/meson.build > > > b/drivers/mempool/nb_stack/meson.build > > > new file mode 100644 > > > index 000000000..4a699511d > > > --- /dev/null > > > +++ b/drivers/mempool/nb_stack/meson.build > > > @@ -0,0 +1,8 @@ > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2019 Intel > > > +Corporation > > > + > > > +if arch_subdir !=3D 'x86' or cc.sizeof('void *') =3D=3D 4 > > > +build =3D false > > > +endif > > > + > > > > Minor suggestion: > > Can be simplified to "build =3D dpdk_conf.has('RTE_ARCH_X86_64')", I > believe. > > > > /Bruce > > Sure, I'll switch to that check in v4. > > Thanks, > Gage IMPORTANT NOTICE: The contents of this email and any attachments are confid= ential and may also be privileged. If you are not the intended recipient, p= lease notify the sender immediately and do not disclose the contents to any= other person, use it for any purpose, or store or copy the information in = any medium. Thank you.