From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 61E67A055A; Tue, 25 Feb 2020 12:45:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 60A6B1BF8D; Tue, 25 Feb 2020 12:45:15 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 97B361BC24 for ; Tue, 25 Feb 2020 12:45:12 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Feb 2020 03:45:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,484,1574150400"; d="scan'208";a="271278842" Received: from orsmsx103.amr.corp.intel.com ([10.22.225.130]) by fmsmga002.fm.intel.com with ESMTP; 25 Feb 2020 03:45:11 -0800 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX103.amr.corp.intel.com (10.22.225.130) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 25 Feb 2020 03:45:11 -0800 Received: from orsmsx602.amr.corp.intel.com (10.22.229.15) by ORSMSX607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 25 Feb 2020 03:45:10 -0800 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by orsmsx602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 25 Feb 2020 03:45:10 -0800 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.174) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 25 Feb 2020 03:45:10 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hdfcd82NVVc37LULAiyWIhFJzrKXj3PVCJ5O3jMQVTSuSKCmQv0Tu1HFCT2m0G8IG1vc1oAyS+yj+0Szl+JPMgmD1m4yy9d1RmvH4Qph0UnG4MHfHnBy6CEAQ61Re9ut0ct7qwpylH0fCPBt0H5S61XhiCRFoOaAjxmS2HeL+vSkg612G7pPcfnIDuyf6icGgBNcI2/SSoSdkEK2rBde2GDHW/PEpSN/pIKmSGhUDneFde2LMOlsQai0V6IlV20IX1rHh88jeX63ppjDGxb2nubNC5bDn36ebcnS2wcRaLqKE98ht2WsDVwTbKEyxshuziABRTt4zVwGRyKkdRol3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ashfSG3zEqyUWTehrumwZvJK9mfWx/oQczz8TeDtzUo=; b=jdf4yLbqp6drOUm6swmrXcQZ1gqTZ5d2u/hmko0z2bLzem+een77lINPzEeJvztP4a8U0Ioq6R2z9YJfa/QHdWUKzPegqo/OkxpDY1ZwNPdxToaIzmm1RZyNWl2n1Oz91UTPhSjV1KQcWsfDfHukLkDye35rJq3VWAZxB7rR+TTeU/T1QyEpcNoOJzem1GrEew+QFxnMUNXk9X6AA0uYay8NHxu3Xz1IEgyfUrmAJkDMNH18Ura0w+VSt7FxVe/LFqqhRyrYPOJv2/rtWrjrPGA8h7cqu1rzbzyKA1sWCVBfFa4wddKCJo4hERuAndbb+2hBTnIP2ekHw8sRWzyJXQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ashfSG3zEqyUWTehrumwZvJK9mfWx/oQczz8TeDtzUo=; b=NwKjTtBv9Ahefezm659OMPbszmuEfbRpD4VoQv3HEvgXb8PvNR7wfyxN/9dwX9lMEWJ0sx1jxbxNS0PNU+tpIR6RCTZBSZAABi1INUSfO5NcCOCQJJs8BQgK/sR7ROhpEaFpBg+D/rzywuLTzgka2cbnYoRuDb0sldefRTWjXSg= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (2603:10b6:805:5d::19) by SN6PR11MB2542.namprd11.prod.outlook.com (2603:10b6:805:60::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2750.21; Tue, 25 Feb 2020 11:45:08 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::395e:eb75:6ab7:2ba5]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::395e:eb75:6ab7:2ba5%3]) with mapi id 15.20.2750.021; Tue, 25 Feb 2020 11:45:08 +0000 From: "Ananyev, Konstantin" To: Honnappa Nagarahalli , Stephen Hemminger , Jerin Jacob CC: dpdk-dev , Olivier Matz , nd , nd Thread-Topic: [dpdk-dev] [RFC 0/6] New sync modes for ring Thread-Index: AQHV6waQaBpJG6uFVkOED+rLZZWNPqgqkXSAgAAQ8YCAABqxgIAAFZ6AgAD3jmA= Date: Tue, 25 Feb 2020 11:45:07 +0000 Message-ID: References: <20200224113515.1744-1-konstantin.ananyev@intel.com> <20200224085919.3e73fda7@hermes.lan> <20200224113529.4c1c94ab@hermes.lan> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZWQwYjZlNDctOGI3YS00MWFjLThmNDctODM3MTVlNmM0YWJmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoibGRKaUlVRGhQd2pDa21yTUFlZ3BneWVYTmgxMTRCK1ZLcXhVUWgyU0hEYkJQRE9UdDRvREoyaHBJcVlneHhjViJ9 dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 x-ctpclassification: CTP_NT authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.174] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: eb6a00f2-9e0e-439c-0878-08d7b9e829da x-ms-traffictypediagnostic: SN6PR11MB2542: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 0324C2C0E2 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39860400002)(346002)(136003)(376002)(396003)(366004)(189003)(199004)(8676002)(81166006)(966005)(81156014)(71200400001)(76116006)(8936002)(66946007)(33656002)(2906002)(5660300002)(66556008)(66476007)(4326008)(66446008)(64756008)(54906003)(110136005)(316002)(9686003)(53546011)(478600001)(186003)(26005)(52536014)(7696005)(55016002)(86362001)(6506007); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB2542; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ukshvQOoxj8t98XUy4NBlUoVoFMIu6A6JFh3f+cA690M0DpJL2jDb0o+Phixfg+av8+lPUd+Q3lVbPdf2FUWviAAG9LGfJsGJZPSumuyRRsmyLUcoEylYPYx8vNGYKFlQ5JjzwWdKOYK5eQy0EZ3jiKa36SfHVfW6sa4MvmoVXZqGycUMm5NfiP/eQidWuSIBk2htPLGRW2qK0j0llbG0Wl78tgUIkW16DV29hKP9gYIW7m398cENJe2NMnMENf4mD/u0V+1BWEACXBAndCnLosFrm6RxgBToRpbzMmie4C+kv5T8eTY38K+iemKlB+4xQD2UyCmh+4GYFKeG/swOvT5iuACvC76PGbGUBt1vr0jx0vcNVIQ7O2Mce1WZFOKZlTaZW7XTZSjy/bvFRL0yOPCluwipYmrEvn2f+/fM6G1vCur2pVbBUJNJwWkwDzs8UXomcEMuULbEX27wqD0rmCGgdesByooEagztLn/zPJjYlLBJMkQuLZZzInLSlipyzJcPGnwh3XOw7nN3HXxEg== x-ms-exchange-antispam-messagedata: 5BbFNE1CU9S7ypz0bLJ/AcAYa35elWA3RZPD06ZWmKbcgWStnSGDB26QQtFMa52NxLcyniUuHQSWC/39SEoBaj9QdS0Rl0gp7rimLhPJw68vOsTJn8R3wfuh9H+X6Jz/ciUY60RduOtTya3I9fYvoA== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: eb6a00f2-9e0e-439c-0878-08d7b9e829da X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Feb 2020 11:45:08.0131 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: EhjWeDUfUSeZ6AmPLvkMzmL7HZj9EgRimsOLjQonHhq7xOUT/ws/vRu/RTijlIFEvg38SeruHF0jSf0/XWxE+G7otCV6CVTTBhmcCElyPdY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2542 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [RFC 0/6] New sync modes for ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > -----Original Message----- > > From: dev On Behalf Of Stephen Hemminger > > Sent: Monday, February 24, 2020 1:35 PM > > To: Jerin Jacob > > Cc: Konstantin Ananyev ; dpdk-dev > > ; Olivier Matz > > Subject: Re: [dpdk-dev] [RFC 0/6] New sync modes for ring > > > > On Mon, 24 Feb 2020 23:29:57 +0530 > > Jerin Jacob wrote: > > > > > On Mon, Feb 24, 2020 at 10:29 PM Stephen Hemminger > > > wrote: > > > > > > > > On Mon, 24 Feb 2020 11:35:09 +0000 > > > > Konstantin Ananyev wrote: > > > > > > > > > Upfront note - that RFC is not a complete patch. > > > > > It introduces an ABI breakage, plus it doesn't update ring_elem > > > > > code properly, etc. > > > > > I plan to deal with all these things in later versions. > > > > > Right now I seek an initial feedback about proposed ideas. > > > > > Would also ask people to repeat performance tests (see below) on > > > > > their platforms to confirm the impact. > > > > > > > > > > More and more customers use(/try to use) DPDK based apps within > > > > > overcommitted systems (multiple acttive threads over same pysical > > cores): > > > > > VM, container deployments, etc. > > > > > One quite common problem they hit: Lock-Holder-Preemption with > > rte_ring. > > > > > LHP is quite a common problem for spin-based sync primitives > > > > > (spin-locks, etc.) on overcommitted systems. > > > > > The situation gets much worse when some sort of fair-locking > > > > > technique is used (ticket-lock, etc.). > > > > > As now not only lock-owner but also lock-waiters scheduling order > > > > > matters a lot. > > > > > This is a well-known problem for kernel within VMs: > > > > > http://www-archive.xenproject.org/files/xensummitboston08/LHP.pdf > > > > > https://www.cs.hs-rm.de/~kaiser/events/wamos2017/Slides/selcuk.pd= f > > > > > The problem with rte_ring is that while head accusion is sort of > > > > > un-fair locking, waiting on tail is very similar to ticket lock > > > > > schema - tail has to be updated in particular order. > > > > > That makes current rte_ring implementation to perform really pure > > > > > on some overcommited scenarios. > > > > > > > > Rather than reform rte_ring to fit this scenario, it would make mor= e > > > > sense to me to introduce another primitive. The current lockless > > > > ring performs very well for the isolated thread model that DPDK was > > > > built around. This looks like a case of customers violating the > > > > usage model of the DPDK and then being surprised at the fallout. > > > > > > I agree with Stephen here. > > > > > > I think, adding more runtime check in the enqueue() and dequeue() wil= l > > > have a bad effect on the low-end cores too. > > > But I agree with the problem statement that in the virtualization use > > > case, It may be possible to have N virtual cores runs on a physical > > > core. > > > > > > IMO, The best solution would be keeping the ring API same and have a > > > different flavor in "compile-time". Something like liburcu did for > > > accommodating different flavors. > > > > > > i.e urcu-qsbr.h and urcu-bp.h will identical definition of API. The > > > application can simply include ONE header file in a C file based on > > > the flavor. > > > If need both at runtime. Need to have function pointer or so in the > > > application and define the function in different c file by including > > > the approaite flavor in C file. > > > > This would also be a good time to consider the tradeoffs of the heavy u= se of > > inlining that is done in rte_ring vs the impact that has on API/ABI sta= bility. > > > I was working on few requirements in rte_ring library for RCU defer APIs.= RFC is at https://patchwork.dpdk.org/cover/66020/. Yep, noticed your patch, seems we sort of collided here. As I understand you patch aims to provide functionality similar to my HTS o= ne. Will try to look at yours one in next fee days, hopefully we can end-up wit= h some common denominator. Konstantin