From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0043.outbound.protection.outlook.com [104.47.34.43]) by dpdk.org (Postfix) with ESMTP id 07E0B3977 for ; Fri, 15 Jul 2016 08:29:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=pdEA7ueHotvXgpO1SHZYnUhma15k/CcUVjDiUl/YIyU=; b=aR7TWusE6IyEpWikM4dsR8KEWLanOOOYGVDLl+MpMoZipIEwYgalQFObY0X7DMya960j0AvF7p6z8hmUMBtP6+NzRKzaNiL3f4CHEVTIZuMw7leuNOWmwOr0I6PaaSbF3RVNUtpDZTXlYgdpUy8Vu2W2orf/hr4HWrZ9wMg2tR4= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.localdomain (122.179.70.91) by BY1PR0701MB1721.namprd07.prod.outlook.com (10.162.111.140) with Microsoft SMTP Server (TLS) id 15.1.534.14; Fri, 15 Jul 2016 06:29:24 +0000 Date: Fri, 15 Jul 2016 11:59:06 +0530 From: Jerin Jacob To: "Ananyev, Konstantin" CC: "Kuusisaari, Juhamatti" , "'dev@dpdk.org'" , "Jan Viktorin (viktorin@rehivetech.com)" , "Chao Zhu (bjzhuc@cn.ibm.com)" Message-ID: <20160715062905.GA6473@localhost.localdomain> References: <2601191342CEEE43887BDE71AB97725836B7C858@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB97725836B7C916@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB97725836B7D20B@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB97725836B7D628@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB97725836B7D850@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB97725836B7DD85@irsmsx105.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <2601191342CEEE43887BDE71AB97725836B7DD85@irsmsx105.ger.corp.intel.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [122.179.70.91] X-ClientProxiedBy: PN1PR01CA0052.INDPRD01.PROD.OUTLOOK.COM (10.164.136.152) To BY1PR0701MB1721.namprd07.prod.outlook.com (10.162.111.140) X-MS-Office365-Filtering-Correlation-Id: b5f03c3d-92b4-426f-25ea-08d3ac795ec5 X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1721; 2:JYHfGAebq5FvLFpSEg8ozbEOwM7ER5k1lWAH8hpsZMrBb3illbKE6ec8edqTflPf9UyRT0hFLc+dderGY+rr+QG3I+uA9+k5erN1ge8NLc8mR0K7GJhDOksLCFf+j61Y6nkIyqt8Dg2/nbejaISJkWRfOEZ2UxoizzTh3Ps9jEzLJ1OB6V7M2tfdtX16qFyW; 3:X/m+/7F83rP5j2hHIlpBmC7nx7r8odPmM4tKHvUvy3Ymg+nrwONDUPc0VO6Q4DibdWRkneHSWVfYInPsLE29ctUuQ76Rm820YFZuJkG566UF8hC7CrJUIob67lNmlggU X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR0701MB1721; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1721; 25:T4DkiGv8RHYhL9fQVVaW3MWMkYLmtTUfId0gi1KJHcuOJr9cAzP1jAywvjOj00Ot5iYnJx2OyMMdvaLqn47DRdZqifnMPRAu31kishtEvck78oMEd0Po7+LyBXYmV8Dnr7uVIfCJquNyzPYjN6Y/H8+sdeIzIuVNsMWJNz6kdqmQK86rpl0hqbgu9L0SJMKHCsdDg/5R/xg0/f6LExft0MlIVysmaqjie5BoEINmcS0GFP3k7Nfzo07DTqmglqSCXf7KJSv8KxlZaaWcoGFzO7BT8VTshVcowsdxmH025lr/5EC5hDys6Jx3VPzK09JEScNOkszgYAneUakpazSX/c7PQGNpjNOIGb389gwp+5jeyCdiJL/E1QPY1AV/W05wnVnWOzM5ZQhsbZSsBU0tmI6jCo5jX2FFK7ZKL/xtrXK64MLfosYQ298F/WWuoliJxtxSukNNH8z96Z1lnOZ8mgz6perWWktDnLAT2Ic5jjy4BnIpKQsJFtwxNQbEioMw9c2HuuCwptsDVc5vvvPQKefCBijUUtHPvLR1BC8lpEEVIXQf+4Yq8QriQf6j89v2UDXlcyAsZJtqgh+sYW0iN9rDgpuVALgJdgB2TjXlxfadKsAO9yOt/IWGgYRCzw+UfFqyzVGFZo0sjgyaCAflqf6di8hxjg/1oIurjtYNKGuS2+rC4pLGmfyS9WPZQ3BtEzorEsU8yPgtyL7J6ObKYk9TEx2sYBy1yvINRg60gM8o3r1Ajxe56W8kV0zrAV+6Oh2+MygkI4vCEp8/Rae2tFkOOvU9qkmn7HKH/Kb09jPeSNQHBY95JxqCFWhAGdj1zFRzQjntJFQ2wiGs/mUAQEQGBm1dM1CFgmNC8+g3yEMJI548Ox5FTHUjJWNjQeoo X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1721; 31:mSeLfPHlG3ax9I85quuH0UEm8tllXPCu/DldCtGaENB/op3YWbpqfQ+lNhXFa30qFDdy5ikD56KjGDJ+WvkJIExPdkgLAuYK9vxtHPaZf//EZVhCC9eI9iwfiECbqjOiifvMqgHG1Cv3d8j8nLdG+P6VCq6fIIvsVxmvH0v+mm4jstT3IbvD9SYhRUyofrwuyUKrykVxSKNcZuN0l8zd3A==; 20:FGGDIDc58dWhwP0eJRTX6Na4OU5Kbc1PT1Qq6D/GlM0ZKrENHM6FEct4nytaGxSNKXQnn259a3or7lnbHcSSuyhee2hV+pmTrsLsbh8XXaTLXy62cTwg7Ua4azjiqvpxGWMVdfZLfGnoo+H5Y3ioO0dCACNfdlGnJ4ansALy5pAOBvTUhIvaY0y+lXwxtcVOCmNsMhfuXJdN1K7K0DceLWZxZIeRN8Wr4x6OX3eQ7V7sZMQWwSL72VL0xwB/1flvSLVogM3CqVKa470IKbh22GRWv6TnlaluHHIcI1JCV3ohbl5Wbs4rwL9sjRfc4wA3a6vl1hi1TMrbaivhfKvjsUh1vbw8NOiYdJxuV2tP9YTTdhdK/Fp+IM/aNSW9vTZKOagZdVwfPwoQMDI3FIbr9IZetEYjRute5hRV+QumVU1+pojNIpcfFQ3OVJNFeh7CGJ6HfeEKHxt7j2ydUaQ29PL/MtPrsGJrOucEaaI7Sr1Isxxs4DZo3UsZm5xEEzKhmv9+OILrUDUkZb9RejDqeGlQGUISv/yDAy7Y3EnX5PhB5v+HJAM98ROcsQiaPx7CINiQUHyKQHukQ7nlZ6Z1XoXPUvjMayjj19sl6yj8UrA= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(22074186197030)(183786458502308); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:BY1PR0701MB1721; BCL:0; PCL:0; RULEID:; SRVR:BY1PR0701MB1721; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1721; 4:2oDsq5N87oEmh/fge7R3/H5U2ARfkOq7SLNZNcltmoH3WjKLEze1H1nIxWE7v6o0e8K7HovZ4y5qVE8nnWcNWek9t+co9bt46080pa4AKJtZCQWDoGfvULeOZPJ1trXIutHCHo+wrgQ20oFDq0/AzD7hc5NxvBOvoqiq3U8ig5utbJ6v4wKwEaqulHbNcXnbn+hCQkHjethzUedx/6Fmp82j6rUpQ3wksv2whfdubPbE+nOHG1mBZmVDGv+nuXggIqzc9Mm7jQ/c0G6XFQcDx3fyFfwDCpoKqu4S/Hcyo1Ej2v5blWLe9qtIlKVZLVwb126Xj25b2oxul3WJ8aViPHftH4SNX8GC2KhKSmFfdTMk2OIAxwg4Z+WzH70PN5AxxaC6M4QgEI1Wx5eGVLI3puvhpQ6MqzJuO+SlHBV4WxAnMVpmad1pNj+7pWjLmsBZ X-Forefront-PRVS: 00046D390F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(51444003)(189002)(199003)(24454002)(23726003)(6116002)(101416001)(15975445007)(586003)(105586002)(561944003)(97756001)(97736004)(93886004)(3846002)(68736007)(110136002)(2950100001)(2906002)(83506001)(42186005)(92566002)(81166006)(7736002)(66066001)(8666005)(106356001)(33656002)(50466002)(77096005)(19580395003)(81156014)(47776003)(305945005)(7846002)(50986999)(15395725005)(46406003)(4001350100001)(8676002)(76176999)(1076002)(61506002)(54356999)(4326007)(9686002)(189998001)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0701MB1721; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0701MB1721; 23:TVwVIKb1hwS0/iG3UZewfHw7gGbWNDQ3NThDr8m?= =?us-ascii?Q?7MUlHKhKR27TiPtd9xJ3YOd5OH6ERksghuR0C8NfWZ7z5QaoDI1/KGkylOKy?= =?us-ascii?Q?gh36MKqMMI0yBGpGBrYZ3r1FOxJF7lvdzdTrlq27CZ2dLZoSk9ZwNXiPXomU?= =?us-ascii?Q?mF7v2NzXYNEYl3kz80dqFqrYyGAMZJ9tkdQxzaeAMWN89Y+eBkhWuN+PFmuA?= =?us-ascii?Q?JbRmDUIWl2FMLvxV/lSFGNQAie5+184qmL/KdjUtlEK8aKVuY2fWnUbR9Pze?= =?us-ascii?Q?/quWix+42bSUadcaYVDw7P97dMZ8nHUtLlYbZj90MTo5K3Uin+vNRMXAoKyV?= =?us-ascii?Q?50VBs0RlGhXOs+d5uB0Dxs/57ekhMb9bz4/KLLYd6pZ+OM3sK1GFv83TK7D6?= =?us-ascii?Q?iUoYn9WumMIG4D9vM2ADtWwSjGxZpg6t7KCx1SNlpylLnRuGH6W29pnYeCOT?= =?us-ascii?Q?mE8JjRnZNOVqEqFa7WjNma7lMvkqd4o8x4VEdio/fNzVP6qkWInBTfCfMcHx?= =?us-ascii?Q?OMr6poijIYyz8Wzlw9Yd2uY1j1X7AII+AfiUa5Tx+xepB2WBRpxnTdvSS9id?= =?us-ascii?Q?iOcJu4D75Fapw1wwJKcePd5Eq+dT541LSPZXZ07aQUL4+BR58dXFnlE9VAg7?= =?us-ascii?Q?RfeCXNGSvyBYCRnMhBHquVU49j5oRetOHnOL6/x4Bs91RWqZSwr4Xcik8srT?= =?us-ascii?Q?jrkUNc82Ie7BZd+uflz9h4RmKwlfGFnksnE3jP731XZ5ivb2zmxKsFAxOMfQ?= =?us-ascii?Q?HTPEi5XIglJXn9uWD1QmykRWyMthVjE6gsEV3N0Si3gU9v0DSaMnnjXlWY3C?= =?us-ascii?Q?CYRVGwblaov+8ZOTBoACq/DILG6BoOY1uy4Ug7i9zq5Ma3K9+4i95dO0IusG?= =?us-ascii?Q?rkCi9i/aDLmGwLtXaUeFVfsljVZCyJ2Yuy2xyJXKYWtc/Ozywm9vpJ0fnU9c?= =?us-ascii?Q?QHh5gzxqrtdof2XLDUWJyn6qtoggBnehbK49yymL219xFXXws7I6cGrHH0fg?= =?us-ascii?Q?rdepwFgv7IOCPthj2gZZC6jJCf6n7A2luBx8OZlHs9P130nsU8c0WjWAFVpi?= =?us-ascii?Q?SbYLsB/MtZ2Wx8kz5D4+12hGYh0sM5SJzSUe7bix1a5GNTyVqOA3ISYOYuH9?= =?us-ascii?Q?P9sFNDrXkBa13lsuoJfYldHNj/kTtLJQfGu20eSmVCwhZIkPMP/QzjLMOAcg?= =?us-ascii?Q?jCCV/do1+f63LAcyD8hrlxigMlsfQt6cTNhp9k1RxyDBgtJZ8QaLypT6KTyP?= =?us-ascii?Q?Rmgry4KgIb4E9DJ4wbIlA9Rihz0pzyJD9lwh45N2Z467zxCyAFjkhHeOtnKo?= =?us-ascii?Q?p5A=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1721; 6:NnXytG81CT6qVPUY4rvKfzDXonpnrKSDtA/ubCGU+nwsnyK2ZRRI0sdXWlBQ6caKJ4hfTdaP6copPiLFQ7+9AWI/eSFyVHT/160Wddowf6rpCs61kkH4N1EkNszCxYmD7ceQHGvmvFA8BFw2DDiaVbNO4aIggVooJqA3RDpKx7YSWNbQh/JemAsuJzMdj95I0cMCwoWh5Yur+95vWWnmFzzuD4PHJ7hO57ZNGONKE19n6htMQ2aKLUMO/fl9DZmYX5jkBv/x+unwF4vVFNDEhZj5PUwJM3STAD59N5LGrC8=; 5:g3P4Zb3MxlcWFzK1gHx+XpZG3teylFPHZyqwjp74qooyfoLOZ0sh+GFDnwm6qGrAoMRFDcXNgITpP45Dr9z+1wnFV90v759cTepkeH8gISqxkyZ2oOj4xND4g6RwNWpUJUWGGBoJ69jtPzPhQ66HGg==; 24:CXX+jE53RrBxjrbAPovC5E1gUD9L6ZJQp0LGeN0vCGMXrdBag1B7hnPQubuVm1TrMRNh+ib7xnRHnZe1ucEpHuSjB9yCe5IFVSfbQbglMFk=; 7:zVKcff7rlzvTxuIXe7EUVSN0T7Ng0Jf22C6jfuY2TcHuKGNX+Ey9zeE17MJaDPHp/26H4iOblhIOBWgAITTEfrTkl/Q1rvXsg2S1qvu1vkIcd6Dp9iBaa6Rh4yk5z3Vczl/DSsmQxH/dDNQFXxW50MOfDdaKobAbBfMKXOOqaolMxjkXMiIJckhQZ/FfU9zOx0Kyb2O5mND149D6d2vRinR59kBg+FOpQnIurp0d3IOtYXDsJ6+SsaibkeocrhR4 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2016 06:29:24.9233 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0701MB1721 Subject: Re: [dpdk-dev] [PATCH] lib: move rte_ring read barrier to correct location X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 06:29:29 -0000 On Thu, Jul 14, 2016 at 12:56:11PM +0000, Ananyev, Konstantin wrote: > > > > The CPU also > > knows already the value that will be written to cons.tail and that value does not depend on the previous read either. The CPU does not > > know we are planning to do a spinlock there, so it might do things out-of-order without proper dependencies. > > > > > For __rte_ring_sc_do_dequeue(), I think you right, we might need > > > something stronger. > > > I don't want to put rte_smp_mb() here as it would cause full HW > > > barrier even on machines with strong memory order (IA). > > > I think that rte_smp_wmb() might be enough here: > > > it would force cpu to wait till writes in DEQUEUE_PTRS() are become > > > visible, which means reads have to be completed too. > > > > In practice I think that rte_smp_wmb() would work fine, even though it is not strictly according to the book. Below solution would be my > > proposal as a fix to the issue of sc dequeueing (and also to mc dequeueing, if we have the problem of CPU completely ignoring the spinlock > > in reality there): > > > > DEQUEUE_PTRS(); > > .. > > rte_smp_wmb(); > > r->cons.tail = cons_next; > > As I said in previous email - it looks good for me for _rte_ring_sc_do_dequeue(), > but I am interested to hear what ARM and PPC maintainers think about it. > Jan, Jerin do you have any comments on it? Actually it is NOT performance effective and difficult to capture the ORDER dependency with plane store and load barriers on WEAK ordered machines. Beyond plane store and load barriers, We need to express #LoadLoad, #LoadStore,#StoreStore barrier dependency with Acquire and Release Semantics in Arch neutral code(Looks like this is compiler barrier on IA) http://preshing.com/20120913/acquire-and-release-semantics/ For instance, Full barrier CAS(__sync_bool_compare_and_swap) will not be required for weak ordered machine in MP case. I can send out a RFC version of ring implementation changes required with acquire-and-release-semantics. If it has performance degradation on IA then we can separate it out through conditional compilation flag. GCC Built-in Functions for Memory Model Aware Atomic Operations https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html Thoughts ? Jerin > Chao, sorry but I still not sure why PPC is considered as architecture with strong memory ordering? > Might be I am missing something obvious here. > Thank > Konstantin >