From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <Jerin.Jacob@cavium.com> Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0062.outbound.protection.outlook.com [104.47.40.62]) by dpdk.org (Postfix) with ESMTP id 852925594 for <dev@dpdk.org>; Sat, 23 Jul 2016 13:49:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=G8ImX9yPKyIg0Ykew8GmgMQsFhJeTR1HIrHvBdEFmSE=; b=Bxe1SchVCXcNEIUuLyzlBKwCXEIvPjTY/1hOX1QBBl5gUcmYNf6iOksno56aAWiCN5UU95NO4QKDuKfjbZV7CVMvUhwiniPw/q61PWpQqGT8qIe3urdWfhxks5+qlxmXoWVtdKUNN18NHa1yrMXTYPcGavYoGFpAWsybmM2vcgM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.localdomain (111.93.218.67) by BY1PR0701MB1722.namprd07.prod.outlook.com (10.162.111.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.544.10; Sat, 23 Jul 2016 11:49:46 +0000 Date: Sat, 23 Jul 2016 17:19:29 +0530 From: Jerin Jacob <jerin.jacob@caviumnetworks.com> To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com> CC: Thomas Monjalon <thomas.monjalon@6wind.com>, Juhamatti Kuusisaari <juhamatti.kuusisaari@coriant.com>, "dev@dpdk.org" <dev@dpdk.org> Message-ID: <20160723114928.GA21364@localhost.localdomain> References: <20160715043951.32040-1-juhamatti.kuusisaari@coriant.com> <2601191342CEEE43887BDE71AB97725836B7E32F@irsmsx105.ger.corp.intel.com> <14017551.U6D1dIIx0P@xps13> <20160723060515.GA13747@localhost.localdomain> <CA+PrjLGmaYppXVLR190Qn3B-JQVsvY1ZHGQexvKXs0_MOt8FmA@mail.gmail.com> <20160723093621.GA18376@localhost.localdomain> <2601191342CEEE43887BDE71AB97725836B81292@irsmsx105.ger.corp.intel.com> <20160723103847.GB18376@localhost.localdomain> <2601191342CEEE43887BDE71AB97725836B812E8@irsmsx105.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <2601191342CEEE43887BDE71AB97725836B812E8@irsmsx105.ger.corp.intel.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: PN1PR01CA0061.INDPRD01.PROD.OUTLOOK.COM (10.164.136.161) To BY1PR0701MB1722.namprd07.prod.outlook.com (10.162.111.141) X-MS-Office365-Filtering-Correlation-Id: e58e34e5-58cd-4ed5-9c9a-08d3b2ef72f4 X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 2:N8yR0+cW1ih4yPti6YX3XYRwvrwXuaegr1/hFbiX80wHlqhX3iQJEC7NatoVnOkfnNMy/6qArY26hYZn6gAlx/6IQQRpf+UnLFBTkhPr4n9/cTFLs+EQ7SIVtAK2bF0SKrBuyvhE3zgq4nyqYgtASUkuTraKrlvBaQF7iRtxJR87DQuuG6q4lVbg0JaZYVo0; 3:jGtVFcfg2XFVs+b2q49rOG538OHwLaAW9hXMg5+sYiv8me7vUvZym76etn7cLFo6TYi9KfN01sJFeBUVVtuqtx+2W4ep9dtp5y+KCkAPdg1a2T/HLqbCZCe0BvwGHjUI X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR0701MB1722; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 25:0mNY1RUO+zbhNaMe7VO4aHrmRKqldJ4zip4/V9PB0hjjBjNcjjywZf+YKcctrxjGaSwmREPQ0o0G2NMg+k/cW6NmK6WDgps1AN/ijGQo8PhgzG5Z45YKHWCwZaHPnbPXakNBs/fJKWLJL56LbPQDFruNRRJkCX2dX4fcRdFaV8iv2EAWXy6VMG5oXU6WtfBMmE6ngOLy3GMZABfcDdv6l0ZJcA2yYVGsnIXyyH57dGt/UEvJJAHSxRkwAYKYpVYfekXqQ+XvvBIz3U0maJ4/9A4RGhcoZUBaB4Up25p/Q1cGuAV5AniCjrGRTGNLmug32uDeNeYiOltzcMzyNqUu2YUbR6htPyPfAZCHQBPbWRrfpLBZCPVRFVYtRm7bAwH43F1vst6D/7P1/k/FcuuMeQysXuosxBRre9IHgzPnhUf2xGn+RXrjSlKX11BdvuxXAa/K5xtwAEJfrBQNN9JDSEqvn0bIlr1Lz9G2u9APQzoHQ97TGDkjBDG0GgJG0aY1Uz+dGfG5h9dyXSSw48/1mjce58LlY/EVFdCel+YBwyjbFCFToO8UH9IlvMJFyqE6JugJb1njad94BzDoQYg+VLxKjHGBoLGTtc0Je7P0U1+tQfca/cftTdkCOA1fjcBfCvfKONE9XA+wH344csaekZIiHBac6dm7KdA+d0r1Z1amNUpr0x4VNLc1JzpdHRuBGHF+zClLKuuUwiD1HnwSWewOGxYwalXq9WE81JWZh7tMemke6I5MSwLtU/fil+mec9VLbJw6H2RANQ8B9y4s8JNsWv6pytcfcYkbXpOc+FcFVFFEGrZb+vI43SidwT/DT86SS0p5Z+2YrdL1YPjAsw== X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 31:5gj2ZDye+Kg7mDLx61hKhKLCtKPXYdHPhaKkgiShZzH1LEfTewFA4i8MDPFoLIYWGbTazzBVTrx1teCgjRSckRVQsJvLlim2D+qnlyuEDKcKyW7uo3pATmuZM/HBUICYTd0HH1anHZkcA/ZEB7vsCEPA5lVeC10cFI1uE9kz50/LrufOhPlIzub+iHm323/iyBnyB0l+oavSsGkB2nv1rg==; 20:AmMJrH6//50FpoKP6yicxbGohdjCz63qCkNczgXP7Hqz/b7Jdgq/5g+Wyco93m1IHrCSZ0McZHFu5SHZ1pXHZ7tK/Gy2fpjafDbUIL+Crmc+JiEr5+qjhjrtAMv/m1bdvwZEaS9qro1VhckdiU+/XWlVDalmcoUAyVTg345oNsFzbz4bzu9vFtko9g0EmCy12wpWlipBAFRw+wtE0nZJx5sAdQieeUo0Ri2vnZsQfCGWMHpJuKAAJtqA55bn+XPhpu9/1oV8dGAJrcU8pyi8omspDQJQ2yQ0JKkazUW7WeFQotM8u/9KIELBSbu2bt+1BzlMB7/mUhWE9HdTVIwJB04oqZ5C8lVHKFy4alGn9UUzmDIjRz2O1mC1lGS6YrxVglR8aOul6Q3/k3kYwzsgdXisfmoe3RGk1EaUP/q++ckEaoOV+LqdGUnLooe8T5DDfxNCKE7rydFOYm2STbrVVyHFxravgMTOYlxN1e6LClZVeE/kRn0heFuVXJgXwaSp8JC66CYPZziXRAhx7dH1ICwVhZZpB9sGt9EOE49YmpBpNPKY19yd3KwXJXE+eol8WzZp1z1NrtEJQAiLmGwiwsG9hoSejOk3mXxG9uGwpPc= X-Microsoft-Antispam-PRVS: <BY1PR0701MB1722A619597553EB96B0F0D7810B0@BY1PR0701MB1722.namprd07.prod.outlook.com> X-Exchange-Antispam-Report-Test: UriScan:(51653755401839)(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001); SRVR:BY1PR0701MB1722; BCL:0; PCL:0; RULEID:; SRVR:BY1PR0701MB1722; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 4:FLdkphIzEwCUJprj4EnLRN4npbZCPZzI15DA/S1XBeUTh1QG62XpZeJxddnrNhx7Kqxb08ano4EnOmd9Fwe4IHPGOZKrBd8cxJuF08ttvhyRWHeFlyuF38baE1/7i6hZffGFCOOk54f4oxKdfjjtvIoe/nDXG6VBKcx8qIumn2j9W3hnzgZDJ15XJRu8h5Q2KjsO5wPpKz5ehf4W8QrB3nSz8EhktOPo7/OM8xCbznluloT1tVdRK/4l7NQp8zAPAeVj4Juk5c/UWbKkdoc4EcDWvD32csTkDfEB7CPg97mckxa3gPGiWSRTN10y+r8jMFRwtRZza6jBc0taFj7jnFx7ATcfAiklRgRzO8u7Jh34/0x+pQ3vWTP2YE9GfvEMkYUzkc+86XbtZcP1O2HenrEzZqUr2c7H38Pon/SeQ2Hx/gYsw1oMzmSQVNGcVQi4 X-Forefront-PRVS: 0012E6D357 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(6069001)(7916002)(24454002)(13464003)(377424004)(51234002)(199003)(189002)(377454003)(19580395003)(5009440100003)(2950100001)(19580405001)(46406003)(42186005)(66066001)(50986999)(54356999)(47776003)(76176999)(4326007)(1076002)(77096005)(10710500007)(8666005)(7736002)(2906002)(23726003)(93886004)(83506001)(7846002)(3846002)(586003)(15975445007)(97756001)(6116002)(61506002)(7110500001)(189998001)(15395725005)(81156014)(81166006)(92566002)(50466002)(33656002)(4001350100001)(305945005)(9686002)(106356001)(101416001)(97736004)(105586002)(8676002)(2420400007)(15650500001)(110136002)(68736007)(7099028)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0701MB1722; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0701MB1722; 23:7OY+05lbErVAzfdKbHPwGfO+dR5ufH66qtwTinV?= =?us-ascii?Q?YsTWEObQsng+ETtGK7Ef2ov5GpYXDbePtj2zsSaQflICc+NGXfWnv8kPLY++?= =?us-ascii?Q?l0X01/bE2ToAZxAXHb0dBxSlPA2pPriv5nzfJ4IykKOM7jKAG2Ed1HlPzzPQ?= =?us-ascii?Q?yAOId1id47uUYLsHpgztqX/muM0SA0W3y7X7of1YqETx6eh4ZLjXRCexC2ra?= =?us-ascii?Q?pQ807mflyzaS4BEkfM8keyJMiqAhvmy5Y/nU9MEUAjxCCCD2epCMmAi5uzii?= =?us-ascii?Q?SawpGmZgyvqq4Z1jD0rav5F33tIvsrimIbuuM7M5XYJSI2sjP7jYm00YsHDE?= =?us-ascii?Q?wrWZKjBlTZrB5Hki10d0A15wzmvdN7FBF7Rdqg0fgQStdwXvgiVnu6aOoPhu?= =?us-ascii?Q?qp78Duzu97C47QttMstLSbW633bahURNLguMiWKmQqQNv+ooMaQO7UGO8rFf?= =?us-ascii?Q?UqdjKLujELXF+SQR8VOyjsA5TgVAtFgsWUAQCYayDpA3S2xOCoWvXcUm13lx?= =?us-ascii?Q?dA0r24ifV8Gg0Ox156atqXHxw2Y6xQmiApxBDMOnvfV0MpiUev0Sa9hbhU4y?= =?us-ascii?Q?FBPnJzUopwUPVneg8SAqvzPx6vnIeZced2rn6zFdEHmXSo+gOr5ia2qHIIfe?= =?us-ascii?Q?cIMf0wRyTtpxeACPqPPa1sx7ta/991kdiNGZ7zVgjRCUEO9lssZB2rot/aKC?= =?us-ascii?Q?eQUYv+iG8A51KvAdp34BR8mvNZcQvAxPtXSiP55aj4E0nK2sGot8b2Hagv4t?= =?us-ascii?Q?WTLfkzRj5jKXrzGcAfXHmZDEZpDYq4fG0HngwWaYmGYPdN+Fq6ffUkWiG4fs?= =?us-ascii?Q?c3cVnPSnxZ3QupP9MoQmNv/BQ/RRggBBqWZlYu5DfHAkxgh3iAV6QKLMwkTx?= =?us-ascii?Q?xEAtnNXQEVbPpU+Z1jo4QoOeVd7EZvBk++3+ZhmSQ8MUoDQ7SBZV6tBk14Ht?= =?us-ascii?Q?yvoJNeG0bI4gtQ/OCyx0v6ufLj2l7vxcZkXpDz7S1iQGkIIenIELTfewYheD?= =?us-ascii?Q?VwU6eJE2OdWPa51ZBkUPq22sytbFccHCpsZOluC9o9wTpbS6e7pwXyWOewoH?= =?us-ascii?Q?kIp5Y9FEuWX9/TzRMlU9OsFdgiEMFEXjO29SoYXYVW4MA+PQrt2T65LyLulb?= =?us-ascii?Q?PtTyGxUxBWpHuzKX6I10E1wSJceNnsWwZKAWQJoG9p29oG6h6IZT2m+ndW3+?= =?us-ascii?Q?6ZvGS1qJyJ9cUxiVtVrYPkQNTgQyLzgmTqV/t5X5zmj0PODABCd63K3vXd/J?= =?us-ascii?Q?wLs9ozBkDjI+IYxBKESnRReD5kH09mPy3lU8f7oNs64YmXcafSCzLKRiRUxi?= =?us-ascii?Q?cUprkG4J8X1Md/xcRoww3NpLhWoA/x1j335SX21EK0DlgrpKELyzAhQ5F34+?= =?us-ascii?Q?kkJGkDeMzkQbnPrwFnAEFG4C8M5wDNQTTjvTodl7qKT5ia97FishLOMdzToO?= =?us-ascii?Q?ZHUEXbk//8InAN2CkRv6rT1bVEL78+4LvBU8ZuxHPH78XeXspF4V+aIfM9an?= =?us-ascii?Q?Jih5qk5btLS2Dhla3cnZk4bG1cyDDTNKRjRM=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0701MB1722; 6:4GgFmuX37nGRsmAeNngAnvcFVF9D6kRVa+DxuuQ7B3k2/zr3H1KpTYmDfugfVOBPcxE4zMYmBHu4mP8eqWjPGpCZDkBlI2afp0kfnWeBLOzjeYJAovWatw3pHQYqWYLqxVpKA+U61XqxVzE1nk/w1G4ut1F2e60Ir2HPor7ftv8cWiTCOoOnw8ODSGltbClMOtBBs20piIqjgVdIBx2GB+0yqUP3K3GRH9lgCmCgmq+gMT3Lkqlx/9oQombKuBSSvGlV90NpUuL4p3Hk1DzcmrKZKzXVE+sMKrWckh9DJAo=; 5:/xkZWOywLk7EzuiSSnIhJ3zRuYrkLR+gtU3xr/JyM+jBg+RGMFA558zQ4rFwKBLlB/L+rCmi2xax75EGkWsp2nIirPTZ4KXXMFg3vqL1PhasSOR2HgbX0BkyVaJdey9DA7NBM1qO3TNqY+PU/TNRTg==; 24:S0g7BvGt9W1NaZ/VMvi6bcvCgGpS5vHM+0x6/qi2MUpal1Ae+T6Q2TtcHM9VW/hzWopVAz6iVZlgdyxjiMosP0aZ07re6BK4f5M0WXHSoeU=; 7:UAWa+z4hZvE2qB/yvsWx5CWzwaNyjG7wYbXty5AWuLyD55nA+JTSNyocgcaqrza7BqtPVuLYRpxOCZTaeX9VTkgurOHVBT8RPpfvwREFjNKT1RHjaWk1FLTO3TPv5ZMYHclmEPwqCQzhwkIo4iC7+Qm8h5PBgzvGAN5QJlLV2Ei2zyjWnNdD27rXsmTBXu34f8xkImSJ2TXD2HAz6P7cu34AQSOwQjbLEGcv8GzhLodjIBRE7/9Yee0dwPuDRUPS SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jul 2016 11:49:46.1764 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0701MB1722 Subject: Re: [dpdk-dev] [PATCH] lib: change rte_ring dequeue to guarantee ordering before tail update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Sat, 23 Jul 2016 11:49:52 -0000 On Sat, Jul 23, 2016 at 11:15:27AM +0000, Ananyev, Konstantin wrote: > > > > -----Original Message----- > > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > > Sent: Saturday, July 23, 2016 11:39 AM > > To: Ananyev, Konstantin <konstantin.ananyev@intel.com> > > Cc: Thomas Monjalon <thomas.monjalon@6wind.com>; Juhamatti Kuusisaari <juhamatti.kuusisaari@coriant.com>; dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH] lib: change rte_ring dequeue to guarantee ordering before tail update > > > > On Sat, Jul 23, 2016 at 10:14:51AM +0000, Ananyev, Konstantin wrote: > > > Hi lads, > > > > > > > On Sat, Jul 23, 2016 at 11:02:33AM +0200, Thomas Monjalon wrote: > > > > > 2016-07-23 8:05 GMT+02:00 Jerin Jacob <jerin.jacob@caviumnetworks.com>: > > > > > > On Thu, Jul 21, 2016 at 11:26:50PM +0200, Thomas Monjalon wrote: > > > > > >> > > Consumer queue dequeuing must be guaranteed to be done > > > > > >> > > fully before the tail is updated. This is not guaranteed > > > > > >> > > with a read barrier, changed to a write barrier just before > > > > > >> > > tail update which in > > > > practice guarantees correct order of reads and writes. > > > > > >> > > > > > > > >> > > Signed-off-by: Juhamatti Kuusisaari > > > > > >> > > <juhamatti.kuusisaari@coriant.com> > > > > > >> > > > > > > >> > Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > > > >> > > > > > >> Applied, thanks > > > > > > > > > > > > There was ongoing discussion on this > > > > > > http://dpdk.org/ml/archives/dev/2016-July/044168.html > > > > > > > > > > Sorry Jerin, I forgot this email. > > > > > The problem is that nobody replied to your email and you did not > > > > > nack the v2 of this patch. > > > > > > It's probably my bad. > > > I acked the patch before Jerin response, and forgot to reply later. > > > > > > > > > > > > > > This change may not be required as it has the performance impact. > > > > > > > > > > We need to clearly understand what is the performance impact > > > > > (numbers and use cases) on one hand, and is there a real bug fixed > > > > > by this patch on the other hand? > > > > > > > > IHMO, there is no real bug here. rte_smb_rmb() provides the > > > > LOAD-STORE barrier to make sure tail pointer WRITE happens only after prior LOADS. > > > > > > Yep, from what I read at the link Jerin provided, indeed it seems rte_smp_rmb() is enough for the arm arch here... > > > For ppc, as I can see both rte_smp_rmb()/rte_smp_wmb() emits the same instruction. > > > > > > > > > > > Thoughts? > > > > > > Wonder how big is a performance impact? > > > > With this change we need to wait for addtional STORES to be completed to local buffer in addtion to LOADS from ring buffers memory. > > I understand that, just wonder did you see any real performance difference? Yeah... > Probably with ring_perf_autotest/mempool_perf_autotest or something? W/O change RTE>>ring_perf_autotest ### Testing single element and burst enq/deq ### SP/SC single enq/dequeue: 4 MP/MC single enq/dequeue: 16 SP/SC burst enq/dequeue (size: 8): 0 MP/MC burst enq/dequeue (size: 8): 2 SP/SC burst enq/dequeue (size: 32): 0 MP/MC burst enq/dequeue (size: 32): 0 ### Testing empty dequeue ### SC empty dequeue: 0.35 MC empty dequeue: 0.60 ### Testing using a single lcore ### SP/SC bulk enq/dequeue (size: 8): 0.93 MP/MC bulk enq/dequeue (size: 8): 2.45 SP/SC bulk enq/dequeue (size: 32): 0.58 MP/MC bulk enq/dequeue (size: 32): 0.97 ### Testing using two physical cores ### SP/SC bulk enq/dequeue (size: 8): 1.89 MP/MC bulk enq/dequeue (size: 8): 4.28 SP/SC bulk enq/dequeue (size: 32): 0.90 MP/MC bulk enq/dequeue (size: 32): 1.19 Test OK RTE>> With change RTE>>ring_perf_autotest ### Testing single element and burst enq/deq ### SP/SC single enq/dequeue: 6 MP/MC single enq/dequeue: 16 SP/SC burst enq/dequeue (size: 8): 1 MP/MC burst enq/dequeue (size: 8): 2 SP/SC burst enq/dequeue (size: 32): 0 MP/MC burst enq/dequeue (size: 32): 0 ### Testing empty dequeue ### SC empty dequeue: 0.35 MC empty dequeue: 0.60 ### Testing using a single lcore ### SP/SC bulk enq/dequeue (size: 8): 1.28 MP/MC bulk enq/dequeue (size: 8): 2.47 SP/SC bulk enq/dequeue (size: 32): 0.64 MP/MC bulk enq/dequeue (size: 32): 0.97 ### Testing using two physical cores ### SP/SC bulk enq/dequeue (size: 8): 2.08 MP/MC bulk enq/dequeue (size: 8): 4.29 SP/SC bulk enq/dequeue (size: 32): 1.24 MP/MC bulk enq/dequeue (size: 32): 1.19 Test OK > Konstantin > > > > > > If there is a real one, I suppose we can revert the patch? > > > > Request to revert this one as their no benifts for other architectures and indeed it creates addtional delay in waiting for STORES to complete > > in ARM. > > Lets do the correct thing by reverting it. > > > > Jerin > > > > > > > > > Konstantin > > > > > > > > > > > > > > > > > Please guys make things clear and we'll revert if needed.