From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0053.outbound.protection.outlook.com [104.47.32.53]) by dpdk.org (Postfix) with ESMTP id 8CEFF567B for ; Sat, 23 Jul 2016 14:36:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=q+sFsmTBTspO6C24Xu8T5GJxu94I/hvdyrw2Owp3QKc=; b=mCKZiZZnGVhDw196BGJp7dgQs2xuoNzl/O0BpuCriqiLFcbuoJzefpHzcLAjWsqx5+CuQz1mpe7hqv5NIJ9tDZ61c6ndHjdaMCFVsH6p+Tm5fZRlEVTCoakDwwopv6fy268SsV2Ab/mOISIcOrUqz5w2PV4eM0zkiWYkhH++EHs= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.localdomain (171.48.19.161) by BN3PR0701MB1717.namprd07.prod.outlook.com (10.163.39.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.544.10; Sat, 23 Jul 2016 12:36:07 +0000 Date: Sat, 23 Jul 2016 18:05:50 +0530 From: Jerin Jacob To: "Ananyev, Konstantin" CC: Thomas Monjalon , Juhamatti Kuusisaari , "dev@dpdk.org" Message-ID: <20160723123549.GA24183@localhost.localdomain> References: <2601191342CEEE43887BDE71AB97725836B7E32F@irsmsx105.ger.corp.intel.com> <14017551.U6D1dIIx0P@xps13> <20160723060515.GA13747@localhost.localdomain> <20160723093621.GA18376@localhost.localdomain> <2601191342CEEE43887BDE71AB97725836B81292@irsmsx105.ger.corp.intel.com> <20160723103847.GB18376@localhost.localdomain> <2601191342CEEE43887BDE71AB97725836B812E8@irsmsx105.ger.corp.intel.com> <20160723114928.GA21364@localhost.localdomain> <2601191342CEEE43887BDE71AB97725836B81316@irsmsx105.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <2601191342CEEE43887BDE71AB97725836B81316@irsmsx105.ger.corp.intel.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [171.48.19.161] X-ClientProxiedBy: MAXPR01CA0051.INDPRD01.PROD.OUTLOOK.COM (10.164.146.151) To BN3PR0701MB1717.namprd07.prod.outlook.com (10.163.39.16) X-MS-Office365-Filtering-Correlation-Id: 7ed1cad9-6222-46f0-4e61-08d3b2f5eca4 X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1717; 2:hjjAEVEUqxCOoxzYjOltnGZ9yArdk9+hBBi9VTvTcTSeEvfT+OTcgDwTOBwZGr0qnxXfS+39Ue/+CC9Bf7JSiMPrvQ2+j9KabGhrDQL3gGcwZoyUjiEHP2G7SjKi9LcissCOi3ECMKGbM10CLHAQqPSPPeQfVF8Kp3bqMUZNIFt26oxxWB4P4qDQM05WW5+q; 3:7LJiOAHfgQMb/2e92GafHn6ArMnl691qQJpRwl8AMCUOBrj2KPH9lQqr2lFm27++c8i1qekRRuQXIpFPk9fhFh7o3C9uZlWEzY7PyEOWlJ4+Y8jf91ZkPtHNbxhJkj8v; 25:ac/tv81M8I/3OZnkf1dyRCKL3UAFj7yrPlhmk3ah4Wd3IxjnS72sVw/ou2N7x9zkPORitopD01bR3u3pG6amuOApCfEMGneLh5ZJ1wwePIj70oK8mPUOaaABJ5n2KiJ3E9E66C99zaCQqlS4nW51zWfn7P6yT3KrjJYBuVTXbnfbV4+ui3Uik6NJ2aot3dUZwWsLwI2+aw4M7bwm6dcpK1O2H0RCkUfI1ZRukMLOjU/m5V23yaamzMVAsIz1LRid2nNZ9nhnpof5oQVE1I2uo5jbv/Y7h7Pm4TlBO78dGDZbgo5qxYKp3y45y4E22NkB4BahIwyirr200oMTEMiwSXuH0RbgiaB2+ehzhSfCpNP5t1nD430LrkSjVgYbsw3P69X+ZLEsinNaeAdvW+kxIyIoIULuOCPielV/4pJox3E= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1717; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1717; 31:mSufARVT8HOYH34YmFxAMaeFaxuewYE/uJiddPtZLljnHidVIj+cKBwPJw0r6eBzR6xCXk7rtapdnTi4BNfTxYvEHJV6whAlrccUtkX7BTmrcEgUSgWGhl/mbTfJ4jGjvSalD+waz5wsfy7liTtMaGvuVzk0kFOasJ0abP83vTFPrpusbilOO9Nl32kCtFLwkwjR2UnCgsZdn2whC3/hYw==; 20:o0vVUmOecd/OX8GwaHXB+yXW9LlIMZfG8Q5ZnzoY9Htbj2eY4eZOmz/in7/bD2ytpFeq0mwjgmzzsCNndAkQM1C40t2lP0wAYazN7YMxXnKfTZsQ04c6tj4aeeaFTVT1LxdtCBIE/ZWXFOlWXP8RYjM0MbQJgwbjmYJSEKP/SYCmGvUxZOnJhOili5KcQR49V2WiKVp38K0bxc9r0kPlFpKelxrPlbtXSkwKt8IQ1QRbxfcYllBIHrkt1AawMa4bdFtawUx9I6n19E9TvHWR40eBG6xxPm8WksFwzdh/g5706If+zV3J5Gi72/WE6bUzdIdMuw0kzXgQnn82Thca+ZAkHdc61cwo/NNoQSpSXAMmxDqtszoprItzzc6SIjuRIgbJkdhXyF27KM230lGz61BawSVj7EEeVu63uYiofco/bcmmo/F/ltFY/V/U5SmKRhQWstWeRakRqrIkOxMz7sgq+tq0/gcyvKCWsVYtd5vFbhqJNzKaxuM4ZP7OUPWHYftUzZEQauo/IAdgGfaocerOKiD4n+PFjg4u600nC3rtIdaIm7cp/a7mHMSZtwX8xxscvstDnYKCmMxLWGttfiLq7vW/xNOkuIOr3MZ4f5U= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(51653755401839)(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:BN3PR0701MB1717; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1717; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1717; 4:VKjSbmLludON1XwNJ4Z6qqqihK/PNpbuGnnIxNJZBw5B8XBVsdYrkZENL4Dh0Eq3YQELp/5hmp1lZH6VYsU8N9o4lphD3C1QT4Ido4Z+jfyQP6/uxYfcaLMrsw8zHMbvXzsw7pBmV1QbZb5BFDJ5QuPxe5RY4miUeF86FVaeFfg2VAieqx9KQPRNMxJnUkFYEMAh9LS9QWnTHtqTCFh2oJiRc0KNUIMAAvAGfRrGjMFJRmBEH3C4QhLXJD8CKV39MJbGsycm8ui1esKRmlH+VU7BGxVoRcEkoMDVaznqCQjAp0W1+n0MMfUPfwn74DDmyakn9sluG8a5SPm3SzOBeiH1OZVxHr0GRZu2WdBEHsvHsI7149WBwdbRzsfFSf1BURoCKtM1KcSLhu4472f6RQjdSRgVWlXvqA97zbveg8wZSl8QJQAaX0ZGsiAw9lsB X-Forefront-PRVS: 0012E6D357 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(189002)(51234002)(377424004)(13464003)(377454003)(24454002)(199003)(93886004)(46406003)(83506001)(97756001)(106356001)(2420400007)(7736002)(101416001)(7846002)(54356999)(4326007)(305945005)(50986999)(76176999)(19580395003)(19580405001)(15650500001)(110136002)(97736004)(189998001)(7110500001)(8666005)(15395725005)(6116002)(3846002)(8676002)(61506002)(50466002)(2906002)(10710500007)(586003)(23726003)(81166006)(66066001)(42186005)(68736007)(105586002)(4001350100001)(92566002)(1076002)(77096005)(9686002)(2950100001)(33656002)(15975445007)(81156014)(47776003)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1717; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1717; 23:q4S2zyTxQQtUDO/Q+V/JvJtHGZM2xv6AazFVRex?= =?us-ascii?Q?y1QSh/oidSli23r9J+3dxQXdDMxVEUljen8m0vOMKPg5ieciOUVjW06wamHk?= =?us-ascii?Q?zp8yW6pPxWaOCf6Rn4MRtvy0zBJYfgTDIrypxnChQxcvbT2rjUU4mZTSCEtg?= =?us-ascii?Q?c6Ol2cJ/TBwQvFjIKQhsyBezns6Dd2xMkt+6yQgIjSfKocaQgplacoijNLAW?= =?us-ascii?Q?hKXem0TWW3t7+DtmitmgZSDMtpRKVTZ+UC2IhrnnZaR4cuAgulB4mChxLPyU?= =?us-ascii?Q?PBQAWCbzBlVX0z5lsJ6qsRJY+E3bh6GZ4Mo12INbor7eFENYQ4AAsBEA13B/?= =?us-ascii?Q?oftmkfAYJV8329mHnNHtfcveH8g5u0OWpTowb5OSjNg8YIcg8EwkAqDZMs25?= =?us-ascii?Q?Sl+4x7PwhJkfnn033Z5VT77jtj3tpZpcd/ZX5fgvDrK3KySqATsE5xzAvzy4?= =?us-ascii?Q?hOYD4YDxiFmGjEJyo/LJtvVG0np64Zlgz43DaaGskKy+n4Lb+H7eMcIq1mQL?= =?us-ascii?Q?75JQB3eKbup/Om43hns7nzO2cmliZ5z0qht7jZFMcg5WPJ4ECkLmcIXCg3vc?= =?us-ascii?Q?139GArTyiicCxAL+8BGUVf400X8tpK8yaYK8ZpDYrP3isZ5IiZR+id7LFIqN?= =?us-ascii?Q?yVATuJBjk4ELpSOpB+804xhB/+raLivyiq1QRiojGsTvMiJSPdTZC2ypGAWm?= =?us-ascii?Q?vQCctVAxZTvHJ1MYOeVfpMOquoUK+U90CTIa1gwJzrunc2Hs+L/wHM2in+aV?= =?us-ascii?Q?WY2RQjALXhjLsfniU2cSpcnEJrvFEfVIHbZsLCzbFYisBTCv0XqGMyGYeof2?= =?us-ascii?Q?CvW1oS9auE1w/zHOllZ9mNxCdW7y+eO0nQURWCVZRkOWG7d8FyzV6zGyRqFm?= =?us-ascii?Q?Oc2OhhV4uO/GCSbAQzqi4fTvM+BVxDT1t5vYTCwTXhHX6Arp5Hmq96YeWCue?= =?us-ascii?Q?vs5WVSQhKAJWPSWkdl4FbZC4PJUeq6ceOBchEmSoSedLSNUyAxwyTTKCLuwM?= =?us-ascii?Q?pjm8p3JXROI3zyW5avBO0GbORu75GdJ5WovZI8Z5KkyAQI/OiJlGcLToVbqX?= =?us-ascii?Q?6SzVeLL8GGVLUpkhPhqobJX+2gIgGturnMSL0+O1eXqQswswlt9x7CU8bI8Z?= =?us-ascii?Q?X+YD44a/9SOMtgWiSBB1GvLPE6QA+iDKs0lFQwjVaezjeHKjT+evd+UAN2UI?= =?us-ascii?Q?TH0YgwRuL78OaIguLurQqGVui8RG1Plxh5RZXtVF4qZIQ1PVCHU1X8zwYDty?= =?us-ascii?Q?2NnbvfvTOWxC6TRN9yyiP83/8mHsv0rerGtp1yCSRzvTdbSAz/MV7uNQC7aG?= =?us-ascii?Q?OwYgYDoREcCToXeA4rJbYIWhzH4cH6L0J/HmuduIi9VyqZhuMHzY0OhXIGOO?= =?us-ascii?Q?FYpil0bTDls/QLxiOwYy8tNtcx6NT2IOMMhDiD1Za3JLOrRZQGGGLGogr+s6?= =?us-ascii?Q?5isXkJBD7pEgGdi831jV10K+t6VA+gQUw3xbAVOMjSIqODnGufmcT?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1717; 6:ygxJT94NANLfJDuk8R9nrIHhw6X/fAcGIHEnEc0vh9e8quemSYkcLan9TAVpIbfayUGvQSr+s3gUaRf+u4ttD2MCDhCRmWXGIMTP2vjuOs6ysMj7iNs+rnimFFE7suyH5liSPUjJ0IDwmtO1GHJAxEE2YIDnmg82oDYOU97Jk7jNXllV5I6Trm7RW9TNAO65Ze0t7+WvG+Gab2CYsbH61I2oPz2d5oR3Hzn325ee773o+F4abZQIftyFxo+nKzjsqPMeOnhB0uz1k9pqA1O2uIwx03uYrQrEO1Fffhss7WI=; 5:u6aIGXXSdkJaImbkQYMLDZcBzoKuOGkm+uhQLyYo56Ct5woCPvAIgQN8m04NLFRFLOXv73XiGUpefUrjstY3pVhIrxOSwaMl344SvJVDIQPH2lCldqAJSUiBhKCn+xM0uP5qXD/GcCgqeXvcDR3AKA==; 24:23quNstffOpK2CA6m2CwSjyhCMgUxhkf0w7fXSYTTmLM1nZf0PvO1hoTrHepZQCAdAN9ueOZFwIAJgynnoJLUe1KnKQybYHrB9MFthV3ZFQ=; 7:2dEwrDKAss0K4xiEbyOxodEBM6wcksOatsmu9SjDYZ/jzHaxkUthQDAG/YumkpZFzH96+pxGcm2d/au4v9viYjc/Hl4f+2d7q+feBvxJ3mmDuN0Tdo2QeCXAZYOmSOT0P7ZtTw1Xnpxq1W9T3JSHJ8u6MWPdD4XiCwZLcjJ0e1MJBoRasNwzJgR7AeMvdVwSckK6Qpo0QyxlpRqn035PMjEAQfJ/yWWv1pJyGiwV/M+AKxBMlnZSFiCep8ftFKVY SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jul 2016 12:36:07.7119 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1717 Subject: Re: [dpdk-dev] [PATCH] lib: change rte_ring dequeue to guarantee ordering before tail update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Jul 2016 12:36:14 -0000 On Sat, Jul 23, 2016 at 12:32:01PM +0000, Ananyev, Konstantin wrote: > > > > -----Original Message----- > > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > > Sent: Saturday, July 23, 2016 12:49 PM > > To: Ananyev, Konstantin > > Cc: Thomas Monjalon ; Juhamatti Kuusisaari ; dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH] lib: change rte_ring dequeue to guarantee ordering before tail update > > > > On Sat, Jul 23, 2016 at 11:15:27AM +0000, Ananyev, Konstantin wrote: > > > > > > > > > > -----Original Message----- > > > > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > > > > Sent: Saturday, July 23, 2016 11:39 AM > > > > To: Ananyev, Konstantin > > > > Cc: Thomas Monjalon ; Juhamatti > > > > Kuusisaari ; dev@dpdk.org > > > > Subject: Re: [dpdk-dev] [PATCH] lib: change rte_ring dequeue to > > > > guarantee ordering before tail update > > > > > > > > On Sat, Jul 23, 2016 at 10:14:51AM +0000, Ananyev, Konstantin wrote: > > > > > Hi lads, > > > > > > > > > > > On Sat, Jul 23, 2016 at 11:02:33AM +0200, Thomas Monjalon wrote: > > > > > > > 2016-07-23 8:05 GMT+02:00 Jerin Jacob : > > > > > > > > On Thu, Jul 21, 2016 at 11:26:50PM +0200, Thomas Monjalon wrote: > > > > > > > >> > > Consumer queue dequeuing must be guaranteed to be done > > > > > > > >> > > fully before the tail is updated. This is not > > > > > > > >> > > guaranteed with a read barrier, changed to a write > > > > > > > >> > > barrier just before tail update which in > > > > > > practice guarantees correct order of reads and writes. > > > > > > > >> > > > > > > > > > >> > > Signed-off-by: Juhamatti Kuusisaari > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > Acked-by: Konstantin Ananyev > > > > > > > >> > > > > > > > > >> > > > > > > > >> Applied, thanks > > > > > > > > > > > > > > > > There was ongoing discussion on this > > > > > > > > http://dpdk.org/ml/archives/dev/2016-July/044168.html > > > > > > > > > > > > > > Sorry Jerin, I forgot this email. > > > > > > > The problem is that nobody replied to your email and you did > > > > > > > not nack the v2 of this patch. > > > > > > > > > > It's probably my bad. > > > > > I acked the patch before Jerin response, and forgot to reply later. > > > > > > > > > > > > > > > > > > > > This change may not be required as it has the performance impact. > > > > > > > > > > > > > > We need to clearly understand what is the performance impact > > > > > > > (numbers and use cases) on one hand, and is there a real bug > > > > > > > fixed by this patch on the other hand? > > > > > > > > > > > > IHMO, there is no real bug here. rte_smb_rmb() provides the > > > > > > LOAD-STORE barrier to make sure tail pointer WRITE happens only after prior LOADS. > > > > > > > > > > Yep, from what I read at the link Jerin provided, indeed it seems rte_smp_rmb() is enough for the arm arch here... > > > > > For ppc, as I can see both rte_smp_rmb()/rte_smp_wmb() emits the same instruction. > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > Wonder how big is a performance impact? > > > > > > > > With this change we need to wait for addtional STORES to be completed to local buffer in addtion to LOADS from ring buffers memory. > > > > > > I understand that, just wonder did you see any real performance difference? > > > > Yeah... > > Ok, then I don't see any good reason why we shouldn't revert it. > I suppose the best way would be to submit a new patch for RC5 to revert the changes. > Do you prefer to submit it yourself and I'll ack it or visa-versa? OK. I will submit it then > Thanks > Konstantin > > > > > > Probably with ring_perf_autotest/mempool_perf_autotest or something? > > > > W/O change > > RTE>>ring_perf_autotest > > ### Testing single element and burst enq/deq ### SP/SC single enq/dequeue: 4 MP/MC single enq/dequeue: 16 SP/SC burst enq/dequeue > > (size: 8): 0 MP/MC burst enq/dequeue (size: 8): 2 SP/SC burst enq/dequeue (size: 32): 0 MP/MC burst enq/dequeue (size: 32): 0 > > > > ### Testing empty dequeue ### > > SC empty dequeue: 0.35 > > MC empty dequeue: 0.60 > > > > ### Testing using a single lcore ### > > SP/SC bulk enq/dequeue (size: 8): 0.93 > > MP/MC bulk enq/dequeue (size: 8): 2.45 > > SP/SC bulk enq/dequeue (size: 32): 0.58 > > MP/MC bulk enq/dequeue (size: 32): 0.97 > > > > ### Testing using two physical cores ### SP/SC bulk enq/dequeue (size: 8): 1.89 MP/MC bulk enq/dequeue (size: 8): 4.28 SP/SC bulk > > enq/dequeue (size: 32): 0.90 MP/MC bulk enq/dequeue (size: 32): 1.19 Test OK > > RTE>> > > > > With change > > RTE>>ring_perf_autotest > > ### Testing single element and burst enq/deq ### SP/SC single enq/dequeue: 6 MP/MC single enq/dequeue: 16 SP/SC burst enq/dequeue > > (size: 8): 1 MP/MC burst enq/dequeue (size: 8): 2 SP/SC burst enq/dequeue (size: 32): 0 MP/MC burst enq/dequeue (size: 32): 0 > > > > ### Testing empty dequeue ### > > SC empty dequeue: 0.35 > > MC empty dequeue: 0.60 > > > > ### Testing using a single lcore ### > > SP/SC bulk enq/dequeue (size: 8): 1.28 > > MP/MC bulk enq/dequeue (size: 8): 2.47 > > SP/SC bulk enq/dequeue (size: 32): 0.64 > > MP/MC bulk enq/dequeue (size: 32): 0.97 > > > > ### Testing using two physical cores ### SP/SC bulk enq/dequeue (size: 8): 2.08 MP/MC bulk enq/dequeue (size: 8): 4.29 SP/SC bulk > > enq/dequeue (size: 32): 1.24 MP/MC bulk enq/dequeue (size: 32): 1.19 Test OK > > > > > Konstantin > > > > > > > > > > > > If there is a real one, I suppose we can revert the patch? > > > > > > > > Request to revert this one as their no benifts for other > > > > architectures and indeed it creates addtional delay in waiting for STORES to complete in ARM. > > > > Lets do the correct thing by reverting it. > > > > > > > > Jerin > > > > > > > > > > > > > > > > > Konstantin > > > > > > > > > > > > > > > > > > > > > > > > > Please guys make things clear and we'll revert if needed.