From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0057.outbound.protection.outlook.com [104.47.33.57]) by dpdk.org (Postfix) with ESMTP id 89BC51B13B; Sat, 29 Sep 2018 12:57:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HYPLan9i4103ZAkLIq4WXkWC8DZmkhmpY0WzyouKXpo=; b=Ig/OT1ZGLMKiowSSOkGtVIlOrIkcQO/bu09TEY8keVZVpYP3Qp8paanTt7HUjmdZ2AfrurRcGLyry/cNMeAKnREvaiefbEGNEAYX6iz5gIm/YBLufYQY0YRRCMDtjmRxZXxD2EAW7WGlyJg8Sz0X9hphPDfxTHiQIpCWcqAbqTc= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from jerin (115.113.156.3) by DM6PR07MB5001.namprd07.prod.outlook.com (2603:10b6:5:25::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1185.20; Sat, 29 Sep 2018 10:57:28 +0000 Date: Sat, 29 Sep 2018 16:27:12 +0530 From: Jerin Jacob To: Gavin Hu Cc: dev@dpdk.org, Honnappa.Nagarahalli@arm.com, steve.capper@arm.com, Ola.Liljedahl@arm.com, nd@arm.com, stable@dpdk.org Message-ID: <20180929105651.GB30457@jerin> References: <20180807031943.5331-1-gavin.hu@arm.com> <1537172244-64874-1-git-send-email-gavin.hu@arm.com> <1537172244-64874-2-git-send-email-gavin.hu@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1537172244-64874-2-git-send-email-gavin.hu@arm.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Originating-IP: [115.113.156.3] X-ClientProxiedBy: BM1PR0101CA0016.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:18::26) To DM6PR07MB5001.namprd07.prod.outlook.com (2603:10b6:5:25::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8b1bda42-67a7-4fa3-14f7-08d625fa5aa2 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534165)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:DM6PR07MB5001; X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB5001; 3:MQVH8kG4j9XPTQU/CB9/hAqQ+SPzGbdmUmb1Iy/1GVS0bcYPyef5oHDeDJZZFSeUS8YKvB2tUO70XaFb6hjZ4LBl+CzI9jERSBH8RkD8BmfmSgY1cMadLnKI9Q6aUB564PtXtwnEycgj9nadc3sdBN4A8Siduozwksxsa2z7yBEZle0iQTEho5NbYgJXpGJa+ntZjj3X0ZD59JsIE9fBYK4VYCqofdH/inSAe616NGpDzfJJ9T6CpHAvvLDA0Txn; 25:18FLks5CX7ZXdH/x0b85EDy63sMplYx/6OkZ7NXVvCJjZB/HmrRvuRMeuxOVefYmnbo4MydmYi3n+ra9cfe0mKXY+bcF+o1ZswPh1HjE3+45JEdiW/J5hrX4/Qx56j1uqGhJIxsWRaiQm+Idxb+Ip+ngVrO+MGT7Jb34yqSX5L//udiaeRjdG8RvRI4Sqi2ZvIV7+PWL/uID1ffj9u9QWr5wTfzAXycCl+IXPchp0IHaLU4H4cX3YiYA0oOAh1UkOwI/xBC2+3jKZc8o/No8hxav8ILLL1q9q1l7gHm0ohZEIuEev4osBQmbQxhCTHYyMojUhnM09g6sIsLnCpvMng==; 31:X+/F82kPq/Bsy9UYMJbqW0mTx4QA5Yzn+eb1aVvoX2eVqyMfh8AGZ75bp+fT2n1doogwC3MP+dxfZ1oX+nJ31yhvvjSVscXdQV1+F+ho/r/nnHdtcD8RheJn1omyMjeIEXZQlnsb9w0wI2z+Ki7bX9M11KlW5y9Bbjd8apae1UD7cXeMSbYaDBsEbxDeH8P7lz4PkDcyyz5rQM31auLGHirwC7x6GmCD4B+oBQlf1PE= X-MS-TrafficTypeDiagnostic: DM6PR07MB5001: X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB5001; 20:vCqL8gRFyNTnTTGVIHScWIKiXOrKr4pxY+Vcw6rcbD8CZK0pWdx/v4sWJVgFQYZhvPhbfqK5fyOnYjmxwzQyLD6vSgUnrjFoRKvAANshbRCoRp3g56ZoSEkMrMTh0c+OGxuC7lD2VTkhSsDfizi5kkdq7Ru3uGIgs+hTW+IsQBIqXyO5eWl7CIbbT90V6+a+eYfV9bWTrkNXLJR9Lp2evvQ88buGTyAHvyaV5aCTONrlAPSjvvXfnZW78ScYKx4AfnjPawTrOE5d7ndIXG3HlG9gtffEOmGtVqT6rCUQxvlWnetlN2Y5oN/UDyLILufWwd8e8UZJ8DPGOM14VaYY8/PzXv4lXQsZ8oHw0OkZk8CnHbUHtyZ4sZqW+GmHZ5q9v2/4xdFnryMgJnJvRsafM7TX53XKoqGCZfB+oVlEEp8w0wfwRxTwUs6KXlW9h7qdF3X10ZrO6raH6wbvY1rOR9p4iBV0H1dJs7fFbADMCnG79ropqEZlmhz+lVFK6+pxURWkW4hIR5FhqXaRjl4oChbHGiN5YGXT5NecrujeJ1joZeGGRgI2c1IFVJZoWiCYxc/FPa4qBBIPLi8Bis6uoj2gzAMHMVdq+/S2LgUqQMw= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(180628864354917)(158342451672863); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(10201501046)(3002001)(3231355)(944501410)(52105095)(149066)(150057)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(201708071742011)(7699051); SRVR:DM6PR07MB5001; BCL:0; PCL:0; RULEID:; SRVR:DM6PR07MB5001; X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB5001; 4:vJoEOV8s7dULfPXm3DWAxBEGqhoxKBODAItsqDjSrFBSKrEqatUR/vFQesccGQlIF5ZLQvu98xMHN36KxQtFON8qo0kdCGkeiL7xhZ7Aqmf+gRlhgGaCtMbGSVgANfQ08Vg9IRzRoBa6Bt2oOKHKDAKH6RTBstmOYy+KDMTh5yFhN/KRixtFr+t6QuW5KpcMg+FmC2MZu8+fH5vj2Fy/fhdSlJyZFq/+LwR2IKlGPKEzFrl+pL31Nh7fhTmjLj4LeXwSnwQW0vxcyL//Z/89s/MeAz0jF0FMHotetyMOOcNNviHcbzP34vkZbTZURbXVDkfnZSKFlh3gP49MW3nBqtkBsINsRC+0fs0NtQRMtfM= X-Forefront-PRVS: 0810818DA0 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(136003)(396003)(366004)(376002)(39860400002)(346002)(13464003)(199004)(189003)(55016002)(53936002)(9686003)(7736002)(4326008)(33896004)(229853002)(33656002)(5660300001)(81156014)(81166006)(8676002)(66066001)(6666003)(72206003)(47776003)(76176011)(6496006)(8936002)(52116002)(97736004)(305945005)(6916009)(478600001)(6246003)(33716001)(44832011)(316002)(26005)(34290500001)(14444005)(106356001)(2906002)(105586002)(25786009)(50466002)(386003)(486006)(58126008)(16586007)(11346002)(446003)(956004)(16526019)(186003)(476003)(68736007)(3846002)(42882007)(6116002)(23726003)(1076002)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR07MB5001; H:jerin; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM6PR07MB5001; 23:YWXLHXOJTAyq1tYorYwnCEGGHaEfoy1kd/vWDcgxh?= =?us-ascii?Q?O/PMKEjoa1iuaFgknbDaM57GFbDNw6haraRKnZkgxELuTkGzYGu2UziDV/uQ?= =?us-ascii?Q?OjyLIkoXG+8kEKcKrf2HsBG9x9rtJFAa510+58ics1dOI+vsW/74tksSrF1v?= =?us-ascii?Q?HG9zLpL0ynPH/msab+uwAg1FMD1PGTmffXIWdcIIMV5mDlybBYnGomDx6S+u?= =?us-ascii?Q?m1xUllMWdBRfnQ7SnLb+V0aeBYil3obaSJL4hTIAM9emexcmJthUdGaxVejX?= =?us-ascii?Q?h0wFbvr9lm5MJ6eGkCsEJOblh1qWwa2kOCHGV0RO1UTbHG3ru32wcGhaVd2P?= =?us-ascii?Q?ZrkEnsiQmcQl1VvMKN25x03sYY0o1zGNJdsXeHrPlTcje8HLbNHC7EIlMFEk?= =?us-ascii?Q?wBRLqy2b4AtF32Hqgrj8aEeu5P0fLzXQe7dUIcP0LGBPSw2mLSQ/c3MXWBTe?= =?us-ascii?Q?P0sfGNwyV9YHZRQOAPxsm2EvAy5HdSpriSlXIMXOG+iF5TxrNnUPGEOLemwU?= =?us-ascii?Q?AxqSLVuuOmi6vVBDlsUGZNdwgno1a9SjyQOtmmMIpabNaxQb4NP5Q6iW837p?= =?us-ascii?Q?X0B/PNhB2cf8OmLY3BSDDacJ3ajizlXGgHqqcBA8vNWUvC2R6MgvaHXWM+o7?= =?us-ascii?Q?d5iKm7lqzdvnbrKSMBrOe5uWbm2+ffWQ6wNLZyRwLnIUd9likgjquPFhAIMT?= =?us-ascii?Q?vlE8EmroycZxK7CsfXXiyslxgvbckN2zIMOZXk4Ucz+ptek6msMGtc+9ss9m?= =?us-ascii?Q?mQ+tKLaHddBh4IQmQ6pSUFmr6cMM9jLjdN1fP7uL17bRbEUFow7Fnahm7FwR?= =?us-ascii?Q?MfO/Z/TpAziVtXcL/kHOZ9xZ59ImCWUpNxCLooNM9Y3vlWSPzcX+jWfLTyJQ?= =?us-ascii?Q?65FhY0op0q8Lds6m50vdh3G6cefZ9BySEkqAPbxONL8fiHOAUAjn9eSqU4da?= =?us-ascii?Q?E57fHPOj+L2REra0r412ujjvPbV8qUb5a5vB1enRVSIzSKevvlV988CAliLD?= =?us-ascii?Q?J+D/Y6LC2KMUrc+ahTQKWTK8G7BSqs27GYd6IesU9BhQuTpYjCCz4yaebmil?= =?us-ascii?Q?dGrLl5cGx3U2Udbt10Z5m+/+EKkXp5t32Ywg3D4TltDJCaZUKeVKyYHLBUG9?= =?us-ascii?Q?wTKE2YhFRlOxAMccVYnGtKtfeJDeTgrLm2jGcFR/XMfin3cHZRlrhCcmUtwu?= =?us-ascii?Q?2Fq9ttsH8kKTfJHXV1SZIIz5SndDM2mW4vASKwrqhUqzpuA/9M5BWhaiv39r?= =?us-ascii?Q?w3NUmi7G9Z3NbTybeIZhHR7elRu4sPgqNq+SvRCR0Ts4Qv7laxRqDhaJ3LEX?= =?us-ascii?Q?5vvsRHBHnXc8hom2pRquwhXEDJPJ66zpK36/fonJDCCMQp8Zx78rh1qgoxzz?= =?us-ascii?Q?Z+64g=3D=3D?= X-Microsoft-Antispam-Message-Info: YJKO1YEnr20res1CNtH69bsx2T/jshKngFRjPV6QyuMQb6Eogu1GGNLhK70A+BONJazO37K7nVILvpm9i5av++n5PmGHpLxVx1BHMZKmvYwgqm//WyfX3L7+nKUBdD5xdNeLrIYEksyPNnGzjT1JhDga5EGBNNQC45guUIKZzUwLvuMVWokRih2hNI0YkM95UtkTTq1PZIK3I9woofsRw5VqchdCWW8kj6cQjC6fJjGFAkJVPhDGh2/As8YN52W3BylBcE6iVkmoGN+noHhU00aratT34CC0Xl7AuFamOLVlllJQHcejMFmt0iEh5hMLpspZDh4FyQ21HdkD6er9Bx6TVtjb91PuKZ/gE90y9nY= X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB5001; 6:/p1LWnkbZNwIXmloqWdMlSrm7PuNLPVHOcrlNhSDj6MeYro5JwMmCUQd+68QmNbbG8Ekz9NUx/rYawE9aaNC6BBjJrhvRd6KJL5DUmmk7HH2rYq5Gap54zZG0aMj2kLymF/BcFC/jpQ2wy1C3785Cqfs4O1dXpDraP4KwJ8ca23J3mSX1uYkHZejLbLquAQHqGz1NO+X1jsvc5Mj28m5j8mkR/O7I10hyJvwJyL3prTltpDfPVCS071RJp4Rtlim9EwNiVxV0gkkUER2FNI+0gweucAsN1Wo/sC1453H7GSPKP3HHctClJ3HQGsKAlN8ZL6a4wKdT6IYbkCaL/DP1s4yPLjR4vhgdvGrvVCbpCN7RwnPAi1s2vG9lWncebjPEkamtHNVROE8L8ehU182eSqwDud7aN1g3lRqQNPgkMdgTY9gHOHtDswvcJ+ncBeYUVSezUKsS6tvB/3HvXiU9w==; 5:pwvVlHNzYlvNR19J8f+O9ffV0lR3NRAx7nq2zzLytfhX/BhUN+WZGn9OjKR9gHK+BhtefpYO4FHZ+KbiW5BFZQ/czykjJ2MiC0JsJp/cWb2eNQco9a1L71KEy7HCat+SSOmwmYp84bP1cix3tMdIBCcfLE1ANtj+bN3HhJGeObc=; 7:VjtZfs8JtmYd4EVPldYdfJ8UFeh3RUl2NFtBcALYCU40iPgp5RNK4CRutcyPQ0PhDqIJ+PAdYdxhAwLUzWXY2rXWpaWUuoHd5AIIulnIpLO/++vVC3poesHCc6iRU9G9vDX5KTXcHjWSUic6jN13rImYwF3wGNYxSEyxcrR0M/Kg0Lv0oFpWTjM8nwKk/2rwhg4A4PESFsQ2JqdNyN2G1GufEUyaI2/bR8ZnThKmzKNMOb01T3RX2Rqqd2PC9OQG SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2018 10:57:28.3220 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8b1bda42-67a7-4fa3-14f7-08d625fa5aa2 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR07MB5001 Subject: Re: [dpdk-dev] [PATCH v3 2/3] ring: synchronize the load and store of the tail X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2018 10:57:34 -0000 -----Original Message----- > Date: Mon, 17 Sep 2018 16:17:23 +0800 > From: Gavin Hu > To: dev@dpdk.org > CC: gavin.hu@arm.com, Honnappa.Nagarahalli@arm.com, steve.capper@arm.com, > Ola.Liljedahl@arm.com, jerin.jacob@caviumnetworks.com, nd@arm.com, > stable@dpdk.org > Subject: [PATCH v3 2/3] ring: synchronize the load and store of the tail > X-Mailer: git-send-email 2.7.4 > > > Synchronize the load-acquire of the tail and the store-release > within update_tail, the store release ensures all the ring operations, > enqueue or dequeue, are seen by the observers on the other side as soon > as they see the updated tail. The load-acquire is needed here as the > data dependency is not a reliable way for ordering as the compiler might > break it by saving to temporary values to boost performance. > When computing the free_entries and avail_entries, use atomic semantics > to load the heads and tails instead. > > The patch was benchmarked with test/ring_perf_autotest and it decreases > the enqueue/dequeue latency by 5% ~ 27.6% with two lcores, the real gains > are dependent on the number of lcores, depth of the ring, SPSC or MPMC. > For 1 lcore, it also improves a little, about 3 ~ 4%. > It is a big improvement, in case of MPMC, with two lcores and ring size > of 32, it saves latency up to (3.26-2.36)/3.26 = 27.6%. > > This patch is a bug fix, while the improvement is a bonus. In our analysis > the improvement comes from the cacheline pre-filling after hoisting load- > acquire from _atomic_compare_exchange_n up above. > > The test command: > $sudo ./test/test/test -l 16-19,44-47,72-75,100-103 -n 4 --socket-mem=\ > 1024 -- -i > > Test result with this patch(two cores): > SP/SC bulk enq/dequeue (size: 8): 5.86 > MP/MC bulk enq/dequeue (size: 8): 10.15 > SP/SC bulk enq/dequeue (size: 32): 1.94 > MP/MC bulk enq/dequeue (size: 32): 2.36 > > In comparison of the test result without this patch: > SP/SC bulk enq/dequeue (size: 8): 6.67 > MP/MC bulk enq/dequeue (size: 8): 13.12 > SP/SC bulk enq/dequeue (size: 32): 2.04 > MP/MC bulk enq/dequeue (size: 32): 3.26 > > Fixes: 39368ebfc6 ("ring: introduce C11 memory model barrier option") > Cc: stable@dpdk.org > > Signed-off-by: Gavin Hu > Reviewed-by: Honnappa Nagarahalli > Reviewed-by: Steve Capper > Reviewed-by: Ola Liljedahl Tested with ThunderX2 server platform. Though it has minor performance impact on non burst variants. For Burst variant, I could see similar performance improvement and it is C11 semantically correct too. Acked-by: Jerin Jacob Tested-by: Jerin Jacob