From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <Jerin.JacobKollanukkaran@cavium.com>
Received: from NAM03-CO1-obe.outbound.protection.outlook.com
 (mail-co1nam03on0067.outbound.protection.outlook.com [104.47.40.67])
 by dpdk.org (Postfix) with ESMTP id 638A91B600
 for <dev@dpdk.org>; Fri, 13 Oct 2017 03:49:43 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=TRx9B+GtyqG3Ixh7GphT2Qm+sDkA+v4LDRjXvc6ziNU=;
 b=EN2jZ9nYLok65YiuZz7SKYWzcOTCXAID6yy08HVisLETiA0dQXdFzQWlyvq/dQkUB5yasMm60dJwLx2EYTPR4w2jPzbfASrfzKr6PU+m6ErJWAcwiImmYftNjO5GZWH9n+6YR4xT65I7nwWmQhQNhUuC66k4DjJUgDIAWUwiy/g=
Received: from jerin (106.201.58.193) by
 SN2PR07MB2525.namprd07.prod.outlook.com (10.167.14.153) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id
 15.20.77.7; Fri, 13 Oct 2017 01:49:37 +0000
Date: Fri, 13 Oct 2017 07:19:16 +0530
From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
To: Jia He <hejianet@gmail.com>
Cc: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
 Olivier MATZ <olivier.matz@6wind.com>, "dev@dpdk.org" <dev@dpdk.org>,
 "jia.he@hxt-semitech.com" <jia.he@hxt-semitech.com>,
 "jie2.liu@hxt-semitech.com" <jie2.liu@hxt-semitech.com>,
 "bing.zhao@hxt-semitech.com" <bing.zhao@hxt-semitech.com>
Message-ID: <20171013014914.GA2067@jerin>
References: <20171010095636.4507-1-hejianet@gmail.com>
 <20171012155350.j34ddtivxzd27pag@platinum>
 <2601191342CEEE43887BDE71AB9772585FAA859F@IRSMSX103.ger.corp.intel.com>
 <20171012172311.GA8524@jerin>
 <c3517bf8-95f1-0aa4-fc64-47922c35ce1f@gmail.com>
 <d48351a5-2c43-4fff-0a66-fd06707a530d@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <d48351a5-2c43-4fff-0a66-fd06707a530d@gmail.com>
User-Agent: Mutt/1.9.1 (2017-09-22)
X-Originating-IP: [106.201.58.193]
X-ClientProxiedBy: BMXPR01CA0048.INDPRD01.PROD.OUTLOOK.COM (10.174.214.34) To
 SN2PR07MB2525.namprd07.prod.outlook.com (10.167.14.153)
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: eee36abf-1342-4d0a-fb9b-08d511dcab4d
X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0;
 RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075);
 SRVR:SN2PR07MB2525; 
X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2525;
 3:u8MTHgJA8hIZri6po4NIniwVwrTkTfF/0tBoiyOKm8kyDTpgg1/A3loDzdi/vQgTNf0RX45ZIhOjHTKnKB1+oirkFKymZszqza/3drLNj1frTr+bFZTRMrclqfvLNi4kwJLPGFYCqQW12PTVkhIzPxQDRxkY9LH9Ju83TmwQm6kvYSntqgroXLRXp2hJj543aEo9MNABjdl2KKn5Q09vtPXH0zrRDyXt2hiyj0WN2brIagx5ifb55F4Qq+Wq5+Ta;
 25:u5FQO7aTYm7Fb3bXYA7iq5XovpQ1o7918VQFGr0oxkoqR+Mvyx6KPrYZSzPhd5/eVJfWKPzgRq65LhmpxqllkGTqiUW/uiWjSIE3yqI2UKN9EZMnmm8iYEoEEb3zLEHpm5Lg5RzdPoHn8YAYvMRQMWRejv6JHt78fOErdvMg1iBPDzWCNxY1xNgqIS9cMYV7/Gi+m2GN13/oi0DBQDixQrGDbPqPVesaxSC8XUX8W3z0Xtv7nj1O2dW4bEn+1Q/aDNQoiDGBCV8RCghVarAlpYL/HhOHFh0DXz1blpicymERbFMHM8cRSkYkBjkyE9W+5zG85LpWwhDYrpwRzPNqTQ==;
 31:bJ+M8mlfo4Oke+enpqCcPTYY3c2eUiwwxQ53k0alDWsN51kTAYwRs2HRHiZeXxdm/jh75btpOa3qaedMfpoyMk1GYCgQrP/KaU9tbb7acTya1CYfgzIzEBw6tqZerYkEkLaoeNuYBFhY7HSvjsHaakQxoM/GioKHmWPfJcbOqyQ5FCIxVtFPo8cBeF532Zc3tJyILIGX9sRjee6oMekGVEEDHl63oXTrUVw6MGH6iNU=
X-MS-TrafficTypeDiagnostic: SN2PR07MB2525:
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; 
X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2525;
 20:tZACWYdHsENBAwGc0LkwLebyfu4eiDCa8m/25I6EgUZDFHjZlg+Kev8XBvr46w4YFFAhXfN51yMgTDvFwFk02hlX287MHDjrPJEsgoupUz2dAvFMGgvNykPlpbyYZbWMD35lNF6/N16PjVRFIexLp2Jj/cLIADwyrq9H9ppISxwC+ovDJQF0ZBMbG1uFBv/qnO2qafmzeCZGkoBkO4DNTWlf88vOsHvRE7SBwt1mOiJDUkgOkNTFYBRioc1mhz550jwK29FPaHImhSCRMUXuJWRwwjootGYmVEsQcYMzjb3vABcto+TC1AWUG7rTVhxWXzHDo6VjUdZR0cU/xvJVUhskKCB7hNGZK7DujjdCmeXZSQMQHzpV6QhCDQOieL5+JJ+Lj9dGgDWMNenC764s/Ul3bZnImgJu5izDemS5QbRIaSTH3Y78YhhdMLY60lh4J+cm2W+vnoYZCoKBtq/VA4yKJp/O8yI4bDIGNEMm5/X8nLKhVMcQutn/vzGTjO1FIe4QT6JlGRkLeyJskqfR82VFqH3VOeBNBvcuZavI9OEqVb/Mur0CJiLSg6IyvDYBswlDUDubKLxKMVBNs3DxlREySkVrdRhoFvtD08+bhmM=
X-Exchange-Antispam-Report-Test: UriScan:(158342451672863)(22074186197030)(166708455590820)(183786458502308)(228905959029699);
X-Microsoft-Antispam-PRVS: <SN2PR07MB25254407181B58303D1307E5E3480@SN2PR07MB2525.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0;
 RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(3002001)(10201501046)(93006095)(6041248)(20161123564025)(20161123558100)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);
 SRVR:SN2PR07MB2525; BCL:0; PCL:0;
 RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);
 SRVR:SN2PR07MB2525; 
X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2525;
 4:WeFi2kxv1rUI3TQZvui7H/qg+BCPkglUxq/5WZ3WuPG7WZess3amSuNKagZYZJ01QiRoLXwFljWo7t+f80mhsfXxAm02zGSGH71iHG+VJsrq+O0pEiNMlcBUVob77FYinflZGbhf1XXT79BrA/ajGMDXdcA6Fq+pfd76q3UFHhxhjD13ZDCFmbrBYwpYlV7rKzJ46hLSirOC3zNCM/HkiWg5WuXM+nXJLPiwvb0N1jdXXw8zkp1oz94TghfrC6b3cWb5OsKDaPqC3stwhKTP3l+xPE9ToqK4CPs8yBtOyHT1GGSyXsViJu4KdunWp4QklvkA2Rlt/q62CR7kI+PrZbW3991ppIEKbqZCfkFMJ9dJNOSdZvEUbdrVaOZ18w+jKyzvc4vqfQyjKlA/J3Gi7pzcDmUdL/fW8oPDl5VbbUbQbDKrie5+8DTCGqNMRG65
X-Forefront-PRVS: 04599F3534
X-Forefront-Antispam-Report: SFV:NSPM;
 SFS:(10009020)(6009001)(376002)(346002)(24454002)(199003)(13464003)(377454003)(189002)(47776003)(58126008)(25786009)(4326008)(76176999)(1411001)(68736007)(316002)(478600001)(16526018)(66066001)(97736004)(966005)(6666003)(72206003)(83506001)(93886005)(5009440100003)(42882006)(2950100002)(53936002)(229853002)(50466002)(5660300001)(2870700001)(55016002)(6916009)(23756003)(6306002)(9686003)(3846002)(2906002)(33656002)(50986999)(305945005)(7736002)(39060400002)(53376002)(6246003)(101416001)(106356001)(54356999)(33716001)(6496005)(189998001)(8936002)(105586002)(1076002)(54906003)(81156014)(8676002)(6116002)(81166006)(18370500001);
 DIR:OUT; SFP:1101; SCL:1; SRVR:SN2PR07MB2525; H:jerin; FPR:; SPF:None;
 PTR:InfoNoRecords; A:1; MX:1; LANG:en; 
Received-SPF: None (protection.outlook.com: cavium.com does not designate
 permitted sender hosts)
X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1; SN2PR07MB2525;
 23:Qcil9gGZ2nlTwK7AvMBtlxesBg6hMIHX5ou+5+S?=
 =?iso-8859-1?Q?O8CctTBzJNDiy7h/EjRuAKm9IPy4gJHEywTys/XDGatf0ODgiwiL8/lxsu?=
 =?iso-8859-1?Q?jAe85pVanYbsbonrNINWP+daYGxfbqebc2QkswFGbM1KAgeCzcElvd+VG0?=
 =?iso-8859-1?Q?kO3prGtdEK7awPV+3mC3IAuUGTzswEOOcidHi7Xq1PzA5zIGKyf5XWCs9x?=
 =?iso-8859-1?Q?uUE0Kc6EgOUWFgzbO46n20pDIMioxTZ2Yb5wlQ9SZnC3+o4kT5TXj2ZX2K?=
 =?iso-8859-1?Q?fTcO6W9IsTAVacWFbbOPdrkYKltz1rA18KWMtKYBtC4FyxPNGLZ7jiipSD?=
 =?iso-8859-1?Q?Cfb5FxwBVVFqPmG39BN6GYjFN59FzXtoQv3/sE8zCt4RCQH6qvSFvIZq+i?=
 =?iso-8859-1?Q?N9Za5fHpOtPaIdjG1/x6iUotjyk5nLYvk8yF91/ki6d41tawo09ptvjeG3?=
 =?iso-8859-1?Q?QOje1gVFXmaLir8eukYTstTG4M/nnIab/wP4lqUjBA77PFEc6onwkPjzEj?=
 =?iso-8859-1?Q?680e5mgZ6L0adPQGAXCFhxhtj5fFvNNqXFa9ea8+FDQCOYqn8r406QOkNf?=
 =?iso-8859-1?Q?sQ/TrOaLHM6qZmIialc910wX9VWqmHwn+o2ZzD9rkpv3IaPeeDI7Fra1Li?=
 =?iso-8859-1?Q?gwnkAmaybWSklv2OR1NKkuIlU5h/bGGz2Z9p2shMgTr65QhicbDPHT4D7f?=
 =?iso-8859-1?Q?1uq0QGWpAU0fy5mddUosSFC9usoSUedjZYbGFwv68APsNAGxjdAjMzp6zN?=
 =?iso-8859-1?Q?/t5TMbSvqzy3vIbRlTjnPHWJySpWvRt+h403sjyUszAzsWJ9BuSlRqK+dn?=
 =?iso-8859-1?Q?XgqHWHngQHMDxjnF9WYuFU20R2pWDM5xI6DiUglZ6M3zzsSnInd7ECKdn3?=
 =?iso-8859-1?Q?e0WhZXsyr80f7Ar7TDdzA8WEPwo1hbGaubEyI3OuPmHJ04ydQIijxb4wIf?=
 =?iso-8859-1?Q?fcHOwvObak2tItiJmIAEIo9Jrsu5ZjVPBJuISJ2KT7Fflf3LqcnVZTKTO8?=
 =?iso-8859-1?Q?EY8U0YGufWhVApEmVDJkstc/loLyvxXbVfULKB6Dgw5wDcj/kXt8HW2+wh?=
 =?iso-8859-1?Q?KCEs8+vqh2aPBvr4Sn4xioktXbTCgj6bZxlSifJFeEd/bcwg48VqtVZgtg?=
 =?iso-8859-1?Q?OP0OoXMCWmQm/J2fdIRUzFcQXX4DZ/Gg7Ji2TPxQYfqkKQ80xp5THBqXPk?=
 =?iso-8859-1?Q?IfdefduCwL5UoOn3obdcq5jviErXv71ukTT5QlbaLAB9LcQ6FIAfDMxyRj?=
 =?iso-8859-1?Q?SEnKEOtcp0Q+1/McXQCLaw42zwT6UnFFB8hgOJeQEvYU3J9n22QrtIm9hE?=
 =?iso-8859-1?Q?r6uNwYGokDEcQMwFND2bqePrEds8rKfp+w5j1yis8HosQTfekWDO/SdBeY?=
 =?iso-8859-1?Q?lCDJhzRAJTzV4rs9HTZbQJ8VMdpVfzgqkcysQAmvbqQekW6AnI5b+7y6by?=
 =?iso-8859-1?Q?vvj85Bgc1xPY6SX875+k2cPta9cLq5MTNIP3mkO90cwNMFejUPrMWl0KA?=
 =?iso-8859-1?Q?=3D=3D?=
X-Microsoft-Exchange-Diagnostics: 1; SN2PR07MB2525;
 6:xK3ynexLUvEk3SJYSAVOK0Y4fXJwJpe1dtYnypcwv4vkoJwQ5OKblHVdE9SeIvaY9SaGYI4HGUsV1ZT+PEjDdk2F0UqQmKQV/d8YVt/PDcNRd6rvFUsDdkJ5qFkeuzxB9fHlPAFhXFvI7EETpBgbrfUc43L5El8oiclXZ9egdrPFqC+wYyi9q5dX8/ZUK/k1Nh33eW7wpnd3sc0WD9T/nlYmyrnvUhVIzJ6hUS432QdziT7ilC2/IN8cBvSQPXv3GOn3QhP1NpxObZHecVbxm9ufZQtWxK/FOnh4KZvJBW+m8gBgN/YLU/VWFvCquMriDVjkR7Z6Hyz+HeOfFp60RA==;
 5:bMt3ASDZHjwvScer8dY0BHB6tF3RXeH7rTHHGwrTYpg2J4Ze+ns+Vn5tgOh4TXcBhEFf/NRPWrv5laR5VbigwjwIFP+gwlZ+OJhD/3mb/sFwRZMgeaOm+gZCZ3bCveJ/a3Q4eYoFZ7ns2UQHFLEVNQ==;
 24:OmLOxW+RFQgEroR9SQpB78pTlRu6rPRnyUNuWKPKQtWI2HSFIVC8jS9CDvicIMhQna7uSCoAzXDs8IK5PKkYi8nQhIbt2HCEXNI0245DZv4=;
 7:unOJFUkKTKVEl8r0c4NFdkH6nmoEtc8+7tsMpoYM2+gLKt+WqKGLGTH9ruAysKwRG5sC8+5h0V8ivHwbjRjRs/kwq4AUudurUl74v3aTwW0Md/vrXrSGmmvaCXtORyZJo0M5g+Ernx2nYfsj4NcHlj17z6iajq53LXOEKAomkg16F1F8bztdAVrAfzXEDtXbbH1WLQClYA26N2nGXDOKnc38hnClydPS7yhpr8HRcYg=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: caviumnetworks.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Oct 2017 01:49:37.9521 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR07MB2525
Subject: Re: [dpdk-dev] [PATCH] ring: guarantee ordering of cons/prod
 loading when doing enqueue/dequeue
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Oct 2017 01:49:44 -0000

-----Original Message-----
> Date: Fri, 13 Oct 2017 09:16:31 +0800
> From: Jia He <hejianet@gmail.com>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>, "Ananyev, Konstantin"
>  <konstantin.ananyev@intel.com>
> Cc: Olivier MATZ <olivier.matz@6wind.com>, "dev@dpdk.org" <dev@dpdk.org>,
>  "jia.he@hxt-semitech.com" <jia.he@hxt-semitech.com>,
>  "jie2.liu@hxt-semitech.com" <jie2.liu@hxt-semitech.com>,
>  "bing.zhao@hxt-semitech.com" <bing.zhao@hxt-semitech.com>
> Subject: Re: [PATCH] ring: guarantee ordering of cons/prod loading when
>  doing enqueue/dequeue
> User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101
>  Thunderbird/52.3.0
> 
> Hi
> 
> 
> On 10/13/2017 9:02 AM, Jia He Wrote:
> > Hi Jerin
> > 
> > 
> > On 10/13/2017 1:23 AM, Jerin Jacob Wrote:
> > > -----Original Message-----
> > > > Date: Thu, 12 Oct 2017 17:05:50 +0000
> > > > 
> [...]
> > > On the same lines,
> > > 
> > > Jia He, jie2.liu, bing.zhao,
> > > 
> > > Is this patch based on code review or do you saw this issue on any
> > > of the
> > > arm/ppc target? arm64 will have performance impact with this change.
> sorry, miss one important information
> Our platform is an aarch64 server with 46 cpus.

Is this an OOO(Out of order execution) aarch64 CPU implementation?

> If we reduced the involved cpu numbers, the bug occurred less frequently.
> 
> Yes, mb barrier impact the performance, but correctness is more important,
> isn't it ;-)

Yes.

> Maybe we can  find any other lightweight barrier here?

Yes, Regarding the lightweight barrier, arm64 has native support for acquire and release
semantics, which is exposed through gcc as architecture agnostic
functions.
https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html
http://preshing.com/20130922/acquire-and-release-fences/

Good to know,
1) How much overhead this patch in your platform? Just relative
numbers are enough
2) As a prototype, Is Changing to acquire and release schematics
reduces the overhead in your platform?

Reference FreeBSD ring/DPDK style ring implementation through acquire
and release schematics
https://github.com/Linaro/odp/blob/master/platform/linux-generic/pktio/ring.c

I will also spend on cycles on this.


> 
> Cheers,
> Jia
> > Based on mbuf_autotest, the rte_panic will be invoked in seconds.
> > 
> > PANIC in test_refcnt_iter():
> > (lcore=0, iter=0): after 10s only 61 of 64 mbufs left free
> > 1: [./test(rte_dump_stack+0x38) [0x58d868]]
> > Aborted (core dumped)
> > 
> > Cheers,
> > Jia
> > > 
> > > 
> > > > Konstantin
> > 
>