From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <Jerin.Jacob@cavium.com>
Received: from na01-by2-obe.outbound.protection.outlook.com
 (mail-by2on0071.outbound.protection.outlook.com [207.46.100.71])
 by dpdk.org (Postfix) with ESMTP id 77C162BD2
 for <dev@dpdk.org>; Thu,  2 Jun 2016 11:40:00 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;
 h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=a/vmKjkT/lHVwip2cmeLJgQg2DMTmodX9oehNo8L5rY=;
 b=Ay0cp2WC3hZQ5Wf23q+U8dKdbJqSyJtR1c4ZhJHwsdnIDA/M9u1XadOxeXiCrpTdtGgO0LSw1ydvcD5TXLvZGrv3aBcNRl8h043BrdHjgGTBso/VJCgAZpdeT8z/SDmJkH1C8vPVgb2TA/fGZSqqLWNI+rb3I8OHAFAuA5O7+j0=
Authentication-Results: 6wind.com; dkim=none (message not signed)
 header.d=none;6wind.com; dmarc=none action=none
 header.from=caviumnetworks.com;
Received: from localhost.localdomain (111.93.218.67) by
 BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18) with Microsoft SMTP
 Server (TLS) id 15.1.506.9; Thu, 2 Jun 2016 09:39:56 +0000
Date: Thu, 2 Jun 2016 15:09:37 +0530
From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
To: Olivier MATZ <olivier.matz@6wind.com>
CC: <dev@dpdk.org>, <thomas.monjalon@6wind.com>, <bruce.richardson@intel.com>, 
 <konstantin.ananyev@intel.com>
Message-ID: <20160602093936.GB6794@localhost.localdomain>
References: <1464101442-10501-1-git-send-email-jerin.jacob@caviumnetworks.com>
 <1464250025-9191-1-git-send-email-jerin.jacob@caviumnetworks.com>
 <574BFD97.2010505@6wind.com>
 <20160531125822.GA10995@localhost.localdomain>
 <574DFC9A.2050304@6wind.com>
 <20160601070018.GA26922@localhost.localdomain>
 <574FE202.2060306@6wind.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <574FE202.2060306@6wind.com>
User-Agent: Mutt/1.6.1 (2016-04-27)
X-Originating-IP: [111.93.218.67]
X-ClientProxiedBy: MAXPR01CA0053.INDPRD01.PROD.OUTLOOK.COM (10.164.146.153) To
 BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18)
X-MS-Office365-Filtering-Correlation-Id: 78e0d4cd-5ac4-4118-045c-08d38ac9dc6a
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719;
 2:rulGZCqN3vux8CbLxHRaRQ9IcQ94gS7TUseIsfA4kg4cwX1oPLN95UOBQI8Wv3Mdjj30f1Gph9sAkaOf0/Fy+tEzRHpeE/8k2JLW8036nWs02sKD5XMFP0Arg/y/QnjzaAQzNUso4TmEOgqeJDyz/o/zSzOOYFOwLuY2Qn7q/m93Zz9yo7tzUhIvEItOGbLa;
 3:tD0duHyGSQFDzvRDw31YSVm2VbJcnIrHwASANjNeMBq4D3vVtd7Ru3azjwiYnf35ba0QfeEo/p+B4mup9xM06c3XrOBaJAuu9nuVEpgrfwYjuklcaugJEiIRoFST8Hy7
X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1719;
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719;
 25:evsUa9YPBUwx4O/HneZNC5seNsjzTuhaDoi6mrX2s+NcNY8bmyz8hK0IXi4RTQMCaz4Qqvq0hdmN26igXG+vzkILYkNclEACDSm8JRKAOwh7OJ4VFEd/yohomFBCMQawILKO5PyIj6fEZx+kH0mmsZ5/0mGwVJ/BAgc14tKcy6GL0PCMRaVl2e7EWKsZRVH0Cfjm+NqTO5ZroammxFfStxhqtd8u9fpFPV51ekqKK1Y+khSgMqg8vh1UqyI0BDCscazFbs4G7rsUeYuaHYjZCWWQotzZx8XJQfjPYJk0CV4T0u3GS3gBdsxh1YMnXVQ/+4CuG8J7piafnxgtoiaIdb8u6QLN5Xb2LTScVPk2Q9wHjgbsQIvJEjN+F/Es4uSOfPluX/QCcaKbczOui2MzqMicKdYZvoUMsIq66S4q2+FiEXvNVUeNdI6qgjj3cO8+aAV4kZ6D0mPXN2ksv8R05RZKp4ERsDfKJ5+FivSsZc5tHE7Sc0vL+KPVpO16NUmHWYL+wtftFmv3K/0gl4N8/sWDVYfF3aMxMSEXclFbAx2X7ju0+x3eGomQIxCRBIoNPWKnAGUsNzsWLkQB6ZUGhIBScRUBY4BncgZES7ZP+eOM9yzJGuifHqFQbJx3wwROxr9P5bwoHBp7aECbnEYp/MUgZUZK1V/mJ0vnAiMXS79D83/OID9rjvsR30ThaDLq8odCDS4bcvTvSovLJ4IBeCN6gQPmZJkUSLig/DEHW2E=
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719;
 20:AOWJKVW7iBOhFkqg52hIKTI3gtJp+doiI5DazQ99u9VzqyBhcpP6LdZuFWOTR2LUQd/SoX95wuBaAXpK3pUyXuddz+Hcxjb3TCaRd8MTdhY7RpUfaHW66D7lW9pwaWFo5ag4FohBHYpT7FSUKnfe2+Xb4Ah2UaPnl424fSPSV0oG877qiA8Mye7588HpYFzP+ZjCZzYFyC2UpxpJ5S8+dZSAUSvuIY4RTs9goYHgeurX+7Ldk9qhDy6tOeTfrlU0B2reSJuVt2ouA7/aqwma2qSxx21H3fCdErs4rg29CdzOxTnek5CCCQ/qNFHnaal/3+n/BtP5cXeClonX8as4mJRpAgsganFzSM30gjgJ8YLeKkRS+/HWRXu/xJEPzcKKmP+pDO0tyK1FiWlrg+U3VJGsrrAgDgzyg1QcVE59QDsdjupt+Tg0OlqfEOIGf4SHG0juvDsjqULdZN6Mx1YeocLsOPi27GB5L+aMnx3mIb3N4qR13UVYAsw4053RtLsczLSTCs+d06+/1ltByGBig9gn6E5UUuB48XsymYGDpACUoCL8gCEGg3RF0cCcL5wmrBW2TqLWny6uItj6T30M7klQDWA6UHDaI+cz1T6+WsU=
X-Microsoft-Antispam-PRVS: <BN3PR0701MB1719423D5E1130B45799CCFB81580@BN3PR0701MB1719.namprd07.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(788757137089);
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0;
 RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);
 SRVR:BN3PR0701MB1719; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1719; 
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719;
 4:U8tasrF0ywDcUrfYkL1NYkZhPKD5pslzzlt3rLOs3cL69Zb7p7l74ztRdWFgmYJ/DNCX9DVGD58XhU/OdQNiaKyLYjlKIB+iWZlIK9mu26nRdEZfD5pYexGONuWrxY+GGyEgTmzR/Kj217N7gzpRnDqTUfDxk6Z717ILUdPGzxsB6pu2aUqZoBo++V34EGln4M5qp0BI0hAlZ0nrBMQU/JHUdbxjrFeVo0Jz2gZORqnHR7lSqObd0iDkcNTrC5FcptmlmvMBBIH9SD8FSoQimyKBAY+Vk/SALT11jAVwoYspfuhVzBFX/fAK1+9O15PLMO9bM9MULQagHYXUjPilNBe45V6IFp0k/uZrVR5QcT4/A7/Gl71jZeC4YEC0lqKMLK+UY2/uVy/TWEqGgSuHJQ==
X-Forefront-PRVS: 0961DF5286
X-Forefront-Antispam-Report: SFV:NSPM;
 SFS:(10009020)(4630300001)(6069001)(6009001)(24454002)(377454003)(2950100001)(77096005)(54356999)(8676002)(81166006)(76176999)(4326007)(9686002)(33656002)(5008740100001)(46406003)(50986999)(47776003)(6116002)(42186005)(3846002)(586003)(23726003)(2906002)(1076002)(66066001)(50466002)(5004730100002)(4001350100001)(92566002)(189998001)(93886004)(110136002)(61506002)(83506001)(5009440100003)(97756001)(7099028);
 DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1719; H:localhost.localdomain; FPR:;
 SPF:None; MLV:sfv; LANG:en; 
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1719;
 23:kHdSodKGyNWdLKEOU31DtMZUxFZcABJp0idTSOI?=
 =?us-ascii?Q?JwvZmvH4/kWuEWU42e8YJa8/lHYIxFbB1n5z58M2lFwsL2CCRVXgh1+guK3D?=
 =?us-ascii?Q?Z+QZrgcTICmt3W928evSvslzbfXcKcDRGVIhRgQELg4szeUVdUyAv+IFieko?=
 =?us-ascii?Q?cCluudFSOVTbAvouBFUhf6nQSl4Y+mwR+yXN8iApXF6zEbn2aDYY1aF/Apht?=
 =?us-ascii?Q?ucDx0rxQ+6vTJ2NawhrAvxhE4fWDdlOHbYo9GfFuBoY0BiTrfLUDMr1AfZ3l?=
 =?us-ascii?Q?ullrQSIIgQcJLjlL5ttvXoIgoaX5RurLSjBFZxAtr198KRWc5d3VsDyoOo4A?=
 =?us-ascii?Q?uXC/5tFRmrhuxouu9KUyk+sCjenlZZS16sBc43iNONj+QJ2H6+Ux6F7HYy6+?=
 =?us-ascii?Q?61aKERQcZpGn8bte6T2Faisv+OLqUtPojjIgYEQlm2yBAHc94hxl285D8M5z?=
 =?us-ascii?Q?Ua2me0Zr72bSL4DEK1tfv448eYaOtyhLSZP0o4+uRM6XlGnmGuieTlNuukZc?=
 =?us-ascii?Q?QyWh7eN+Az2xBGQOE61ScywZ3cmifiuj8BADBtzVoVg9/Tmo2YRugSvXgdvt?=
 =?us-ascii?Q?O/58esUiXsRjUQYaFaGIsZ5iNaF1CcgKe0JJa7NxlkhW6tnzDPD0CXeSH0lH?=
 =?us-ascii?Q?m3aHGd0KFfmgp89gVkwqZ1frM+xdYWgLW0uReUcbC5TWhRekTJoF/0fdXDMV?=
 =?us-ascii?Q?wN7qodCyakv2wdQ2IncX4Rm+8kDwhkTsmhJwKA6uEiL0e7jwEoQe2BLUyVVk?=
 =?us-ascii?Q?naloiac05yCY6QGsRDXOz7Egprs36/1FJUnDrJY1AAo9pQBLpXHGVDxO1h61?=
 =?us-ascii?Q?xX8X16NZPFGc6/bUvSPBjpfwczj+bRhTEC8sYfgF3flBsq9Cns3qBC+kca80?=
 =?us-ascii?Q?wVVT4TlwbQBWHlXqu3Dm0I2PiFm5MkgxVPT11oKYR9JiZZeds49wo42ic+Bl?=
 =?us-ascii?Q?W9rzIm1ZKnwTiz8V+Po4sTOGVYbDq4tmEj2XmRn08nN8NHBHnHmIdNXRTxAM?=
 =?us-ascii?Q?g3XCpP5RLFJoeDX2VxwzurdNX?=
X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719;
 5:7gSmAcVMbaX1DlaUHIo6sI5GdEydvpCzKe9a8eSTsne3JbZU4+/mION00+yLaOrFLHi0XWCKsMW2hxNn+nTcA659a1CrCm5MYFo0mkA6I5VFX7FDBXiY8hf3EO73s6zXxOtzQO6jA8WMawIDqSD37g==;
 24:QFiHKf4SarkLHrHktGCAcxXRrG/RCTGN7EcYyzCVeUjY5Yg7LGd7TvH7a+iTkMmgFWZYzUtrMn8aw3r/myb6BHGp2q+uWV+Rt+s3hVwMWjE=;
 7:DrasxV0zPGv7r77BHABs8I+cFfXc3r9mc9Iu97WfRhgomqeAt+6bySJ94D6kEcdpDq5KB1N3vGNCvJy/voJ0M6ZN1EWYgztaSPCdKf8HzSUHLQkUBtomiHKChNuWRJOmd2hTU5l0bT4g+SSyT2scd9lgKCeF80EZOGM96bDCu+3PV0JVGhA+GLEIPj1r8e2g
SpamDiagnosticOutput: 1:23
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: caviumnetworks.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2016 09:39:56.1460 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1719
Subject: Re: [dpdk-dev] [PATCH v2] mempool: replace c memcpy code semantics
 with optimized rte_memcpy
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 02 Jun 2016 09:40:01 -0000

On Thu, Jun 02, 2016 at 09:36:34AM +0200, Olivier MATZ wrote:
> Hi Jerin,
> 
> On 06/01/2016 09:00 AM, Jerin Jacob wrote:
> > On Tue, May 31, 2016 at 11:05:30PM +0200, Olivier MATZ wrote:
> >> Today, the objects pointers are reversed only in the get(). It means
> >> that this code:
> >>
> >> 	rte_mempool_get_bulk(mp, table, 4);
> >> 	for (i = 0; i < 4; i++)
> >> 		printf("obj = %p\n", t[i]);
> >> 	rte_mempool_put_bulk(mp, table, 4);
> >>
> >>
> >> 	printf("-----\n");
> >> 	rte_mempool_get_bulk(mp, table, 4);
> >> 	for (i = 0; i < 4; i++)
> >> 		printf("obj = %p\n", t[i]);
> >> 	rte_mempool_put_bulk(mp, table, 4);
> >>
> >> prints:
> >>
> >> 	addr1
> >> 	addr2
> >> 	addr3
> >> 	addr4
> >> 	-----
> >> 	addr4
> >> 	addr3
> >> 	addr2
> >> 	addr1
> >>
> >> Which is quite strange.
> > 
> > IMO, It is the expected LIFO behavior. Right ?
> > 
> > What is not expected is the following, which is the case after change. Or Am I
> > missing something here?
> > 
> > addr1
> > addr2
> > addr3
> > addr4
> > -----
> > addr1
> > addr2
> > addr3
> > addr4
> > 
> >>
> >> I don't think it would be an issue to replace the loop by a
> >> rte_memcpy(), it may increase the copy speed and it will be
> >> more coherent with the put().
> >>
> 
> I think the LIFO behavior should occur on a per-bulk basis. I mean,
> it should behave like in the exemplaes below:
> 
>   // pool cache is in state X
>   obj1 = mempool_get(mp)
>   obj2 = mempool_get(mp)
>   mempool_put(mp, obj2)
>   mempool_put(mp, obj1)
>   // pool cache is back in state X
> 
>   // pool cache is in state X
>   bulk1 = mempool_get_bulk(mp, 16)
>   bulk2 = mempool_get_bulk(mp, 16)
>   mempool_put_bulk(mp, bulk2, 16)
>   mempool_put_bulk(mp, bulk1, 16)
>   // pool cache is back in state X
> 

Per entry LIFO behavior make more sense in _bulk_ case as recently en-queued buffer
comes out for next "get" makes more chance that buffer in Last level cache.

> Note that today it's not the case for bulks, since object addresses
> are reversed only in get(), we are not back in the original state.
> I don't really see the advantage of this.
> 
> Removing the reversing may accelerate the cache in case of bulk get,
> I think.

I tried in my setup, it was dropping the performance. Have you got
improvement in any setup?

Jerin

> 
> Regards,
> Olivier