From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0082.outbound.protection.outlook.com [157.56.110.82]) by dpdk.org (Postfix) with ESMTP id 0FC285683 for ; Wed, 4 May 2016 17:20:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-caviumnetworks-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=JTL0gy2p0++jA1dh0apRlmfmsW2kNe4NGKzuA79MSUs=; b=M5G6r5iWLq3Ap+PReen73a1iNdtG3XDduecZOA2nWAAD9Nwu+pd5TbyvI0mKRNmzHMnL+UTyLh9ND4GrqQDT/RjKxrgJ2DIgCclTKe02QKmxoJkiK7yv7hHU/+yp+qd4ad1QGodBPtFv7jR+XX9Ef96iqsELS+jyP6DsI3t19ts= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from localhost.localdomain (122.171.53.0) by BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18) with Microsoft SMTP Server (TLS) id 15.1.485.9; Wed, 4 May 2016 15:20:01 +0000 Date: Wed, 4 May 2016 20:49:38 +0530 From: Jerin Jacob To: "Richardson, Bruce" CC: "dev@dpdk.org" , "thomas.monjalon@6wind.com" Message-ID: <20160504151937.GA14876@localhost.localdomain> References: <1462176465-21562-1-git-send-email-jerin.jacob@caviumnetworks.com> <1462279327-9876-1-git-send-email-jerin.jacob@caviumnetworks.com> <20160504110950.GB25492@bricha3-MOBL3> <20160504134231.GA13071@localhost.localdomain> <59AF69C657FD0841A61C55336867B5B035A4D9C7@IRSMSX103.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <59AF69C657FD0841A61C55336867B5B035A4D9C7@IRSMSX103.ger.corp.intel.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Originating-IP: [122.171.53.0] X-ClientProxiedBy: BM1PR01CA0057.INDPRD01.PROD.OUTLOOK.COM (10.163.199.29) To BN3PR0701MB1719.namprd07.prod.outlook.com (10.163.39.18) X-MS-Office365-Filtering-Correlation-Id: e3afe65f-cd43-4985-daa0-08d3742f90ac X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 2:q0Xk3BwK0Qv5pZ7cyxqAXZEXjlr6vgFdP+miIiMCA3vhyeBR5PRcvYUz67PwBULlNRlxf57azR14uVqHr0Y7z/M3HrsjfsNyLwqf4t7f5JRSLFH4U9Jbn/0nOllyDurjfjLZf4bysspW1p5TSccVslQAnr+9CJP6t29c2x+dyG4Fa7+loGnPIVV74bGAXW9E; 3:k+81IIs8Pq+pZ2pdoT71sSly6fALuzdnyghn8W7B1gCF0bsB/ItT+lmQ4ScSjI2VfpNIA2C0vIJQNxCIn6RND802dqMwHpuTMetRZJaDZOctQi9ncKN9noJz0JrDs5j7 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1719; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 25:lrqBN4Huk0TXozgosJDoI5+SWu2tZZYAQ0b0Fvp9+usk61s5xdxLNBXEP2d/X7EoMu3nytth9tUT7tU0s+SgllnEWjqd30Z9CHotCrXK5OF44baUbtw0a+lEdKM3bSura4F+S+qGAbNoP9N4oWnxDTUQD02oy47K5nPtCdaFPj9ZsojI9GMfYfiugiEfobVJ44/d3E9IzS2tvSaIQMfYhWXknGPKgM1ffALZX1gzx5KVoJEUliLPVK3f+ITgVcX0eSMm3isN4HUPGPg61Q3qL8XeZcof/0AZxf11ysX0wr5lqn3i17nCch1maAfChtTb42/IrZo2Q3EqsaV27AxOtXaFykgA5ETLK4lAyH9HvWjfmRvT3ib6n5cpIUyDwq55RFx1XV4EOUBcpzZCe637xjcNrelDsO29+xmqPjwhv1Qvq9dnRdZU/IZPfzvPFT+x0z1ahmaks2IgEFpRoH8ZAAWhoZSJJeRukikOWyxsxmV1ZIwBEFXz4SJjBTSTYkeyU80l10JwJnBvjrgbcedlw+PWM9ElE2jT05IalcechtKt0zCKrKKlvandj3yfgLv4CagF734Fp1OpDZ+UkflNbCM4S7wWUC/V7MPtYxDT2M7qqgHB3W1TyPVYLknDkJeDfdg39T6u4LoSIxtbTHVIguyveQCwLkPpGzpQii+nB2K19VIo2xt6wBw/yXvZyve0NEIoCVSnhYUCkgVmPvj1IbWogEk417xWmmoXy2zBGfq8dSIgzN881Zg64IC2h36s6fj68DQSntKVqbCbXKa8JAeFtXvsM0H2ghyzAe1AJc4= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 20:e4CHJyeVEc177pqeIagqPFs2MwYAHgO3UtAkAM41NyV54L6llt5x2a1CdSqDMk4VGgPj4PdOFCU37eHoa19mk5C2yH65D6+TARhCXonpvuy20RyBrWGwKclrE6nl6skFAZzl+fr8LN5et6YdrdmEVeQovQm7bY0sSnrcwCcdvbLaZ7hZVc0KbbcerLt+/VvvWVEAXnAqr5uhogI4fechJgD/P4ugOZTUDWRPQsVR8pKw/IwnDeh3N5Bfi3H5F17lMQA0wlF08wUWJCZnPcnV1AvMhyl/SJ6BBdZ7m3VSYsNAlH8VajLFnRpr0TN1e4dUEyp220pp8Dru2qmeiJJ/iceC+wMOZYuTbxVfZCj+PftcCNgbVyMsEW/a8aCTzMM8M3cQE7pj22zqivHzIbnf3xuIW6+mxeuhrve/I3S87UiiwPAdXYq23fO+Whr6jgAYqzkgui/AFIrMEdEWxIAeoE5tzfY4kom3uFXOEZ4HihBCNkzbxDbWedfr1LhjWEmdcPW9sEWAMr3TXDUVEZoAAxIIpJCXJW5UP6kBVhLSCZh+MAnIwLnu/UuqpIsKnmtM8DKk3VjwwuAQZUn5LUCIWrVgMSP1QKtmD1z7QmFmfeI= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(9101528026)(9101521098)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001); SRVR:BN3PR0701MB1719; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1719; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 4:7cFUBqJYYeCnYseFKqBB0ksLXgOanRk0do0N3Yowt2m0YihZCKWOm9PZPGWMj+2OPqRwlXZgSTGRSfbdbR4MaVTvfI9Lr6RdVjUTS5+6s3Vy40TbmTDjXEHe1WGwnYVXmdz7lopE1upEAZjRnjQsoMnFJMzBiuWv14H6q/ufYZeyNioMGPMnRlpaKllKau7/a84+LCmXP5FUVAxON7kBhBi2rYI/mwJamZvNQXeYFUXC9EEEHf/NT86wMHk7wk2gfLzb0DsqZBtF0wjqfkswYLY6nKggsxbsWoKgmo9baY6ZZIA9aUoosUTSp45ru2tl4s111WlsMTtNXAeeIarp2cH2eX4F2nABq4NwF4M4kW9ykIoRPe1B/r7AzWBzYSTAi5Bu+wQOomUvGuZvV/l9KBHZmrvO2/SEwh70usp9eqo= X-Forefront-PRVS: 093290AD39 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(13464003)(24454002)(377454003)(6116002)(23726003)(1076002)(2906002)(2950100001)(4326007)(3846002)(5004730100002)(66066001)(110136002)(47776003)(586003)(77096005)(50466002)(5008740100001)(92566002)(19580405001)(19580395003)(93886004)(46406003)(9686002)(83506001)(33656002)(4001350100001)(76176999)(54356999)(42186005)(50986999)(86362001)(189998001)(81166005)(97756001)(61506002); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1719; H:localhost.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1719; 23:YT8oLOCsk9YruLBP5vD1pSkJfJvrNfHMAo+eX1n?= =?us-ascii?Q?0PfkxzgDFg0dNotVNh/ctI9A0o6WBeqEyRRJNvDRoaDViMuScbDtYlf9HRPF?= =?us-ascii?Q?K7ta9HzxvNk/TKu5jckGfdfveuMLv9C//pzFNX/lwhrmrC3od30hWeIcHN9w?= =?us-ascii?Q?JslhhnnC/ZGN703O2gx54eMTaFtjd6c2OtRkjYS+E0uAjtfue9Td0aJfFofA?= =?us-ascii?Q?JtPIUlKn6QiRW8bcyblwj3IAE0Uz0YJQVF5+uJKi2i0kgR0+B+A5MwKalW2V?= =?us-ascii?Q?4pPZA8Te0bMJfvrm4a1Yx9Px5PwVAeTbfezqtNexMeRNAGXUXr8y0FsyNzrx?= =?us-ascii?Q?qE9q1D8L+rfUEXMQZkAbgwL9aavelGXXcNCrvQshOu+cgm8ak5bXpuzfSUBF?= =?us-ascii?Q?WjUhGtyunp0tN9bZqSvVuTkjsvyJQ233wMKoEqH2Zvs9rzDHXN1I2uc/Ljr3?= =?us-ascii?Q?K8o1z5r8q987kxm0tYmWoztTYYUbX7ETMPs3ebgvOE+n6ZB2cpd3jjVaX6MS?= =?us-ascii?Q?Xb2b59aRd+HxlPtncInLAr/TlghW9WCXgUIPhjzBryKjjNjSuLQhKB81BUQ/?= =?us-ascii?Q?qmFWm7BiuyOz/6DUyjbwr0KygVKr2XzwYNkYlnBv3GU9eri4+4cWYlTErtfi?= =?us-ascii?Q?+iKbH2XgGr2lPPDBEQxstkl2TgSFyLPimzlVgnbz1YhnjvEDlovWIGQHKWa5?= =?us-ascii?Q?BNAWtzfWOBcj8EkyzYV6VDf9ePc9Zb/vdvLHsiUyV4GbjwoO8OWhQ2KMFckH?= =?us-ascii?Q?FolmctX042hAPZvwPZMm1NvamroOdNDJtrwse0BN1RH5cYuwmjsckwdHQAmB?= =?us-ascii?Q?yMxpxw1prBBDTUy78PBPTSNVwK+Jhzmrh+9nrVh1GaEmNC/PKu8NyOXFvfuq?= =?us-ascii?Q?ydERlpkPP+EGvGW+cKLoT5Jb5WHliRHtEptL1hqZAK9cO7nG+YY4JrhByTk/?= =?us-ascii?Q?+N37zsccdMP4jv2hrwA6wdHBnWJO8xJLqVxgJZqzGPUoQ9alzPAUvw9uUQqT?= =?us-ascii?Q?bzNpZgMNzGJ3dUZkznADojxktiyXM8Abqvq5WglVf1xWmWA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1719; 5:qSQwR9BJ4NjAyJOQ9WJCQ3Tu2SnZRiYI7OAPOxZdR0QOHc3SSvMcvDCaLHDjO7WQWUqtf/qanvYwcgoh0w5k4R7jjd167qS5Z2hi9eYsm5BOE6oS0Pa1Ak/gMnpIhrPG9hfxmO1hFb7QLE7y6q4wPQ==; 24:wJI4em/LCnUcp9zH/5VR6SksFVG2Bf7pXAFGh6Z561rwPUd23DhTirpY7eOAWqIPV/XUbe/R/VOAGUyN8NiejFAyIy6U3HwFiTu2ZDCHf8Y=; 7:EpoGcn6n+dtKHbJS7VddMtbIzgZXFE+ddm9doLIN+9rpEqot5tgOetWBuYtolOvw+Ecd6tdJJyYN0Wea7gWkbUq5niqzsWBEu4/rDi5U+KuKAEJFJszB+y3luWTy6biZ5txR5bGqhUwy2qNgePz/0AmNoPpwnAH1nBlO6KJlctc4tRecKdCO6qlgLIk3jKu6 SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 May 2016 15:20:01.1716 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1719 Subject: Re: [dpdk-dev] [PATCH v2] ethdev: make struct rte_eth_dev cache aligned X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 May 2016 15:20:04 -0000 On Wed, May 04, 2016 at 01:53:39PM +0000, Richardson, Bruce wrote: > > > > -----Original Message----- > > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > > Sent: Wednesday, May 4, 2016 2:43 PM > > To: Richardson, Bruce > > Cc: dev@dpdk.org; thomas.monjalon@6wind.com > > Subject: Re: [dpdk-dev] [PATCH v2] ethdev: make struct rte_eth_dev cache > > aligned > > > > On Wed, May 04, 2016 at 12:09:50PM +0100, Bruce Richardson wrote: Snip > > > > > > Hi Jerin, > > > > Hi Bruce, > > > > > > > > have you seen a performance degradation due to ethdev elements sharing > > > a cache > > > > No. Not because of sharing the cache line. > > > > > line? I ask because, surprisingly for me, I actually see a performance > > > regression > > > > I see performance degradation in PMD in my setup where independent changes > > are causing the performance issue in PMD(~<100k). That's the reason I > > thought making aligned cache line stuff where ever it makes sense so that > > independent change shouldn't impact the PMD performance and this patch was > > an initiative for the same. > > > > > when I apply the above patch. It's not a big change - perf reduction > > > of <1% - but still noticable across multiple runs using testpmd. I'm > > > using two 1x40G NICs using i40e driver, and I see ~100kpps less > > > traffic per port after applying the patch. [CPU: Intel(R) Xeon(R) CPU > > > E5-2699 v3 @ 2.30GHz] > > > > This particular patch does not have any performance degradation in my > > setup. > > CPU: ThunderX > > Ok, so I take it that this patch is performance neutral on your setup, then? > If that's the case, can we hold off on merging it on the basis that it's not needed and does cause a slight regression. OK Can you share the base dpdk.org upstream change set where this patch creates the slight regression? I will test it the same on IA and ThunderX platform. In my testpmd build, rte_eth_devices(0x0000000000751ef8) share the cache line with inactive "notify_ops" that the reason for its not creating any benefit. I guess the case would have been different if its active write location. COMMON 0x0000000000751ef0 0x8 /home/jerin/dpdk-thunderx/build/lib/librte_vhost.a(virtio-net.o) 0x0000000000751ef0 notify_ops COMMON 0x0000000000751ef8 0x80900 /home/jerin/dpdk-thunderx/build/lib/libethdev.a(rte_ethdev.o) 0x0000000000751ef8 rte_eth_devices Thanks, Jerin > > Thanks, > /Bruce