From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bon0081.outbound.protection.outlook.com [157.56.111.81]) by dpdk.org (Postfix) with ESMTP id 7F5F06CC8 for ; Wed, 18 May 2016 20:50:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=sHLUcBBklc53MO3iIARgxI8rz/BsYXdevXoOpXFHL8s=; b=Q2Dmhm8+KiXvpulSAzQp7X9pzfENFs7PDh1gYn1LJyq+zbS2CnToDRndAamMRuMNPPZ0ueA78JPm2p4ag4A6TZtZ2TTQEOOyXwpgDRwRCG1dIpCXQ9M8U1GV/kxWlQPeORBJuBwlFBE7PWLxlgP3nTheCcFKJQUgQmv4cFhC4gA= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from localhost.localdomain (122.171.205.87) by CY1PR0701MB1726.namprd07.prod.outlook.com (10.163.21.140) with Microsoft SMTP Server (TLS) id 15.1.497.12; Wed, 18 May 2016 18:50:35 +0000 Date: Thu, 19 May 2016 00:20:16 +0530 From: Jerin Jacob To: Bruce Richardson CC: , , , , Message-ID: <20160518185011.GA4432@localhost.localdomain> References: <1463579863-32053-1-git-send-email-jerin.jacob@caviumnetworks.com> <20160518164300.GA12324@bricha3-MOBL3> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20160518164300.GA12324@bricha3-MOBL3> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [122.171.205.87] X-ClientProxiedBy: MA1PR01CA0063.INDPRD01.PROD.OUTLOOK.COM (10.164.116.163) To CY1PR0701MB1726.namprd07.prod.outlook.com (10.163.21.140) X-MS-Office365-Filtering-Correlation-Id: 01e2f1ed-2f4b-438c-0908-08d37f4d4d6e X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 2:1qpxSiQLWpgzpICPcaK6LtBEIdSC/8kjFijvBTpTBWu+A+1Cq5sG/126aHixdBR7ZyAZZ4PUiVJW1a0JFQ2IVLeU3UEhim9HkqGhi+u21hCbh5oP17990vjv8uMzvPf7oRMySmaCeKEsMFwRvv5kqSBFJzbgR3hrkkQm4ykB8qJ7RPbGMpDbhGbUjOZVvOVL; 3:3DXM/Yyhq2KXj4yBJbyIzH034v67T66sB5hga1lqhoGY1AgTz62quQLcbr/czd/BoYbRDPPPjFtNFH/RQrBa1kUk/InYf3v/+5H7nawkmbrp52MHXpKD03xPgwry9ssE X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR0701MB1726; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 25:K9rwZYIYSPd8a7B+wYYK3r6ShI3Z2UuzqpoJ6E75qgCI03M5cuhe4Y9TamOgrEMGuTielzTQ1wxbF9QpnpV0YSOkNvt5IB4JyQgymXareRZ16adA7LkqOpMePzlqU3/8YPQo1ajLTmEaFvJL+0YMI/diSUQhK2ihnoJRhgUazBHq4uXA3E5QZP30PiK3wHJlEU/OWPWLJcg2TVTWSWtua0xGhO1Pm4TwHZxkNK8De9M1XNan+Ge8A+j5RECMBs6BEqh9KGGLnFuiksw+lRnEVLIgxe8e8HReZyEmAiExLUShd/+4lQ80QEY7kwOpe6NyqV1cyL4Uy1Ji+/7UJ3knkSHWQOBhUprDv3U1MaKVFWsVCQj0YWYOniNc5OQKNURRLdOcO0VfEZJnOPZvppa0nL+pWz7+SZwKuCDc1u/Cy1LpOIa14JChMQVa+017uD0F+ZgB++IOHpBedYmObAnAm2YExVhmzWGLOpODXPHivNYy5mgHspYEbHLH67j9B/MaYPDsdfjGxQGeQiTF1OvEx6yWYfIJbidJl5QEpCg42yzmRbHvKy3g8/WZWFEgP16l7MGoGZhLQPheLaV12TbLAhDKer10WAYG0DT2LdJR+AtLt8u3b2j/ckXp2tfQdk8JK2Ebq5C40zKPd2o7wQOZ/bXMF2D65Oh1eKtMn+kxxcNObLPrYnVxnqoVS2QraHYG99rXWdz8n0HmMLfLGkMGj7J5YtbgAScSZK5TwqkHO1H67dXBCwUZBlD7k1q1JMDrRbQCgBn7I/bLWBYQ9XZQhHmm+ZMsTZFh9jBWQTZ+MA/D8apIVnu29atec9LjpJTgi8THtQsjC9aDuwwE1QnjHw== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 20:U/iTCAbMS8vMPVUCzyyfe5FIvlDmNleVDB8Fls6cG6eeyicygg/3y1hkzYDGXPJknKxCK/p4/SdJzFDyiTUJM6uVB1l/GsF9X+VfgoOT1IRXdrQ9QIxx8eVzBxzqlOomd9ets8zO4o1FlBisS5NvBWnlJAqQ2h1pRWim0lc1IXAmyJZfSbpAMhRfSxNk6NfMj8ZCcH7bxMeIqGv5o6iCPuQasxE7TzMxVml6h89kXDjVqLV9pd7xCurHhOC1EgaPNbD7wp6xE5TXCcMg7j31dXtr7Pq6t4pnJyi2k1WG/fLH5ACL6HNQOrTyOArJoUe5rIDcpogQlThha1oxvdSNBIxoaiQdRlZHtHW+4ylFIylfNpNHz4sBy8OCuNsSyJGchHN3+9fzMTAjh9UmQilBtHTb9FgifkPYdKcReb/w4ymjFY5oA0/MoZXNxhXCwSSyA58KjvpkiqiRAnnoWPM4sTr8Rl37Vo3sWtVniSpwG17srQ/DvBnb412z2rT8mLN4izCvxCwGGrOhc1nBtGTyKB+elIRKmEKm/Ll9mLg9FyoLPtJzrgRkjGCfpZjqtKHcouvvvjFWmpDhLY3cNl6+UZwk+kdXE7qPpZ0cmTzSWkk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:CY1PR0701MB1726; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0701MB1726; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 4:BCX09kN1uztRGpnLbD9Pp+uXwCk4E07SXFsFNnwqOCmrbFaxlmP6sOaI9r+K7SWA5JI2tGE9Fje7flaJoN3+GddUlAllgC2xRNQrTLzmR06facRxWVxJ0j69/fgyHxMsg8HQ7jnxAMmr9tK5GMgq8vsr4XJEgNN0jtPUVtEnYpr8IKXY9UjtEIMQj3uH0EJusuqC/6Z6LF3eejMyImZPfHbsoDvaRasCW8zLI6f9kiF/1DP/wZUPCquZfY8mZoEWXnZ1D9L5s5Xs/iRb65uSVvD06Snwsvwk9xN7hfztpsZTcFEDdehmDcu6wnRQZu+PpDWyMA6CbfX+CbplljbBWkQsjxylFM1scVziyUn8DVt2VgIYQe4skr5YFO4+TKyE X-Forefront-PRVS: 0946DC87A1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(6069001)(51444003)(24454002)(9686002)(47776003)(5004730100002)(97756001)(6116002)(81166006)(23726003)(66066001)(8676002)(50466002)(77096005)(33656002)(15975445007)(1076002)(586003)(5008740100001)(2906002)(83506001)(50986999)(110136002)(54356999)(76176999)(61506002)(4326007)(19580395003)(19580405001)(42186005)(46406003)(92566002)(15395725005)(4001350100001)(189998001)(2950100001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0701MB1726; H:localhost.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0701MB1726; 23:STXUaS5EAEf7ThlHiy/PbfpFhk45bRmAQytKGc0?= =?us-ascii?Q?QDkQISYIk+6A06AU8KNrYj9MBSCBdqI59Rg4OJp6p/3uOVaXgOet531yFjLO?= =?us-ascii?Q?X2lIQ8b72R97fazt6NqAzA6P/21qNojy5qcZcC95vYBI1Ef9qt9QNo6iRAV9?= =?us-ascii?Q?bbCufAf2I/sSgX2XkVgUe0XAeFYK9B5ZPWiwg0MSL/zgSkVI4HENhSbibU8y?= =?us-ascii?Q?QFP2Cx1tYgOD/Vsdhk3tSq2S0K1aegRf/xdDEp+v+xdDYaE/rnVlhygSj5fP?= =?us-ascii?Q?qLvVctq9W9QaY2zA8yFFCodjfxFeLkm7olK0ee6AKh7+yRW/7MV8Km+Z2Nuv?= =?us-ascii?Q?H6YwVYu5l6zruukwjZntScFj0Jf6qfAQo+bPRySJ0WyrLPwpucAjQN1EJr66?= =?us-ascii?Q?HQ2AsktHsbPHScf/iOf0m4KA1P3Kjx/KxRfbcKxhVq/L9Q8/zXtmOnIyLcGN?= =?us-ascii?Q?NCIR0AMPhK/jmmVU+LW8vR8xJdgcSe6HGwZvD93Jrf74NdeN7qWKZYLkX/xC?= =?us-ascii?Q?ajEKz0cyginPZiY7ER4kvlCiS/U0h6IOO4vIjOVdcgJw/55EfYzzToXxkDZA?= =?us-ascii?Q?4XZKM10Zixeg3CZo+sop9ml1PmD0jJDap4PHNQSGiaFlXcoGwzJHG0/ijcZ9?= =?us-ascii?Q?2MOY8oZA2L2QojJQNgI1zHUUkLdg7ja0hutkQO6gqNj126ULzpAb54ShnvtP?= =?us-ascii?Q?BGkgQ+DyQu0BT2sifL9Ur6GoEkR7mbhRKGvmjmCdA3394VZS28M/5xwv3T2r?= =?us-ascii?Q?58IetAXbYkxmNNaxuccrA6Lml0I7I3CeSI7hEnG9BSSBnH/cUYqJox81v1oJ?= =?us-ascii?Q?JQDleUo5I5wrPiX/03j0c/pg38PDerUNinPRzN+VUUVXtLBJfsqJj6esdwJg?= =?us-ascii?Q?iwd+htZ9MC8zFbm57BVoSJnTuBqbI8Rl1EKO+SV+HDMKX3MKLVJpe/DTbNIo?= =?us-ascii?Q?LOj2DvataR5puKOZBL9sz+TStpwTS4znci+fMDhsitJoEciNJlapCGzoCKOo?= =?us-ascii?Q?ugCpcY+i2GU/8MCXGle0e0FA39JOfTmyRTGFCBMQfSi0E8Q=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 5:h+ISXZDTJwFSxq82GA8L4QYbsmKBGJrbd6aHl9T4GPHszDOyz+X+VSwaCmUw3Lei2KTSC+1rk9AAbCS19fxyEZ0Peplo8QtYvky1xABij2iGcJNUt8Drr0ftuAaYXRj9nimRUBIdGYAkEZtuQy9WPg==; 24:2NUdzrGoWnxKYJzPKW/E9PhfgZeWSeLx20V8E1T8KKlREWC7n67MOzP3XsT/sXBsqi57Mi6Sw8l6lwaQ846FY/Eq9BWdkRerupz4JSMcYJ0=; 7:GKOiTSS1EF1BfhIuryXqbQ8pMSKwCFhJrDWvQ0SpVgYeTSHFRsTmPpmwB4Fi6lCIHtM9VYyRIyIXLoHyDq+o6LI1RDe0dSYLG+tRoZ2F1ZsRWhNYiNzicByXLbs+CtRshqP+yiLxvNmeY2tdD3DFZ1tfF5j2qDYk/qeKOk6JECWEDyxdHFz96fsY1Rsw+biN SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 May 2016 18:50:35.1387 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0701MB1726 Subject: Re: [dpdk-dev] [PATCH] mbuf: make rearm_data address naturally aligned X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 May 2016 18:50:41 -0000 On Wed, May 18, 2016 at 05:43:00PM +0100, Bruce Richardson wrote: > On Wed, May 18, 2016 at 07:27:43PM +0530, Jerin Jacob wrote: > > To avoid multiple stores on fast path, Ethernet drivers > > aggregate the writes to data_off, refcnt, nb_segs and port > > to an uint64_t data and write the data in one shot > > with uint64_t* at &mbuf->rearm_data address. > > > > Some of the non-IA platforms have store operation overhead > > if the store address is not naturally aligned.This patch > > fixes the performance issue on those targets. > > > > Signed-off-by: Jerin Jacob > > --- > > > > Tested this patch on IA and non-IA(ThunderX) platforms. > > This patch shows 400Kpps/core improvement on ThunderX + ixgbe + vector environment. > > and this patch does not have any overhead on IA platform. > > > > Have tried an another similar approach by replacing "buf_len" with "pad" > > (in this patch context), > > Since it has additional overhead on read and then mask to keep "buf_len" intact, > > not much improvement is not shown. > > ref: http://dpdk.org/ml/archives/dev/2016-May/038914.html > > > > --- > While this will work and from your tests doesn't seem to have a performance > impact, I'm not sure I particularly like it. It's extending out the end of > cacheline0 of the mbuf by 16 bytes, though I suppose it's not technically using > up any more space of it. Extending by 2 bytes. Right ?. Yes, I guess, Now we using only 56 out of 64 bytes in the first 64-byte cache line. > > What I'm wondering about though, is do we have any usecases where we need a > variable buf_len for packets for RX. These mbufs come directly from a mempool, > which is generally understood to be a set of fixed-sized buffers. I realise that > this change was made in the past after some discussion, but one of the key points > there [at least to my reading] was that - even though nobody actually made a > concrete case where they had variable-sized buffers - having support for them > made no performance difference. > > The latter part of that has now changed, and supporting variable-sized mbufs > from an mbuf pool has a perf impact. Do we definitely need that functionality, > because the easiest fix here is just to move the rxrearm marker back above > mbuf_len as it was originally in releases like 1.8? And initialize the buf_len with mp->elt_size - sizeof(struct rte_mbuf). Right? I don't have a strong opinion on this, I can do this if there is no objection on this. Let me know. However, I do see in future, "buf_len" may belong at the end of the first 64 byte cache line as currently "port" is defined as uint8_t, IMO, that is less. We may need to increase that uint16_t. The reason why I think that because, Currently in ThunderX HW, we do have 128VFs per socket for built-in NIC, So, the two node configuration and one external PCIe NW card configuration can easily go beyond 256 ports. > > Regards, > /Bruce > > Ref: http://dpdk.org/ml/archives/dev/2014-December/009432.html >