From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0076.outbound.protection.outlook.com [207.46.100.76]) by dpdk.org (Postfix) with ESMTP id C46316CA2 for ; Thu, 12 May 2016 14:17:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=zxldPMEMJR1X+uk6XXkE7bKAt4MOswbDEwFGbmimVUk=; b=nQSJoleISocTTKWkSGZvZenXBx8iC6houzCHQZZ/ijMmHemoupVHdisDKmSWoG5un1x85LzePZdgd1IG0p9TlE3PoayuabSh1ouUO/3eksMeusyA4IA9PVURCNv6wcl1W9CuTc7nqrO2vzwxHy+Fzxp7oxAjIcPj+FIjV42J+UU= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from localhost.localdomain (111.92.123.202) by CY1PR0701MB1728.namprd07.prod.outlook.com (10.163.21.142) with Microsoft SMTP Server (TLS) id 15.1.492.11; Thu, 12 May 2016 12:17:45 +0000 Date: Thu, 12 May 2016 17:47:21 +0530 From: Jerin Jacob To: "Ananyev, Konstantin" CC: "dev@dpdk.org" , "Richardson, Bruce" , "thomas.monjalon@6wind.com" Message-ID: <20160512121719.GA1806@localhost.localdomain> References: <20160512091349.GA10395@localhost.localdomain> <2601191342CEEE43887BDE71AB97725836B4FFE2@irsmsx105.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2601191342CEEE43887BDE71AB97725836B4FFE2@irsmsx105.ger.corp.intel.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Originating-IP: [111.92.123.202] X-ClientProxiedBy: PN1PR01CA0018.INDPRD01.PROD.OUTLOOK.COM (10.164.137.25) To CY1PR0701MB1728.namprd07.prod.outlook.com (10.163.21.142) X-MS-Office365-Filtering-Correlation-Id: 1ecf6f0f-d8af-43cd-6508-08d37a5f6e68 X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1728; 2:dsAF757q/L1RYuZRHV24QJGmq+dJFlG5BfocKGPtn7hfj+zo9eKavQ6rdTDO2aBZSzSm8t69ys3lo01M/J1Mp1S652Zp0bbdkvmm2y1nMfvHZehleYtf97loWurIQ/end6FEv3QEct7OEV1pq+V8JpXxCZ1J4BzibYGpTFcZd0RjTZ/2qokdkcTNAkCNqvxX; 3:tzjNwD10rxSwND3E59urXOH/wrW3DhxIsyPZw72zTRlhmWC9OjVHMcRz9BuJlqGgitReEnGnZRvfLsPM1kNsGyuIcMz7rKTOr4qT8yx1O1oIoPFaelkQAva36RMqBOkU X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR0701MB1728; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1728; 25:08HgtN2GK3D0psg/qgIDi4Ccqvbd+gcQrnwp2vdFYEkw8jGyasKt0SeAI7REssB0XblcAD4d3xENiwkF+J2u4nm81zFLINkYGxrVmijOJwuRJSBScGcK7qh8X89TxWhtbNLlSR4sBftuelEROl/Oz+oog+nC+i3ygwk8FOzpiNSrc8ESIK8qZynYa7lP9iU+iF7+qmSDFRbnDROk1E3+Va+OHebFQ0h46/rJPYlA/2T2FvDyWunFwAnrRWLlenzy4M+/sBZhzGhYW4CqmlfiHQdklqRJtGJaPAT6L6B7AXbL057yBj85nJenAHbWqt9krCDQ8ZYNUQfO37e4CqkYJSIs/A4wiLlHLGdFsxQXfvsMFlrgg52wnXlWuhKxlUBT+grLbXKOLkp3HJg5jwYH6yLS39nGsZ7cskng15mtTBZaFuhqbdIPm9gYGvqUJZu3IBHntjPG+ENzSywdKbE5/855UUtYa/jVAPl9iwdVOzz3OwgHWAA+E8Lety+rpb0humh6HCr+3Wp8MKezF0i0VkLn3wXqQPupgIV5LgWGtYjB8wJZsptqGYlX8EJdxy6bNkdvzm9PJqb/6R9lwmePrMqCXYpmwMdn3xwhHGmWOpbJKNqWlOvPOaVrwI0vfDmrLfy2NQzI5oFxgLV1/u7z2Ov2yxi/BI1D06KBpOmn4OxZ6Tnu2IAwmc1cYz1HD4yNJrNfgYqpXfnA+e5WEipJFiODrpqG/3LSRvOo+VeDE+YSLmhHbxmc+DKTCfjutOZvqeTC8fm0IkYE7Ro17rKkKw== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1728; 20:897wpPYBKh5a8AmMsWGMh3G1T1ETb9Go7Yt37bjYM+gTtsmkUV9Qz94fpzHtH1WAUua48JICYDfgnyQJutgiXdbFk17Trmop9qmx1qbgJAbu+UmVS0wDNiQ6jRw7rCE90CIqI8y9pznrBfvkM4Jcnr47UveI/bYi9wJi4wQd45DSLIJk59qqYrrIqnlyoDNt0XRCgCPH7kvbu8p2TxuFXoeQwm/a6kZlFAbM+etYdmlAcCIUd8sdGbe/lDME8cyEAGh2ALhtitk0S1ouJugg5SaIQ96TSQudQ4EGX/Pm57QkhpRiaZbPKh7cGBDvRK45PRtFcBFSxNJbYF7cCZ+JFe0D4a8Mx3do3bngYaJ4Kj7NaVGO8fwuMiOySnc+IO2Q3lzdU5DKUDC++3UFgoDmmtJ1yi+8gKp3eelUm+nk8dCPt5AMYohmK9iWVuKQKsgxv23pdqYFbfwnJBRZYin5y2KpwM6kAEMfaANZiRmK+lRaBgiDgk+41r2MU4hzTQ3y2D9UHbhR2nnHfZujbA1+AO5LPiNyQXVyt7wHNvvjeqRczgpAFV523IQKVVJWKqHhGd8ecemxzxDEXG54V4dopINhjyKrmHeXCcjSaG5AsGs= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046); SRVR:CY1PR0701MB1728; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0701MB1728; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1728; 4:I4KODb2coG0N0MDnHEeXY4BLHq56gRsR5vXJ0eJxD/blsKyft4thYJsuaAQEzOHuxusVZmszhM4aEhtbNorTD+jtm/QZalx6GOz3IQRVRq2/14klitdXvAIAEjyLG5i87iL+CU0McK0BUpeQDYH5E5XTsXx7LwTWsn2hnorQRuKeXtIG2ZTVvHAz/BSKuE7wYDk3Moyz81A2F8Az1nbDSr3gpgA8ToVMzrQtroevmNB8ekBq40GZ8bHQSwYDsnF4dEF9Aq5aTvm91KAqg0RmroaH8jTZ6zNv5pbSqAt5DMuBQNh69tbSfiKdN7Vl5tE22Q1N959OGbY0P3H6plNLSKa0BvtTQo9LnI5YoBmTAqa62QYeBMqeCXiT07fyw7rW X-Forefront-PRVS: 0940A19703 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(6069001)(53754006)(24454002)(3846002)(6116002)(189998001)(83506001)(110136002)(2950100001)(5004730100002)(2870700001)(586003)(77096005)(4001350100001)(5009440100003)(2906002)(47776003)(81166006)(9686002)(50466002)(23676002)(1076002)(5008740100001)(61506002)(66066001)(76176999)(54356999)(33656002)(42186005)(92566002)(50986999); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0701MB1728; H:localhost.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1728; 23:uh29y9r9g3rBMHoQ+gX5Koft6LtNhVKSijdw0tUfN3KabgBEgfl+riaev/MDK3MhlcyM0ts4rHAF/hjThJSJdiofUh5yVNCl6GVXMOqCyVtLG15wOzzH2deWcz+7CbwRNIg+LqsUk5yluxeeNPMUSRSpyBCzCM8F1uTCzACjonsLyr/8GST4tYow9R0RCAnNtNsBfaWwtQqEyKabeyWb+0c2OFQVQfJS2Ur2JTuxsTtJmgPEM5fXpxkBnFDL7oVp1BcwZF7TJsunmuA+9r+l5bn0CGqX/TBQSkLlLSREE1Aqeycsso9OK4H8gihSLoaIVUfsxWIzhuPFri4UfzYhsZ9F1xwLK4wM3pqMNN6vR5WWPrsxtHn+1LHALWvaZ7mf2MRWxmyJ9VliZ1She4iAimCXDaJ26B1GTQzKKHSy9reBw9fn16/DTfMLPDgLh1ZpgN08IURqetkC0g7jnH8F97byu5WInIsq0sxF95nIZuepPRYO5FM6x05Etkxf6OArnx2BWDy/I/VNSk7faIu0KHgDXL5MQwS5MctNwBlmvihUzBF3pQSvBfHIhgidXIVEZUB02lsP8rKlUBD9P0T224cHXBhZUsMaPWwbGzJdnYAHU45tCmudfwNnr7/Yvg3sETdCqFyCdJPfVTS0EXerfS4IOnTPPq2HPt3K7RDcWBSylyocQ4NQztLyUhVOtLwkEzFqTynn3vxoc5hLhbIT9dnSOPdQcQUd5g1FgkBD0Dqae5Er4d0KlYcD/QSk3w2BBmsz67EhaOoZgqnW6H39gk0jgbQFBQx7fc3A9odU5/2w2km3njrXpvGwduU2GQW4NntDVHNyzZxw5HwaaRV65AikXOv8ztn2DWBYv7k9yWDC+zLUEVfXyNjUewgHPul1wZ9b6XlXMswkZVNd2BInvA== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1728; 5:cSzheJe9OB6GjHioe/ZoYAvx/GU9OwLz2jerNMgqmzLLOBprtjcAPTIKQGcMx5mDjRN5KlXhXa379+oSXqbkCbfhFWxrN3ljwLvxGuYrYn4yg3q5rwe1JixvGpnfLIb75SRiEPcHGnvGKEpKrx/mjw==; 24:IW2yMEsVgH6GfXCX9JS0ADE8HdkItfuHKVaiXmiujY5SmtWKnezeetWSkXGGoSbckmmELwlZ4HDbno6z5I/QxP36T/Z9XBk+P7dP7UIYDDk=; 7:mLtjskI8B/gdJLhZWwnbFOs+OBaCKmiw9afYyueKrBc9Id6PJFX6fi0j4vFWnC888HqdULZjvavtM38hrnkyhwYXAyIG71uWQqvg9c1aC39xMgHl4YAuZ+E9nrCbOcKfKyjMe+k/tqLuXLENI85oVhLKQk2XnDq280tDiykvtd+JSRnZzTrloiEYJQlyXi/S SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2016 12:17:45.1250 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0701MB1728 Subject: Re: [dpdk-dev] mbuff rearm_data aligmenet issue on non x86 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 12:17:50 -0000 On Thu, May 12, 2016 at 10:07:09AM +0000, Ananyev, Konstantin wrote: > Hi Jerrin, > > > > > Hi All, > > > > I would like align mbuff rearm_data field to 8 byte aligned so that > > write to mbuf->rearm_data with uint64_t* will be naturally aligned. > > I am not sure about IA but some other architecture/implementation has overhead > > in non-naturally aligned stores. > > > > Proposed patch is something like this below, But open for any change to > > make fit for all other architectures/platform. > > > > Any thoughts ? > > > > ➜ [master] [dpdk-master] $ git diff > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h > > index 529debb..5a917d0 100644 > > --- a/lib/librte_mbuf/rte_mbuf.h > > +++ b/lib/librte_mbuf/rte_mbuf.h > > @@ -733,10 +733,8 @@ struct rte_mbuf { > > void *buf_addr; /**< Virtual address of segment > > buffer. */ > > phys_addr_t buf_physaddr; /**< Physical address of segment > > buffer. */ > > > > - uint16_t buf_len; /**< Length of segment buffer. */ > > - > > > There is no need to move buf_len itself, I think. > Just move rearm_data marker prior to buf_len is enough. > Though how do you suggest to deal with the fact, that right now we blindly > update the whole 64bits pointed by rearm_data: > > drivers/net/ixgbe/ixgbe_rxtx_vec.c: > /* > * Flush mbuf with pkt template. > * Data to be rearmed is 6 bytes long. > * Though, RX will overwrite ol_flags that are coming next > * anyway. So overwrite whole 8 bytes with one load: > * 6 bytes of rearm_data plus first 2 bytes of ol_flags. > */ > p0 = (uintptr_t)&mb0->rearm_data; > *(uint64_t *)p0 = rxq->mbuf_initializer; > > ? > > If buf_len will be inside these 64bits, we can't do it anymore. > > Are you suggesting something like: > > uint64_t *p0, v0; > > p0 = &mb0->rearm_data; > v0 = *p0 & REARM_MASK; > *p0 = v0 | rxq->mbuf_initializer; > ? Due to unaligned rearm_data issue, In ThunderX platform, we need to write multiple half word of aligned stores(so masking was better us). But I think, if we can put 16bit hole between port and ol_flags then we may not need the masking stuff in ixgbe. Right? OR Even better, if we can fill in a uint16_t variable which will replaced later in the flow like "data_len"? and move buf_len at end the first cache line? or any other thoughts to fix unaligned rearm_data issue? Jerin > > If so I wonder what would be the performance impact of that change. > Konstantin > > > > /* next 6 bytes are initialised on RX descriptor rearm */ > > - MARKER8 rearm_data; > > + MARKER64 rearm_data; > > uint16_t data_off; > > > > /** > > @@ -754,6 +752,7 @@ struct rte_mbuf { > > uint8_t nb_segs; /**< Number of segments. */ > > uint8_t port; /**< Input port. */ > > > > + uint16_t buf_len; /**< Length of segment buffer. */ > > uint64_t ol_flags; /**< Offload features. */ > > > > /* remaining bytes are set on RX when pulling packet from > > * descriptor > > > > /Jerin