From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0067.outbound.protection.outlook.com [104.47.42.67]) by dpdk.org (Postfix) with ESMTP id 9D0D1388F for ; Tue, 22 Nov 2016 19:49:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ktyjUIRhLgaAHhVhTiOJIibAuIGUA1gYywmMITFNjuQ=; b=gMLxd2rNm88cZy4fDzTj81epIeLUhuJcY9aNStYhGc2CONOskTfTYJLKP74rEvEgh2cv3c6HIuY1BJ8t459RlkM7gnLrT1troEfHlznKnpOh/3gv9ejdIDEdIs1oqynCDlCCHaUjN9/DtfMC08kcXiPPgS5Ccbw/ptGoVVS7BTo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from svelivela-lt.caveonetworks.com (50.233.148.156) by CY1PR0701MB1725.namprd07.prod.outlook.com (10.163.21.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.693.12; Tue, 22 Nov 2016 18:49:36 +0000 Date: Wed, 23 Nov 2016 00:19:31 +0530 From: Jerin Jacob To: Bruce Richardson CC: "Ananyev, Konstantin" , Satha Rao , "Zhang, Helin" , "Wu, Jingjing" , "jianbo.liu@linaro.org" , "dev@dpdk.org" Message-ID: <20161122184930.GA11048@svelivela-lt.caveonetworks.com> References: <1479473533-9393-1-git-send-email-skoteshwar@caviumnetworks.com> <2601191342CEEE43887BDE71AB9772583F0DE265@irsmsx105.ger.corp.intel.com> <20161121221638.GA17380@svelivela-lt.caveonetworks.com> <20161122134654.GA58712@bricha3-MOBL3.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20161122134654.GA58712@bricha3-MOBL3.ger.corp.intel.com> User-Agent: Mutt/1.7.1 (2016-10-04) X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: SN1PR17CA0045.namprd17.prod.outlook.com (10.163.3.141) To CY1PR0701MB1725.namprd07.prod.outlook.com (10.163.21.14) X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 2:3mc0onaemPrP52Z8lBaGOXlhc6AJByHCP0kUp2oXbMZDAMi/invfjXJvTykHGqtUNyh86wUQHaFiH7WoBSQ0MH8u/ml7rxzypQ9iJI4rTYcULd9QJR1g/UFroxK8j0C7nLZ/KsA13uYa2HZPimqNMJPCVGZl33XcSMqTL1Mf/DM=; 3:V/FIGaPy6Hp00+/RE+gIMswiMh8uaY0gcrsLR0urkpr+LlyXDbTDNuVtZzuOctgeRBwKtywNhBN2Nwloxl2UCP4lACVX5cevJfsZT/V6SGGTeEQTPrSJAQVf8aQ+E5KdyQCA78mPBGTn3m/SCqbyKf/DjYPvJo4MmOS1yLC3lwo= X-MS-Office365-Filtering-Correlation-Id: 53ce3dc0-8e51-4422-8883-08d413084ea8 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CY1PR0701MB1725; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 25:tyPD5DD8kSldxx264HSki2+wDuJmi5hlhevFbKG36FjTyHGJ8zALyYWY0S27iOG0htJRYlenfn15gsd7FEMiF0coNgFoLPv1vAUuAXFpN164YtWCZjSjv1fMX7bcsxp5Z+j81lKQtqyFEYbaVhtvA0o9LBt+scBzjMPjopwE9dSCGqhYgVSkc97JMfkBFD2LwFNLk6fWnwcFyUKb8B95CzMaECccYGdVC1YyQeO6UPswo0IoLCykRgzvTUDSGMsiIMckvHydi1HjXUwCcJNsj3Wg4CFMPtoE+v4FIOasqUHgMHziIQpauwWB4vIPiRbs884sK8h9o06rHh9r5hc/UyLE4d22b6+7Monrx2AZKndoMER0sjy9R36bsiB/FTbCTZPXCyXXAkYoAWhWYWYw8Tot10lmfLKCEncd4oHh/TiB7CtiKEX5dxcZDB3Vty4jvF7Zg9DNT+IlDVLoc1P8snE36owF4iXQwxwLYFYg8jzu+a1QUUIZGjuBJu720cpI3fa8MZuadXCBqb/88TyHu3CiZ7OKSmeIKMiwsdIfjP2dKJUhY7mWHr2vnBttYF8X47VQQJwFHnMqLnH3kL6XPFn9WfoGSX3xRWwCWrRwvHxINK3c9qIyQr2O3l5mWE19f6wRkUtxXPP5+HC6YWeznhK4WEwFcX3sJM1RsnDICLeL44eoPCqDuWk3r1OUMwsSV5D3fdQ9COpTX+hrtoZwiQklBpEwkbp/qmbQkJuIKJbWjF0LnfY211RW2OKBqg7T4QqXRRp78ZjM6sRtafnT3lj16XxfeXKsmFa+OoVYrJGOpn1k9f7OQoNX9XcaN/P3CGHA/jn+omPXhyRL7bSogHJOHabtpo9CkORK2c9ajiZaiRoUQnhNvfDn5NORYeaq X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 31:EJx/gEwzWPX7nt6wDz+Hgj+YxKsRXW4VzU/x0tBSEFVk9fCkhyYm9dKaizUMQ8Ks/O3KrF47GYUTvOAUxORFkXKv5iyuYk1zjGicBLTVSmgSZ+LqpUAuAiWrzyVKo3VCD3juFASIW422t/S2gm57u2JF/7O6MXw6EQNYJEg8bPKhaeQCbSyHaiTrbsG33kWvAuEuVVmqTC7eTkcBE8SsCRmU9ahO1NWZUcu3OIu1XAVzlAs7FjiK4g/TmfDwiQyhpt4AG30wH39AmRHdqIE9rlfa7fGJO350KiEwevVyTXg=; 20:PBSs065U5E+CE72s0FA2BClgsoba3flHnVpqdl4nJrGOrAiSd50DzqnP9TgyQ6rzyEP/vxzfcRlmbWgOr/h2UPXtkVNJ1vf/Hv4PXOmPHUrO2Rdq5mRcRIaQ0IHWjkKCis0jgZvpwO2/n3EjINBaF8mxSYJ8PG2D+uwLgeAInfg5W4LGbrcPEg2ZsEpMsklcGYeNe9PcF1U1P6gpoqcsbcVPYp/8BGMAbYYk/ecIY30ARUUt0o12D07ynXNnotn4uVobuI/fxdP2mrj7+aEyJUrsudjMclGUOTtcm0lhWTZ/fHz85uaFLAQhzjbdp9RtTyCns/eumVC6X46Lxwf6xWnU+bjaJ3UP36PNSn9MRAMPNTlmwLerVRJaautTTwevcvsom3TL7Ho5ywZLy58F9rZrC5UGnbcc8IlD+9zfBf9lw7txlltJpjKFJ3fiWwBs1mxyW2V8qOtVLy/zogJ8i3fIv+UQTMHKlUGVSqIYDiP1iOPEFspYB8/oZL4GBAt1ffK+pf8zL3Mstyptb+GqRTH/1la7XVHDlfO7ZSPnJLtHNWaIME+qq84cjp9TRaD3YRtkbv9p79YCjo1Q8/g+eNSSp2Mx7fDz6c6eUiRLsaM= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(180628864354917)(11711142671493)(58145275503218); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040307)(6060326)(6045199)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6061324)(6041248)(6042181); SRVR:CY1PR0701MB1725; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0701MB1725; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 4:woR1OZqyPNNcFPoi1/TtUISNXOKjIplsybPWG+SilrXTxfqzGdvht/fq8jI+C8jwjSFWQOJ30P5N8oAIiYyy5LIBmJD4Q4GaP0w6pNu/Lidirt0ZUYM+TB93Yga08p8uqQJFwjlIX1/SGv2ezWjAgpiccxpb98a/ul8E1kNGtAO2wqtAXDVRjasQZ2X2n1uM5PzsIr9VoC44NCuT0936H5HWti2byS6nnoff9f3rg4Tzdu+6+NeD6GLi6Q1mPNcIrjSdxG+z0VmFqZYzoExyJAhvHf813MXZeuEYOUfFyYFYAhdbt5tWxqpLJ7ZeLiK4sRaLYNu4Y3qopDdWPGW4B58B2UFQOAUzjXaYpoqQ2jj2z1TF3/uSdoACBFVgOf3BOQHYx443VTQx9rRxkob/rzr2327NEtQ23cQqgNYxjulEpdUbiK0IiKvXKHiIysrCVVhZaqpCx28z3k1+qV2r3knB1TO2xoI1mA0kkTcFA0ECt9xf5oZzPWh/tbQ5og1YPN+2JPnE3WjirHbEL1eO5AhZYQ0JGv3bmxXxE2TrdhNC+h/AHQGK+3srPEbz6AgbgKdGxXGiwVeuLj77R3hMHbM72lBe5fCNaYgnMmRCWOU= X-Forefront-PRVS: 0134AD334F X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(979002)(6009001)(7916002)(24454002)(189002)(199003)(7736002)(53416004)(7846002)(6916009)(46406003)(81156014)(105586002)(92566002)(106356001)(50986999)(33656002)(6666003)(54356999)(305945005)(110136003)(5660300001)(2950100002)(101416001)(15380165006)(42882006)(76176999)(93886004)(9686002)(68736007)(97756001)(97736004)(69596002)(42186005)(50466002)(6116002)(83506001)(23726003)(47776003)(189998001)(8676002)(4326007)(1076002)(3846002)(77096005)(38730400001)(66066001)(229853002)(81166006)(4001350100001)(2906002)(18370500001)(15760500001)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0701MB1725; H:svelivela-lt.caveonetworks.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0701MB1725; 23:tPl7LOyo9MxPBb3hxYMudxZV9CukVFd+iwCfQQl?= =?us-ascii?Q?560rv6SHqfmDxO+/9fRhS4JTYJxl3tclglprqwu+BSYnXMME5gbtrZ6wrTBX?= =?us-ascii?Q?CLd0xGI+SuR2HYydsGjYeSgowNULtklVs2hS4AVrCNeJiV1OVNJ3bSSLaDj+?= =?us-ascii?Q?ghnkQfSgYmLk+cZ0AX44wqSEh8WuJQVwTZz9sIJAVQgtqdXDFX76LhiWdDFB?= =?us-ascii?Q?GDJhjG2Y7DCEXC6TfTySa80eBmPSMosBFYux3DeChOdGcf65Glh6G7IQ20n4?= =?us-ascii?Q?xRggM7N7vRfZqhWa8zLBbFVoiSKaUEz+EJar0or5v1+OWYDdHEbvbKAFY7cj?= =?us-ascii?Q?lH47IVeEpkzvWK9bTXVN/eRc6pzZSMuBguqeTZWhvP1rJpl/BNKuSYh0537m?= =?us-ascii?Q?ePjsvFNxLkpPhT1YvsfOD/iNM1DGcC0dmNSyssJpVK1N25zYRJPmm0wVPSyA?= =?us-ascii?Q?48KzADfhqWabVsFyyrpwFDaw2oqVx7jQBX9N8hjPaKGrr5Xs9atPAwZezD8x?= =?us-ascii?Q?LfV5uh2FJve0IkgSJSyUwcqGl8+2ifUSVqaYbDm0IQS+XsXDJ/NAFskCSIaf?= =?us-ascii?Q?zFU4pl+cxqOlOPQiMnUezwqdysmYDjvfaFFM+GLdwYoQ3tO50ls2jQ+71iK+?= =?us-ascii?Q?TkJ4AArIkuLk5/7rMT6YJItLXFKEozSOBPI4AD07UvPxrStUkMI5CM3YJAyU?= =?us-ascii?Q?liyFU7YLc+s0bYUyqxxyNJqKY9H1MxMoC16kDlekHdz258XylAcMMzEVtoD0?= =?us-ascii?Q?PdrD/Z7SgIJglTHo+ceuyrceoHaao1+nm+ovX6i33N0Tv0OYM2mQyi5jKXqB?= =?us-ascii?Q?3WLDW19ayM8ICbO4SDHStOIpR48D9XGE9ypc7G0R1KhxZmlgqK4/hEWI6Kbg?= =?us-ascii?Q?9MXaeom3V5+Z1EHQjqKPXXnKsFz14i1eJRrtR6+aZX9sDZYfoXJNT3n1BMP5?= =?us-ascii?Q?9KSgb/3tygTwJ3ttwe+NK5AJZ9lfTYn9KYf9UevvbI6qWwBL/cTk+Hcj29GI?= =?us-ascii?Q?G2e5J/HeGEuT3HXq5Aq3Zrdnr9oKSSIa7FS4yKuvpHYKaxUCF2MfQxPjBvOR?= =?us-ascii?Q?eu3Mr6GR6Y3u+/8DYmTPGKbf6efQyyQhe4qsqghC8SW/Z37nm0VeAYieDfhw?= =?us-ascii?Q?IukVRf/yLRQUOEYJq2kiAz04zeccv6UWJqGqdHxHcGYNQQgMYOY1rQU3F8VJ?= =?us-ascii?Q?HBIPfm3o6fgIjLexq5s2ZQW0hvPtiBfufKcCrd5qGjARaGDblldrsPkWp9jO?= =?us-ascii?Q?tSQRQEX8mAH02PMNPciWfAmtX1Ki++Tfh4EGVXMzlJB81HsprOiFimolT/yK?= =?us-ascii?Q?WyhSxl9foJxfS1YsStRm7GCicBAjUnaEIxyLKHhKKbnVO+oU7D/HCsXvdsLU?= =?us-ascii?Q?X7nBZMFe3i+bIWMJ3kI6u9pNOzlIxFNboGPJuPwE3843sG1vM2sRAauysGBK?= =?us-ascii?Q?BO9Blc41GNw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 6:INgZoIjgsR/cU0zUBGlbNiXPSPh20Gm43zajYcPnOmWqC2Jq+O6VdbcLCRD1xPyGforJumaiRuN3w3gOaJIrHndVcEx5pM2bqB3QZ+EFGcEDixAUv/tNvGdE2/cXw68XdsSuOVTEz90vB0hIjgU7ByBS/4//uo3GQH7mx6fXifR8tDbjQe4fKFbxtYqY3avm1XQhAujA46jRlAxk/htOjeFLTPH2hUPbBd70Tm4miOSjEYvylARB0oiP1nXTADreMFqEPZySAhtRXZFEp5puqw9bmqnBwiZw9L1K77UjH/rU9/iJYTtFrdOtz8CSY8X7LtrcHFtShaJSFcxUTmA6J9UVa13/hwN683Qi7820Of8=; 5:QIuUhCxD2SUHG0ZxkBPTQ7IFMGHhDUgZb7aHD+TmBP1GOcDwliKAMdB8gc7bHeELp8O3fdQzIML/Q1/WE8MjDBPmks4cnPADxVciQom70FqXBsSSA0+iQI7ugc80epB9R+Sjuv/g1sQ7XYmxj4U8vw==; 24:M9zsU0/ToBVzOdxLLDgpoppqLxWm/slCfRdQ/Z1LdSpTA9VYdrhUkl2q+h4pgtjm2suAapP5mkNmoC/KrOUA55OWhU9/9fn4cATSPGiPpJg= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1725; 7:uDfQbrKpuV0XN3s0xYdqbLNphNHxXqBuakQozGsmmi9AeuWye7BErMI2ff+6mFloJqkacJMTUhOLWOZucYsUmuiSDpU98oJ1LzwDqxXnl5URtYZN+WPPD1Y3zdbsjFc1hLta1SE8RK7ZHs88gxdlmuQ0h7Ayw7nJoFuNiRj0VHl8VJoOJ3XaWfKvD1toulOfbX+GbdqkLsGkXpvLW6a4hpa8R5Wq0oaBwnbb2VukeI8MbxfE7UwVaSFTZe7qNq7hJWiVNE6TI4DN7AoxHx7YwhIHqxnLuz4AGEyXz1QP4+dDyo2+4ZvZYMMgai0H5PMBsI+Ab6dofAMyYpq/TbDdve5uwxzfQKvoZzigJu35zZY= X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2016 18:49:36.0322 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0701MB1725 Subject: Re: [dpdk-dev] [PATCH v2] i40e: Fix eth_i40e_dev_init sequence on ThunderX X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2016 18:49:43 -0000 On Tue, Nov 22, 2016 at 01:46:54PM +0000, Bruce Richardson wrote: > On Tue, Nov 22, 2016 at 03:46:38AM +0530, Jerin Jacob wrote: > > On Sun, Nov 20, 2016 at 11:21:43PM +0000, Ananyev, Konstantin wrote: > > > Hi > > > > > > > > i40e_asq_send_command: rd32 & wr32 under ThunderX gives unpredictable > > > > results. To solve this include rte memory barriers > > > > > > > > Signed-off-by: Satha Rao > > > > --- > > > > drivers/net/i40e/base/i40e_osdep.h | 14 ++++++++++++++ > > > > 1 file changed, 14 insertions(+) > > > > > > > > diff --git a/drivers/net/i40e/base/i40e_osdep.h b/drivers/net/i40e/base/i40e_osdep.h > > > > index 38e7ba5..ffa3160 100644 > > > > --- a/drivers/net/i40e/base/i40e_osdep.h > > > > +++ b/drivers/net/i40e/base/i40e_osdep.h > > > > @@ -158,7 +158,13 @@ do { \ > > > > ((volatile uint32_t *)((char *)(a)->hw_addr + (reg))) > > > > static inline uint32_t i40e_read_addr(volatile void *addr) > > > > { > > > > +#if defined(RTE_ARCH_ARM64) > > > > + uint32_t val = rte_le_to_cpu_32(I40E_PCI_REG(addr)); > > > > + rte_rmb(); > > > > + return val; > > > > > > If you really need an rmb/wmb with MMIO read/writes on ARM, > > > I think you can avoid #ifdefs here and use rte_smp_rmb/rte_smp_wmb. > > > BTW, I suppose if you need it for i40e, you would need it for other devices too. > > > > Yes. ARM would need for all devices(typically, the devices on external PCI bus). > > I guess rte_smp_rmb may not be the correct abstraction. So we need more of > > rte_rmb() as we need only non smp variant on IO side. I guess then it make sense to > > create new abstraction in eal with following variants so that each arch > > gets opportunity to make what it makes sense that specific platform > > > > rte_readb_relaxed > > rte_readw_relaxed > > rte_readl_relaxed > > rte_readq_relaxed > > rte_writeb_relaxed > > rte_writew_relaxed > > rte_writel_relaxed > > rte_writeq_relaxed > > rte_readb > > rte_readw > > rte_readl > > rte_readq > > rte_writeb > > rte_writew > > rte_writel > > rte_writeq > > > > Thoughts ? > > > > That seems like a lot of API calls! > Perhaps you can clarify - why would the rte_smp_rmb() not work for you? Currently arm64 mapped DMB as rte_smp_rmb() for smp case. Ideally for io barrier and non smp case, we need to map it as DSB and it is bit heavier than DMB The linux kernel arm64 mappings http://lxr.free-electrons.com/source/arch/arm64/include/asm/io.h#L142 DMB vs DSB https://community.arm.com/thread/3833 The relaxed one are without any barriers.(the use case like accessing on chip peripherals may need only relaxed versions) Thoughts on new rte EAL abstraction? > > /Bruce