From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 834F7558C for ; Thu, 1 Dec 2016 12:38:55 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP; 01 Dec 2016 03:38:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,724,1477983600"; d="scan'208";a="1066603424" Received: from irsmsx109.ger.corp.intel.com ([163.33.3.23]) by orsmga001.jf.intel.com with ESMTP; 01 Dec 2016 03:38:28 -0800 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.43]) by IRSMSX109.ger.corp.intel.com ([169.254.13.158]) with mapi id 14.03.0248.002; Thu, 1 Dec 2016 11:38:27 +0000 From: "Ananyev, Konstantin" To: Jerin Jacob CC: "Richardson, Bruce" , Satha Rao , "Zhang, Helin" , "Wu, Jingjing" , "jianbo.liu@linaro.org" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v2] i40e: Fix eth_i40e_dev_init sequence on ThunderX Thread-Index: AQHSRET1xy4DjrTkUkKXMwHPEfUdnKDlBToAgABUjYCADIBIAIAANWkAgADyisA= Date: Thu, 1 Dec 2016 11:38:27 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772583F0E26BC@irsmsx105.ger.corp.intel.com> References: <1479473533-9393-1-git-send-email-skoteshwar@caviumnetworks.com> <2601191342CEEE43887BDE71AB9772583F0DE265@irsmsx105.ger.corp.intel.com> <20161121221638.GA17380@svelivela-lt.caveonetworks.com> <20161122134654.GA58712@bricha3-MOBL3.ger.corp.intel.com> <20161122184930.GA11048@svelivela-lt.caveonetworks.com> <2601191342CEEE43887BDE71AB9772583F0E2477@irsmsx105.ger.corp.intel.com> <20161130205455.GA5535@svelivela-lt.caveonetworks.com> In-Reply-To: <20161130205455.GA5535@svelivela-lt.caveonetworks.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2] i40e: Fix eth_i40e_dev_init sequence on ThunderX X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Dec 2016 11:38:56 -0000 Hi Jerin, =20 > > > > > > > > > > > > > > i40e_asq_send_command: rd32 & wr32 under ThunderX gives unpre= dictable > > > > > > > results. To solve this include rte mem= ory barriers > > > > > > > > > > > > > > Signed-off-by: Satha Rao > > > > > > > --- > > > > > > > drivers/net/i40e/base/i40e_osdep.h | 14 ++++++++++++++ > > > > > > > 1 file changed, 14 insertions(+) > > > > > > > > > > > > > > diff --git a/drivers/net/i40e/base/i40e_osdep.h b/drivers/net= /i40e/base/i40e_osdep.h > > > > > > > index 38e7ba5..ffa3160 100644 > > > > > > > --- a/drivers/net/i40e/base/i40e_osdep.h > > > > > > > +++ b/drivers/net/i40e/base/i40e_osdep.h > > > > > > > @@ -158,7 +158,13 @@ do { = \ > > > > > > > ((volatile uint32_t *)((char *)(a)->hw_addr + (reg))) > > > > > > > static inline uint32_t i40e_read_addr(volatile void *addr) > > > > > > > { > > > > > > > +#if defined(RTE_ARCH_ARM64) > > > > > > > + uint32_t val =3D rte_le_to_cpu_32(I40E_PCI_REG(addr)); > > > > > > > + rte_rmb(); > > > > > > > + return val; > > > > > > > > > > > > If you really need an rmb/wmb with MMIO read/writes on ARM, > > > > > > I think you can avoid #ifdefs here and use rte_smp_rmb/rte_smp_= wmb. > > > > > > BTW, I suppose if you need it for i40e, you would need it for o= ther devices too. > > > > > > > > > > Yes. ARM would need for all devices(typically, the devices on ext= ernal PCI bus). > > > > > I guess rte_smp_rmb may not be the correct abstraction. So we nee= d more of > > > > > rte_rmb() as we need only non smp variant on IO side. I guess the= n it make sense to > > > > > create new abstraction in eal with following variants so that eac= h arch > > > > > gets opportunity to make what it makes sense that specific platfo= rm > > > > > > > > > > rte_readb_relaxed > > > > > rte_readw_relaxed > > > > > rte_readl_relaxed > > > > > rte_readq_relaxed > > > > > rte_writeb_relaxed > > > > > rte_writew_relaxed > > > > > rte_writel_relaxed > > > > > rte_writeq_relaxed > > > > > rte_readb > > > > > rte_readw > > > > > rte_readl > > > > > rte_readq > > > > > rte_writeb > > > > > rte_writew > > > > > rte_writel > > > > > rte_writeq > > > > > > > > > > Thoughts ? > > > > > > > > > > > > > That seems like a lot of API calls! > > > > Perhaps you can clarify - why would the rte_smp_rmb() not work for = you? > > > > > > Currently arm64 mapped DMB as rte_smp_rmb() for smp case. > > > > > > Ideally for io barrier and non smp case, we need to map it as DSB and= it is > > > bit heavier than DMB > > > > Ok, so you need some new macro, like rte_io_(r|w)mb or so, that would e= xpand into dmb > > for ARM, correct? >=20 > The io barrier expands to dsb. > http://lxr.free-electrons.com/source/arch/arm64/include/asm/io.h#L110 Sorry, yes I meant DSB here. >=20 > > > > > > > > The linux kernel arm64 mappings > > > http://lxr.free-electrons.com/source/arch/arm64/include/asm/io.h#L142 > > > > > > DMB vs DSB > > > https://community.arm.com/thread/3833 > > > > > > The relaxed one are without any barriers.(the use case like accessing= on > > > chip peripherals may need only relaxed versions) > > > > > > Thoughts on new rte EAL abstraction? > > > > Looks like a lot of macros but if you guys think that would help - NP w= ith that :) >=20 > I don't have strong opinion here. If there is concern on a lot of macros > then, I can introduce only "rte_io_(r|w)mb" instead of read[b|w|l|q]/writ= e[b|w|l|q]/relaxed. > let me know? I think we can have both. The question is in the amount of work need to be done. >=20 > > Again, in that case we probably can get rid of driver specific pci reg = read/write defines. > Yes. But, That's going to have a lot of change :-( Yes I agree, the changes would be quite significant. >=20 > If there is no objection then I will introduce > "read[b|w|l|q]/write[b|w|l|q]/relaxed" and then change all external pcie = drivers > with new macros. That seems like a good idea to me. Though as you said that seems quite a significant change. Probably make sense to do it in 2 stages (just a suggestion):=20 First introduce rte_io_(r|w)mb and fix with it existing issues in the parti= cular drivers. Second replace existing PMD specific xxx_read/write_addr() with your new ge= neric=20 Konstantin