Re: [dpdk-dev] [PATCH] mk: change TLS model for ARMv8 and DPAA machine

[Sachin Saxena <sachin.saxena@nxp.com>]

> -----Original Message-----
> From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> Sent: Monday, June 11, 2018 1:15 PM
> To: Sachin Saxena <sachin.saxena@nxp.com>
> Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; dev@dpdk.org;
> nitin.saxena@cavium.com; narayanaprasad.athreya@cavium.com
> Subject: Re: [dpdk-dev] [PATCH] mk: change TLS model for ARMv8 and DPAA
> machine
> 

[....]

> > > > Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
> > > > ---
> > > >  mk/machine/armv8a/rte.vars.mk | 3 +++
> > > >  mk/machine/dpaa/rte.vars.mk   | 3 +++
> > > >  mk/machine/dpaa2/rte.vars.mk  | 3 +++
> > > >  3 files changed, 9 insertions(+)
> > > >
> > > > diff --git a/mk/machine/armv8a/rte.vars.mk
> > > > b/mk/machine/armv8a/rte.vars.mk index 8252efb..6897cd6 100644
> > > > --- a/mk/machine/armv8a/rte.vars.mk
> > > > +++ b/mk/machine/armv8a/rte.vars.mk
> > > > @@ -29,3 +29,6 @@
> > > >  # CPU_ASFLAGS =
> > > >
> > > >  MACHINE_CFLAGS += -march=armv8-a+crc+crypto
> > > > +
> > > > +# To avoid TLS corruption issue.
> > > > +MACHINE_CFLAGS += -mtls-dialect=trad
> > >
> > > This issue is not reproducible on Cavium ARMv8 platforms. Just
> > > wondering, Do we need to change default ARMv8 config?
> > [Sachin Saxena]  The issue is currently visible On NXP platforms with VPP-
> dpdk solution only. Similar behavior like random crashes or initialization
> failures have been seen by Cavium guys on VPP but they are still
> investigating whether the issues are related to TLS corruption.
> 
> I checked with Cavium-VPP team. According to them, they are not facing any
> issue related to TLS
> 
[Sachin Saxena] Some more information. - The issue is appearing on NXP ARM platforms as DPDK drivers are also using __thread TLS variables. If the total number of TLS variables (main application + dpdk shared Lib) increases beyond Static TLS Size limit, one will start facing issue like Corruption in TLS variable values. 

> > Also, issue will not be there with statically linked dpdk applications
> >
> > >
> > > The GNU (descriptor) dialect for TLS is the default has been since
> > > for a while on aarch64.
> > [Sachin Saxena] I agree but this model only applies to Shared mode
> compilation. As per my knowledge, the "initial-exec" model is default for
> static compilation or when -fPIC is not used. For shared dpdk or when -fPIC is
> used, the default is "global-dynamics" and tls-dialect=desc.
> 
> But shared mode compilation is important too. Right? We are concerned
> about performance and stability aspects of "changing default" with out any
> proper root cause.
[Sachin Saxena] I understand changes in " armv8a/rte.vars.mk " are common but we need it for Virtualization scenario where DPDK running in VM.
So, may I request you to please validate whether this patch results in any performance impact on your platform? If there will be any impact we will try to omit common changes.
One reason for other ARM platform not facing this issue could be that their overall TLS usage is somehow under limit. This can be verified using "initial-exec" model. Using "initial-exec" model with shared dpdk-plugin will force TLS space to be static.
If your TLS usage increase beyond the static TLS limit The application will give initialization error & exit. This is true for our case.
> 
> >
> > >
> > > I think, it will be mostly a glibc issue with your SDK based toolchain.
> > > Are you able to reproduce this issue with Linaro toolchain +
> > > standard OS distribution environments? if so, could you please share
> more details.
> > [Sachin Saxena] Yes, issue is happening with both SDK & Linaro 7.2
> toolchain.
> 
> We tested and it works with gcc version 5.4.0 20160609 (Ubuntu/Linaro
> 5.4.0-6ubuntu1~16.04.9).
>
> If it is bug from Linaro on the latest toolchain, lets report and try to have this
> fix based on runtime attributes.
> 
> > >
> > > I am only concerned about, any performance issue with traditional
> > > tls dialect model vs descriptor dialect.
> > [Sachin Saxena] No performance impact is expected for statically build
> dpdk. For shared mode, minor impact is expected but performance analysis is
> yet to be done. The Fix is suggested because right now it is functionally
> broken with VPP.
> 
> Is it possible to check with "gcc version 5.4.0 20160609 (Ubuntu/Linaro
> 5.4.0-6ubuntu1~16.04.9)"? so that we can identity it is an toolchain issue or
> not?

[Sachin Saxena]  Yes, we are also using ubuntu 16.04 and even with native compilation on board using 5.4 standard toolchain, issue is there. 

> 
> > >
> > > I think, we have two options,
> > > 1) If you can identify if it is due a specific glibc version then we
> > > could detect at runtime
> > > 2) In a worst case, it can be a conditional compilation option.
> > >
> > > /Jerin
> >