From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <sachin.saxena@nxp.com> Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70075.outbound.protection.outlook.com [40.107.7.75]) by dpdk.org (Postfix) with ESMTP id 8AD471EB56 for <dev@dpdk.org>; Thu, 14 Jun 2018 08:42:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h+OjgLuqXGnJ9BjBZGEiKE3Pwezt5BrAbhpLVaucV/w=; b=MD/WQQrFQcjYaU2nvpPsJTKT07OJGc1BYbHXFPGRJUlMTmLq2OFBi5nwRbdbwjD1IgSHubI7Jgd/C5AQPrqo0KtGV7wfnv4yp5852fGb3gfJRi7GNagOZXdont/pDIsEuC3zThBt7UGmQagkQzczTRkUB4+sfhgUOUc/N6AriPY= Received: from HE1PR0401MB1772.eurprd04.prod.outlook.com (10.169.118.150) by HE1PR0401MB2505.eurprd04.prod.outlook.com (10.168.147.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.841.19; Thu, 14 Jun 2018 06:42:40 +0000 Received: from HE1PR0401MB1772.eurprd04.prod.outlook.com ([fe80::5911:66dd:1e20:9a53]) by HE1PR0401MB1772.eurprd04.prod.outlook.com ([fe80::5911:66dd:1e20:9a53%8]) with mapi id 15.20.0863.016; Thu, 14 Jun 2018 06:42:40 +0000 From: Sachin Saxena <sachin.saxena@nxp.com> To: Jerin Jacob <jerin.jacob@caviumnetworks.com> CC: Hemant Agrawal <hemant.agrawal@nxp.com>, "dev@dpdk.org" <dev@dpdk.org>, "nitin.saxena@cavium.com" <nitin.saxena@cavium.com>, "narayanaprasad.athreya@cavium.com" <narayanaprasad.athreya@cavium.com> Thread-Topic: [dpdk-dev] [PATCH] mk: change TLS model for ARMv8 and DPAA machine Thread-Index: AQHT/JdZyIIY7fpNJ0OLS5pCyUZt/6RZXSOAgAEZCDCAAED8gIAEnMjA Date: Thu, 14 Jun 2018 06:42:40 +0000 Message-ID: <HE1PR0401MB1772105D5596E248DF5C8E68E37D0@HE1PR0401MB1772.eurprd04.prod.outlook.com> References: <1528180425-27937-1-git-send-email-hemant.agrawal@nxp.com> <20180610110700.GA6990@jerin> <VI1PR0401MB17747AEE317C0362B6CE2235E3780@VI1PR0401MB1774.eurprd04.prod.outlook.com> <20180611074527.GA18217@jerin> In-Reply-To: <20180611074527.GA18217@jerin> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=sachin.saxena@nxp.com; x-originating-ip: [14.142.187.166] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0401MB2505; 7:6BoYXh9SP19T1q4N6HxCqM66C2PSZInBoTslPy2P70shX3k7aoVKPRD89+NwqEMOZP7nEj0zpcr4s0/OJqr4i2yiqOBDhCh3TZuvk2xfLG6uLIyB5Zh7PGqS5SygJHslvfwbiTe9ib4WsZXIj8ZjQcdMpSabrgO8izIzH0ttVUu4nJWrEMJbQLU+MdqIWlk2u9LYtr7UafQ9EmChORiuQHucxlEpx2bdTqexmR0vMl4SQnFk1l0otdDaS31erKnT x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 7fc3e41b-4b57-4e47-4d37-08d5d1c20659 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(5600026)(711020)(48565401081)(2017052603328)(7153060)(7193020); SRVR:HE1PR0401MB2505; x-ms-traffictypediagnostic: HE1PR0401MB2505: x-microsoft-antispam-prvs: <HE1PR0401MB2505D82C4C42AEF24C1F8AFEE37D0@HE1PR0401MB2505.eurprd04.prod.outlook.com> x-exchange-antispam-report-test: UriScan:(278428928389397)(185117386973197); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(10201501046)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123564045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:HE1PR0401MB2505; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0401MB2505; x-forefront-prvs: 0703B549E4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39380400002)(39860400002)(376002)(346002)(396003)(366004)(199004)(189003)(57704003)(13464003)(316002)(81156014)(8676002)(102836004)(74316002)(93886005)(33656002)(6916009)(9686003)(99286004)(66066001)(68736007)(5660300001)(59450400001)(6506007)(54906003)(7696005)(55236004)(81166006)(8936002)(97736004)(305945005)(7736002)(53546011)(76176011)(5250100002)(446003)(14454004)(55016002)(2900100001)(6246003)(44832011)(476003)(3280700002)(478600001)(3846002)(105586002)(53936002)(186003)(11346002)(26005)(229853002)(2906002)(6116002)(86362001)(25786009)(4326008)(106356001)(3660700001)(6436002)(486006); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0401MB2505; H:HE1PR0401MB1772.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: +E/4VY6NgMSQAqFw+VrYviHzoJTK3IFBeU20r8tzW/Zmq/Cry5La7uQYsSfdJT841Jdq8DBbmzLA5xlDsXShv8Eo+90CjNcycXpA8adBOaS1v6xABh9ca4VvIicAL0p7rwHkuzl8QCng7mhynb15Q7rpfRVgr3OchSw6texZm88SyykByOB48nGfz6nINTh8 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7fc3e41b-4b57-4e47-4d37-08d5d1c20659 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jun 2018 06:42:40.1588 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0401MB2505 Subject: Re: [dpdk-dev] [PATCH] mk: change TLS model for ARMv8 and DPAA machine X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Thu, 14 Jun 2018 06:42:42 -0000 > -----Original Message----- > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > Sent: Monday, June 11, 2018 1:15 PM > To: Sachin Saxena <sachin.saxena@nxp.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; dev@dpdk.org; > nitin.saxena@cavium.com; narayanaprasad.athreya@cavium.com > Subject: Re: [dpdk-dev] [PATCH] mk: change TLS model for ARMv8 and DPAA > machine >=20 [....] > > > > Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com> > > > > --- > > > > mk/machine/armv8a/rte.vars.mk | 3 +++ > > > > mk/machine/dpaa/rte.vars.mk | 3 +++ > > > > mk/machine/dpaa2/rte.vars.mk | 3 +++ > > > > 3 files changed, 9 insertions(+) > > > > > > > > diff --git a/mk/machine/armv8a/rte.vars.mk > > > > b/mk/machine/armv8a/rte.vars.mk index 8252efb..6897cd6 100644 > > > > --- a/mk/machine/armv8a/rte.vars.mk > > > > +++ b/mk/machine/armv8a/rte.vars.mk > > > > @@ -29,3 +29,6 @@ > > > > # CPU_ASFLAGS =3D > > > > > > > > MACHINE_CFLAGS +=3D -march=3Darmv8-a+crc+crypto > > > > + > > > > +# To avoid TLS corruption issue. > > > > +MACHINE_CFLAGS +=3D -mtls-dialect=3Dtrad > > > > > > This issue is not reproducible on Cavium ARMv8 platforms. Just > > > wondering, Do we need to change default ARMv8 config? > > [Sachin Saxena] The issue is currently visible On NXP platforms with V= PP- > dpdk solution only. Similar behavior like random crashes or initializatio= n > failures have been seen by Cavium guys on VPP but they are still > investigating whether the issues are related to TLS corruption. >=20 > I checked with Cavium-VPP team. According to them, they are not facing an= y > issue related to TLS >=20 [Sachin Saxena] Some more information. - The issue is appearing on NXP ARM = platforms as DPDK drivers are also using __thread TLS variables. If the tot= al number of TLS variables (main application + dpdk shared Lib) increases b= eyond Static TLS Size limit, one will start facing issue like Corruption in= TLS variable values.=20 > > Also, issue will not be there with statically linked dpdk applications > > > > > > > > The GNU (descriptor) dialect for TLS is the default has been since > > > for a while on aarch64. > > [Sachin Saxena] I agree but this model only applies to Shared mode > compilation. As per my knowledge, the "initial-exec" model is default for > static compilation or when -fPIC is not used. For shared dpdk or when -fP= IC is > used, the default is "global-dynamics" and tls-dialect=3Ddesc. >=20 > But shared mode compilation is important too. Right? We are concerned > about performance and stability aspects of "changing default" with out an= y > proper root cause. [Sachin Saxena] I understand changes in " armv8a/rte.vars.mk " are common b= ut we need it for Virtualization scenario where DPDK running in VM. So, may I request you to please validate whether this patch results in any = performance impact on your platform? If there will be any impact we will tr= y to omit common changes. One reason for other ARM platform not facing this issue could be that their= overall TLS usage is somehow under limit. This can be verified using "init= ial-exec" model. Using "initial-exec" model with shared dpdk-plugin will fo= rce TLS space to be static. If your TLS usage increase beyond the static TLS limit The application will= give initialization error & exit. This is true for our case. >=20 > > > > > > > > I think, it will be mostly a glibc issue with your SDK based toolchai= n. > > > Are you able to reproduce this issue with Linaro toolchain + > > > standard OS distribution environments? if so, could you please share > more details. > > [Sachin Saxena] Yes, issue is happening with both SDK & Linaro 7.2 > toolchain. >=20 > We tested and it works with gcc version 5.4.0 20160609 (Ubuntu/Linaro > 5.4.0-6ubuntu1~16.04.9). > > If it is bug from Linaro on the latest toolchain, lets report and try to = have this > fix based on runtime attributes. >=20 > > > > > > I am only concerned about, any performance issue with traditional > > > tls dialect model vs descriptor dialect. > > [Sachin Saxena] No performance impact is expected for statically build > dpdk. For shared mode, minor impact is expected but performance analysis = is > yet to be done. The Fix is suggested because right now it is functionally > broken with VPP. >=20 > Is it possible to check with "gcc version 5.4.0 20160609 (Ubuntu/Linaro > 5.4.0-6ubuntu1~16.04.9)"? so that we can identity it is an toolchain issu= e or > not? [Sachin Saxena] Yes, we are also using ubuntu 16.04 and even with native c= ompilation on board using 5.4 standard toolchain, issue is there.=20 >=20 > > > > > > I think, we have two options, > > > 1) If you can identify if it is due a specific glibc version then we > > > could detect at runtime > > > 2) In a worst case, it can be a conditional compilation option. > > > > > > /Jerin > >