From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by dpdk.org (Postfix) with ESMTP id 0FA224C9C; Fri, 9 Nov 2018 12:35:14 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id B87A72311B; Fri, 9 Nov 2018 06:35:10 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Fri, 09 Nov 2018 06:35:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=cDBbr786yJy7m5sxQelBIcoY7/sP2lEzWY6cFPh/SeM=; b=Yld0bj9j2F1n 2RyRZg33B3/ayYkUqdaGmkWitwi66LR5H/mWI9fDf2k34xfNHjwG+O0FhqczidiV VUueeG8eM5ztfaeInbMowRNKJwqo+2fYG+nvJBjK1w3WTL1ADqR/fcrlXcbnVLsD qqMeBcPMbI6b7dF4Ij6DGtLoH57hSaM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=cDBbr786yJy7m5sxQelBIcoY7/sP2lEzWY6cFPh/S eM=; b=bWw+JBQ5uCU4l35RjfuqT5i+pP61iwe9F+EMVHbisu/mQHC7SNVx6Dj1Q pdeg4uWA9rqVNZg3RZ1L0FBXy0nc944z2oRmqdQX2ByQghRL5zqA7J3FPMoIFSJv xdGO1ixJuKDnhVYeZblxqiZuEGfB0uSzmFlGPWnuYc5I4bbUGM7Eqzuzf+oLA/6V 59H12lKSTGzapjxWcm60qbIOsHzg1GTMXsr8dEQT6mQLKzsOm3qP0UjXFm8mQNS/ 4IF8acsZejb942CGNAgNDnq3J50Ehf8lkOe784fPUMYy6bzMzAliOiw4URlf4oZf P8K4BCwTUgqtZYvia86mWs4hgqMfA== X-ME-Sender: X-ME-Proxy: Received: from xps.localnet (212.18.136.77.rev.sfr.net [77.136.18.212]) by mail.messagingengine.com (Postfix) with ESMTPA id 22C8E102F0; Fri, 9 Nov 2018 06:35:07 -0500 (EST) From: Thomas Monjalon To: Ferruh Yigit , Christian Ehrhardt Cc: dev@dpdk.org, yskoh@mellanox.com, keith.wiles@intel.com, Bruce Richardson , Shahaf Shuler , "Ananyev, Konstantin" , anatoly.burakov@intel.com, stable@dpdk.org, justin.parus@microsoft.com, David Coronel , Josh Powers , Jay Vosburgh , Dan Streetman Date: Fri, 09 Nov 2018 12:35:03 +0100 Message-ID: <1713583.cXMO6sc2bi@xps> In-Reply-To: References: <20181023212318.43082-1-yskoh@mellanox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] AVX512 bug on SkyLake X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2018 11:35:15 -0000 09/11/2018 10:49, Ferruh Yigit: > On 11/9/2018 6:27 AM, Christian Ehrhardt wrote: > > On Fri, Nov 9, 2018 at 12:01 AM Yongseok Koh wrote: > >> > >> > >>> On Nov 8, 2018, at 9:21 AM, Ferruh Yigit wrote: > >>> > >>> On 11/8/2018 3:59 PM, Thomas Monjalon wrote: > >>>> Hi, > >>>> > >>>> We need to gather more information about this bug. > >>>> More below. > >>>> > > > > Thanks Thomas for looping us in! > > > >>>> 07/11/2018 10:04, Wiles, Keith: > >>>>>> On Nov 6, 2018, at 9:30 PM, Yongseok Koh wrote: > >>>>>>> On Nov 5, 2018, at 6:06 AM, Wiles, Keith wrote: > >>>>>>>> On Nov 2, 2018, at 9:04 PM, Yongseok Koh wrote: > >>>>>>>> > >>>>>>>> This is a workaround to prevent a crash, which might be caused by > >>>>>>>> optimization of newer gcc (7.3.0) on Intel Skylake. > >>>>>>> > >>>>>>> Should the code below not also test for the gcc version and > >>>>>>> the Sky Lake processor, maybe I am wrong but it seems it is > >>>>>>> turning AVX512 for all GCC builds > >>>>>> > >>>>>> I didn't want to check gcc version as 7.3.0 is very new. Only gcc 8 is newly up since then (gcc 8.2). > >>>>>> Also, I wasn't able to test every gcc versions and I wanted to be a bit conservative for this crash. > >>>>>> Performance drop (if any) by disabling a new (experimental) feature would be less risky than unaccountable crash. > >>>>>> And, it does disable the feature only if CONFIG_RTE_ENABLE_AVX512=n. Please refer to v3. > >>>>> > >>>>> Are you not turning off all of the GCC versions for AVX512. > >>>>> And you can test for range or greater then GCC version and > >>>>> it just seems like we are turning off every gcc version, is that true? > >>>> > >>>> Do we know exactly which GCC versions are affected? > >>>> > >>>>>>> Also bug 97 seems a bit obscure reference, maybe you know > >>>>>>> the bug report, but more details would be good? > >>>>>> > >>>>>> I sent out the report to dev list two month ago. > >>>>>> And I created the Bug 97 in order to reference it > >>>>>> in the commit message. > >>>>>> I didn't want to repeat same message here and there, > >>>>>> but it would've been better to have some sort of summary > >>>>>> of the Bug, although v3 has a few more words. > >>>>>> However, v3 has been merged. > >>>>> > >>>>> Still this is too obscure if nothing else give a link to > >>>>> a specific bug not just 97. > >>>> > >>>> The URL is > >>>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D97&data=02%7C01%7Cyskoh%40mellanox.com%7C90ff6c361faf422b976108d6459eb490%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636772945282345908&sdata=2o%2Fg203aWrKCYg16S6oI4BcS41igpLu1DloS%2FrRnknc%3D&reserved=0 > >>>> The bug is also pointing to an email: > >>>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmails.dpdk.org%2Farchives%2Fdev%2F2018-September%2F111522.html&data=02%7C01%7Cyskoh%40mellanox.com%7C90ff6c361faf422b976108d6459eb490%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636772945282345908&sdata=NCFKxaREd69iZ8eyFKg%2FWBP73CLTXkxrNQQeii%2Bbsao%3D&reserved=0 > >>>> > >>>> Summary: > >>>> - CPU: Intel Skylake > >>>> - Linux environment: Ubuntu 18.04 > >>>> - Compiler: gcc-7.3 (Ubuntu 7.3.0-16ubuntu3) > >>> > >>> Is it possible to test a few other gcc versions to check if the issue is > >>> specific to this compiler version? > >> > >> Nothing's impossible but even with my quick search in gcc.gnu.org, > >> I could find the following documents mention mavx512f support: > >> > >> GCC 4.9.0 > >> April 22, 2014 (changes, documentation) > >> > >> GCC 5.1 > >> April 22, 2015 (changes, documentation) > >> > >> GCC 6.4 > >> July 4, 2017 (changes, documentation) > >> > >> GCC 7.1 > >> May 2, 2017 (changes, documentation) > >> > >> GCC 8.1 > >> May 2, 2018 (changes, documentation) > >> > >> We altogether have to put quite large resource to verify all of the versions. > >> > >> I assumed older than gcc 7 would have the same issue. I know it was a speculation > >> but like I mentioned I wanted to be more conservative. I didn't mean this is a permanent fix. > >> For two months, we couldn't have any tangible solution (actually nobody cared including myself), > >> so I submitted the patch to temporarily disable mavx512f. > >> > >> I'm still not sure what the best option is... > > > > What I wonder in all of this as I don't understand that part of it yet is this. > > I assume you are building on Ubuntu as that is your gcc reference. > > FYI: as people asked for bug references, there also is [1] which seems > > pretty much the same issue. > > > > It builds with mostly defaults, that means per > > mk/machine/default/rte.vars.mk and similar it sets -march=corei7 > > > > But when I look at what that implies all avx512 is disabled > > $ gcc -Q --help=target -m64 -march=corei7 | grep avx512f > > -mavx512f [disabled] > > This is output is the "corei7" architecture. Defect is for "native" build on > Skylake. > > For "native" arch: > gcc -Q --help=target -m64 -march=native | grep avx512f > -mavx512f [enabled] I confirm. > Also I can confirm from disassembly output that avx512 instructions are used > without '-mno-avx512f' provided. > > > So I wonder what/why -mno-avx512f should help at all. > > I used the full list of gcc args we have for the build (e.g. [2] of a > > 18.05 build), but that doesn't change that (mostly -W, -I and -D). > > So I wonder, did people do a custom build and bump up march or enable > > -mavx512f on their own to hit that? > > Or are we facing a real gcc issue where " -mavx512f [disabled]" is not > > the same as -mno-avx512f ? > > Maybe someone who hit the bug could clarify that please? > > > > BTW: per reports I've seen it also seems to apply to the latest > > compiler update of the same series - at least it was said to be fully > > updated, that would be 7.3.0-27ubuntu1~18.04 > > But this is 2nd grade information as I don't have a system with the > > right combo MLX5+Skylake available atm, so I can't confirm for sure > > :-/ I tried latest gcc 7.3.0-27ubuntu1~18.04, and the bug is still there.