DPDK patches and discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
Cc: "Yigit, Ferruh" <ferruh.yigit@intel.com>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"stable@dpdk.org" <stable@dpdk.org>,
	"Wiles, Keith" <keith.wiles@intel.com>,
	Yongseok Koh <yskoh@mellanox.com>, "dev@dpdk.org" <dev@dpdk.org>,
	Shahaf Shuler <shahafs@mellanox.com>,
	"Burakov, Anatoly" <anatoly.burakov@intel.com>,
	"justin.parus@microsoft.com" <justin.parus@microsoft.com>,
	"christian.ehrhardt@canonical.com"
	<christian.ehrhardt@canonical.com>,
	"david.coronel@canonical.com" <david.coronel@canonical.com>,
	"josh.powers@canonical.com" <josh.powers@canonical.com>,
	"jay.vosburgh@canonical.com" <jay.vosburgh@canonical.com>,
	"dan.streetman@canonical.com" <dan.streetman@canonical.com>
Subject: Re: [dpdk-dev] [dpdk-stable] AVX512 bug on SkyLake
Date: Sun, 11 Nov 2018 19:15:02 +0100	[thread overview]
Message-ID: <14171327.rkP5k0YJWv@xps> (raw)
In-Reply-To: <2601191342CEEE43887BDE71AB977258010CE46DE0@IRSMSX106.ger.corp.intel.com>

11/11/2018 15:15, Ananyev, Konstantin:
> Hi Thomas,
> 
> > Below is my conclusion for this bug.
> > An expert of x86 is required to follow-up.
> > 
> > Summary:
> > 	- CPU: Intel Skylake
> > 	- Linux environment: Ubuntu 18.04
> > 	- Compiler: GCC 7 or 8
> > 	- Scenario: testpmd crashes when it starts forwarding
> > 	- Behaviour: AVX2 version of rte_memcpy() fails if optimized for AVX512
> > 	- Context: inline rte_memcpy() is called from
> > 			inline rte_mempool_put_bulk(), called from
> > 			mlx5_tx_complete() (inline or not)
> > 	- Analysis: AVX512 optimization changes vmovdqu to vmovdqu8
> > 
> > Latest status can be found in Bugzilla:
> > 	https://bugs.dpdk.org/show_bug.cgi?id=97#c35
> 
> 
> Looking at dissamled output from the bug report, it seems that the
> problem is not in vmovdqu8 instruction itself, but in the wrong offsets
> generated by the compiler:
> 
>    vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x2]
>    vinserti128 ymm0,ymm0,XMMWORD PTR [rax*8+0x30],0x1
>     vmovups XMMWORD PTR [rsi+0x20],xmm0
>     vextracti128 XMMWORD PTR [rsi+0x30],ymm0,0x1
>     vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x4]
>     vinserti128 ymm0,ymm0,XMMWORD PTR [rax*8+0x50],0x1
>     vmovups XMMWORD PTR [rsi+0x40],xmm0
>     vextracti128 XMMWORD PTR [rsi+0x50],ymm0,0x1
>     vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x6]
> 
> Should be:
> vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x20]
> I think.
> 
> Same for next two offsets: 0x4 and 0x6 respectively should be 0x40 and 0x60.

Yes, you're right, I missed it, thank you!

The full diff is below:

--- bad-avx512-enabled
+++ good-avx512-disabled
-    vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x0]
+    vmovdqu xmm0,XMMWORD PTR [rax*8+0x0]
     vinserti128 ymm0,ymm0,XMMWORD PTR [rax*8+0x10],0x1
     vmovups XMMWORD PTR [rsi],xmm0
     vextracti128 XMMWORD PTR [rsi+0x10],ymm0,0x1
-    vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x2]
+    vmovdqu xmm0,XMMWORD PTR [rax*8+0x20]
     vinserti128 ymm0,ymm0,XMMWORD PTR [rax*8+0x30],0x1
     vmovups XMMWORD PTR [rsi+0x20],xmm0
     vextracti128 XMMWORD PTR [rsi+0x30],ymm0,0x1
-    vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x4]
+    vmovdqu xmm0,XMMWORD PTR [rax*8+0x40]
     vinserti128 ymm0,ymm0,XMMWORD PTR [rax*8+0x50],0x1
     vmovups XMMWORD PTR [rsi+0x40],xmm0
     vextracti128 XMMWORD PTR [rsi+0x50],ymm0,0x1
-    vmovdqu8 xmm0,XMMWORD PTR [rax*8+0x6]
+    vmovdqu xmm0,XMMWORD PTR [rax*8+0x60]
     vinserti128 ymm0,ymm0,XMMWORD PTR [rax*8+0x70],0x1
     vmovups XMMWORD PTR [rsi+0x60],xmm0
     vextracti128 XMMWORD PTR [rsi+0x70],ymm0,0x1

> Not sure what causing compiler behaves that way.
> BTW, looking though testpmd objdump output - it seems that only mlx5 driver
> exhibits such problem (I didn't enable mlx4 actually, probably same problem here).
> Which looks a bit weird to me.

Yes it's weird. I don't see how the mlx5 code could influence
the compiler to generate this bad code in AVX512 mode.

  reply	other threads:[~2018-11-11 18:15 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-23 21:23 [dpdk-dev] [PATCH] build: disable compiler AVX512F support Yongseok Koh
2018-11-01 23:11 ` Thomas Monjalon
2018-11-02 12:42 ` [dpdk-dev] [dpdk-stable] " Ferruh Yigit
2018-11-02 13:48   ` Ferruh Yigit
2018-11-02 20:59     ` Yongseok Koh
2018-11-02 21:46       ` Ferruh Yigit
2018-11-02 23:31         ` Yongseok Koh
2018-11-02 21:04 ` [dpdk-dev] [PATCH v2] " Yongseok Koh
2018-11-05 14:06   ` Wiles, Keith
2018-11-06 21:30     ` Yongseok Koh
2018-11-07  9:04       ` Wiles, Keith
2018-11-08 15:59         ` [dpdk-dev] AVX512 bug on SkyLake Thomas Monjalon
2018-11-08 17:21           ` Ferruh Yigit
2018-11-08 23:01             ` Yongseok Koh
2018-11-09  6:27               ` Christian Ehrhardt
2018-11-09  9:49                 ` Ferruh Yigit
2018-11-09 11:35                   ` Thomas Monjalon
2018-11-09 10:03               ` Ferruh Yigit
2018-11-09 13:17                 ` [dpdk-dev] [dpdk-stable] " Thomas Monjalon
2018-11-09 14:27                   ` Thomas Monjalon
2018-11-09 20:06                     ` Ferruh Yigit
2018-11-09 18:46           ` [dpdk-dev] " Stephen Hemminger
2018-11-10  2:13           ` [dpdk-dev] [dpdk-stable] " Thomas Monjalon
2018-11-11 14:15             ` Ananyev, Konstantin
2018-11-11 18:15               ` Thomas Monjalon [this message]
2018-11-12  9:09                 ` Christian Ehrhardt
2018-11-12  9:21                   ` Thomas Monjalon
2018-11-12  9:26                 ` Ananyev, Konstantin
2018-11-03  1:06 ` [dpdk-dev] [PATCH v3] build: disable gcc AVX512F support Yongseok Koh
2018-11-04 20:56   ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=14171327.rkP5k0YJWv@xps \
    --to=thomas@monjalon.net \
    --cc=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=christian.ehrhardt@canonical.com \
    --cc=dan.streetman@canonical.com \
    --cc=david.coronel@canonical.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=jay.vosburgh@canonical.com \
    --cc=josh.powers@canonical.com \
    --cc=justin.parus@microsoft.com \
    --cc=keith.wiles@intel.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=shahafs@mellanox.com \
    --cc=stable@dpdk.org \
    --cc=yskoh@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).