* [dpdk-users] Segfault while running on older CPU @ 2019-02-06 10:47 Filip Janiszewski 2019-02-14 13:04 ` Filip Janiszewski 0 siblings, 1 reply; 6+ messages in thread From: Filip Janiszewski @ 2019-02-06 10:47 UTC (permalink / raw) To: users Hi Everybody, We have one 'slightly' older machine (well, very old CPU.) in our Lab that seems to crash DPDK on every execution attempt, I was wondering if anybody encountered a similar issue and if there's a change in the DPDK config that might remedy the problem, this is the stack trace of the fault: . #0 0x000000000057bd19 in rte_cpu_get_flag_enabled () #1 0x000000000046067e in rte_acl_init () #2 0x000000000088359d in __libc_csu_init () #3 0x00007ffff6642cf5 in __libc_start_main () from /lib64/libc.so.6 #4 0x0000000000464fee in _start () . The CPU on this machine is: "Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (fam: 06, model: 3a, stepping: 09)" running with "3.10.0-862.el7.x86_64". Our builds are running fine on newest hardware, nevertheless the segfault seems a bit weird even for an unsupported CPU (some error prompt would be more friendly), any suggestion on what the problem might be? Thanks -- BR, Filip +48 666 369 823 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Segfault while running on older CPU 2019-02-06 10:47 [dpdk-users] Segfault while running on older CPU Filip Janiszewski @ 2019-02-14 13:04 ` Filip Janiszewski 2019-02-14 13:34 ` Wiles, Keith 0 siblings, 1 reply; 6+ messages in thread From: Filip Janiszewski @ 2019-02-14 13:04 UTC (permalink / raw) To: users Hi All, We've just encountered the same issue on a new server with a couple of Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz, stack trace the same: . Program received signal SIGILL, Illegal instruction. 0x0000000000557ce9 in rte_cpu_get_flag_enabled () Missing separate debuginfos, use: debuginfo-install glibc-2.17-260.el7.x86_64 libaio-0.3.109-13.el7.x86_64 libgcc-4.8.5-36.el7.x86_64 libxml2-2.9.1-6.el7_2.3.x86_64 numactl-libs-2.0.9-7.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-18.el7.x86_64 (gdb) bt #0 0x0000000000557ce9 in rte_cpu_get_flag_enabled () #1 0x000000000046076e in rte_acl_init () #2 0x000000000085f56d in __libc_csu_init () #3 0x00007ffff6636365 in __libc_start_main () from /lib64/libc.so.6 #4 0x00000000004650de in _start () . I'm building DPDK with x86_64-native-linuxapp-gcc, nothing else, plain config from DPDK. I've attempted to recompile with CONFIG_RTE_EXEC_ENV set to 'native' instead of 'linuxapp' but nothing changes. Did anybody had a similar issue? Any suggestion? Thanks Il 06/02/19 11:47, Filip Janiszewski ha scritto: > Hi Everybody, > > We have one 'slightly' older machine (well, very old CPU.) in our Lab > that seems to crash DPDK on every execution attempt, I was wondering if > anybody encountered a similar issue and if there's a change in the DPDK > config that might remedy the problem, this is the stack trace of the fault: > > . > #0 0x000000000057bd19 in rte_cpu_get_flag_enabled () > #1 0x000000000046067e in rte_acl_init () > #2 0x000000000088359d in __libc_csu_init () > #3 0x00007ffff6642cf5 in __libc_start_main () from /lib64/libc.so.6 > #4 0x0000000000464fee in _start () > . > > The CPU on this machine is: "Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz > (fam: 06, model: 3a, stepping: 09)" running with "3.10.0-862.el7.x86_64". > > Our builds are running fine on newest hardware, nevertheless the > segfault seems a bit weird even for an unsupported CPU (some error > prompt would be more friendly), any suggestion on what the problem might be? > > Thanks > -- BR, Filip +48 666 369 823 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Segfault while running on older CPU 2019-02-14 13:04 ` Filip Janiszewski @ 2019-02-14 13:34 ` Wiles, Keith 2019-02-14 14:15 ` Filip Janiszewski 0 siblings, 1 reply; 6+ messages in thread From: Wiles, Keith @ 2019-02-14 13:34 UTC (permalink / raw) To: Filip Janiszewski; +Cc: users > On Feb 14, 2019, at 7:04 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote: > > Hi All, > > We've just encountered the same issue on a new server with a couple of > Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz, stack trace the same: > What version of DPDK? > . > Program received signal SIGILL, Illegal instruction. > > 0x0000000000557ce9 in rte_cpu_get_flag_enabled () > > Missing separate debuginfos, use: debuginfo-install > glibc-2.17-260.el7.x86_64 libaio-0.3.109-13.el7.x86_64 > libgcc-4.8.5-36.el7.x86_64 libxml2-2.9.1-6.el7_2.3.x86_64 > numactl-libs-2.0.9-7.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 > zlib-1.2.7-18.el7.x86_64 > > (gdb) bt > > #0 0x0000000000557ce9 in rte_cpu_get_flag_enabled () > #1 0x000000000046076e in rte_acl_init () > #2 0x000000000085f56d in __libc_csu_init () > #3 0x00007ffff6636365 in __libc_start_main () from /lib64/libc.so.6 > #4 0x00000000004650de in _start () > . > > I'm building DPDK with x86_64-native-linuxapp-gcc, nothing else, plain > config from DPDK. I've attempted to recompile with CONFIG_RTE_EXEC_ENV > set to 'native' instead of 'linuxapp' but nothing changes. Not sure what this means it looks like you already compiled it as native. > > Did anybody had a similar issue? Any suggestion? maybe compile with -g for debug symbols. i use ‘make install T=$RTE_TARGET EXTRA_CFLAGS=“-g”’ you can also try reducing the optimization, but I do not think that is the problem. Does this happen every time, what OS version, what is the application yours or one of the examples? If testpmd works then it maybe your application. I can not tell for sure but could this be the first time this routine is called? Are you calling some thing in DPDK before rte_eal_init() is called? With -g you should them be able to location the line in DPDK where it is failing. > > Thanks > > Il 06/02/19 11:47, Filip Janiszewski ha scritto: >> Hi Everybody, >> >> We have one 'slightly' older machine (well, very old CPU.) in our Lab >> that seems to crash DPDK on every execution attempt, I was wondering if >> anybody encountered a similar issue and if there's a change in the DPDK >> config that might remedy the problem, this is the stack trace of the fault: >> >> . >> #0 0x000000000057bd19 in rte_cpu_get_flag_enabled () >> #1 0x000000000046067e in rte_acl_init () >> #2 0x000000000088359d in __libc_csu_init () >> #3 0x00007ffff6642cf5 in __libc_start_main () from /lib64/libc.so.6 >> #4 0x0000000000464fee in _start () >> . >> >> The CPU on this machine is: "Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz >> (fam: 06, model: 3a, stepping: 09)" running with "3.10.0-862.el7.x86_64". >> >> Our builds are running fine on newest hardware, nevertheless the >> segfault seems a bit weird even for an unsupported CPU (some error >> prompt would be more friendly), any suggestion on what the problem might be? >> >> Thanks >> > > -- > BR, Filip > +48 666 369 823 Regards, Keith ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Segfault while running on older CPU 2019-02-14 13:34 ` Wiles, Keith @ 2019-02-14 14:15 ` Filip Janiszewski 2019-02-14 14:46 ` Van Haaren, Harry 0 siblings, 1 reply; 6+ messages in thread From: Filip Janiszewski @ 2019-02-14 14:15 UTC (permalink / raw) To: Wiles, Keith; +Cc: users Hi, I prepared a simple test application with only a call to rte_eal_init and it crashes, DPDK version is 18.05 and apparently the problem is with the BMI2 instruction set, the faulty line is: . │0x57fec9 <rte_cpu_get_flag_enabled+89> shrx %eax,%edx,%eax . I'm attempting to disable those instructions by setting march to corei7 which according to gcc does not include BMI2, but still when opening the asm list of the binary can see that instruction. If I understand correctly how the DPDK building process works then I should be able to set this 'corei7' switch by just changing mk/machine/default/rte.vars.mk and then setting CONFIG_RTE_MACHINE="default" in the x86_64-native-linuxapp-gcc/.config file, but I'm not sure whether it's picking it up while building (make T=x86_64-native-linuxapp-gcc DESTDIR=. -j28). Il 14/02/19 14:34, Wiles, Keith ha scritto: > > >> On Feb 14, 2019, at 7:04 AM, Filip Janiszewski <contact@filipjaniszewski.com> wrote: >> >> Hi All, >> >> We've just encountered the same issue on a new server with a couple of >> Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz, stack trace the same: >> > > What version of DPDK? >> . >> Program received signal SIGILL, Illegal instruction. >> >> 0x0000000000557ce9 in rte_cpu_get_flag_enabled () >> >> Missing separate debuginfos, use: debuginfo-install >> glibc-2.17-260.el7.x86_64 libaio-0.3.109-13.el7.x86_64 >> libgcc-4.8.5-36.el7.x86_64 libxml2-2.9.1-6.el7_2.3.x86_64 >> numactl-libs-2.0.9-7.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 >> zlib-1.2.7-18.el7.x86_64 >> >> (gdb) bt >> >> #0 0x0000000000557ce9 in rte_cpu_get_flag_enabled () >> #1 0x000000000046076e in rte_acl_init () >> #2 0x000000000085f56d in __libc_csu_init () >> #3 0x00007ffff6636365 in __libc_start_main () from /lib64/libc.so.6 >> #4 0x00000000004650de in _start () >> . >> >> I'm building DPDK with x86_64-native-linuxapp-gcc, nothing else, plain >> config from DPDK. I've attempted to recompile with CONFIG_RTE_EXEC_ENV >> set to 'native' instead of 'linuxapp' but nothing changes. > > Not sure what this means it looks like you already compiled it as native. >> >> Did anybody had a similar issue? Any suggestion? > > maybe compile with -g for debug symbols. i use ‘make install T=$RTE_TARGET EXTRA_CFLAGS=“-g”’ you can also try reducing the optimization, but I do not think that is the problem. > > Does this happen every time, what OS version, what is the application yours or one of the examples? > If testpmd works then it maybe your application. I can not tell for sure but could this be the first time this routine is called? > Are you calling some thing in DPDK before rte_eal_init() is called? > > With -g you should them be able to location the line in DPDK where it is failing. >> >> Thanks >> >> Il 06/02/19 11:47, Filip Janiszewski ha scritto: >>> Hi Everybody, >>> >>> We have one 'slightly' older machine (well, very old CPU.) in our Lab >>> that seems to crash DPDK on every execution attempt, I was wondering if >>> anybody encountered a similar issue and if there's a change in the DPDK >>> config that might remedy the problem, this is the stack trace of the fault: >>> >>> . >>> #0 0x000000000057bd19 in rte_cpu_get_flag_enabled () >>> #1 0x000000000046067e in rte_acl_init () >>> #2 0x000000000088359d in __libc_csu_init () >>> #3 0x00007ffff6642cf5 in __libc_start_main () from /lib64/libc.so.6 >>> #4 0x0000000000464fee in _start () >>> . >>> >>> The CPU on this machine is: "Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz >>> (fam: 06, model: 3a, stepping: 09)" running with "3.10.0-862.el7.x86_64". >>> >>> Our builds are running fine on newest hardware, nevertheless the >>> segfault seems a bit weird even for an unsupported CPU (some error >>> prompt would be more friendly), any suggestion on what the problem might be? >>> >>> Thanks >>> >> >> -- >> BR, Filip >> +48 666 369 823 > > Regards, > Keith > -- BR, Filip +48 666 369 823 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Segfault while running on older CPU 2019-02-14 14:15 ` Filip Janiszewski @ 2019-02-14 14:46 ` Van Haaren, Harry 2019-02-15 5:45 ` Filip Janiszewski 0 siblings, 1 reply; 6+ messages in thread From: Van Haaren, Harry @ 2019-02-14 14:46 UTC (permalink / raw) To: Filip Janiszewski, Wiles, Keith; +Cc: users > -----Original Message----- > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Filip Janiszewski > Sent: Thursday, February 14, 2019 2:15 PM > To: Wiles, Keith <keith.wiles@intel.com> > Cc: users@dpdk.org > Subject: Re: [dpdk-users] Segfault while running on older CPU > > Hi, > > I prepared a simple test application with only a call to rte_eal_init > and it crashes, DPDK version is 18.05 and apparently the problem is with > the BMI2 instruction set, the faulty line is: > > . > │0x57fec9 <rte_cpu_get_flag_enabled+89> shrx %eax,%edx,%eax > . > > I'm attempting to disable those instructions by setting march to corei7 > which according to gcc does not include BMI2, but still when opening the > asm list of the binary can see that instruction. > > If I understand correctly how the DPDK building process works then I > should be able to set this 'corei7' switch by just changing > mk/machine/default/rte.vars.mk and then setting > CONFIG_RTE_MACHINE="default" in the x86_64-native-linuxapp-gcc/.config > file, but I'm not sure whether it's picking it up while building (make > T=x86_64-native-linuxapp-gcc DESTDIR=. -j28). Possibly related crashing bugs; DPDK rte memcpy / binutils 2.30: https://bugs.dpdk.org/show_bug.cgi?id=97#c48 Gcc bug report; https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88096 And a related/duplicate: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86735 Please check if you're using binutils 2.30, and if so try latest DPDK code which has merged a fix for disabling certain features as a workaround. As Keith mentioned, what OS / Distro / Release, and what version of gcc / binutils is in use? Regards, -Harry <snip previous discussion> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Segfault while running on older CPU 2019-02-14 14:46 ` Van Haaren, Harry @ 2019-02-15 5:45 ` Filip Janiszewski 0 siblings, 0 replies; 6+ messages in thread From: Filip Janiszewski @ 2019-02-15 5:45 UTC (permalink / raw) To: Van Haaren, Harry, Wiles, Keith; +Cc: users Il 14/02/19 15:46, Van Haaren, Harry ha scritto: >> -----Original Message----- >> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Filip Janiszewski >> Sent: Thursday, February 14, 2019 2:15 PM >> To: Wiles, Keith <keith.wiles@intel.com> >> Cc: users@dpdk.org >> Subject: Re: [dpdk-users] Segfault while running on older CPU >> >> Hi, >> >> I prepared a simple test application with only a call to rte_eal_init >> and it crashes, DPDK version is 18.05 and apparently the problem is with >> the BMI2 instruction set, the faulty line is: >> >> . >> │0x57fec9 <rte_cpu_get_flag_enabled+89> shrx %eax,%edx,%eax >> . >> >> I'm attempting to disable those instructions by setting march to corei7 >> which according to gcc does not include BMI2, but still when opening the >> asm list of the binary can see that instruction. >> >> If I understand correctly how the DPDK building process works then I >> should be able to set this 'corei7' switch by just changing >> mk/machine/default/rte.vars.mk and then setting >> CONFIG_RTE_MACHINE="default" in the x86_64-native-linuxapp-gcc/.config >> file, but I'm not sure whether it's picking it up while building (make >> T=x86_64-native-linuxapp-gcc DESTDIR=. -j28). > > > Possibly related crashing bugs; > > DPDK rte memcpy / binutils 2.30: > https://bugs.dpdk.org/show_bug.cgi?id=97#c48 > > Gcc bug report; > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88096 > And a related/duplicate: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86735 > > > Please check if you're using binutils 2.30, and if so try latest DPDK code which has > merged a fix for disabling certain features as a workaround. > I'm running version 2.27, will try to upgrade to the latest DPDK code and check if things are getting fixed. > As Keith mentioned, what OS / Distro / Release, and what version of gcc / binutils is in use? > OS CentOS 7.5.1804 with 3.10.0-862.14.4.el7.x86_64, gcc 4.8.5 (20150623) and binutils 2.27. Thanks. > > Regards, -Harry > > > <snip previous discussion> > -- BR, Filip +48 666 369 823 ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-02-15 5:45 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-02-06 10:47 [dpdk-users] Segfault while running on older CPU Filip Janiszewski 2019-02-14 13:04 ` Filip Janiszewski 2019-02-14 13:34 ` Wiles, Keith 2019-02-14 14:15 ` Filip Janiszewski 2019-02-14 14:46 ` Van Haaren, Harry 2019-02-15 5:45 ` Filip Janiszewski
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).