Re: [dpdk-dev] [PATCH] acl: If build does not support sse4.2, emulate missing instructions with C code

["Ananyev, Konstantin" <konstantin.ananyev@intel.com>]

> -----Original Message-----
> From: Neil Horman [mailto:nhorman@tuxdriver.com]
> Sent: Wednesday, August 06, 2014 1:12 PM
> To: Ananyev, Konstantin
> Cc: dev@dpdk.org; Thomas Monjalon; Richardson, Bruce
> Subject: Re: [PATCH] acl: If build does not support sse4.2, emulate missing instructions with C code
> 
> On Wed, Aug 06, 2014 at 10:52:09AM +0000, Ananyev, Konstantin wrote:
> >
> > > > For ACL there is a simple UT, that could be run as:
> > > > ./x86_64-native-linuxapp-gcc/app/test ...
> > > > RTE>>acl_autotest
> > > > It takes just few seconds to run.
> > >
> > > It doesn't seem to work properly, at least not on any of my systems.  With a
> > > system allocation 100 pages to hugepage memory:
> > >
> > > [root@hmsreliant app]# ./test -c 0x3 -n 2
> > > EAL: Detected lcore 0 as core 0 on socket 0
> > > EAL: Detected lcore 1 as core 1 on socket 0
> > > EAL: Detected lcore 2 as core 2 on socket 0
> > > EAL: Detected lcore 3 as core 3 on socket 0
> > > EAL: Detected lcore 4 as core 0 on socket 0
> > > EAL: Detected lcore 5 as core 1 on socket 0
> > > EAL: Detected lcore 6 as core 2 on socket 0
> > > EAL: Detected lcore 7 as core 3 on socket 0
> > > EAL: Support maximum 64 logical core(s) by configuration.
> > > EAL: Detected 8 lcore(s)
> > > EAL:   cannot open VFIO container, error 2 (No such file or directory)
> > > EAL: VFIO support could not be initialized
> > > EAL: Setting up memory...
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7fef07000000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7fef06c00000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x400000 bytes
> > > EAL: Virtual area found at 0x7fef06600000 (size = 0x400000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7fef06200000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7fef05e00000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x600000 bytes
> > > EAL: Virtual area found at 0x7fef05600000 (size = 0x600000)
> > > EAL: Ask a virtual area of 0x600000 bytes
> > > EAL: Virtual area found at 0x7fef04e00000 (size = 0x600000)
> > > EAL: Ask a virtual area of 0x800000 bytes
> > > EAL: Virtual area found at 0x7fef04400000 (size = 0x800000)
> > > EAL: Ask a virtual area of 0x400000 bytes
> > > EAL: Virtual area found at 0x7fef03e00000 (size = 0x400000)
> > > EAL: Ask a virtual area of 0xa00000 bytes
> > > EAL: Virtual area found at 0x7fef03200000 (size = 0xa00000)
> > > EAL: Ask a virtual area of 0x400000 bytes
> > > EAL: Virtual area found at 0x7fef02c00000 (size = 0x400000)
> > > EAL: Ask a virtual area of 0x400000 bytes
> > > EAL: Virtual area found at 0x7fef02600000 (size = 0x400000)
> > > EAL: Ask a virtual area of 0x1800000 bytes
> > > EAL: Virtual area found at 0x7fef00c00000 (size = 0x1800000)
> > > EAL: Ask a virtual area of 0x1200000 bytes
> > > EAL: Virtual area found at 0x7feeff800000 (size = 0x1200000)
> > > EAL: Ask a virtual area of 0x2600000 bytes
> > > EAL: Virtual area found at 0x7feefd000000 (size = 0x2600000)
> > > EAL: Ask a virtual area of 0xc00000 bytes
> > > EAL: Virtual area found at 0x7feefc200000 (size = 0xc00000)
> > > EAL: Ask a virtual area of 0x2200000 bytes
> > > EAL: Virtual area found at 0x7feef9e00000 (size = 0x2200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7feef9a00000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7feef9600000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7feef9200000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7feef8e00000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7feef8a00000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7feef8600000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x200000 bytes
> > > EAL: Virtual area found at 0x7feef8200000 (size = 0x200000)
> > > EAL: Ask a virtual area of 0x600000 bytes
> > > EAL: Virtual area found at 0x7feef7a00000 (size = 0x600000)
> > > EAL: Requesting 100 pages of size 2MB from socket 0
> > > EAL: TSC frequency is ~3392297 KHz
> > > EAL: Master core 0 is ready (tid=73cf880)
> > > EAL: Core 1 is ready (tid=f71fe700)
> > > APP: HPET is not enabled, using TSC as default timer
> > > RTE>>acl_autotest
> > > ACL: allocation of 25166720 bytes on socket 9 for ACL_acl_ctx failed
> > >
> > > This hangs forever (well, at least 30 minutes, but thats sufficiently
> > > close to forever for me to wait).
> > >
> >
> > Ok that's unusual.
> > Never seen before.
> > I suppose that happens with unmodified dpdk 1.7?
> If I build unmodified dpdk 1.7 (which I can only build for the native platform),
> it works, however it doesn't work when built with modifications for the default
> platform, but see below.
> 
> > I do run acl_autotest with at least 256M of huge-pages as ACL is quite memory consuming  and, as I remember, requires at least
> 2X32M areas of contiguous hugepages.
> > But in that case (not enough hugepages) it should just fail with something like:
> > ACL: allocation of 25953152 bytes on socket -1 for ACL_acl_ctx failed
> > Any other details, so I can try to reproduce it:
> > arch you build/run it on?
> x86_64 default build (has to be to test the new code).
> 
> > amount of free memory in the system?
> Its an 8G workstation, should have most of that free (haven't checked yet).

That's should be enough.
I run it on 8GB workstation with ~3GB reported as 'free' by linux.

> 
> > Wonder what pstack says?
> It says that we're stuck in the eal library, before we've touched the ACL
> library it seems, so I'm not sure whats going on here:
> #0  0x0000003d11ef53c3 in epoll_wait () from /lib64/libc.so.6
> #1  0x00000000004d315a in eal_intr_thread_main ()
> #2  0x0000003d12607f33 in start_thread () from /lib64/libpthread.so.0
> #3  0x0000003d11ef4ded in clone () from /lib64/libc.so.6
> Thread 2 (Thread 0x7fee73ffe700 (LWP 14529)):
> #0  0x0000003d1260e87d in read () from /lib64/libpthread.so.0
> #1  0x00000000004ced0a in eal_thread_loop ()
> #2  0x0000003d12607f33 in start_thread () from /lib64/libpthread.so.0
> #3  0x0000003d11ef4ded in clone () from /lib64/libc.so.6
> Thread 1 (Thread 0x7fee84100880 (LWP 14527)):
> #0  0x000000000041adde in acl_match_check_x2.constprop ()
> #1  0x000000000041b1af in search_sse_2 ()
> #2  0x00000000004baab9 in rte_acl_classify ()
> #3  0x000000000047d12d in test_acl ()
> #4  0x000000000041c535 in cmd_autotest_parsed ()
> #5  0x00000000004d8503 in cmdline_parse ()
> #6  0x00000000004d7600 in cmdline_valid_buffer ()
> #7  0x00000000004da43a in rdline_char_in ()
> #8  0x00000000004d7809 in cmdline_interact ()
> #9  0x000000000041bccb in main ()

Ok, seems that we are in the indefinite loop.
Probably here:
acl_match_check_x2 
{
  ...
  while (!MM_TESTZ(temp, temp)) {..}
}
 
Did I get it right, that it works ok with x86_64-default-linuxapp-gcc binary, but hangs with modified x86_64-default-linuxapp-gcc?
If yes, then I wonder what modifications you made?

Konstantin