Re: [dpdk-dev] Running DPDK Binaries on a different Target

["Ananyev, Konstantin" <konstantin.ananyev@intel.com>]

Hi Venkat,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Venkat Thummala
> Sent: Thursday, April 09, 2015 10:07 AM
> To: Neil Horman
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Running DPDK Binaries on a different Target
> 
> I have the following ACL rule configured in the ACL Table.
> 
> Priority - 20
> SIP - 0
> SIP MASK - 0
> DIP - 0
> DIP MASK - 0
> PROTO - 17 [UDP]
> PROTO MASK - 0xFF
> SPORT - 0
> SPORT RANGE - 0xFFFF
> DPORT - 0
> DPORT RANGE - 0xFFFF
> 
> And pumping UDP traffic.
> 
> On Machine 1, it always HITS the rule as expected.
> 
> But On Machine 2, the same traffic never HITS the rule.
> 
> I have tried this with both SCALAR and SSE classification algorithms, but
> the result is same.

So did I get it right:
You build a binary with RTE_MACHINE="default" on a box with AVX2 support.
Then run that binary on 
a) native box (machine with AVX2 support)
b) machine without AVX2 support

For the same input and same rules, same binary you got different results for a) and b).  
If that is the case, then there is something wrong here, either within our code or with the compiler you are using.
Could you provide steps to reproduce it?
I did a quick test with l3fwd-acl for that case, 'default' binary that I built on HSW board,
works as expected on IVB box for me. 

Build box: HSW (E3-1285), gcc 4.8.3 
SUT: IVB 2.80GHz, gcc 4.8.2

# cat acl_ipv4.rule1
R0.0.0.0/0 0.0.0.0/0 0 : 0xffff 0 : 0xffff 17/0xff 0
# cat acl_ipv6.rule1
R0:0:0:0:0:0:0:0/0 0:0:0:0:0:0:0:0/0 0 : 0xffff 0 : 0xffff 17/0xff 0

./dpdk.org-acl/examples/l3fwd-acl/x86_64-default-linuxapp-gcc/l3fwd-acl --lcores=3 -n 4 --socket-mem=1024,0 -w 0000:04:00.1 -- -p 1 --config "(0, 0, 3)" --rule_ipv4=./acl_ipv4.rule1 --rule_ipv6=acl_ipv6.rule1 -P

Forwards UDP packets, drops all others.

Anything else in your setup, that I am missing here?

Konstantin

> 
> Could some one, please help me on this?
> 
> DPDK Version:
> ===========
> 2.0
> 
> Machine 1 CPU INFO:
> ================
> vendor_id    : GenuineIntel
> cpu family    : 6
> model        : 69
> model name    : Intel(R) Core(TM) i5-4200U CPU @ 1.60GHz
> stepping    : 1
> microcode    : 0x16
> cpu MHz        : 759.000
> cache size    : 3072 KB
> physical id    : 0
> siblings    : 4
> core id        : 1
> cpu cores    : 2
> apicid        : 3
> initial apicid    : 3
> fpu        : yes
> fpu_exception    : yes
> cpuid level    : 13
> wp        : yes
> flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
> pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor
> ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt
> tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb
> xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase
> tsc_adjust bmi1 avx2 smep bmi2 erms invpcid
> bogomips    : 4589.70
> clflush size    : 64
> cache_alignment    : 64
> address sizes    : 39 bits physical, 48 bits virtual
> power management:
> 
> MACHINE 1 OS:
> =============
> Linux vthummala-HP-Pavilion-15-Notebook-PC 3.13.0-48-generic #80-Ubuntu SMP
> Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
> 
> MACHINE 2 CPU INFO:
> ==================
> vendor_id    : GenuineIntel
> cpu family    : 6
> model        : 62
> model name    : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
> stepping    : 4
> microcode    : 0x416
> cpu MHz        : 2599.890
> cache size    : 20480 KB
> physical id    : 1
> siblings    : 16
> core id        : 7
> cpu cores    : 8
> apicid        : 47
> initial apicid    : 47
> fpu        : yes
> fpu_exception    : yes
> cpuid level    : 13
> wp        : yes
> flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
> pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor
> ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic
> popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat
> xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep
> erms
> bogomips    : 5201.35
> clflush size    : 64
> cache_alignment    : 64
> address sizes    : 46 bits physical, 48 bits virtual
> power management:
> 
> MACHINE 2 OS:
> =============
> Linux vthummala-PowerEdge-R720 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10
> 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
> 
> 
> On 8 April 2015 at 17:17, Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Wed, Apr 08, 2015 at 02:03:05PM +0530, Venkat Thummala wrote:
> > > Hi Neil,
> > >
> > > Thanks for the quick response.
> > >
> > > The issue got fixed by setting CONFIG_RTE_MACHINE to "default".
> > >
> > Default is the lowest common demoninator system that dpdk supports (core2
> > duo).
> > So if you're system is superior to that processor, you're ok.
> >
> > > Though, this fixed the CRASH issue, I am observing inconsistent behavior
> > > with ACL tables.
> > >
> > > With the same traffic and same ACL rule, on ONE machine the ACL rule is
> > > being HIT, where as on the OTHER machine the rule never HITS.
> > >
> > > Is this because of the "default" machine option?
> > >
> > Possibly, the machine type changes the method of lookup for ACL rules,
> > though it
> > should remain consistent in terms of which rules are hit.
> > > Regards
> > > Venkat
> > >
> > >
> > > On 7 April 2015 at 20:05, Neil Horman <nhorman@tuxdriver.com> wrote:
> > >
> > > > On Tue, Apr 07, 2015 at 05:30:15PM +0530, Venkat Thummala wrote:
> > > > > Attaching the CPU Info.
> > > > >
> > > > > On 7 April 2015 at 17:28, Venkat Thummala <
> > > > venkat.thummala.1978@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I have built a DPDK application [based on version 2.0] and run on
> > the
> > > > > > native machine successfully.
> > > > > >
> > > > > > I have tried running the binary on a different machine, but it
> > > > resulted in
> > > > > > a CRASH with the following back trace.
> > > > > >
> > > > > > Please find the CPU info of the machines from the attachment.
> > > > > >
> > > > > > cpuinfo-1 - Native Machine
> > > > > > cpuinfo-2 - Non Native Machine
> > > > > >
> > > > > > Could someone please help me in understanding the issue here and
> > > > making it
> > > > > > work?
> > > > > >
> > > > > > Regards
> > > > > > Venkat
> > > > > >
> > > > > > Program terminated with signal SIGILL, Illegal instruction.
> > > > > > #0  0x00000000004209f2 in rte_cpu_get_flag_enabled
> > (feature=2147483656,
> > > > > > feature@entry=RTE_CPUFLAG_EM64T)
> > > > > >     at
> > > > > >
> > > >
> > /home/vthummala/src/vwlc/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_cpuflags.h:303
> > > > > > 303        return (regs[feat->reg] >> feat->bit) & 1;
> > > >
> > > > Looks like this somehow compiled into an instruction that the native
> > system
> > > > understands but the system you're running on doesn't.  disassemble the
> > > > code here
> > > > to see what instruction that is to confirm, then you either need to:
> > > >
> > > > 1) Change the machine you're compiling for to be more specific to your
> > > > target
> > > > system
> > > >
> > > > or
> > > >
> > > > 2) Understand that the hardware you're targeting doesn't support dpdk
> > > >
> > > > Neil
> > > >
> > > > > > (gdb) bt
> > > > > > #0  0x00000000004209f2 in rte_cpu_get_flag_enabled
> > (feature=2147483656,
> > > > > > feature@entry=RTE_CPUFLAG_EM64T)
> > > > > >     at
> > > > > >
> > > >
> > /home/vthummala/src/vwlc/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_cpuflags.h:303
> > > > > > #1  0x0000000000420a1b in rte_hash_crc_set_alg (alg=6 '\006')
> > > > > >     at
> > > > > >
> > > >
> > /home/vthummala/src/vwlc/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_hash_crc.h:429
> > > > > > #2  rte_hash_crc_init_alg () at
> > > > > >
> > > >
> > /home/vthummala/src/vwlc/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_hash_crc.h:445
> > > > > > #3  0x000000000054968d in __libc_csu_init ()
> > > > > > #4  0x00007fc3ca474e55 in group_nodes_into_DFAstates
> > > > > > (dests_ch=0x940f507ab10ff0c8, dests_node=0x4a8b44de74c084c0,
> > > > > > state=0x4420528b48a8ebc9,
> > > > > >     dfa=<optimized out>) at regexec.c:3614
> > > > > > #5  build_trtable (dfa=0x840fc139c0014468,
> > state=0x4420528b48a8ebc9) at
> > > > > > regexec.c:3354
> > > > > > #6  0x41d589495541f689 in ?? ()
> > > > > > #7  0x00251630258d4c54 in ?? ()
> > > > > > #8  0x002517302d8d4855 in ?? ()
> > > > > > #9  0xc148db31e5294c53 in ?? ()
> > > > > > #10 0x65e808ec834803fd in ?? ()
> > > > > > #11 0x1e74ed8548ffed48 in ?? ()
> > > > > > #12 0x0000000000841f0f in ?? ()
> > > > > >
> > > > >
> > > >
> >