From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from nm6-vm1.bullet.mail.ne1.yahoo.com (nm6-vm1.bullet.mail.ne1.yahoo.com [98.138.91.71]) by dpdk.org (Postfix) with ESMTP id 7A8198DB1 for ; Fri, 21 Aug 2015 17:13:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1440170024; bh=q2hu0Y/UZXGPl6dp/cxStXyUMj9Ac1b7VLYU8IQsZ3g=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:From:Subject; b=YlO5suQkPaSQzKnWV/34Nhvv5I5SZ/edOMhAy4aGHS/lc78O9ou4DWTyAfrVG0Lu8j/Q30RVv1iPU+fwUbQS8D5lExxEmUlVd4jI7b+bZlUGUuORF9/Cg3M8VwQiF/evKU7hHvJ2bnEXhiF+nB1Lh1d54e1gvUgmP3g6MzCJsN+2d0qdwbhHoZ51KSZB8Ez/b4j3OyzILrldclBzqG6MDqdWThv6zFZB4Tlh31rA15BVHsU3Yiet7+PzF2WFIUkjuWSHEJEOHAB0NrIZG/6PV0lxkv3FcjQxomTt2N+gCzVUlwp/kikNi1U4MUS+GP4+p0ZV9ALR2/6iElwWCm+TTA== Received: from [98.138.100.114] by nm6.bullet.mail.ne1.yahoo.com with NNFMP; 21 Aug 2015 15:13:44 -0000 Received: from [98.138.89.249] by tm105.bullet.mail.ne1.yahoo.com with NNFMP; 21 Aug 2015 15:13:44 -0000 Received: from [127.0.0.1] by omp1041.mail.ne1.yahoo.com with NNFMP; 21 Aug 2015 15:13:44 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 535510.95413.bm@omp1041.mail.ne1.yahoo.com X-YMail-OSG: W37Q6FkVM1muXk4LMQrWSCFrnOeM6N1NAEX2zq8zTA5oUQJBVXoAAxthYc3whPg SsYVCW2Z9zEuGP2obrR..OSQhKm1zLlXeEaNnOZ_j.TomGT6g57U0apjrleAfIbtNW6S0nPIF0UV woi2wroubHngQdlaeJcNg2nVGi6KD1kBqCAnf8C.eY0KLPvtZyNmfgt3dpvBf4UK5EVNWRku8VoA _LYozhwH5vyR5srVAfMOAZGJZr3WIXxZbgnybABXDuVhilNCmMMVNeGUX_D2fWQdVEbi5ciqUuyW KZkOHaXt.lU1forKa29IxbX919siFkswDdnAOQqVzSQmSn_gdoQb2WCPg9wfrq0VasybtN0c.E6V qK7MsnGAHDdIX9EHgF4pxvU02BZrlnJ3g_ioIaWEWKl8WrgBaUd2tTK2I1VO.nMslplZxmmnStzU GzJTeztZNtUfIjk0UWX9NtxDtKziwFwGfH.OFvWb0p8P2lN0AdfW6u3WKzffFdC0YzeYdeiaU.Sl hiXqI_zw- Received: by 98.138.105.192; Fri, 21 Aug 2015 15:13:44 +0000 Date: Fri, 21 Aug 2015 15:13:40 +0000 (UTC) From: "Keith E. Fleming" To: Bruce Richardson Message-ID: <590738896.8600735.1440170020765.JavaMail.yahoo@mail.yahoo.com> In-Reply-To: <20150821105432.GA3240@bricha3-MOBL3> References: <20150821105432.GA3240@bricha3-MOBL3> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] DPDK 2.1.0 build error: inlining failed in call to always_inline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: "Keith E. Fleming" List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 15:13:46 -0000 Hi Bruce, It is Fedora 21 [root@neutron dpdk-2.1.0]# cat /proc/cpuinfo | more processor=C2=A0=C2=A0=C2=A0 : 0 vendor_id=C2=A0=C2=A0=C2=A0 : AuthenticAMD cpu family=C2=A0=C2=A0=C2=A0 : 16 model=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 : 10 model name=C2=A0=C2=A0=C2=A0 : AMD Phenom(tm) II X6 1055T Processor stepping=C2=A0=C2=A0=C2=A0 : 0 microcode=C2=A0=C2=A0=C2=A0 : 0x10000dc cpu MHz=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 : 800.000 cache size=C2=A0=C2=A0=C2=A0 : 512 KB physical id=C2=A0=C2=A0=C2=A0 : 0 siblings=C2=A0=C2=A0=C2=A0 : 6 core id=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 : 0 cpu cores=C2=A0=C2=A0=C2=A0 : 6 apicid=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 : 0 initial apicid=C2=A0=C2=A0=C2=A0 : 0 fpu=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 : yes fpu_exception=C2=A0=C2=A0=C2=A0 : yes cpuid level=C2=A0=C2=A0=C2=A0 : 6 wp=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 : yes flags=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0 : fpu vme de pse tsc msr pae mce= cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht sysc= all nx mmxe xt fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl non= stop_tsc extd_apicid aperfmperf pni monitor cx16 popcnt lah f_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch = osvw ibs skinit wdt cpb hw_pstate npt lbrv svm_lock nrip_sa ve pausefilter vmmcall [root@neutron dpdk-2.1.0]# uname -a Linux neutron 4.1.5-100.fc21.x86_64 #1 SMP Tue Aug 11 00:24:23 UTC 2015 x86= _64 x86_64 x86_64 GNU/Linux [root@neutron dpdk-2.1.0]# gcc -v Using built-in specs. COLLECT_GCC=3Dgcc COLLECT_LTO_WRAPPER=3D/usr/libexec/gcc/x86_64-redhat-linux/4.9.2/lto-wrappe= r Target: x86_64-redhat-linux Configured with: ../configure --prefix=3D/usr --mandir=3D/usr/share/man --i= nfodir=3D/usr/share/info --with-bugurl=3Dhttp://bugzilla.redhat.com/bugzill= a --enable-bootstrap --enable-shared --enable-threads=3Dposix --enable-chec= king=3Drelease --enable-multilib --with-system-zlib --enable-__cxa_atexit -= -disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-bu= ild-id --with-linker-hash-style=3Dgnu --enable-languages=3Dc,c++,objc,obj-c= ++,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-lib= gcj --with-isl=3D/builddir/build/BUILD/gcc-4.9.2-20150212/obj-x86_64-redhat= -linux/isl-install --with-cloog=3D/builddir/build/BUILD/gcc-4.9.2-20150212/= obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with= -tune=3Dgeneric --with-arch_32=3Di686 --build=3Dx86_64-redhat-linux Thread model: posix gcc version 4.9.2 20150212 (Red Hat 4.9.2-6) (GCC)=20 Let me know if there's anything else you need. Thanks! On Friday, August 21, 2015 6:54 AM, Bruce Richardson wrote: =20 On Thu, Aug 20, 2015 at 05:35:10PM +0000, Keith E. Fleming wrote: > make config T=3Dx86_64-native-linuxapp-gccsed -ri 's,(PMD_PCAP=3D).*,\1y,= ' build/.configmake > [root@neutron dpdk-2.1.0]# make 2>&1 | more > =3D=3D Build lib > =3D=3D Build lib/librte_compat > =3D=3D Build lib/librte_eal > =3D=3D Build lib/librte_eal/common > =3D=3D Build lib/librte_eal/linuxapp > =3D=3D Build lib/librte_eal/linuxapp/igb_uio > (cat /dev/null;=C2=A0=C2=A0 echo kernel//root/dpdk-2.1.0/build/build/lib/= librte_eal/linuxapp/igb_uio/igb_uio.ko;) > /root/dpdk-2.1.0/build/build > /lib/librte_eal/linuxapp/igb_uio/modules.order > =C2=A0 Building modules, stage 2. > =C2=A0 MODPOST 1 modules > =3D=3D Build lib/librte_eal/linuxapp/eal > =C2=A0 CC eal_common_options.o > In file included from /usr/lib/gcc/x86_64-redhat-linux/4.9.2/include/x86i= ntrin.h:37:0, > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 from /root/dpdk-2.1.0/build/include/rte_vect.h:67, > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 from /root/dpdk-2.1.0/build/include/rte_memcpy.h:4= 6, > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 from /root/dpdk-2.1.0/lib/librte_eal/common/eal_co= mmon_options.c:48: > /root/dpdk-2.1.0/build/include/rte_memcpy.h: In function =E2=80=98rte_mem= cpy=E2=80=99: > /usr/lib/gcc/x86_64-redhat-linux/4.9.2/include/tmmintrin.h:185:1: error: = inlining failed in call to always_inline =E2=80=98_mm_alignr_epi8=E2=80=99:= t > arget specific option mismatch > =C2=A0_mm_alignr_epi8(__m128i __X, __m128i __Y, const int __N) > =C2=A0^ > In file included from /root/dpdk-2.1.0/lib/librte_eal/common/eal_common_o= ptions.c:48:0: > /root/dpdk-2.1.0/build/include/rte_memcpy.h:425:9: error: called from her= e >=20 > I can't even guess at what the problem is here. It's a total trainwreck. = I assume this has compiled successfully for someone somewhere? Thanks What OS platform is this on - from the gcc path, it looks like gcc 4.9 on R= edhat 7? Is that correct? /Bruce >From zoltan.kiss@linaro.org Fri Aug 21 20:05:08 2015 Return-Path: Received: from mail-wi0-f174.google.com (mail-wi0-f174.google.com [209.85.212.174]) by dpdk.org (Postfix) with ESMTP id E48C65A6F for ; Fri, 21 Aug 2015 20:05:08 +0200 (CEST) Received: by wicne3 with SMTP id ne3so21987139wic.0 for ; Fri, 21 Aug 2015 11:05:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:content-type:content-transfer-encoding; bh=rqYwk902DLsXf0R3RUwusQ7DPdPQ4midnM9D77V9L9U=; b=bGVw6A0RFUXBF8MqiE6Orwkt4JMMoe8GNPfuHY2iJ8dAYhZi0pRttqLe3+BuUGu2uy VMeeYSpxbEJh0jEGkcd4EX0jhhjcKLzM7wQ8YqwwRPMl7nSvrKjb55lQSBCJvb/r9VqN IA3myVwJg/SEr3m2LyMGbpW3zW8skO8G5a6dFxO4Ktp+pbV7YFuvQEzqJMF6cZso+vm1 jZX8ip5XBv4dwepJSHj4mUua+vIn53KbpQkl3kRAi3EzGENlCGUHFGfr2nRNeQ9KJO+Z AECoCnb5wdyFHObu4uzQzLau2V1rwChWxTbHB/DiN2Dz9Bavm+Y+FpK7FF7aencEjrYN NIzA== X-Gm-Message-State: ALoCoQll17WSGi8GU0MFhBRvGJb55DJIHydAlLHJwaZIbbA3mkwhBHMNCFhA7o270clF4mxBX8rx X-Received: by 10.194.93.198 with SMTP id cw6mr18043662wjb.113.1440180308659; Fri, 21 Aug 2015 11:05:08 -0700 (PDT) Received: from [192.168.0.101] ([90.152.119.35]) by smtp.googlemail.com with ESMTPSA id i6sm10950493wje.33.2015.08.21.11.05.07 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 21 Aug 2015 11:05:07 -0700 (PDT) Message-ID: <55D76854.5010306@linaro.org> Date: Fri, 21 Aug 2015 19:05:08 +0100 From: Zoltan Kiss User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 To: "dev@dpdk.org" , dev@openvswitch.org Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: [dpdk-dev] OVS-DPDK performance problem on ixgbe vector PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 18:05:09 -0000 Hi, I've set up a simple packet forwarding perf test on a dual-port 10G 82599ES: one port receives 64 byte UDP packets, the other sends it out, one core used. I've used latest OVS with DPDK 2.1, and the first result was only 13.2 Mpps, which was a bit far from the 13.9 I've seen last year with the same test. The first thing I've changed was to revert back to the old behaviour about this issue: http://permalink.gmane.org/gmane.comp.networking.dpdk.devel/22731 So instead of the new default I've passed 2048 + RTE_PKTMBUF_HEADROOM. That increased the performance to 13.5, but to figure out what's wrong started to play with the receive functions. First I've disabled vector PMD, but ixgbe_recv_pkts_bulk_alloc() was even worse, only 12.5 Mpps. So then I've enabled scattered RX, and with ixgbe_recv_pkts_lro_bulk_alloc() I could manage to get 13.98 Mpps, which is I guess as close as possible to the 14.2 line rate (on my HW at least, with one core) Does anyone has a good explanation about why the vector PMD performs so significantly worse? I would expect that on a 3.2 GHz i5-4570 one core should be able to reach ~14 Mpps, SG and vector PMD shouldn't make a difference. I've tried to look into it with oprofile, but the results were quite strange: 35% of the samples were from miniflow_extract, the part where parse_vlan calls data_pull to jump after the MAC addresses. The oprofile snippet (1M samples): 511454 19 0.0037 flow.c:511 511458 149 0.0292 dp-packet.h:266 51145f 4264 0.8357 dp-packet.h:267 511466 18 0.0035 dp-packet.h:268 51146d 43 0.0084 dp-packet.h:269 511474 172 0.0337 flow.c:511 51147a 4320 0.8467 string3.h:51 51147e 358763 70.3176 flow.c:99 511482 2 3.9e-04 string3.h:51 511485 3060 0.5998 string3.h:51 511488 1693 0.3318 string3.h:51 51148c 2933 0.5749 flow.c:326 511491 47 0.0092 flow.c:326 And the corresponding disassembled code: 511454: 49 83 f9 0d cmp r9,0xd 511458: c6 83 81 00 00 00 00 mov BYTE PTR [rbx+0x81],0x0 51145f: 66 89 83 82 00 00 00 mov WORD PTR [rbx+0x82],ax 511466: 66 89 93 84 00 00 00 mov WORD PTR [rbx+0x84],dx 51146d: 66 89 8b 86 00 00 00 mov WORD PTR [rbx+0x86],cx 511474: 0f 86 af 01 00 00 jbe 511629 51147a: 48 8b 45 00 mov rax,QWORD PTR [rbp+0x0] 51147e: 4c 8d 5d 0c lea r11,[rbp+0xc] 511482: 49 89 00 mov QWORD PTR [r8],rax 511485: 8b 45 08 mov eax,DWORD PTR [rbp+0x8] 511488: 41 89 40 08 mov DWORD PTR [r8+0x8],eax 51148c: 44 0f b7 55 0c movzx r10d,WORD PTR [rbp+0xc] 511491: 66 41 81 fa 81 00 cmp r10w,0x81 My only explanation to this so far is that I misunderstand something about the oprofile results. Regards, Zoltan