From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B60EA04DB; Thu, 15 Oct 2020 19:08:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6A0B51DCE9; Thu, 15 Oct 2020 19:08:13 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 3360D1DCD4 for ; Thu, 15 Oct 2020 19:08:10 +0200 (CEST) IronPort-SDR: 9rcDhUqt0nhJFN+77SZlpZY2t8705WLvuSdkdc+ChRMf6g6IhQE315rpHx/lP244XEIUhK7Khx Ci0ssVNtqVNQ== X-IronPort-AV: E=McAfee;i="6000,8403,9775"; a="228052004" X-IronPort-AV: E=Sophos;i="5.77,379,1596524400"; d="scan'208";a="228052004" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Oct 2020 10:08:09 -0700 IronPort-SDR: KCrg9rSmBC6NHHx1v1OyilSWLhRkCnJt+d+bGaJE10wFQgFeBQjNXyoTvXAN2fROie/v5tf8NS UbHFJCFeiRcg== X-IronPort-AV: E=Sophos;i="5.77,379,1596524400"; d="scan'208";a="531335126" Received: from bricha3-mobl.ger.corp.intel.com ([10.213.245.94]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 15 Oct 2020 10:08:08 -0700 Date: Thu, 15 Oct 2020 18:08:04 +0100 From: Bruce Richardson To: Ali Alnubani Cc: "dev@dpdk.org" , NBU-Contact-Thomas Monjalon , Asaf Penso Message-ID: <20201015170804.GG554@bricha3-MOBL.ger.corp.intel.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Subject: Re: [dpdk-dev] performance degradation with fpic X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Oct 15, 2020 at 04:00:44PM +0000, Ali Alnubani wrote: > Hi Bruce, > > > We have been seeing in some cases that the DPDK forwarding performance > is up to 9% lower when DPDK is built as static with meson compared to a > build with makefiles. > > > The same degradation can be reproduced with makefiles on older DPDK > releases when building with EXTAR_CFLAGS set to “-fPIC”, it can also be > resolved in meson when passing “pic: false” to meson’s static_library > call (more tweaking needs to be done to prevent building shared > libraries because this change breaks them). > > > I can reproduce this drop with the following cases: > * Baremetal / NIC: ConnectX-4 Lx / OS: RHEL7.4 / CPU: Intel(R) > Xeon(R) Gold 6154. Testpmd command: > > testpmd -c 0x7ffc0000 -n 4 -w d8:00.1 -w d8:00.0 --socket-mem=2048,2048 > -- --port-numa-config=0,1,1,1 --socket-num=1 --burst=64 --txd=512 > --rxd=512 --mbcache=512 --rxq=2 --txq=2 --nb-cores=1 --no-lsc-interrupt > -i -a --rss-udp > * KVM guest with SR-IOV passthrough / OS: RHEL7.4 / NIC: ConnectX-5 / > Host’s CPU: Intel(R) Xeon(R) Gold 6154. Testpmd command: > testpmd --master-lcore=0 -c 0x1ffff -n 4 -w > 00:05.0,mprq_en=1,mprq_log_stride_num=6 --socket-mem=2048,0 -- > --port-numa-config=0,0 --socket-num=0 --burst=64 --txd=1024 > --rxd=1024 --mbcache=512 --rxq=16 --txq=16 --nb-cores=8 > --port-topology=chained --forward-mode=macswap --no-lsc-interrupt > -i -a --rss-udp > * Baremetal / OS: Ubuntu 18.04 / NIC: ConnectX-5 / CPU: Intel(R) > Xeon(R) CPU E5-2697A v4. Testpmd command: > testpmd -n 4 -w 0000:82:00.0,rxqs_min_mprq=8,mprq_en=1 -w > 0000:82:00.1,rxqs_min_mprq=8,mprq_en=1 -c 0xff80 -- --burst=64 > --mbcache=512 -i --nb-cores=8 --rxq=8 --txq=8 --txd=1024 > --rxd=1024 --rss-udp --auto-start > > The packets being received and forwarded by testpmd are of IPv4/UDP > type and 64B size. > > Should we disable PIC in static builds? > > Hi Ali, thanks for reporting, though it's strange that you see such a big impact. In my previous tests with i40e driver I never noticed a difference between make and meson builds, and I and some others here have been using meson builds for any performance work for over a year now. That being said let me reverify what I see on my end. In terms of solutions, disabling the -fPIC flag globally implies that we can no longer build static and shared libs from the same sources, so we would need to revert to doing either a static or a shared library build but not both. If the issue is limited to only some drivers or some cases, we can perhaps add in a build option to have no-fpic-static builds, to be used in a cases where it is problematic. However, at this point, I think we need a little more investigation. Is there any testing you can do to see if it's just in your driver, or in perhaps a mempool driver/lib that the issue appears, or if it's just a global slowdown? Do you see the impact with both clang and gcc? I'll retest things a bit tomorrow on my end to see what I see. Regards, /Bruce