From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 966B2A00BE;
Mon, 28 Oct 2019 15:22:09 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 9BB041BF2D;
Mon, 28 Oct 2019 15:21:59 +0100 (CET)
Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com
[67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 224221BEDF
for <dev@dpdk.org>; Mon, 28 Oct 2019 15:21:56 +0100 (CET)
Received: from pps.filterd (m0045851.ppops.net [127.0.0.1])
by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id
x9SEKfnP005757; Mon, 28 Oct 2019 07:21:55 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com;
h=from : to : cc :
subject : date : message-id : in-reply-to : references : mime-version :
content-transfer-encoding : content-type; s=pfpt0818;
bh=w4aP24ocp2dDtDsXH/5Yh5k4VBIY1EIJvfIcVZpsx4w=;
b=x2BAG5b6msbgpmeRfdZcxucrTccG6ZzNcYvsXkQDh0FU+iT25kcEfcisco5KyLXjDuf6
vuLw1x1ISc8DxxDK1uLne7PJAn0lp2CqQUQdcP6qfaRIgNvXSJl6nEH28IWz2HuaCLjl
Q4UEmJrQKRW6LdHJkgd2nPze71zH0iuhV4w96718RwZPRfQQ+Ar4gg4dJQme8C32uucG
i6BfhaMisvuVjMaTAlqr1p0i1fSwI3JsBQEnQWQMluRp/JbUkl4dr4Y/1YtIyjpxucqi
VrnCCJdToygLxk4BoQ4foePOFiAcMvkKC8fqawuhxFCaa2nG4CvdBfCkOch4gzCZKabX /w==
Received: from sc-exch04.marvell.com ([199.233.58.184])
by mx0b-0016f401.pphosted.com with ESMTP id 2vvnnnwneh-2
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT);
Mon, 28 Oct 2019 07:21:55 -0700
Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com
(10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 28 Oct
2019 07:21:53 -0700
Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com
(10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend
Transport; Mon, 28 Oct 2019 07:21:53 -0700
Received: from amok.marvell.com (unknown [10.95.130.103])
by maili.marvell.com (Postfix) with ESMTP id 521983F703F;
Mon, 28 Oct 2019 07:21:51 -0700 (PDT)
From: Andrzej Ostruszka <aostruszka@marvell.com>
To: <dev@dpdk.org>, Aaron Conole <aconole@redhat.com>, Michael Santana
<maicolgabriel@hotmail.com>, Thomas Monjalon <thomas@monjalon.net>, John
McNamara <john.mcnamara@intel.com>, Marko Kovacevic
<marko.kovacevic@intel.com>
CC: <mattias.ronnblom@ericsson.com>, <stephen@networkplumber.org>
Date: Mon, 28 Oct 2019 15:21:36 +0100
Message-ID: <20191028142145.3758-3-aostruszka@marvell.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20191028142145.3758-1-aostruszka@marvell.com>
References: <20191022115412.8837-1-aostruszka@marvell.com>
<20191028142145.3758-1-aostruszka@marvell.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,1.0.8
definitions=2019-10-28_05:2019-10-25,2019-10-28 signatures=0
Subject: [dpdk-dev] [PATCH v5 02/11] build: add an option to enable LTO build
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
This patch adds an option to enable link time optimization. In addition
to LTO option itself (-flto) fat-lto-objects are being used. This is
because during the build pmdinfogen scans the generated ELF objects to
find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc
produces ELF only with extra symbols for internal use during linking.
Signed-off-by: Andrzej Ostruszka <aostruszka@marvell.com>
---
.travis.yml | 7 +++++
config/common_base | 5 +++
config/meson.build | 13 ++++++++
doc/guides/prog_guide/index.rst | 1 +
doc/guides/prog_guide/lto.rst | 40 ++++++++++++++++++++++++
doc/guides/rel_notes/release_19_11.rst | 9 ++++++
mk/toolchain/gcc/rte.toolchain-compat.mk | 4 +++
mk/toolchain/gcc/rte.vars.mk | 12 +++++++
mk/toolchain/icc/rte.vars.mk | 8 +++++
9 files changed, 99 insertions(+)
create mode 100644 doc/guides/prog_guide/lto.rst
diff --git a/.travis.yml b/.travis.yml
index 3d6ef2959..3cd746dba 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -34,6 +34,7 @@ env:
- DEF_LIB="static" OPTS="-Denable_kmods=false"
- DEF_LIB="shared" OPTS="-Denable_kmods=false"
- DEF_LIB="shared" RUN_TESTS=1
+ - DEF_LIB="shared" OPTS="-Db_lto=true"
matrix:
include:
@@ -105,6 +106,12 @@ matrix:
apt:
packages:
- *extra_packages
+ - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+ compiler: gcc
+ addons:
+ apt:
+ packages:
+ - *extra_packages
script: ./.ci/${TRAVIS_OS_NAME}-build.sh
diff --git a/config/common_base b/config/common_base
index b2be3d96a..0d1207166 100644
--- a/config/common_base
+++ b/config/common_base
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
#
CONFIG_RTE_ARCH_STRICT_ALIGN=n
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
#
# Compile to share library
#
diff --git a/config/meson.build b/config/meson.build
index e1ebdad26..2b1cb92e7 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -225,3 +225,16 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
if is_freebsd
add_project_arguments('-D__BSD_VISIBLE', language: 'c')
endif
+
+if get_option('b_lto')
+ if cc.has_argument('-ffat-lto-objects')
+ add_project_arguments('-ffat-lto-objects', language: 'c')
+ else
+ error('compiler does not support fat LTO objects - please turn LTO off')
+ endif
+ # workaround for gcc bug 81440
+ if cc.get_id() == 'gcc' and cc.version().version_compare('<8.0')
+ add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+ add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+ endif
+endif
diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
index 692409af8..dc4851c57 100644
--- a/doc/guides/prog_guide/index.rst
+++ b/doc/guides/prog_guide/index.rst
@@ -65,5 +65,6 @@ Programmer's Guide
ext_app_lib_make_help
perf_opt_guidelines
writing_efficient_code
+ lto
profile_app
glossary
diff --git a/doc/guides/prog_guide/lto.rst b/doc/guides/prog_guide/lto.rst
new file mode 100644
index 000000000..50aecc9e5
--- /dev/null
+++ b/doc/guides/prog_guide/lto.rst
@@ -0,0 +1,40 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2019 Marvell International Ltd.
+
+Link Time Optimization
+======================
+
+The DPDK supports compilation with link time optimization turned on.
+This depends obviously on the ability of the compiler to do "whole
+program" optimization at link time and is available only for compilers
+that support that feature.
+To be more specific, compiler (in addition to performing LTO) have to
+support creation of ELF objects containing both normal code and internal
+representation (called fat-lto-objects in gcc and icc).
+This is required since during build some code is generated by parsing
+produced ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.
+However LTO is also useful for additional code analysis done by the
+compiler.
+In particular due to interprocedural analysis compiler can produce
+additional warnings about variables that might be used uninitialized.
+Some of these warnings might be "false positives" though and you might
+need to explicitly initialize variable in order to silence the compiler.
+
+Please note that turning LTO on causes considerable extension of
+build time.
+
+When using make based build, link time optimization can be enabled for
+the whole DPDK by setting:
+
+.. code-block:: console
+ CONFIG_ENABLE_LTO=y
+
+in config file.
+For the meson based build it can be enabled by setting meson built-in
+'b_lto' option:
+
+.. code-block:: console
+ meson build -Db_lto=true
diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index ae8e7b2f0..b11abe2fe 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -231,6 +231,15 @@ New Features
* Added a console command to testpmd app, ``show port (port_id) ptypes`` which
gives ability to print port supported ptypes in different protocol layers.
+* **Added build support for Link Time Optimization.**
+
+ LTO is an optimization technique used by the compiler to perform whole
+ program analysis and optimization at link time. In order to do that
+ compilers store their internal representation of the source code that
+ the linker uses at the final stage of compilation process.
+
+ See :doc:`../prog_guide/lto` for more information:
+
Removed Items
-------------
diff --git a/mk/toolchain/gcc/rte.toolchain-compat.mk b/mk/toolchain/gcc/rte.toolchain-compat.mk
index ea40a11c0..ad4fad83c 100644
--- a/mk/toolchain/gcc/rte.toolchain-compat.mk
+++ b/mk/toolchain/gcc/rte.toolchain-compat.mk
@@ -88,6 +88,10 @@ else
MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
endif
+ ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1)
+ CONFIG_RTE_ENABLE_LTO=n
+ endif
+
# Disable thunderx PMD for gcc < 4.7
ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d
diff --git a/mk/toolchain/gcc/rte.vars.mk b/mk/toolchain/gcc/rte.vars.mk
index b852fcfd7..9fc704193 100644
--- a/mk/toolchain/gcc/rte.vars.mk
+++ b/mk/toolchain/gcc/rte.vars.mk
@@ -62,6 +62,18 @@ endif
# process cpu flags
include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+# workaround for GCC bug 81440
+ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1)
+WERROR_FLAGS += -Wno-lto-type-mismatch
+endif
+endif
+
# workaround GCC bug with warning "missing initializer" for "= {0}"
ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
WERROR_FLAGS += -Wno-missing-field-initializers
diff --git a/mk/toolchain/icc/rte.vars.mk b/mk/toolchain/icc/rte.vars.mk
index aa1422bf1..8aa87aa1e 100644
--- a/mk/toolchain/icc/rte.vars.mk
+++ b/mk/toolchain/icc/rte.vars.mk
@@ -54,5 +54,13 @@ endif
# process cpu flags
include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
--
2.17.1