From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 966B2A00BE; Mon, 28 Oct 2019 15:22:09 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9BB041BF2D; Mon, 28 Oct 2019 15:21:59 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 224221BEDF for ; Mon, 28 Oct 2019 15:21:56 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x9SEKfnP005757; Mon, 28 Oct 2019 07:21:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=w4aP24ocp2dDtDsXH/5Yh5k4VBIY1EIJvfIcVZpsx4w=; b=x2BAG5b6msbgpmeRfdZcxucrTccG6ZzNcYvsXkQDh0FU+iT25kcEfcisco5KyLXjDuf6 vuLw1x1ISc8DxxDK1uLne7PJAn0lp2CqQUQdcP6qfaRIgNvXSJl6nEH28IWz2HuaCLjl Q4UEmJrQKRW6LdHJkgd2nPze71zH0iuhV4w96718RwZPRfQQ+Ar4gg4dJQme8C32uucG i6BfhaMisvuVjMaTAlqr1p0i1fSwI3JsBQEnQWQMluRp/JbUkl4dr4Y/1YtIyjpxucqi VrnCCJdToygLxk4BoQ4foePOFiAcMvkKC8fqawuhxFCaa2nG4CvdBfCkOch4gzCZKabX /w== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2vvnnnwneh-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 28 Oct 2019 07:21:55 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 28 Oct 2019 07:21:53 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Mon, 28 Oct 2019 07:21:53 -0700 Received: from amok.marvell.com (unknown [10.95.130.103]) by maili.marvell.com (Postfix) with ESMTP id 521983F703F; Mon, 28 Oct 2019 07:21:51 -0700 (PDT) From: Andrzej Ostruszka To: , Aaron Conole , Michael Santana , Thomas Monjalon , John McNamara , Marko Kovacevic CC: , Date: Mon, 28 Oct 2019 15:21:36 +0100 Message-ID: <20191028142145.3758-3-aostruszka@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20191028142145.3758-1-aostruszka@marvell.com> References: <20191022115412.8837-1-aostruszka@marvell.com> <20191028142145.3758-1-aostruszka@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,1.0.8 definitions=2019-10-28_05:2019-10-25,2019-10-28 signatures=0 Subject: [dpdk-dev] [PATCH v5 02/11] build: add an option to enable LTO build X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking. Signed-off-by: Andrzej Ostruszka --- .travis.yml | 7 +++++ config/common_base | 5 +++ config/meson.build | 13 ++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/lto.rst | 40 ++++++++++++++++++++++++ doc/guides/rel_notes/release_19_11.rst | 9 ++++++ mk/toolchain/gcc/rte.toolchain-compat.mk | 4 +++ mk/toolchain/gcc/rte.vars.mk | 12 +++++++ mk/toolchain/icc/rte.vars.mk | 8 +++++ 9 files changed, 99 insertions(+) create mode 100644 doc/guides/prog_guide/lto.rst diff --git a/.travis.yml b/.travis.yml index 3d6ef2959..3cd746dba 100644 --- a/.travis.yml +++ b/.travis.yml @@ -34,6 +34,7 @@ env: - DEF_LIB="static" OPTS="-Denable_kmods=false" - DEF_LIB="shared" OPTS="-Denable_kmods=false" - DEF_LIB="shared" RUN_TESTS=1 + - DEF_LIB="shared" OPTS="-Db_lto=true" matrix: include: @@ -105,6 +106,12 @@ matrix: apt: packages: - *extra_packages + - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1 + compiler: gcc + addons: + apt: + packages: + - *extra_packages script: ./.ci/${TRAVIS_OS_NAME}-build.sh diff --git a/config/common_base b/config/common_base index b2be3d96a..0d1207166 100644 --- a/config/common_base +++ b/config/common_base @@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n # CONFIG_RTE_ARCH_STRICT_ALIGN=n +# +# Enable link time optimization +# +CONFIG_RTE_ENABLE_LTO=n + # # Compile to share library # diff --git a/config/meson.build b/config/meson.build index e1ebdad26..2b1cb92e7 100644 --- a/config/meson.build +++ b/config/meson.build @@ -225,3 +225,16 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c') if is_freebsd add_project_arguments('-D__BSD_VISIBLE', language: 'c') endif + +if get_option('b_lto') + if cc.has_argument('-ffat-lto-objects') + add_project_arguments('-ffat-lto-objects', language: 'c') + else + error('compiler does not support fat LTO objects - please turn LTO off') + endif + # workaround for gcc bug 81440 + if cc.get_id() == 'gcc' and cc.version().version_compare('<8.0') + add_project_arguments('-Wno-lto-type-mismatch', language: 'c') + add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c') + endif +endif diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 692409af8..dc4851c57 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -65,5 +65,6 @@ Programmer's Guide ext_app_lib_make_help perf_opt_guidelines writing_efficient_code + lto profile_app glossary diff --git a/doc/guides/prog_guide/lto.rst b/doc/guides/prog_guide/lto.rst new file mode 100644 index 000000000..50aecc9e5 --- /dev/null +++ b/doc/guides/prog_guide/lto.rst @@ -0,0 +1,40 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Marvell International Ltd. + +Link Time Optimization +====================== + +The DPDK supports compilation with link time optimization turned on. +This depends obviously on the ability of the compiler to do "whole +program" optimization at link time and is available only for compilers +that support that feature. +To be more specific, compiler (in addition to performing LTO) have to +support creation of ELF objects containing both normal code and internal +representation (called fat-lto-objects in gcc and icc). +This is required since during build some code is generated by parsing +produced ELF objects (pmdinfogen). + +The amount of performance gain that one can get from LTO depends on the +compiler and the code that is being compiled. +However LTO is also useful for additional code analysis done by the +compiler. +In particular due to interprocedural analysis compiler can produce +additional warnings about variables that might be used uninitialized. +Some of these warnings might be "false positives" though and you might +need to explicitly initialize variable in order to silence the compiler. + +Please note that turning LTO on causes considerable extension of +build time. + +When using make based build, link time optimization can be enabled for +the whole DPDK by setting: + +.. code-block:: console + CONFIG_ENABLE_LTO=y + +in config file. +For the meson based build it can be enabled by setting meson built-in +'b_lto' option: + +.. code-block:: console + meson build -Db_lto=true diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst index ae8e7b2f0..b11abe2fe 100644 --- a/doc/guides/rel_notes/release_19_11.rst +++ b/doc/guides/rel_notes/release_19_11.rst @@ -231,6 +231,15 @@ New Features * Added a console command to testpmd app, ``show port (port_id) ptypes`` which gives ability to print port supported ptypes in different protocol layers. +* **Added build support for Link Time Optimization.** + + LTO is an optimization technique used by the compiler to perform whole + program analysis and optimization at link time. In order to do that + compilers store their internal representation of the source code that + the linker uses at the final stage of compilation process. + + See :doc:`../prog_guide/lto` for more information: + Removed Items ------------- diff --git a/mk/toolchain/gcc/rte.toolchain-compat.mk b/mk/toolchain/gcc/rte.toolchain-compat.mk index ea40a11c0..ad4fad83c 100644 --- a/mk/toolchain/gcc/rte.toolchain-compat.mk +++ b/mk/toolchain/gcc/rte.toolchain-compat.mk @@ -88,6 +88,10 @@ else MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS)) endif + ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1) + CONFIG_RTE_ENABLE_LTO=n + endif + # Disable thunderx PMD for gcc < 4.7 ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1) CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d diff --git a/mk/toolchain/gcc/rte.vars.mk b/mk/toolchain/gcc/rte.vars.mk index b852fcfd7..9fc704193 100644 --- a/mk/toolchain/gcc/rte.vars.mk +++ b/mk/toolchain/gcc/rte.vars.mk @@ -62,6 +62,18 @@ endif # process cpu flags include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk +ifeq ($(CONFIG_RTE_ENABLE_LTO),y) +# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX' +# exported in symbol table and without this option only internal +# representation is present. +TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects +TOOLCHAIN_LDFLAGS += -flto +# workaround for GCC bug 81440 +ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1) +WERROR_FLAGS += -Wno-lto-type-mismatch +endif +endif + # workaround GCC bug with warning "missing initializer" for "= {0}" ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1) WERROR_FLAGS += -Wno-missing-field-initializers diff --git a/mk/toolchain/icc/rte.vars.mk b/mk/toolchain/icc/rte.vars.mk index aa1422bf1..8aa87aa1e 100644 --- a/mk/toolchain/icc/rte.vars.mk +++ b/mk/toolchain/icc/rte.vars.mk @@ -54,5 +54,13 @@ endif # process cpu flags include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk +ifeq ($(CONFIG_RTE_ENABLE_LTO),y) +# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX' +# exported in symbol table and without this option only internal +# representation is present. +TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects +TOOLCHAIN_LDFLAGS += -flto +endif + export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS -- 2.17.1