From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3CF44A046B for ; Mon, 19 Aug 2019 15:48:32 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0EFA41BEB2; Mon, 19 Aug 2019 15:48:31 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 5A7CA1BEAD for ; Mon, 19 Aug 2019 15:48:29 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x7JDlV76025795; Mon, 19 Aug 2019 06:48:28 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=X6Tdvf6RWCYTzBwevq2qi3FW02639lL3wYfA38nPlhk=; b=h9F+3M7CyG3US913ILNOoaWXd8ngfQrvfCBlkQWdoUPCAwAGuh3rBKda4QfzYp2iK+aA IeS0uHwhZZC67khsKpE69aGrD9ZWimKEh8CFaKLvvKLGo8cvtUKmpUSOC/y0m4BQZ+FL RhS9lQOWwql5JN0V7IHrKfe0ztYGnbkTjxusr9gQvZWPsiDf79IDN4IvbZI0/w1XOb8c Fpk1e2T7AhIb5VWq8PcueH8K9JigzCw+YHlt3iV/h6AQW3xcu/95dU0BPFWs4HJ5lB3i zS16L//rL0WCf/cPCn8049GtmFWxesXsDE7+h8QoLTw1HF+2BIy7Gv/0BQr7A1YcT7qE LQ== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2ueh3hx3mw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 19 Aug 2019 06:48:28 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 19 Aug 2019 06:48:26 -0700 Received: from NAM05-CO1-obe.outbound.protection.outlook.com (104.47.48.52) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Mon, 19 Aug 2019 06:48:26 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oT2q+Jo8T2+sb7ReC7kf2XIeucWCIGJYrSN++kCo0bGtjSxeJIR0Bsp1oqeCDW19ztRFSGOdKyS55gvRWZmtV1tRTGlUdI1f/5Zsu6YYqGqsR0gbVnGN66h+M4jcG17eO72bHG6Q9gTpOhOoEvv0/mAKyTH1pr07hMzbsQCls3iH6hNrLUYTVHUowi9RFEuzZM0Xc/m92FzOYQ56yJaiVeAkGMXJfnpp3wJCTIKzCXESx0UAO9XmAmjDcJVWBDDzqG5g1wNauZHyAM+jthkR1F/1zUKGtNtyXdpRguqShzpULk5vrqNQx+AwHE/wYtdhw/M6NJ6OAGkhTtWXv5YzwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X6Tdvf6RWCYTzBwevq2qi3FW02639lL3wYfA38nPlhk=; b=MSt+5GFXQQGpvS7AQQH226cjr0m3CfOw+HK7rp/LCh0FTmvIB1R0E9B1p8uLnoBTcqwjNZKlhC1kZODAxs5eWryahJGb81wQnBIr6wHwNZgJCMdN+FMrQe0NQQU4F/MOJ8NVkmc+mprbhg+GtZvQP8/u1Ty8Y2So0WNbVkZ412ehnQPynMuqX35n6TeXlNanaM6Sp0OE5gl7mSkCuUWIjamNxfpAXWd/iLWPXhu4eTqZBZXCZPXZGIhUTxi9cP42P4gJWAwlsQMEfbotWk9R7oR59LU17oSsEtsnSoiYKhspfudIM8fxY8eORPeFRG95iIfNP+zo9RGuvMjxowss2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X6Tdvf6RWCYTzBwevq2qi3FW02639lL3wYfA38nPlhk=; b=V2eKTj/wzlymht5cbd8IOKa3WTMw2A9r9e8awsa5u456lINYhMvs9XG+nhH5WCwj1mPrRHHDKsszP64PYKcKzKnkqAQ7z9vcbCqoq7WtyYhNJOZP7iWeM20nctyHGgYyDmA2cKvSXbywwTvjpLkdIayAlpCUf542McyFrz08MkY= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (20.179.21.149) by MN2PR18MB2990.namprd18.prod.outlook.com (20.179.22.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2178.16; Mon, 19 Aug 2019 13:48:21 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::c593:d343:e2ed:584a]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::c593:d343:e2ed:584a%2]) with mapi id 15.20.2178.018; Mon, 19 Aug 2019 13:48:21 +0000 From: Harman Kalra To: Thomas Monjalon , John McNamara , Marko Kovacevic , "Bruce Richardson" CC: "dev@dpdk.org" , Harman Kalra Thread-Topic: [PATCH] mk: add support for UBSAN Thread-Index: AQHVVpTDs7FnaDuo7UCImNLU+qa6/Q== Date: Mon, 19 Aug 2019 13:48:21 +0000 Message-ID: <1566222457-17342-1-git-send-email-hkalra@marvell.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BM1PR01CA0166.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:68::36) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:3e::21) x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.7.4 x-originating-ip: [115.113.156.2] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 151ca6ec-bbfc-4bc8-b9cb-08d724abe5f0 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600148)(711020)(4605104)(1401327)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:MN2PR18MB2990; x-ms-traffictypediagnostic: MN2PR18MB2990: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0134AD334F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(346002)(136003)(376002)(366004)(39850400004)(396003)(189003)(199004)(52116002)(2616005)(476003)(36756003)(99286004)(86362001)(486006)(107886003)(53936002)(8676002)(186003)(7736002)(6506007)(26005)(14454004)(256004)(478600001)(14444005)(71190400001)(71200400001)(102836004)(6116002)(305945005)(3846002)(66066001)(55236004)(386003)(66476007)(64756008)(5660300002)(81156014)(6436002)(66946007)(66556008)(66446008)(4326008)(54906003)(110136005)(2906002)(81166006)(6512007)(8936002)(316002)(50226002)(25786009)(6486002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB2990; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: h1V4TOTkzWZd5FRwjJ9Eet4y8KE9YHspruPEom3ylxNkF6BsHgRWIuXqkLlMlNFBbLcsxiHByFdC/pw4ZzSShCZoH2ufFWGQCewvH7jkQGZOjhnhKX3fS7IIuYgHgCCZaWqzdhifA/avl8iCYEIK++F3dzb2PPSyjj96axf8brgQWREjnl6Sex+Sqnc2MVkeyG42g5YRZUHmh1SZ7JLWPGJyMm1v31yvNh2y5vNAolX7mHaOlwcMgaSXzFiIhzMf7ZabIjOXpQ6rI8Sp4jSZ544QDLK2i4gEzeucIfgqKQ/RwRyxt1/7Vt75fnUuiy2CfyBHzmoxA8ZTw+RzpjDTgCrzUjAg8M1J/xRoTVwgqRMxjCsUxx/f19QHDDLkRpzCpOfgWii6DUgfL/w47McbOzxaVwf041Am6I/gQf9qPj0= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 151ca6ec-bbfc-4bc8-b9cb-08d724abe5f0 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Aug 2019 13:48:21.3599 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: FBTFcpUrJDbjTqp8GYNqIMMOHXiydLIOADumKtpnnJtaDyNazPO5p2Tk5EUSRs4AotDxwzF522YwKCFeRMDz8w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB2990 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-08-19_03:2019-08-19,2019-08-19 signatures=0 Subject: [dpdk-dev] [PATCH] mk: add support for UBSAN X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" UndefinedBehaviorSanitizer (UBSan) is a fast undefined behavior detector. UBSan modifies the program at compile-time to catch various kinds of undefined behavior during program execution. This patch implements support for UBSan to the DPDK. See: doc/guides/prog_guide/ubsan.rst for more information. Signed-off-by: Harman Kalra --- config/common_base | 6 ++ config/meson.build | 15 ++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/ubsan.rst | 112 +++++++++++++++++++++++++ doc/guides/rel_notes/release_19_11.rst | 7 ++ meson_options.txt | 2 + mk/rte.app.mk | 10 +++ mk/rte.lib.mk | 12 +++ mk/toolchain/clang/rte.vars.mk | 4 + mk/toolchain/gcc/rte.vars.mk | 8 ++ 10 files changed, 177 insertions(+) create mode 100644 doc/guides/prog_guide/ubsan.rst diff --git a/config/common_base b/config/common_base index 8ef75c203..b47a6f28e 100644 --- a/config/common_base +++ b/config/common_base @@ -1067,3 +1067,9 @@ CONFIG_RTE_APP_CRYPTO_PERF=3Dy # Compile the eventdev application # CONFIG_RTE_APP_EVENTDEV=3Dy + +# +# Enable undefined behavior sanitizer +# +CONFIG_RTE_UBSAN=3Dn +CONFIG_RTE_UBSAN_SANITIZE_ALL=3Dn diff --git a/config/meson.build b/config/meson.build index 2bafea530..6a2fa117d 100644 --- a/config/meson.build +++ b/config/meson.build @@ -196,3 +196,18 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c') if is_freebsd add_project_arguments('-D__BSD_VISIBLE', language: 'c') endif + +# enable ubsan +if get_option('enable_ubsan') + if cc.has_argument('-fsanitize=3Dundefined') + ubsan_dep =3D cc.find_library('libubsan', required: false) + if ubsan_dep.found() + add_project_arguments('-fsanitize=3Dundefined', language: 'c') + add_project_link_arguments('-fsanitize=3Dundefined', language: 'c') + else + message('libubsan not found, UBSAN cannot be enabled') + endif + else + message('gcc version does not support UBSAN') + endif +endif diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.= rst index 692409af8..970901e9a 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -66,4 +66,5 @@ Programmer's Guide perf_opt_guidelines writing_efficient_code profile_app + ubsan glossary diff --git a/doc/guides/prog_guide/ubsan.rst b/doc/guides/prog_guide/ubsan.= rst new file mode 100644 index 000000000..cb19f3bd9 --- /dev/null +++ b/doc/guides/prog_guide/ubsan.rst @@ -0,0 +1,112 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Marvell International Ltd. + +The Undefined Behavior Sanitizer - UBSan +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +UndefinedBehaviorSanitizer (UBSan) is a runtime undefined behavior detecto= r. +UBSan uses compile-time instrumentation and modifies the program by adding +some stubs which perform certain checks before operations that might cause +undefined behaviour. If some UB detected, respective _UBSan_handle_* handl= ers +(which are defined in libUBSan library) are called to prints the error mes= sage. + +Some examples of undefined behaviour checks: + +* Misaligned memory access +* Signed integer overflow +* Load from/store to an object with insufficient space. +* Integer divide by zero as well as INT_MIN / -1 division +* Out-of-bounds memory accesses. +* Null argument declared with nonnull attribute, returned null from functi= on + which never returns null, null ptr dereference +* Variable size array with non-positive length + +GCC supports this feature since 4.9, however GCC 5.0 onwards has many more +checkers implemented. + +Example UBSan error +-------------------- + +Following error was reported when UBSan was enabled: + +.. code-block:: console + + drivers/net/octeontx2/otx2_stats.c:82:26: runtime error: left shift of + 1 by 31 places cannot be represented in type 'int' + +Code responsible for this error: + +.. code-block:: c + + if (dev->txmap[i] & (1 << 31)) { + +To fix this error: + +.. code-block:: c + + if (dev->txmap[i] & (1U << 31)) { + +Usage +----- + +make build +^^^^^^^^^^ + +To enable UBSan, enable following configuration: + +.. code-block:: console + + CONFIG_RTE_UBSAN=3Dy + +UBSan framework supports three modes: + +1. Enable UBSan on the entire DPDK source code - set following configurati= on: + +.. code-block:: console + + CONFIG_RTE_UBSAN_SANITIZE_ALL=3Dy + +2. Enable UBSan on a particular library or PMD - add the following line to= the + respective Makefile of the library or PMD + (make sure ``CONFIG_RTE_UBSAN_SANITIZE_ALL=3Dn``). This will instrument= only + the library or PMD and not the entire repository. + +.. code-block:: console + + UBSAN_SANITIZE :=3D y + +3. Disable UBSan for a particular library or PMD - add the following line = to + the respective Makefile of the library or PMD. Make sure + ``CONFIG_RTE_UBSAN_SANITIZE_ALL=3Dy`` config is set. This will instrume= nt + entire DPDK repository but not this specific library or PMD. + +.. code-block:: console + + UBSAN_SANITIZE :=3D n + +.. Note:: + + Standard DPDK applications like test, testpmd, etc. cannot be + chosen explicitly for UBSan check, like libraries or PMD. The reason is, + say UBSan is enabled for library X, and ``UBSAN_SANITIZE=3Dy`` is not ad= ded + in Makefile of app Y which uses X APIs. This will lead to undefined + reference to _UBSan_handle_* handlers as Y is not compiled with UBSan fl= ags. + Hence UBSan check is enabled for all standard DPDK applications as soon = as + ``CONFIG_RTE_UBSAN=3Dy`` is set. + +meson build +^^^^^^^^^^^ + +To enable UBSan in meson build system, use following meson build command: + +**Example usage:** + +.. code-block:: console + + meson build -Denable_ubsan=3Dtrue + ninja -C build + +.. Note:: + + Meson build works only in one mode i.e. UBSan can be enabled for + the entire DPDK sources and not individual libraries or PMD, like make b= uild. diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/= release_19_11.rst index 8490d897c..cfefdbeec 100644 --- a/doc/guides/rel_notes/release_19_11.rst +++ b/doc/guides/rel_notes/release_19_11.rst @@ -56,6 +56,13 @@ New Features Also, make sure to start the actual text at the margin. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D =20 +* **Added Undefined Behavior Sanitizer framework.** + + UBSan is a fast runtime undefined behavior detector which uses compile-t= ime + instrumentation and modifies the program by adding some stubs that perfo= rm + certain checks before operations that might cause undefined behavior. + + See :doc:`../prog_guide/ubsan` for more information: =20 Removed Items ------------- diff --git a/meson_options.txt b/meson_options.txt index 448f3e63d..fb0aead00 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -8,6 +8,8 @@ option('enable_docs', type: 'boolean', value: false, description: 'build documentation') option('enable_kmods', type: 'boolean', value: true, description: 'build kernel modules') +option('enable_ubsan', type: 'boolean', value: false, + description: 'Enables undefined behavior sanitizer') option('examples', type: 'string', value: '', description: 'Comma-separated list of examples to build by default') option('flexran_sdk', type: 'string', value: '', diff --git a/mk/rte.app.mk b/mk/rte.app.mk index ba5c39e01..9ea421a83 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -374,6 +374,16 @@ endif =20 MAPFLAGS =3D -Map=3D$@.map --cref =20 +# +# If UBSAN is enabled, all application will be compiled with +# '-fsanitize=3Dundefined' flag +# +ifeq ($(CONFIG_RTE_UBSAN),y) +ifeq ($(UBSAN_ENABLE),y) +CFLAGS +=3D -fsanitize=3Dundefined +endif +endif + .PHONY: all all: install =20 diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk index 4df8849a0..33f5746c8 100644 --- a/mk/rte.lib.mk +++ b/mk/rte.lib.mk @@ -29,6 +29,18 @@ CPU_LDFLAGS +=3D --version-script=3D$(SRCDIR)/$(EXPORT_M= AP) endif endif =20 +# +# If UBSAN is enabled, lib to undergo check can be chosen +# by setting UBSAN_SANITIZE=3Dy in respective lib Makefile +# else set CONFIG_RTE_UBSAN_SANITIZE_ALL=3Dy to enforce check +# on entire repo. +# +ifeq ($(CONFIG_RTE_UBSAN),y) +ifeq ($(UBSAN_ENABLE),y) +CFLAGS +=3D $(if $(patsubst %n,,$(CONFIG_RTE_UBSAN_SANITIZE_ALL)$(UBSAN_SA= NITIZE)) \ + , -fsanitize=3Dundefined) +endif +endif =20 _BUILD =3D $(LIB) PREINSTALL =3D $(SYMLINK-FILES-y) diff --git a/mk/toolchain/clang/rte.vars.mk b/mk/toolchain/clang/rte.vars.m= k index 3c49dc568..623780106 100644 --- a/mk/toolchain/clang/rte.vars.mk +++ b/mk/toolchain/clang/rte.vars.mk @@ -56,5 +56,9 @@ ifeq ($(shell test $(CLANG_MAJOR_VERSION) -ge 4 && echo 1= ), 1) WERROR_FLAGS +=3D -Wno-address-of-packed-member endif =20 +ifeq ($(CONFIG_RTE_UBSAN),y) +UBSAN_ENABLE :=3D y +endif + export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS diff --git a/mk/toolchain/gcc/rte.vars.mk b/mk/toolchain/gcc/rte.vars.mk index b852fcfd7..d40e14d61 100644 --- a/mk/toolchain/gcc/rte.vars.mk +++ b/mk/toolchain/gcc/rte.vars.mk @@ -90,5 +90,13 @@ endif # disable packed member unalign warnings WERROR_FLAGS +=3D -Wno-address-of-packed-member =20 +ifeq ($(CONFIG_RTE_UBSAN),y) +ifeq ($(shell test $(GCC_VERSION) -lt 49 && echo 1), 1) +$(warning UBSAN not supported gcc < 4.9) +else +UBSAN_ENABLE =3D y +endif +endif + export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS --=20 2.18.0