Re: [dpdk-dev] [PATCH v3] mk: add support for UBSAN

[Aaron Conole <aconole@redhat.com>]

Harman Kalra <hkalra@marvell.com> writes:

> UndefinedBehaviorSanitizer (UBSan) is a fast undefined behavior
> detector. UBSan modifies the program at compile-time to catch
> various kinds of undefined behavior during program execution.
>
> This patch introduces support for UBSan to the DPDK.
>
> See: doc/guides/prog_guide/ubsan.rst for more information.
>
> Signed-off-by: Harman Kalra <hkalra@marvell.com>
> ---

Sorry I am coming to this late.

> V2:
> * Addressed review comment regarding combining two
> ifeq into one.
>
> V3:
> * Added version change logs.
>
>  config/common_base                     |   6 ++
>  config/meson.build                     |  15 ++++
>  doc/guides/prog_guide/index.rst        |   1 +
>  doc/guides/prog_guide/ubsan.rst        | 112 +++++++++++++++++++++++++
>  doc/guides/rel_notes/release_19_11.rst |   7 ++
>  meson_options.txt                      |   2 +
>  mk/rte.app.mk                          |   8 ++
>  mk/rte.lib.mk                          |  12 +++
>  mk/toolchain/clang/rte.vars.mk         |   4 +
>  mk/toolchain/gcc/rte.vars.mk           |   8 ++
>  10 files changed, 175 insertions(+)
>  create mode 100644 doc/guides/prog_guide/ubsan.rst
>
> diff --git a/config/common_base b/config/common_base
> index 914277856..f1bb3e0b2 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -1098,3 +1098,9 @@ CONFIG_RTE_APP_CRYPTO_PERF=y
>  # Compile the eventdev application
>  #
>  CONFIG_RTE_APP_EVENTDEV=y
> +
> +#
> +# Enable undefined behavior sanitizer
> +#
> +CONFIG_RTE_UBSAN=n
> +CONFIG_RTE_UBSAN_SANITIZE_ALL=n
> diff --git a/config/meson.build b/config/meson.build
> index 2b1cb92e7..a43c23f50 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -238,3 +238,18 @@ if get_option('b_lto')
>  		add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
>  	endif
>  endif
> +
> +# enable ubsan
> +if get_option('enable_ubsan')
> +	if cc.has_argument('-fsanitize=undefined')
> +		ubsan_dep = cc.find_library('libubsan', required: false)
> +		if ubsan_dep.found()
> +			add_project_arguments('-fsanitize=undefined', language: 'c')
> +			add_project_link_arguments('-fsanitize=undefined', language: 'c')
> +		else
> +			message('libubsan not found, UBSAN cannot be enabled')
> +		endif
> +	else
> +		message('gcc version does not support UBSAN')
> +	endif
> +endif

Why is this needed?  AFAIK, meson supports
-Db_sanitize=undefined,address,... so what do we gain by this?

Especially since for ubsan and asan, -fno-omit-frame-pointer is needed
for useful backtraces (which the meson module does for us).  This
support has been in since 2017, afaict.

> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst
> index dc4851c57..911b82a41 100644
> --- a/doc/guides/prog_guide/index.rst
> +++ b/doc/guides/prog_guide/index.rst
> @@ -67,4 +67,5 @@ Programmer's Guide
>      writing_efficient_code
>      lto
>      profile_app
> +    ubsan
>      glossary
> diff --git a/doc/guides/prog_guide/ubsan.rst b/doc/guides/prog_guide/ubsan.rst
> new file mode 100644
> index 000000000..cb19f3bd9
> --- /dev/null
> +++ b/doc/guides/prog_guide/ubsan.rst
> @@ -0,0 +1,112 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Marvell International Ltd.
> +
> +The Undefined Behavior Sanitizer - UBSan
> +=======================================+
> +UndefinedBehaviorSanitizer (UBSan) is a runtime undefined behavior detector.
> +UBSan uses compile-time instrumentation and modifies the program by adding
> +some stubs which perform certain checks before operations that might cause
> +undefined behaviour. If some UB detected, respective _UBSan_handle_* handlers
> +(which are defined in libUBSan library) are called to prints the error message.
> +
> +Some examples of undefined behaviour checks:
> +
> +* Misaligned memory access
> +* Signed integer overflow
> +* Load from/store to an object with insufficient space.
> +* Integer divide by zero as well as INT_MIN / -1 division
> +* Out-of-bounds memory accesses.
> +* Null argument declared with nonnull attribute, returned null from function
> +  which never returns null, null ptr dereference
> +* Variable size array with non-positive length
> +
> +GCC supports this feature since 4.9, however GCC 5.0 onwards has many more
> +checkers implemented.
> +
> +Example UBSan error
> +--------------------
> +
> +Following error was reported when UBSan was enabled:
> +
> +.. code-block:: console
> +
> +    drivers/net/octeontx2/otx2_stats.c:82:26: runtime error: left shift of
> +    1 by 31 places cannot be represented in type 'int'
> +
> +Code responsible for this error:
> +
> +.. code-block:: c
> +
> +    if (dev->txmap[i] & (1 << 31)) {
> +
> +To fix this error:
> +
> +.. code-block:: c
> +
> +    if (dev->txmap[i] & (1U << 31)) {
> +
> +Usage
> +-----
> +
> +make build
> +^^^^^^^^^^
> +
> +To enable UBSan, enable following configuration:
> +
> +.. code-block:: console
> +
> +    CONFIG_RTE_UBSAN=y
> +
> +UBSan framework supports three modes:
> +
> +1. Enable UBSan on the entire DPDK source code - set following configuration:
> +
> +.. code-block:: console
> +
> +    CONFIG_RTE_UBSAN_SANITIZE_ALL=y
> +
> +2. Enable UBSan on a particular library or PMD - add the following line to the
> +   respective Makefile of the library or PMD
> +   (make sure ``CONFIG_RTE_UBSAN_SANITIZE_ALL=n``). This will instrument only
> +   the library or PMD and not the entire repository.
> +
> +.. code-block:: console
> +
> +    UBSAN_SANITIZE := y
> +
> +3. Disable UBSan for a particular library or PMD - add the following line to
> +   the respective Makefile of the library or PMD. Make sure
> +   ``CONFIG_RTE_UBSAN_SANITIZE_ALL=y`` config is set. This will instrument
> +   entire DPDK repository but not this specific library or PMD.
> +
> +.. code-block:: console
> +
> +    UBSAN_SANITIZE := n
> +
> +.. Note::
> +
> +  Standard DPDK applications like test, testpmd, etc. cannot be
> +  chosen explicitly for UBSan check, like libraries or PMD. The reason is,
> +  say UBSan is enabled for library X, and ``UBSAN_SANITIZE=y`` is not added
> +  in Makefile of app Y which uses X APIs. This will lead to undefined
> +  reference to _UBSan_handle_* handlers as Y is not compiled with UBSan flags.
> +  Hence UBSan check is enabled for all standard DPDK applications as soon as
> +  ``CONFIG_RTE_UBSAN=y`` is set.
> +
> +meson build
> +^^^^^^^^^^^
> +
> +To enable UBSan in meson build system, use following meson build command:
> +
> +**Example usage:**
> +
> +.. code-block:: console
> +
> + meson build -Denable_ubsan=true
> + ninja -C build
> +
> +.. Note::
> +
> +  Meson build works only in one mode i.e. UBSan can be enabled for
> +  the entire DPDK sources and not individual libraries or PMD, like make build.
> diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
> index c0045a91f..61fd1bcc2 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -294,6 +294,13 @@ New Features
>  
>    See :doc:`../prog_guide/lto` for more information:
>  
> +* **Added Undefined Behavior Sanitizer framework.**
> +
> +  UBSan is a fast runtime undefined behavior detector which uses compile-time
> +  instrumentation and modifies the program by adding some stubs that perform
> +  certain checks before operations that might cause undefined behavior.
> +
> +  See :doc:`../prog_guide/ubsan` for more information:
>  
>  
>  Removed Items
> diff --git a/meson_options.txt b/meson_options.txt
> index 89650b0e9..f3b42d2b1 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -10,6 +10,8 @@ option('enable_docs', type: 'boolean', value: false,
>  	description: 'build documentation')
>  option('enable_kmods', type: 'boolean', value: true,
>  	description: 'build kernel modules')
> +option('enable_ubsan', type: 'boolean', value: false,
> +	description: 'Enables undefined behavior sanitizer')
>  option('examples', type: 'string', value: '',
>  	description: 'Comma-separated list of examples to build by default')
>  option('flexran_sdk', type: 'string', value: '',
> diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> index 683e3a4e3..1304227cf 100644
> --- a/mk/rte.app.mk
> +++ b/mk/rte.app.mk
> @@ -385,6 +385,14 @@ endif
>  
>  MAPFLAGS = -Map=$@.map --cref
>  
> +#
> +# If UBSAN is enabled, all application will be compiled with
> +# '-fsanitize=undefined' flag
> +#
> +ifeq ($(CONFIG_RTE_UBSAN)$(UBSAN_ENABLE),yy)
> +CFLAGS += -fsanitize=undefined
> +endif
> +
>  .PHONY: all
>  all: install
>  
> diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
> index 4df8849a0..33f5746c8 100644
> --- a/mk/rte.lib.mk
> +++ b/mk/rte.lib.mk
> @@ -29,6 +29,18 @@ CPU_LDFLAGS += --version-script=$(SRCDIR)/$(EXPORT_MAP)
>  endif
>  endif
>  
> +#
> +# If UBSAN is enabled, lib to undergo check can be chosen
> +# by setting UBSAN_SANITIZE=y in respective lib Makefile
> +# else set CONFIG_RTE_UBSAN_SANITIZE_ALL=y to enforce check
> +# on entire repo.
> +#
> +ifeq ($(CONFIG_RTE_UBSAN),y)
> +ifeq ($(UBSAN_ENABLE),y)
> +CFLAGS += $(if $(patsubst %n,,$(CONFIG_RTE_UBSAN_SANITIZE_ALL)$(UBSAN_SANITIZE)) \
> +		, -fsanitize=undefined)
> +endif
> +endif
>  
>  _BUILD = $(LIB)
>  PREINSTALL = $(SYMLINK-FILES-y)
> diff --git a/mk/toolchain/clang/rte.vars.mk b/mk/toolchain/clang/rte.vars.mk
> index 3c49dc568..623780106 100644
> --- a/mk/toolchain/clang/rte.vars.mk
> +++ b/mk/toolchain/clang/rte.vars.mk
> @@ -56,5 +56,9 @@ ifeq ($(shell test $(CLANG_MAJOR_VERSION) -ge 4 && echo 1), 1)
>  WERROR_FLAGS += -Wno-address-of-packed-member
>  endif
>  
> +ifeq ($(CONFIG_RTE_UBSAN),y)
> +UBSAN_ENABLE := y
> +endif
> +
>  export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
>  export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
> diff --git a/mk/toolchain/gcc/rte.vars.mk b/mk/toolchain/gcc/rte.vars.mk
> index 9fc704193..43e7d139b 100644
> --- a/mk/toolchain/gcc/rte.vars.mk
> +++ b/mk/toolchain/gcc/rte.vars.mk
> @@ -102,5 +102,13 @@ endif
>  # disable packed member unalign warnings
>  WERROR_FLAGS += -Wno-address-of-packed-member
>  
> +ifeq ($(CONFIG_RTE_UBSAN),y)
> +ifeq ($(shell test $(GCC_VERSION) -lt 49 && echo 1), 1)
> +$(warning UBSAN not supported gcc < 4.9)
> +else
> +UBSAN_ENABLE = y
> +endif
> +endif
> +
>  export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
>  export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS