DPDK patches and discussions
 help / color / mirror / Atom feed
From: David Marchand <david.marchand@redhat.com>
To: Zhihong Peng <zhihongx.peng@intel.com>,
	Bruce Richardson <bruce.richardson@intel.com>
Cc: "Burakov, Anatoly" <anatoly.burakov@intel.com>,
	 "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
	 Stephen Hemminger <stephen@networkplumber.org>,
	dev <dev@dpdk.org>,  Xueqin Lin <xueqin.lin@intel.com>,
	Thomas Monjalon <thomas@monjalon.net>
Subject: Re: [dpdk-dev] [PATCH v6 1/2] Enable ASan for memory detector on DPDK
Date: Thu, 30 Sep 2021 10:20:00 +0200	[thread overview]
Message-ID: <CAJFAV8yzYJtwpnx+jsaB+X7q7POT86uKC3RS-FB9t7p=kTyGDw@mail.gmail.com> (raw)
In-Reply-To: <20210930052724.195414-1-zhihongx.peng@intel.com>

Hello,

I see v6 is superseded in pw, I have been cleaning my queue... maybe my fault.


On Thu, Sep 30, 2021 at 7:37 AM <zhihongx.peng@intel.com> wrote:
>
> From: Zhihong Peng <zhihongx.peng@intel.com>
>
> AddressSanitizer (ASan) is a google memory error detect
> standard tool. It could help to detect use-after-free and
> {heap,stack,global}-buffer overflow bugs in C/C++ programs,
> print detailed error information when error happens, large
> improve debug efficiency.
>
> `AddressSanitizer
> <https://github.com/google/sanitizers/wiki/AddressSanitizer>` (ASan)
> is a widely-used debugging tool to detect memory access errors.
> It helps detect issues like use-after-free, various kinds of buffer
> overruns in C/C++ programs, and other similar errors, as well as
> printing out detailed debug information whenever an error is detected.

This patch mixes how to use ASan and instrumenting the DPDK mem allocator.

I would split this patch in two.

The first patch can add the documentation on enabling/using ASan and
describe the known issues on enabling it.
I'd find it better (from a user pov) if we hide all those details
about b_lundef and installation of libasan on Centos.

Something like (only quickly tested):

diff --git a/config/meson.build b/config/meson.build
index 4cdf589e20..7d8b71da79 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -411,6 +411,33 @@ if get_option('b_lto')
     endif
 endif

+if get_option('b_sanitize') == 'address'
+    asan_dep = cc.find_library('asan', required: true)
+    if (not cc.links('int main(int argc, char *argv[]) { return 0; }',
+                     dependencies: asan_dep))
+        error('broken dependency, "libasan"')
+    endif
+    add_project_link_arguments('-lasan', language: 'c')
+    dpdk_extra_ldflags += '-lasan'
+endif
+
 if get_option('default_library') == 'both'
     error( '''
  Unsupported value "both" for "default_library" option.


Bruce, do you see an issue with this approach?


Then a second patch adds the rte_malloc instrumentation, with a check
at configuration time.

     endif
     add_project_link_arguments('-lasan', language: 'c')
     dpdk_extra_ldflags += '-lasan'
+    if arch_subdir == 'x86'
+        asan_check_code = '''
+#ifdef __SANITIZE_ADDRESS__
+#define RTE_MALLOC_ASAN
+#elif defined(__has_feature)
+# if __has_feature(address_sanitizer)
+#define RTE_MALLOC_ASAN
+# endif
+#endif
+
+#ifndef RTE_MALLOC_ASAN
+#error ASan not available.
+#endif
+'''
+        if cc.compiles(asan_check_code)
+            dpdk_conf.set10('RTE_MALLOC_ASAN', true)
+        endif
+    endif
 endif

 if get_option('default_library') == 'both'


Few more comments:


>
> DPDK ASan functionality is currently only supported Linux x86_64.
> Support other platforms, need to define ASAN_SHADOW_OFFSET value
> according to google ASan document.
>
> Here is an example of heap-buffer-overflow bug:
>         ......
>         char *p = rte_zmalloc(NULL, 7, 0);
>         p[7] = 'a';
>         ......
>
> Here is an example of use-after-free bug:
>         ......
>         char *p = rte_zmalloc(NULL, 7, 0);
>         rte_free(p);
>         *p = 'a';
>         ......
>
> If you want to use this feature,
> you need to add below compilation options when compiling code:
> -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address
> "-Dbuildtype=debug": This is a non-essential option. When this option
> is added, if a memory error occurs, ASan can clearly show where the
> code is wrong.
> "-Db_lundef=false": When use clang to compile DPDK, this option must
> be added.
>
> Signed-off-by: Xueqin Lin <xueqin.lin@intel.com>
> Signed-off-by: Zhihong Peng <zhihongx.peng@intel.com>
> ---
>  devtools/words-case.txt         |   1 +
>  doc/guides/prog_guide/ASan.rst  | 108 +++++++++++++++++
>  doc/guides/prog_guide/index.rst |   1 +
>  examples/helloworld/main.c      |   5 +
>  lib/eal/common/malloc_elem.c    |  26 +++-
>  lib/eal/common/malloc_elem.h    | 204 +++++++++++++++++++++++++++++++-
>  lib/eal/common/malloc_heap.c    |  12 ++
>  lib/eal/common/rte_malloc.c     |   9 +-
>  8 files changed, 361 insertions(+), 5 deletions(-)
>  create mode 100644 doc/guides/prog_guide/ASan.rst
>
> diff --git a/devtools/words-case.txt b/devtools/words-case.txt
> index 0bbad48626..3655596d47 100644
> --- a/devtools/words-case.txt
> +++ b/devtools/words-case.txt
> @@ -86,3 +86,4 @@ VXLAN
>  Windows
>  XDP
>  XOR
> +ASan

Alphabetical order please.


> diff --git a/doc/guides/prog_guide/ASan.rst b/doc/guides/prog_guide/ASan.rst

Filenames are lowercase in the doc.


> new file mode 100644
> index 0000000000..7145a3b1a1
> --- /dev/null
> +++ b/doc/guides/prog_guide/ASan.rst
> @@ -0,0 +1,108 @@
> +.. Copyright (c) <2021>, Intel Corporation
> +   All rights reserved.
> +
> +Memory error detect standard tool - AddressSanitizer(ASan)
> +==========================================================
> +
> +AddressSanitizer (ASan) is a google memory error detect
> +standard tool. It could help to detect use-after-free and
> +{heap,stack,global}-buffer overflow bugs in C/C++ programs,
> +print detailed error information when error happens, large
> +improve debug efficiency.
> +
> +By referring to its implementation algorithm
> +(https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm),
> +enabled heap-buffer-overflow and use-after-free functions on DPDK.
> +DPDK ASan function currently only supports on Linux x86_64.
> +
> +AddressSanitizer is a part of LLVM(3.1+)and GCC(4.8+).

missing spaces around ().


> +
> +Example heap-buffer-overflow error
> +----------------------------------
> +
> +Following error was reported when ASan was enabled::
> +
> +    Applied 9 bytes of memory, but accessed the 10th byte of memory,
> +    so heap-buffer-overflow appeared.
> +
> +Below code results in this error::
> +
> +    char *p = rte_zmalloc(NULL, 9, 0);
> +    if (!p) {
> +        printf("rte_zmalloc error.");
> +        return -1;
> +    }
> +    p[9] = 'a';
> +
> +The error log::
> +
> +    ==49433==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f773fafa249 at pc 0x5556b13bdae4 bp 0x7ffeb4965e40 sp 0x7ffeb4965e30 WRITE of size 1 at 0x7f773fafa249 thread T0
> +    #0 0x5556b13bdae3 in asan_heap_buffer_overflow ../app/test/test_asan_heap_buffer_overflow.c:25

Please update this example since the unit test has been removed.


> +    #1 0x5556b043e9d4 in cmd_autotest_parsed ../app/test/commands.c:71
> +    #2 0x5556b1cdd4b0 in cmdline_parse ../lib/cmdline/cmdline_parse.c:290
> +    #3 0x5556b1cd8987 in cmdline_valid_buffer ../lib/cmdline/cmdline.c:26
> +    #4 0x5556b1ce477a in rdline_char_in ../lib/cmdline/cmdline_rdline.c:421
> +    #5 0x5556b1cd923e in cmdline_in ../lib/cmdline/cmdline.c:149
> +    #6 0x5556b1cd9769 in cmdline_interact ../lib/cmdline/cmdline.c:223
> +    #7 0x5556b045f53b in main ../app/test/test.c:234
> +    #8 0x7f7f1eba90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
> +    #9 0x5556b043e70d in _start (/home/pzh/yyy/x86_64-native-linuxapp-gcc/app/test/dpdk-test+0x7ce70d)
> +
> +    Address 0x7f773fafa249 is a wild pointer.
> +    SUMMARY: AddressSanitizer: heap-buffer-overflow ../app/test/test_asan_heap_buffer_overflow.c:25 in asan_heap_buffer_overflow
> +
> +Example use-after-free error
> +----------------------------
> +
> +Following error was reported when ASan was enabled::
> +
> +    Applied for 9 bytes of memory, and accessed the first byte after
> +    released, so heap-use-after-free appeared.
> +
> +Below code results in this error::
> +
> +    char *p = rte_zmalloc(NULL, 9, 0);
> +    if (!p) {
> +        printf("rte_zmalloc error.");
> +        return -1;
> +    }
> +    rte_free(p);
> +    *p = 'a';
> +
> +The error log::
> +
> +    ==49478==ERROR: AddressSanitizer: heap-use-after-free on address 0x7fe2ffafa240 at pc 0x56409b084bc8 bp 0x7ffef62c57d0 sp 0x7ffef62c57c0 WRITE of size 1 at 0x7fe2ffafa240 thread T0
> +    #0 0x56409b084bc7 in asan_use_after_free ../app/test/test_asan_use_after_free.c:26

Idem.


> +    #1 0x56409a1059d4 in cmd_autotest_parsed ../app/test/commands.c:71
> +    #2 0x56409b9a44b0 in cmdline_parse ../lib/cmdline/cmdline_parse.c:290
> +    #3 0x56409b99f987 in cmdline_valid_buffer ../lib/cmdline/cmdline.c:26
> +    #4 0x56409b9ab77a in rdline_char_in ../lib/cmdline/cmdline_rdline.c:421
> +    #5 0x56409b9a023e in cmdline_in ../lib/cmdline/cmdline.c:149
> +    #6 0x56409b9a0769 in cmdline_interact ../lib/cmdline/cmdline.c:223
> +    #7 0x56409a12653b in main ../app/test/test.c:234
> +    #8 0x7feafafc20b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
> +    #9 0x56409a10570d in _start (/home/pzh/yyy/x86_64-native-linuxapp-gcc/app/test/dpdk-test+0x7ce70d)
> +
> +    Address 0x7fe2ffafa240 is a wild pointer.
> +    SUMMARY: AddressSanitizer: heap-use-after-free ../app/test/test_asan_use_after_free.c:26 in asan_use_after_free


-- 
David Marchand


  parent reply	other threads:[~2021-09-30  8:20 UTC|newest]

Thread overview: 117+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-10  2:01 [dpdk-dev] [PATCH] Enable AddressSanitizer feature " zhihongx.peng
2021-09-10  2:47 ` Stephen Hemminger
2021-09-13  5:27   ` Peng, ZhihongX
2021-09-13 15:05     ` Stephen Hemminger
2021-09-13 15:22       ` Bruce Richardson
2021-09-13 15:45         ` Stephen Hemminger
2021-09-14  3:17           ` Lin, Xueqin
2021-09-14  3:11         ` Lin, Xueqin
2021-09-10 17:58 ` David Christensen
2021-09-13  5:35   ` Peng, ZhihongX
2021-09-17 20:50     ` David Christensen
2021-09-18  7:21       ` Peng, ZhihongX
2021-09-20 19:41         ` David Christensen
2021-09-21  8:29           ` David Marchand
2021-09-21 10:15             ` Jerin Jacob
2021-09-22  7:32             ` Peng, ZhihongX
2021-09-16  1:38 ` [dpdk-dev] [PATCH v2] " zhihongx.peng
2021-09-17  8:23   ` David Marchand
2021-09-17  9:12     ` Peng, ZhihongX
2021-09-17 14:58       ` Stephen Hemminger
2021-09-18  6:36         ` Peng, ZhihongX
2021-09-18  7:41   ` [dpdk-dev] [PATCH v3] " zhihongx.peng
2021-09-20 10:03     ` David Marchand
2021-09-22  7:26       ` Peng, ZhihongX
2021-09-24  2:20     ` [dpdk-dev] [PATCH v4 1/2] Enable ASan for memory detector " zhihongx.peng
2021-09-24  2:20       ` [dpdk-dev] [PATCH v4 2/2] lib/pipeline: Fix gcc compilation error using ASan zhihongx.peng
2021-09-24  9:33       ` [dpdk-dev] [PATCH v5 1/2] Enable ASan for memory detector on DPDK zhihongx.peng
2021-09-24  9:33         ` [dpdk-dev] [PATCH v5 2/2] Fix gcc compilation error using ASan zhihongx.peng
2021-09-24 10:03       ` [dpdk-dev] [PATCH v5 1/2] Enable ASan for memory detector on DPDK zhihongx.peng
2021-09-24 10:03         ` [dpdk-dev] [PATCH v5 2/2] lib/pipeline: Fix gcc compilation error using ASan zhihongx.peng
2021-09-30  5:27         ` [dpdk-dev] [PATCH v6 1/2] Enable ASan for memory detector on DPDK zhihongx.peng
2021-09-30  5:27           ` [dpdk-dev] [PATCH v6 2/2] lib/pipeline: Fix gcc compilation error using ASan zhihongx.peng
2021-09-30  8:29             ` [dpdk-dev] [dpdk-stable] " David Marchand
2021-10-12  2:41               ` Peng, ZhihongX
2021-09-30  8:20           ` David Marchand [this message]
2021-10-08  8:07             ` [dpdk-dev] [PATCH v6 1/2] Enable ASan for memory detector on DPDK Peng, ZhihongX
2021-10-08  8:30               ` David Marchand
2021-10-12  5:41                 ` Peng, ZhihongX
2021-10-12  7:17             ` Peng, ZhihongX
2021-10-13  7:59             ` Bruce Richardson
2021-10-14  6:33               ` Peng, ZhihongX
2021-10-14  6:53                 ` Peng, ZhihongX
2021-09-30 12:59         ` zhihongx.peng
2021-09-30 12:59           ` [dpdk-dev] [PATCH v6 2/2] lib/pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-09-30 13:59           ` [dpdk-dev] [PATCH v6 1/2] Enable ASan for memory detector on DPDK Burakov, Anatoly
2021-09-30 18:49           ` David Marchand
2021-10-08  9:17           ` [dpdk-dev] [PATCH v7 1/3] " zhihongx.peng
2021-10-08  9:17             ` [dpdk-dev] [PATCH v7 2/3] DPDK code adapts to ASan zhihongx.peng
2021-10-08  9:17             ` [dpdk-dev] [PATCH v7 3/3] lib/pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-10-11  6:28             ` [dpdk-dev] [PATCH v8 1/3] Enable ASan for memory detector on DPDK zhihongx.peng
2021-10-11  6:28               ` [dpdk-dev] [PATCH v8 2/3] DPDK code adapts to ASan zhihongx.peng
2021-10-11  6:28               ` [dpdk-dev] [PATCH v8 3/3] lib/pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-10-12  9:43               ` [dpdk-dev] [PATCH v9 1/3] Enable ASan for memory detector on DPDK zhihongx.peng
2021-10-12  9:43                 ` [dpdk-dev] [PATCH v9 2/3] DPDK code adapts to ASan zhihongx.peng
2021-10-13 16:45                   ` David Marchand
2021-10-14 11:45                     ` Peng, ZhihongX
2021-10-12  9:43                 ` [dpdk-dev] [PATCH v9 3/3] pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-10-15 14:27                   ` [dpdk-dev] [PATCH v10 1/4] Enable ASan for memory detector on DPDK zhihongx.peng
2021-10-15 14:27                     ` [dpdk-dev] [PATCH v10 2/4] DPDK code adapts to ASan zhihongx.peng
2021-10-15 14:27                     ` [dpdk-dev] [PATCH v10 3/4] pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-10-15 14:27                     ` [dpdk-dev] [PATCH v10 4/4] performance-thread: Fix cross compilation failed zhihongx.peng
2021-10-19  9:02                     ` [dpdk-dev] [PATCH v10 1/4] Enable ASan for memory detector on DPDK Mcnamara, John
2021-10-19  9:28                       ` Peng, ZhihongX
2021-10-15 15:11                   ` zhihongx.peng
2021-10-15 15:11                     ` [dpdk-dev] [PATCH v10 2/4] DPDK code adapts to ASan zhihongx.peng
2021-10-15 15:11                     ` [dpdk-dev] [PATCH v10 3/4] pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-10-18 12:21                       ` Dumitrescu, Cristian
2021-10-18 12:54                         ` Peng, ZhihongX
2021-10-19 11:26                           ` Dumitrescu, Cristian
2021-10-19 12:11                             ` Peng, ZhihongX
2021-10-15 15:11                     ` [dpdk-dev] [PATCH v10 4/4] performance-thread: Fix cross compilation failed zhihongx.peng
2021-10-19  6:02                       ` Peng, ZhihongX
2021-10-19 10:12                       ` [dpdk-dev] [PATCH v11 1/4] Enable ASan for memory detector on DPDK zhihongx.peng
2021-10-19 10:12                         ` [dpdk-dev] [PATCH v11 2/4] DPDK code adapts to ASan zhihongx.peng
2021-10-19 10:12                         ` [dpdk-dev] [PATCH v11 3/4] pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-10-19 10:12                         ` [dpdk-dev] [PATCH v11 4/4] performance-thread: Fix cross compilation failed zhihongx.peng
2021-10-19 10:37                           ` Bruce Richardson
2021-10-19 13:04                           ` [dpdk-dev] [PATCH v12 1/4] Enable ASan for memory detector on DPDK zhihongx.peng
2021-10-19 13:04                             ` [dpdk-dev] [PATCH v12 2/4] DPDK code adapts to ASan zhihongx.peng
2021-10-19 13:04                             ` [dpdk-dev] [PATCH v12 3/4] pipeline: Fix compilation error with gcc ASan zhihongx.peng
2021-10-19 13:04                             ` [dpdk-dev] [PATCH v12 4/4] performance-thread: Fix cross compilation failed zhihongx.peng
2021-10-19 13:58                           ` [dpdk-dev] [PATCH v12 1/4] Enable ASan Address Sanitization zhihongx.peng
2021-10-19 13:58                             ` [dpdk-dev] [PATCH v12 2/4] DPDK code adapts to ASan zhihongx.peng
2021-10-19 13:58                             ` [dpdk-dev] [PATCH v12 3/4] Code changes to avoid the ASan error zhihongx.peng
2021-10-19 14:05                               ` Dumitrescu, Cristian
2021-10-19 13:58                             ` [dpdk-dev] [PATCH v12 4/4] performance-thread: Fix cross compilation failed zhihongx.peng
2021-10-19 14:47                             ` [dpdk-dev] [PATCH v12 1/4] Enable ASan Address Sanitization Peng, ZhihongX
2021-10-19 15:17                               ` Mcnamara, John
2021-10-20  1:55                                 ` Peng, ZhihongX
2021-10-19 15:20                               ` David Marchand
2021-10-20  2:41                                 ` Lin, Xueqin
2021-10-19 15:15                           ` [dpdk-dev] [PATCH v12 1/4] Enable ASan AddressSanitizer zhihongx.peng
2021-10-19 15:15                             ` [dpdk-dev] [PATCH v12 2/4] DPDK code adapts to ASan zhihongx.peng
2021-10-19 15:15                             ` [dpdk-dev] [PATCH v12 3/4] Code changes to avoid the ASan error zhihongx.peng
2021-10-19 15:15                             ` [dpdk-dev] [PATCH v12 4/4] performance-thread: Fix cross compilation failed zhihongx.peng
2021-10-20  7:46                               ` [dpdk-dev] [PATCH v13 1/4] enable ASan AddressSanitizer zhihongx.peng
2021-10-20  7:46                                 ` [dpdk-dev] [PATCH v13 2/4] DPDK code adapts to ASan zhihongx.peng
2021-10-29  9:23                                   ` David Marchand
2021-10-29 12:54                                     ` Peng, ZhihongX
2021-10-20  7:46                                 ` [dpdk-dev] [PATCH v13 3/4] code changes to avoid the ASan error zhihongx.peng
2021-10-20  7:46                                 ` [dpdk-dev] [PATCH v13 4/4] performance-thread: avoid cross compilation fail zhihongx.peng
2021-10-25  1:58                                 ` [dpdk-dev] [PATCH v13 1/4] enable ASan AddressSanitizer Peng, ZhihongX
2021-10-29  9:21                                 ` David Marchand
2021-10-29 12:48                                   ` Peng, ZhihongX
2021-10-29 14:50                                   ` David Marchand
2021-11-01  3:37                                     ` Lin, Xueqin
2021-11-02 17:29                                       ` David Marchand
2021-11-03  9:36                                         ` Lin, Xueqin
2021-11-03  9:44                                           ` David Marchand
2021-10-19 10:24                         ` [dpdk-dev] [PATCH v11 1/4] Enable ASan for memory detector on DPDK Peng, ZhihongX
2021-10-19  5:52                     ` [dpdk-dev] [PATCH v10 " Peng, ZhihongX
2021-10-13 16:44                 ` [dpdk-dev] [PATCH v9 1/3] " David Marchand
2021-10-14  6:46                   ` Peng, ZhihongX
2021-10-14  7:04                     ` Thomas Monjalon
2021-10-14 11:42                   ` Peng, ZhihongX
2021-09-27 12:02     ` [dpdk-dev] [PATCH v3] Enable AddressSanitizer feature " Burakov, Anatoly
2021-09-30  5:18       ` Peng, ZhihongX

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJFAV8yzYJtwpnx+jsaB+X7q7POT86uKC3RS-FB9t7p=kTyGDw@mail.gmail.com' \
    --to=david.marchand@redhat.com \
    --cc=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=konstantin.ananyev@intel.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    --cc=xueqin.lin@intel.com \
    --cc=zhihongx.peng@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).