From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D408AA0C41; Thu, 30 Sep 2021 10:20:19 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 61B28410EB; Thu, 30 Sep 2021 10:20:19 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id 8EC40410EA for ; Thu, 30 Sep 2021 10:20:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1632990017; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gRaHfcYvRSFIc9hSxOcjWUy2PZN4FvJAydBmDGdhEMs=; b=TgoWduNwAjKYPYOIn16RJMqFuefO25pPI4AX2ALAXz4c5Zr7uXbWn07YKsihzRo2qc+TKa lyr9kiwRLduLZ2OyvdoeLWFezYFGbuaFBjCFbisau+kXiYulGQS7TeUAIwjjEygqVwLXLu Mijq8wTWPm/urwC4PIdHBXHCqjNyw2I= Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com [209.85.167.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-425-9O4AadHZMRCWV4tSvHnblQ-1; Thu, 30 Sep 2021 04:20:14 -0400 X-MC-Unique: 9O4AadHZMRCWV4tSvHnblQ-1 Received: by mail-lf1-f72.google.com with SMTP id f21-20020ac25095000000b003fd0f3d19f4so3691532lfm.0 for ; Thu, 30 Sep 2021 01:20:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gRaHfcYvRSFIc9hSxOcjWUy2PZN4FvJAydBmDGdhEMs=; b=EBH37hQuOFL2UJ7sqkMSLL4ZqVCIkxw+PRsFjjyhl+YlmKY/1eVqeXLwtzAxEhbaiO 74qzS6NlA93SU3AJVh1JjIhgnuDLy+of6P6113QwB5ggJAjCdNp5ZaBxb6hJbjfg5Z5s lo1LE3Ce1k8auDZ/wSg+PeSo1hmad8PrVkrP9l0OsHCd0wTbgOopaundl+6h9sCyqY3n ItpQ65uTXfzD3HJ2GJcsi/FYcYOz3NmiR9kNb1lds9w54mJwh0qcW49yjEOxiHSUsfEa j2hd859xMx8qplyQ2bmvLfuz48g85dOifvu3goxQC1wJGEEoI+MGGjTjTcAw8qdV0aRZ OS8g== X-Gm-Message-State: AOAM533WDYkmmiU7b2+qrYmcVOrVKRiyy29xW8i8RFWONto30QnVxqnq ifqIVpA6XsJCuVZu2si7fQYXkRUYyiyIerOc8dVEw5ww/xKVK6VJ33Em5vc6/MwQNjHnKa2NOf+ Vq/+pvw/I8H7/XpnvSWE= X-Received: by 2002:a19:690e:: with SMTP id e14mr4420000lfc.553.1632990012517; Thu, 30 Sep 2021 01:20:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzo9gPoD6M1kbDS2SnPbLrzWBVKc180BvYFE+JDqRq5P9O92LRmAzzUMiBnYcgooOhpgMNhyjC5JviUvfcEwdI= X-Received: by 2002:a19:690e:: with SMTP id e14mr4419980lfc.553.1632990012156; Thu, 30 Sep 2021 01:20:12 -0700 (PDT) MIME-Version: 1.0 References: <20210924100310.4278-1-zhihongx.peng@intel.com> <20210930052724.195414-1-zhihongx.peng@intel.com> In-Reply-To: <20210930052724.195414-1-zhihongx.peng@intel.com> From: David Marchand Date: Thu, 30 Sep 2021 10:20:00 +0200 Message-ID: To: Zhihong Peng , Bruce Richardson Cc: "Burakov, Anatoly" , "Ananyev, Konstantin" , Stephen Hemminger , dev , Xueqin Lin , Thomas Monjalon Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v6 1/2] Enable ASan for memory detector on DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hello, I see v6 is superseded in pw, I have been cleaning my queue... maybe my fault. On Thu, Sep 30, 2021 at 7:37 AM wrote: > > From: Zhihong Peng > > AddressSanitizer (ASan) is a google memory error detect > standard tool. It could help to detect use-after-free and > {heap,stack,global}-buffer overflow bugs in C/C++ programs, > print detailed error information when error happens, large > improve debug efficiency. > > `AddressSanitizer > ` (ASan) > is a widely-used debugging tool to detect memory access errors. > It helps detect issues like use-after-free, various kinds of buffer > overruns in C/C++ programs, and other similar errors, as well as > printing out detailed debug information whenever an error is detected. This patch mixes how to use ASan and instrumenting the DPDK mem allocator. I would split this patch in two. The first patch can add the documentation on enabling/using ASan and describe the known issues on enabling it. I'd find it better (from a user pov) if we hide all those details about b_lundef and installation of libasan on Centos. Something like (only quickly tested): diff --git a/config/meson.build b/config/meson.build index 4cdf589e20..7d8b71da79 100644 --- a/config/meson.build +++ b/config/meson.build @@ -411,6 +411,33 @@ if get_option('b_lto') endif endif +if get_option('b_sanitize') == 'address' + asan_dep = cc.find_library('asan', required: true) + if (not cc.links('int main(int argc, char *argv[]) { return 0; }', + dependencies: asan_dep)) + error('broken dependency, "libasan"') + endif + add_project_link_arguments('-lasan', language: 'c') + dpdk_extra_ldflags += '-lasan' +endif + if get_option('default_library') == 'both' error( ''' Unsupported value "both" for "default_library" option. Bruce, do you see an issue with this approach? Then a second patch adds the rte_malloc instrumentation, with a check at configuration time. endif add_project_link_arguments('-lasan', language: 'c') dpdk_extra_ldflags += '-lasan' + if arch_subdir == 'x86' + asan_check_code = ''' +#ifdef __SANITIZE_ADDRESS__ +#define RTE_MALLOC_ASAN +#elif defined(__has_feature) +# if __has_feature(address_sanitizer) +#define RTE_MALLOC_ASAN +# endif +#endif + +#ifndef RTE_MALLOC_ASAN +#error ASan not available. +#endif +''' + if cc.compiles(asan_check_code) + dpdk_conf.set10('RTE_MALLOC_ASAN', true) + endif + endif endif if get_option('default_library') == 'both' Few more comments: > > DPDK ASan functionality is currently only supported Linux x86_64. > Support other platforms, need to define ASAN_SHADOW_OFFSET value > according to google ASan document. > > Here is an example of heap-buffer-overflow bug: > ...... > char *p = rte_zmalloc(NULL, 7, 0); > p[7] = 'a'; > ...... > > Here is an example of use-after-free bug: > ...... > char *p = rte_zmalloc(NULL, 7, 0); > rte_free(p); > *p = 'a'; > ...... > > If you want to use this feature, > you need to add below compilation options when compiling code: > -Dbuildtype=debug -Db_lundef=false -Db_sanitize=address > "-Dbuildtype=debug": This is a non-essential option. When this option > is added, if a memory error occurs, ASan can clearly show where the > code is wrong. > "-Db_lundef=false": When use clang to compile DPDK, this option must > be added. > > Signed-off-by: Xueqin Lin > Signed-off-by: Zhihong Peng > --- > devtools/words-case.txt | 1 + > doc/guides/prog_guide/ASan.rst | 108 +++++++++++++++++ > doc/guides/prog_guide/index.rst | 1 + > examples/helloworld/main.c | 5 + > lib/eal/common/malloc_elem.c | 26 +++- > lib/eal/common/malloc_elem.h | 204 +++++++++++++++++++++++++++++++- > lib/eal/common/malloc_heap.c | 12 ++ > lib/eal/common/rte_malloc.c | 9 +- > 8 files changed, 361 insertions(+), 5 deletions(-) > create mode 100644 doc/guides/prog_guide/ASan.rst > > diff --git a/devtools/words-case.txt b/devtools/words-case.txt > index 0bbad48626..3655596d47 100644 > --- a/devtools/words-case.txt > +++ b/devtools/words-case.txt > @@ -86,3 +86,4 @@ VXLAN > Windows > XDP > XOR > +ASan Alphabetical order please. > diff --git a/doc/guides/prog_guide/ASan.rst b/doc/guides/prog_guide/ASan.rst Filenames are lowercase in the doc. > new file mode 100644 > index 0000000000..7145a3b1a1 > --- /dev/null > +++ b/doc/guides/prog_guide/ASan.rst > @@ -0,0 +1,108 @@ > +.. Copyright (c) <2021>, Intel Corporation > + All rights reserved. > + > +Memory error detect standard tool - AddressSanitizer(ASan) > +========================================================== > + > +AddressSanitizer (ASan) is a google memory error detect > +standard tool. It could help to detect use-after-free and > +{heap,stack,global}-buffer overflow bugs in C/C++ programs, > +print detailed error information when error happens, large > +improve debug efficiency. > + > +By referring to its implementation algorithm > +(https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm), > +enabled heap-buffer-overflow and use-after-free functions on DPDK. > +DPDK ASan function currently only supports on Linux x86_64. > + > +AddressSanitizer is a part of LLVM(3.1+)and GCC(4.8+). missing spaces around (). > + > +Example heap-buffer-overflow error > +---------------------------------- > + > +Following error was reported when ASan was enabled:: > + > + Applied 9 bytes of memory, but accessed the 10th byte of memory, > + so heap-buffer-overflow appeared. > + > +Below code results in this error:: > + > + char *p = rte_zmalloc(NULL, 9, 0); > + if (!p) { > + printf("rte_zmalloc error."); > + return -1; > + } > + p[9] = 'a'; > + > +The error log:: > + > + ==49433==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f773fafa249 at pc 0x5556b13bdae4 bp 0x7ffeb4965e40 sp 0x7ffeb4965e30 WRITE of size 1 at 0x7f773fafa249 thread T0 > + #0 0x5556b13bdae3 in asan_heap_buffer_overflow ../app/test/test_asan_heap_buffer_overflow.c:25 Please update this example since the unit test has been removed. > + #1 0x5556b043e9d4 in cmd_autotest_parsed ../app/test/commands.c:71 > + #2 0x5556b1cdd4b0 in cmdline_parse ../lib/cmdline/cmdline_parse.c:290 > + #3 0x5556b1cd8987 in cmdline_valid_buffer ../lib/cmdline/cmdline.c:26 > + #4 0x5556b1ce477a in rdline_char_in ../lib/cmdline/cmdline_rdline.c:421 > + #5 0x5556b1cd923e in cmdline_in ../lib/cmdline/cmdline.c:149 > + #6 0x5556b1cd9769 in cmdline_interact ../lib/cmdline/cmdline.c:223 > + #7 0x5556b045f53b in main ../app/test/test.c:234 > + #8 0x7f7f1eba90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2) > + #9 0x5556b043e70d in _start (/home/pzh/yyy/x86_64-native-linuxapp-gcc/app/test/dpdk-test+0x7ce70d) > + > + Address 0x7f773fafa249 is a wild pointer. > + SUMMARY: AddressSanitizer: heap-buffer-overflow ../app/test/test_asan_heap_buffer_overflow.c:25 in asan_heap_buffer_overflow > + > +Example use-after-free error > +---------------------------- > + > +Following error was reported when ASan was enabled:: > + > + Applied for 9 bytes of memory, and accessed the first byte after > + released, so heap-use-after-free appeared. > + > +Below code results in this error:: > + > + char *p = rte_zmalloc(NULL, 9, 0); > + if (!p) { > + printf("rte_zmalloc error."); > + return -1; > + } > + rte_free(p); > + *p = 'a'; > + > +The error log:: > + > + ==49478==ERROR: AddressSanitizer: heap-use-after-free on address 0x7fe2ffafa240 at pc 0x56409b084bc8 bp 0x7ffef62c57d0 sp 0x7ffef62c57c0 WRITE of size 1 at 0x7fe2ffafa240 thread T0 > + #0 0x56409b084bc7 in asan_use_after_free ../app/test/test_asan_use_after_free.c:26 Idem. > + #1 0x56409a1059d4 in cmd_autotest_parsed ../app/test/commands.c:71 > + #2 0x56409b9a44b0 in cmdline_parse ../lib/cmdline/cmdline_parse.c:290 > + #3 0x56409b99f987 in cmdline_valid_buffer ../lib/cmdline/cmdline.c:26 > + #4 0x56409b9ab77a in rdline_char_in ../lib/cmdline/cmdline_rdline.c:421 > + #5 0x56409b9a023e in cmdline_in ../lib/cmdline/cmdline.c:149 > + #6 0x56409b9a0769 in cmdline_interact ../lib/cmdline/cmdline.c:223 > + #7 0x56409a12653b in main ../app/test/test.c:234 > + #8 0x7feafafc20b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2) > + #9 0x56409a10570d in _start (/home/pzh/yyy/x86_64-native-linuxapp-gcc/app/test/dpdk-test+0x7ce70d) > + > + Address 0x7fe2ffafa240 is a wild pointer. > + SUMMARY: AddressSanitizer: heap-use-after-free ../app/test/test_asan_use_after_free.c:26 in asan_use_after_free -- David Marchand