From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 73B8EA09EF; Mon, 21 Dec 2020 19:44:36 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A2457CAC7; Mon, 21 Dec 2020 19:44:34 +0100 (CET) Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by dpdk.org (Postfix) with ESMTP id 252BCCAC5 for ; Mon, 21 Dec 2020 19:44:32 +0100 (CET) Received: by mail-pj1-f51.google.com with SMTP id f14so6632576pju.4 for ; Mon, 21 Dec 2020 10:44:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E9WufEE90vOXxzcUJtJlK+iR5g2TJH396GKReXz2ulE=; b=dEd/zqOv95p+UhnglsTT/hKuSw2b+PeFd30OtV6Gmc28r7igeujKxzZcdUelFQcaUO x8H0uLmLebSz87DwjeibIroWmwRf7PJNjoEaPCT2Xg9ltyXuJhRnSDalYkCwxF8LsnQ7 L/2HYZtXW9KVJ7FxFaUgdDkHkMPiRrCWnLXmDfsYy2ZDsHlHrzgS0fNcP995dPTRB53o 7R9tgmVpOB3UthdQXGQS/ZpqVXiw9qUmbtAMfUsKxAL0XUs3Tq6QJEcMkHVGUCVv5grw k9uG3rjzPNmxCoA0pL9FOwf8SQIwez/GONE/U54sXlGkJ5W3sJqt5HhgHcd+dRu0Nvk9 bn1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E9WufEE90vOXxzcUJtJlK+iR5g2TJH396GKReXz2ulE=; b=N4vyHAF6uk0X0WyNaspkFgaaWIdY3bDjp2BQsS9YOKqQDQefp6Uw8leX7pRL+cAMRs c35iZt8njyziXt5o1fYIirtt33Crqik8iEIQMB5nw2WUnCWncBbV6Q9dwjdDWWH25tE3 ueUvjiV/4rybWJnVFNe0woExfPZdSksKCW4wpWGIenN/oO2RdXXliDFi6I1mtgRWZAbw ae8adovN5ZNaJgs3Its7Si9vg5zixcOHAyRMHMegQiZ7/bWmKRuXKpMSDYdxL/RE0y6B IQVaKY74uFqLyVcuE2rkG0trIKLxKwugyGnH5WJaCugvd4Qk1aMwbt5cfrLGfPVFcFk+ wShA== X-Gm-Message-State: AOAM533Cm4KW3t7YKVBpIEzlmYJdsN8IBUbM6CyugzFRfOncqZQJziPt huY9mkaIlH5AUQtHNT3scaMw4Q== X-Google-Smtp-Source: ABdhPJwCCdkH0e6L+JgcRnndL9k+YGYtFZW8rglxHP3AqP5fK2KkTS5aBIsh+xXYcin8XCyvTWcNlg== X-Received: by 2002:a17:902:c104:b029:da:5206:8b9b with SMTP id 4-20020a170902c104b02900da52068b9bmr17531008pli.46.1608576271075; Mon, 21 Dec 2020 10:44:31 -0800 (PST) Received: from hermes.local (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id gz2sm16887831pjb.2.2020.12.21.10.44.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Dec 2020 10:44:30 -0800 (PST) Date: Mon, 21 Dec 2020 10:44:21 -0800 From: Stephen Hemminger To: "Peng, ZhihongX" Cc: "Wang, Haiyue" , "Zhang, Qi Z" , "Xing, Beilei" , "dev@dpdk.org" , "Lin, Xueqin" , "Yu, PingX" Message-ID: <20201221104421.5912a951@hermes.local> In-Reply-To: References: <20201218192109.50098-1-zhihongx.peng@intel.com> <20201218105424.6731d866@hermes.local> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC] mem_debug add more log X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, 21 Dec 2020 07:35:10 +0000 "Peng, ZhihongX" wrote: > 1. I think this implement doesn't add significant overhead. Overhead only will be occurred in rte_malloc and rte_free. > > 2. Current existing address sanitizer infrastructure only support libc malloc. > > Regards, > Peng,Zhihong > > -----Original Message----- > From: Stephen Hemminger > Sent: Saturday, December 19, 2020 2:54 AM > To: Peng, ZhihongX > Cc: Wang, Haiyue ; Zhang, Qi Z ; Xing, Beilei ; dev@dpdk.org > Subject: Re: [dpdk-dev] [RFC] mem_debug add more log > > On Fri, 18 Dec 2020 14:21:09 -0500 > Peng Zhihong wrote: > > > 1. The debugging log in current DPDK RTE_MALLOC_DEBUG mode is insufficient, > > which makes it difficult to locate the issues, such as: > > a) When a memeory overlflow occur in rte_free, there is a little log > > information. Even if abort here, we can find which API is core > > dumped but we still need to read the source code to find out where > > the requested memory is overflowed. > > b) Current DPDK can NOT find that the overflow if the memory has been > > used and not released. > > c) If there are two pieces of continuous memory, when the first block > > is not released and an overflow is occured and also the second block > > of memory is covered, a memory overflow will be detected once the second > > block of memory is released. However, current DPDK can not find the > > correct point of memory overflow. It only detect the memory overflow > > of the second block but should dedect the one of first block. > > ---------------------------------------------------------------------------------- > > | header cookie | data1 | trailer cookie | header cookie | data2 |trailer cookie | > > > > ---------------------------------------------------------------------- > > ------------ 2. To fix above issues, we can store the requested > > information When DPDK > > request memory. Including the requested address and requested momory's > > file, function and numbers of rows and then put it into a list. > > -------------------- ---------------------- ---------------------- > > | struct list_head |---->| struct malloc_info |---->| struct malloc_info | > > -------------------- ---------------------- ---------------------- > > The above 3 problems can be solved through this implementation: > > a) If there is a memory overflow in rte_free, you can traverse the > > list to find the information of overflow memory and print the > > overflow memory information. like this: > > code: > > 37 char *p = rte_zmalloc(NULL, 64, 0); > > 38 memset(p, 0, 65); > > 39 rte_free(p); > > 40 //rte_malloc_validate_all_memory(); > > memory error: > > EAL: Error: Invalid memory > > malloc memory address 0x17ff2c340 overflow in \ > > file:../examples/helloworld/main.c function:main line:37 > > b)c) Provide a interface to check all memory overflow in function > > rte_malloc_validate_all_memory, this function will check all > > memory on the list. Call this funcation manually at the exit > > point of business logic, we can find all overflow points in time. > > > > Signed-off-by: Peng Zhihong > > Good concept, but doesn't this add significant overhead? > > Maybe we could make rte_malloc work with existing address sanitizer infrastructure in gcc/clang? That would provide faster and more immediate better diagnostic info. Everybody builds there own custom debug hooks, and some of these are worth sharing. But lots of time debug code becomes a technical debt, creates API/ABI issues and causes more trouble than it is worth. Therefore my desire is for DPDK to be better supported by standard tools such as valgrind and address sanitizer. The standard tools catch more errors faster and do not create project maintenance workload. See: https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm