From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6F9AA034C; Tue, 14 Dec 2021 16:41:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB03240041; Tue, 14 Dec 2021 16:41:33 +0100 (CET) Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) by mails.dpdk.org (Postfix) with ESMTP id D99FF4003C for ; Tue, 14 Dec 2021 16:41:31 +0100 (CET) Received: by mail-ot1-f44.google.com with SMTP id x19-20020a9d7053000000b0055c8b39420bso21274473otj.1 for ; Tue, 14 Dec 2021 07:41:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=VB5VABHUE4fg7LkVNXP0MnXUlv2PYW0M9vo3VEzje9s=; b=BgduNHdZEN8xskmwJba3QeYGHtMMU0HOARl0N1LvA24UmcnuVBt8mmmUmE48GA49us gH2cWbXPXy5pcCX4iScHsaF2+Ta2zm9ek+kBKKQwNvIyXhDHk37WCkx3etp4TlQCnXaA bdXAKwrjxRwlP+NGGPDRvthcXTLoiCuY0+Uf4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VB5VABHUE4fg7LkVNXP0MnXUlv2PYW0M9vo3VEzje9s=; b=quG4rfmDtJou3KCV4pyo9jYuBfSS8Eg7NS750zfzTsXevy4KmnlUawNueAhfimwfcl Mvp3PtW4aXDBBzum9kIgW+t49N7NpVC4PJXfIYjxnzqZ5bdmNX4UCGrG/C/161ayJ5l3 Q3O0bAYMtN2VJsxUWKBygDnASHqrZjMpcJaziKXKtuYdaxWlo0XM2V/qHuwACuZ5P9dk 1BFQT8mrSmHhYhHL3ruMml9sdFE3ZCLW+Lo4TtN3MZYhwkykdZ4Nay3MW2fJ8JFOzKBS s2TeqsgS6QG2oWzxHaIAYxADoA2oQjyHkU+a1NY9N9Wd1WGYhx4iZV5vGVrpGYrPFSdl 4o9A== X-Gm-Message-State: AOAM530uJLBSgxTI7e3N3SW+q4jURZscJJycQAbVXgKRqJfw2gLjC4zD XXwRkdbxQHGV9iBipukMXssU05WTU57pG/ier7nigQ== X-Google-Smtp-Source: ABdhPJyx0leXyBurAnsSoiIKWo15LBOvDJkKa5LeGwE0mbkc9JrbOb+V3ZygLwBVwtIZbQa9lfySuhV0F8KJ7CDJsNg= X-Received: by 2002:a9d:5ccc:: with SMTP id r12mr4692732oti.67.1639496491183; Tue, 14 Dec 2021 07:41:31 -0800 (PST) MIME-Version: 1.0 References: <1639130270-232144-1-git-send-email-junx.dong@intel.com> In-Reply-To: <1639130270-232144-1-git-send-email-junx.dong@intel.com> From: Owen Hilyard Date: Tue, 14 Dec 2021 10:40:55 -0500 Message-ID: Subject: Re: [RFC] [PATCH V1] framework/*: Add function to support ASan test To: Dong JunX Cc: dts@dpdk.org, "Tu, Lijuan" , weix.ling@intel.com Content-Type: multipart/alternative; boundary="000000000000f7c63105d31d05e9" X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org --000000000000f7c63105d31d05e9 Content-Type: text/plain; charset="UTF-8" Am I missing where this is disabled for performance tests? Both ASAN and LSAN have a large performance impact, especially in any code that allocates a lot of memory. I have no objections to using this in functional testing, since performance doesn't matter that much there. I also can't tell if this changes the json output for a test suite. If an asan failure does not cause a test case/suite to fail, (which I think it should), then we need to have that information available in test_results.json, probably as a new key under the per-crb output. If an asan failure does cause the test case/suite to fail, then I don't think we need extra output. Finally, I think the config file needs to describe all of the options and what their valid values are, nic_single_core_perf.cfg is a good example of this. Also, I would prefer to avoid embedding json in the config files if possible, and I think that we might be able to have the start and end values for each of the sanitizers be stored as a pair, so we would have something like this: lsan_bounds=LeakSanitizer, SUMMARY asan_bounds=AddressSanitizer, SUMMARY That would avoid needing to use json parsing and you could just access by index. Owen Hilyard --000000000000f7c63105d31d05e9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Am I missing where this is disabled for p= erformance tests? Both ASAN and LSAN have a large performance impact, espec= ially=C2=A0in any code that allocates a lot of memory. I have no objections= to using this in functional testing, since performance doesn't matter = that much there.=C2=A0

I also can't = tell if this changes the json output for a test suite. If an asan failure d= oes not cause a test case/suite to fail, (which I think it should), then we= need to have that information=C2=A0available=C2=A0in test_results.json, pr= obably as a new key under the per-crb output. If an asan failure does cause= the test case/suite to fail, then I don't think we need extra output.= =C2=A0

Finally, I think the config file needs to d= escribe all of the options and what their valid values are, nic_single_core= _perf.cfg is a good example of this. Also, I would prefer to avoid embeddin= g json in the config files if possible, and I think that we might be able t= o have the start and end values for each of the sanitizers be stored as a p= air, so we would have something like this:

lsan_bounds=3DLeakSanitiz= er, SUMMARY
asan_bounds=3DAddressSanitizer, SUMMARY
That would avoid needing to use json parsing and you could just= access by index.=C2=A0

Owen Hilyard
--000000000000f7c63105d31d05e9--