From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31180A054F; Tue, 2 Mar 2021 16:27:15 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6155240693; Tue, 2 Mar 2021 16:27:14 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id E29144068C for <dev@dpdk.org>; Tue, 2 Mar 2021 16:27:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1614698832; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4gQ6Wwy9rxPyVpNPItUW5rYE5AeRcnMcUfL3fiLoi0M=; b=OriWcxd3c7blplCvwrj8LuIg+5mJFNnGpHfPW9+OQAOkMrdW5POOZo7a4VewQQ/4EbJct1 5H+m/BqZYzW4r0R6iGZd+OqjhC4Z0fztfXE39xZVK8hrFHxXJLIU4X+2TrCq+o+nTzsrsn 7L1SAYFiX3S2sEOHts2N71UV2JhxUuE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-72-ZRw3sn4GPveHRiSruS3GyQ-1; Tue, 02 Mar 2021 10:27:07 -0500 X-MC-Unique: ZRw3sn4GPveHRiSruS3GyQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 965B91005501; Tue, 2 Mar 2021 15:27:06 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (ovpn-119-172.rdu2.redhat.com [10.10.119.172]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E143F5C1BB; Tue, 2 Mar 2021 15:27:00 +0000 (UTC) From: Aaron Conole <aconole@redhat.com> To: Bruce Richardson <bruce.richardson@intel.com> Cc: David Marchand <david.marchand@redhat.com>, dev <dev@dpdk.org>, Ray Kinsella <mdr@ashroe.eu> References: <20210208165234.319578-1-aconole@redhat.com> <20210210145559.4090684-1-aconole@redhat.com> <CAJFAV8xm7MA7m7dsV6e+Mf6dX+wPDbK48bw+2Hze-GuxL8Oz2A@mail.gmail.com> <20210302100401.GA1396@bricha3-MOBL.ger.corp.intel.com> Date: Tue, 02 Mar 2021 10:26:59 -0500 In-Reply-To: <20210302100401.GA1396@bricha3-MOBL.ger.corp.intel.com> (Bruce Richardson's message of "Tue, 2 Mar 2021 10:04:01 +0000") Message-ID: <f7t7dmpvanw.fsf@dhcp-25.97.bos.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=aconole@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [dpdk-dev] [PATCH v2] guides: add a testing guide for developing tests X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> Bruce Richardson <bruce.richardson@intel.com> writes: > On Tue, Mar 02, 2021 at 10:07:26AM +0100, David Marchand wrote: >> On Wed, Feb 10, 2021 at 3:56 PM Aaron Conole <aconole@redhat.com> wrote: >> > diff --git a/doc/guides/contributing/testing.rst b/doc/guides/contributing/testing.rst >> > new file mode 100644 >> > index 0000000000..86ca24ce43 >> > --- /dev/null >> > +++ b/doc/guides/contributing/testing.rst >> > @@ -0,0 +1,245 @@ >> > +.. SPDX-License-Identifier: BSD-3-Clause >> > + Copyright 2018 The DPDK contributors >> >> 2021? Whoops, I forgot to update my time machine. >> > + >> > +.. _testing_guidelines: >> >> I can't find a call to the testing_guidelines reference, so this can be removed. done. >> >> [snip] >> >> > +The suites can be selected by adding the ``--suite`` option to the >> > +``meson test`` command. Ex: ``meson test --suite fast-tests``:: >> > + >> > + $ meson test -C build --suite fast-tests >> > + ninja: Entering directory `/home/aconole/git/dpdk/build' >> > + [2543/2543] Linking target app/test/dpdk-test. >> > + 1/60 DPDK:fast-tests / acl_autotest OK 3.17 s >> > + 2/60 DPDK:fast-tests / bitops_autotest OK 0.22 s >> > + 3/60 DPDK:fast-tests / byteorder_autotest OK 0.22 s >> > + 4/60 DPDK:fast-tests / cmdline_autotest OK 0.28 s >> > + 5/60 DPDK:fast-tests / common_autotest OK 0.57 s >> > + 6/60 DPDK:fast-tests / cpuflags_autotest OK 0.27 s >> > + ... >> >> Trying this in my build env, I get all tests failing. >> This is because I run this as a normal user, but the system has >> hugepages configured. >> I figured this out quickly since I know the test framework (simply >> added a echo 0; exit at the top of has-hugepages.sh). >> But I am not sure a reader of this doc would be able to troubleshoot this. >> >> Not sure if this is worth explaining here, or if we can enhance the >> hugepage check (permissions maybe?). I prefer to fix the hugepage check to make the tests SKIP when we don't have hugepages accessible (so we need some kind of permission check in there). I will submit it separately. >> >> [snip] >> >> > +Checking code coverage >> > +---------------------- >> > +The meson build system supports generating a code coverage report >> > +via the `-Db_coverage=true` option, in conjunction with a package >> > +like **lcov**, to generate an HTML code coverage report. Example:: >> > + >> > + $ meson covered -Db_coverage=true >> >> At first, I read "covered" as a meson command :-). >> I prefer an explicit "meson setup covered -Db_coverage=true", but well >> this is more a matter of taste. >> >> > > I also tend to prefer the build directory name at the end of the command, > so I'd suggest: "meson setup -Db_coverage=true covered". Furthermore, > while I can understand the use of "covered" as a build directory name, I > think for consistency across all docs, we should just use "build" here as > the directory name, which again will reduce confusion. "meson setup > -Db_coverage build" Okay - I will paint this bikeshed like: meson setup build -Db_coverage=true It's a little inconsistent everywhere - so I guess a good janitor project would be to clean up all the places we have meson commands. Otherwise, what I've found is that the options generally come after the build directory / command is specified (ex: see octeontx, the arm64 cross build docs, etc.) so I'll keep that for consistency there. Hopefully we will consistently become more consistent :) > /Bruce