From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6919425FE; Tue, 19 Sep 2023 11:07:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9788E4067B; Tue, 19 Sep 2023 11:07:30 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 85897402DA for ; Tue, 19 Sep 2023 11:07:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695114448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k2jTiZl7Nq3uRkslk3qUqlWHjkBCG26HirfD5r/GnKY=; b=gHLEG670RDbmLl2qRGsSToI054ZvQjNGg8KgkBwXgAWjbl9AsvDIqk23H+Z/fHCgLKBFME LtksIgyfCFhZ5n4nyY7CFkaofCqt9JBlBrwIiJAS5axP2uP5Bh0EEqM2xd0ArNit1+qT20 Z3iMk3Cnmkz6S0synyzbkm5+wRoIq/Q= Received: from mail-lj1-f197.google.com (mail-lj1-f197.google.com [209.85.208.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-685-BH2Xd7vSOcqh91WrKe8DAg-1; Tue, 19 Sep 2023 05:07:26 -0400 X-MC-Unique: BH2Xd7vSOcqh91WrKe8DAg-1 Received: by mail-lj1-f197.google.com with SMTP id 38308e7fff4ca-2c01d1d0fe4so17086241fa.2 for ; Tue, 19 Sep 2023 02:07:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695114445; x=1695719245; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k2jTiZl7Nq3uRkslk3qUqlWHjkBCG26HirfD5r/GnKY=; b=MuGLzpreVvKc0UI1PflbJ6BM2a6dKWDY/icbJ6QjjdodY9yKKKa/tt8vhoBkIqIMhB 1RBLVttv3akjwCIIcvPENqsGCN12joamEKWj8W3EULEktilN5mSXi5MmCYW/wmfuzun4 sHq3D0fdcPKcltvruQcO0ZY1XnWnoZ4CDb9/6NQ+vC18pssU0nC5uj2uH5oBkLzRJ9jr +5YteuFDZGF3VdgaQ4wEmEId/xCBdPEDlpp6cnyAF17i+LwOlxZ/AxY4qaZcWOOEqCGJ NISw7Grlqj3iVkwBukmRoP/2HnuJjqN2q8Iiq2XSqQc5kRqoAQ+BuFtYredKN9LlDXJs QlqA== X-Gm-Message-State: AOJu0Yzki5p6jk3sV1Z88vBAvr1KQEn6XoTOLMl2VlKbthgo2B+x9mVp tdOGzR80/p150Yg8AipqM52uiT9UJsDDROrHDzI5bqxK1Xl7iHlyy535xpadg4MhnhVdrFzAebL yX2SOBmntfIwfCeY3UnA= X-Received: by 2002:a2e:8ec1:0:b0:2bc:fffc:7f55 with SMTP id e1-20020a2e8ec1000000b002bcfffc7f55mr9578584ljl.6.1695114445444; Tue, 19 Sep 2023 02:07:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFu8y6XhzwSeChLlZIklVcX2cKJbQ+ijiAgLVJvlpO0Ry87IaVFLgWLXaG2SNfOcTMEInMiVZYmSfvu86514VM= X-Received: by 2002:a2e:8ec1:0:b0:2bc:fffc:7f55 with SMTP id e1-20020a2e8ec1000000b002bcfffc7f55mr9578566ljl.6.1695114445092; Tue, 19 Sep 2023 02:07:25 -0700 (PDT) MIME-Version: 1.0 References: <20230915115206.132198-1-bruce.richardson@intel.com> In-Reply-To: From: David Marchand Date: Tue, 19 Sep 2023 11:07:13 +0200 Message-ID: Subject: Re: [PATCH 0/2] add checks for tests not in a suite To: Bruce Richardson Cc: dev@dpdk.org, Thomas Monjalon , Ferruh Yigit , Jerin Jacob Kollanukkaran , Akhil Goyal , Maxime Coquelin , Aaron Conole X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Sep 19, 2023 at 10:36=E2=80=AFAM Bruce Richardson wrote: > > On Tue, Sep 19, 2023 at 10:29:07AM +0200, David Marchand wrote: > > On Fri, Sep 15, 2023 at 1:52=E2=80=AFPM Bruce Richardson > > wrote: > > > > > > To help ensure that we don't have "orphaned" tests not in any test > > > suites we can add the following checks: > > > > > > * In developer-mode builds, emit a warning for each test defined usin= g > > > REGISTER_TEST_COMMAND > > > * In checkpatches, add a check to prevent the addition of new tests > > > using the REGISTER_TEST_COMMAND macro > > > > > > Bruce Richardson (2): > > > app/test: emit warning for tests not in a test suite > > > devtools: check for tests added without a test suite > > > > > > app/test/suites/meson.build | 13 ++++++++++++- > > > buildtools/get-test-suites.py | 12 +++++++++--- > > > devtools/checkpatches.sh | 8 ++++++++ > > > 3 files changed, 29 insertions(+), 4 deletions(-) > > > > The "non_suite_tests" testsuite returned by > > buildtools/get-test-suites.py is a bit strange, as it is not a > > testsuite from meson pov. > > Yeah, it is a bit strange, and I'm open to new ideas on other solutions. = I > did it that way to avoid having yet another script to scan the files - I > figured it was faster (in terms of runtime, not dev time) to do the I had figured it was "faster dev time" that won :-). I am fine with it, I don't expect more complications in this area in the fu= ture. > scanning when the files are already being opened and processed by this on= e. > > Of course, if we can get the un-suitened [:-)] test cases down to zero, w= e > can theoretically drop this check in future, and just use the checkpatch > one. Well, that's still a question that nobody seems to comment on. What should we do with tests that don't enter one of those testsuites, and are not invoked by the CI? Though we may be removing some level of coverage, I am for cleaning/unused dead code. --=20 David Marchand