From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8A0FE425A1; Fri, 15 Sep 2023 09:56:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7A1AB402AF; Fri, 15 Sep 2023 09:56:05 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 3A09F4029E for ; Fri, 15 Sep 2023 09:56:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694764563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1SSkgAJ255KssVOY7o/CWgOorsLo/PoZ1+cJ9bpFkIk=; b=YRfgReJ1mnX7ZTutylSovIte8WOyR3se18wprfHF46UGJFF3KW3QlEX0IcJHrYOLiVkEsH POsuzQhHdndiK9X/dS5P56L14cRNre4b7A7gVfHV8M3xI6fZAuQzaAt+yIT3FjLNgJB900 KbFYkv+1IvC7Sd/qdM3ZeiUXCnXeo/c= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-68-ssYVHotDOUq4wQOkpckwsA-1; Fri, 15 Sep 2023 03:56:02 -0400 X-MC-Unique: ssYVHotDOUq4wQOkpckwsA-1 Received: by mail-lj1-f200.google.com with SMTP id 38308e7fff4ca-2bfadc70950so22082051fa.2 for ; Fri, 15 Sep 2023 00:56:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694764561; x=1695369361; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1SSkgAJ255KssVOY7o/CWgOorsLo/PoZ1+cJ9bpFkIk=; b=RIDiug8lLknIWUXpsa/LO51sdsDLoIwtYB3egZKXwa5w19bTQgkobwfm+bnj2YBZtb KmfRkuqFMTSpsYy9B64AKma1dCgfq+1YNJ9ACBOC6aR55K57by3hYozcbRN8ezld2dq9 uqPdo1o+THCdfYcBpNXPHhYfxl8LEsrT63LzNa9EcpxsSc3j/nCmGQe6AVIraQRIQkVy lSCPoGoIZKSiF5T7Ft+Qj9Aovbr3ZfD+1md89qQQoJsx6AaI967NdWLkTjm0z4CnWG8L ORmUC1GWtQ8Lqvu1a7sG38z73trX67856t1ADQErT6PYJ9+ElMXAyqQZxiNOwd0itAVW Yf4Q== X-Gm-Message-State: AOJu0YynK1AAPv9uO05jN8RbVrf+jo1RYU6vVaR/I27YlfbL2FWQvPML C66CIgQyVm8x/AqBjFIp/TflVoRcUK5Xc8jEZTLdXWDpm+n+am/66eaxCDQkhSCMoSbfL9mlo7O K5+NzQ0EKpBsVm9eNm7k= X-Received: by 2002:a2e:a285:0:b0:2bc:f4ee:ca57 with SMTP id k5-20020a2ea285000000b002bcf4eeca57mr749690lja.48.1694764561053; Fri, 15 Sep 2023 00:56:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFk8KKVU/Vq8aMTgnhbTZb5QdLL3nOubhsayjIpDCd+k8oZugk2L+/fNynwWsDsi6I2VbFD9r170+hpI3/vSSs= X-Received: by 2002:a2e:a285:0:b0:2bc:f4ee:ca57 with SMTP id k5-20020a2ea285000000b002bcf4eeca57mr749668lja.48.1694764560740; Fri, 15 Sep 2023 00:56:00 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: David Marchand Date: Fri, 15 Sep 2023 09:55:49 +0200 Message-ID: Subject: Re: Unit tests not defined in any test suite To: Bruce Richardson Cc: dev@dpdk.org, Cristian Dumitrescu , Olivier Matz , Fan Zhang , Ashish Gupta , Akhil Goyal , Abhinandan Gujjar , Naga Harish K S V , Erik Gabriel Carrillo , Jerin Jacob , Anatoly Burakov , Kiran Kumar K , Nithin Dabilpuram , Zhirun Yan , Konstantin Ananyev , Vladimir Medvedkin , Reshma Pattan , Chas Williams , "Min Hu (Connor)" , Stephen Hemminger , Anoob Joseph , Volodymyr Fialko , David Hunt , Sachin Saxena , Hemant Agrawal , Honnappa Nagarahalli X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hello Bruce, On Thu, Sep 14, 2023 at 6:16=E2=80=AFPM Bruce Richardson wrote: > > Hi all, > > As part of the recent rework of the build of our unit test binary > (dpdk-test), the individual autotests were updated in the source code to > specify the test suite they belong to. The replacement of the > registration macro was done automatically using the existing suite lists > in the meson.build file, but following the replacement, it appears that > there are quite a number of test cases which are not in any test suite - > and as such as not being run automatically for patches in the IOL test > lab. > > Below is a list of these test cases, and the files each is defined in. > We should endeavour to ensure each test case is in a suite. Currently we > have 3 suites - fast-test, perf-test, and driver-test, but more suites > can be added as necessary. > > I've attempted to automatically identify the maintainers of the various > tests using "get-maintainers" script - hence the list of those on CC. If > possible, can each of you identify what test cases you may be maintainer > for, and add them to a test suite, to help improve our test coverage. > [Apologies for any mis-identifications] > > Thanks, > /Bruce > > alarm_autotest (test_alarm.c) > bitratestats_autotest (test_bitratestats.c) Those two tests above (and the one on the graph library) made me raise an eyebrow as I remembered them being part of the fast-tests list. I thought I had checked the tests lists before/after rework..., but looking again I did not do this properly (the blame is on the reviewers and commiters). Before the rework, GHA would run 106 fast-tests ut (look for fast-tests in follow log): https://pipelines.actions.githubusercontent.com/serviceHosts/aad9342c-d08c-= 4ebc-a424-c37e5c0e7e92/_apis/pipelines/1/runs/1767/signedlogcontent/8?urlEx= pires=3D2023-09-15T07%3A37%3A45.5644150Z&urlSigningMethod=3DHMACV1&urlSigna= ture=3D0g%2BN%2B0Q64RQolmslxH5N%2B91J7kvDAcVAanOmYnvO5cE%3D After the rework, only 92 remain: https://pipelines.actions.githubusercontent.com/serviceHosts/aad9342c-d08c-= 4ebc-a424-c37e5c0e7e92/_apis/pipelines/1/runs/1771/signedlogcontent/8?urlEx= pires=3D2023-09-15T07%3A38%3A01.6216684Z&urlSigningMethod=3DHMACV1&urlSigna= ture=3DuVfBMml1XXXr1CLVggaPDMLN8US6n9TcudVcyqFgsd0%3D I have a fix ready that I will send shortly. And the list of non tagged tests must be reevaluated. > cfgfile_autotest (test_cfgfile.c) > cksum_perf_autotest (test_cksum_perf.c) > compressdev_autotest (test_compressdev.c) > cryptodev_scheduler_autotest (test_cryptodev.c) > cryptodev_dpaa2_sec_raw_api_autotest (test_cryptodev.c) > cryptodev_dpaa_sec_raw_api_autotest (test_cryptodev.c) > cryptodev_mlx5_autotest (test_cryptodev.c) > cryptodev_ccp_autotest (test_cryptodev.c) > cryptodev_virtio_autotest (test_cryptodev.c) > cryptodev_octeontx_autotest (test_cryptodev.c) > cryptodev_caam_jr_autotest (test_cryptodev.c) > cryptodev_nitrox_autotest (test_cryptodev.c) > cryptodev_bcmfs_autotest (test_cryptodev.c) > cryptodev_openssl_asym_autotest (test_cryptodev_asym.c) > cryptodev_octeontx_asym_autotest (test_cryptodev_asym.c) > cryptodev_cn9k_asym_autotest (test_cryptodev_asym.c) > cryptodev_cn10k_asym_autotest (test_cryptodev_asym.c) > cryptodev_crosscheck (test_cryptodev_crosscheck.c) > event_crypto_adapter_autotest (test_event_crypto_adapter.c) > event_eth_rx_adapter_autotest (test_event_eth_rx_adapter.c) > event_eth_rx_intr_adapter_autotest (test_event_eth_rx_adapter.c) > event_eth_tx_adapter_autotest (test_event_eth_tx_adapter.c) > event_timer_adapter_test (test_event_timer_adapter.c) > eventdev_selftest_sw (test_eventdev.c) > eventdev_selftest_octeontx (test_eventdev.c) > eventdev_selftest_dpaa2 (test_eventdev.c) > eventdev_selftest_dlb2 (test_eventdev.c) > eventdev_selftest_cn9k (test_eventdev.c) > eventdev_selftest_cn10k (test_eventdev.c) > external_mem_autotest (test_external_mem.c) > graph_autotest (test_graph.c) > node_list_dump (test_graph.c) > graph_perf_autotest (test_graph_perf.c) > ipsec_sad_autotest (test_ipsec_sad.c) > latencystats_autotest (test_latencystats.c) > link_bonding_autotest (test_link_bonding.c) > link_bonding_mode4_autotest (test_link_bonding_mode4.c) > link_bonding_rssconf_autotest (test_link_bonding_rssconf.c) > metrics_autotest (test_metrics.c) > pcapng_autotest (test_pcapng.c) > pdcp_autotest (test_pdcp.c) > pdump_autotest (test_pdump.c) > ring_pmd_autotest (test_pmd_ring.c) > ring_pmd_perf_autotest (test_pmd_ring_perf.c) > power_caps_autotest (test_power_cpufreq.c) > rawdev_autotest (test_rawdev.c) > red_autotest (test_red.c) > resource_autotest (test_resource.c) > ring_stress_autotest (test_ring_stress.c) > inline_macsec_autotest (test_security_inline_macsec.c) > inline_ipsec_autotest (test_security_inline_proto.c) > inline_ipsec_sg_autotest (test_security_inline_proto.c) > event_inline_ipsec_autotest (test_security_inline_proto.c) > table_autotest (test_table.c) > timer_secondary_autotest (test_timer_secondary.c) > vdev_autotest (test_vdev.c) > --=20 David Marchand