From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EA33941DC9 for ; Fri, 3 Mar 2023 16:15:39 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E4BB240EE3; Fri, 3 Mar 2023 16:15:39 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 9F33D40EE3 for ; Fri, 3 Mar 2023 16:15:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677856537; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tEoWIP5wTsugH/W+rZJW5QAtNZ7DyNfMH9AGTMu/M+Q=; b=I1avm+DLM8aYNnQF/GDm4dWNJmZD7Pg/xLpLYxDh8w7mdIMnDlQmBoxyKEJI5orU6FL2un 5VNyMYndeN9GYl325B57QuEGpepHZ9FzwHmFyd9uKT4LEG/CQ2Ki9n/X30q8qgB3LFaH8x PrQzxN5AGOb5ZB9fbLKPSGG1pHnRZvo= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-629-yWZT6tJDOlacCJzK25-HIg-1; Fri, 03 Mar 2023 10:15:36 -0500 X-MC-Unique: yWZT6tJDOlacCJzK25-HIg-1 Received: by mail-wm1-f70.google.com with SMTP id m31-20020a05600c3b1f00b003e9de8c95easo1062064wms.2 for ; Fri, 03 Mar 2023 07:15:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677856532; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tEoWIP5wTsugH/W+rZJW5QAtNZ7DyNfMH9AGTMu/M+Q=; b=JOIUAnzLA8yjIlv+Z2K4oqCEvfiugHyGiTlqmbmHquEeT2r7soal9YVYbqOCwjT7oY yv62IwLqNqm66pxuZZXrVC1My7k8BLaZUWkeVeQkKCcq5Exo0wZ4UiUO/rSuWDe1pp99 gSXeQEVmqz5OnEZPZhnLNOMZ+0YiFqafaYoWKN4p8+Ry3BYcQR8JtHWlJZPUGrJUeOaN SK/x6azVEi/eP4W0DS5FK5G/nBOuNzk1N4vOlpoumcxjE8c8ZmygTE47nbxuHv60FaMj CoQZ/vkIBqxnhCqSucNbZhG2VrZ0p/JD8ynOflRV5reb6HXMyRPzkoTmv7G+2oR61N6c +Jkg== X-Gm-Message-State: AO0yUKWjfTOiKB562X5ABVyQnKFXZnHmtoIxQyD6Fj+h4k17KudMCYv6 IztckpA56AxoP/EhHex9iF1ARqi6NgJ7ePvbfL8/4piCPCE8YqQi0eJMt41QPpvfZT9hh/qVc0Y gfw== X-Received: by 2002:a05:600c:3b0c:b0:3eb:2b88:999f with SMTP id m12-20020a05600c3b0c00b003eb2b88999fmr1915663wms.5.1677856531832; Fri, 03 Mar 2023 07:15:31 -0800 (PST) X-Google-Smtp-Source: AK7set9Z+Q1LQE3Di00VPCtBhfvItSTMCwb+SOdPcjGYlw/PnM3gibZK80cldR8Zc7I81aIxii6yfA== X-Received: by 2002:a05:600c:3b0c:b0:3eb:2b88:999f with SMTP id m12-20020a05600c3b0c00b003eb2b88999fmr1915648wms.5.1677856531505; Fri, 03 Mar 2023 07:15:31 -0800 (PST) Received: from [192.168.0.36] ([78.16.250.81]) by smtp.gmail.com with ESMTPSA id v7-20020a05600c444700b003eb0d6f48f3sm6965306wmn.27.2023.03.03.07.15.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 03 Mar 2023 07:15:31 -0800 (PST) Message-ID: <44234778-6720-76f4-8546-297994419643@redhat.com> Date: Fri, 3 Mar 2023 15:15:30 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 To: Patrick Robb Cc: Lincoln Lavoie , ci@dpdk.org, Aaron Conole , David Marchand , Jeremy Spewock References: <08944dac-937f-5433-6ce5-fb6fbb2536ed@redhat.com> <7c0fae6e-638a-a806-0029-b228c4956870@redhat.com> From: Kevin Traynor Subject: Re: UNH CI skipped tests In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org On 03/03/2023 14:33, Patrick Robb wrote: > Hi Kevin, > > My thinking was that allowing new testruns simply due to a jenkinsfile > change might produce testrun results which yielded no value for you or > others. But, it sounds like you do not feel this way, and I agree having a > testrun kick off due to Jenkinsfile changes would provide some debugging > information to us UNH folks. So, since it sounds like it's alright with > you, I'm going to go ahead and undo my previous changes on this front. > Fine by me. > And yes I'm sure all of those new tests popping up all of a sudden was > unexpected... Some of those were tests run on a server which was > temporarily down (ARM-Gigabyte), so once that went back online we were able > to run additional tests. And then you know the others which were failing > were never intended to run on 21.11. > ok, I think that's all the differences explained. It might be worth flagging in the dashboard if some valid tests were skipped for a case like this. I've no idea if that's easy to do, so not sure if it's worth the effort. Thanks Lincoln and Patrick for the quick response and explanations, Kevin. > Thanks, > Patrick > > > > On Fri, Mar 3, 2023 at 4:59 AM Kevin Traynor wrote: > >> On 02/03/2023 18:49, Patrick Robb wrote: >>> Hi Kevin, >>> >> >> Hi Patrick, >> >>> The FIPS test has the same issue as the cryptodev tests - it cannot run >> on >>> 21.11 due to a lacking dependency. We disabled the testing on the normal >>> 21.11 branch but I missed 21.11-staging. Sorry about the oversight. All 3 >>> tests are now disabled on 21.11-staging. >>> >> >> Cool, thanks. >> >>> The reason the two runs happened is that we were inadvertently polling >>> DPDK-Stable AND our jenkinsfile repo for commits, and triggering new >> builds >>> for commits to either repo. I have disabled polling to our Jenkinsfile >>> repo, which should limit new builds to just commits to DPDK-Stable. >>> >> >> I'm not sure that was the reason for the second test run. I did that >> deliberately... >> >> - I ran the initial run on the original commit (pass) >> - I added a new dependency/build impacting commit and saw new tests and >> failures >> - I wanted to understand if the new tests and failures were related to >> the single commit allowing additional tests to run, so... >> - I force pushed back to the original commit for a second run on the >> original commit. There I also saw the new tests so was able to confirm >> it was unrelated to the new commit. >> >> Just a thought wrt Jenkins repo, I'm not sure how often it gets updated >> and modifies tests. If a Jenkins repo update triggers a run, then it >> will be the only delta from the previous build, so won't that make it >> easier to debug if there is some issue? >> >> Otherwise dpdk-stable and Jenkins updates will be first tested together, >> which may make it harder to debug failures. >> >> I will leave it to your more better judgement the best approach wrt >> triggering builds on jenkins updates. Either is fine with me. >> >> thanks, >> Kevin. >> >> >>> Best, >>> Patrick >>> >>> On Thu, Mar 2, 2023 at 9:04 AM Kevin Traynor >> wrote: >>> >>>> On 02/03/2023 13:22, Lincoln Lavoie wrote: >>>>> Hi Kevin, >>>>> >>>>> The FIPS and crypto (ZUC / SNOW) testing shouldn't be running on the >>>> older >>>>> LTS branches, because they don't include the required patches that were >>>>> released as part of 22.11. So, you can ignore those failures. We'll >> make >>>>> sure those tests are excluded from future runs on the older >>>>> staging branches. >>>> >>>> ok, cool, thanks. >>>> >>>>> >>>>> In terms of the two runs, I'm not sure of the cause and we'll have to >>>> look >>>>> into that. >>>>> >>>> >>>> No problem, it's not urgent or blocking. I will keep a closer eye on the >>>> tests ran in future and just force a re-run if necessary. >>>> >>>> thanks, >>>> Kevin. >>>> >>>>> Cheers, >>>>> Lincoln >>>>> >>>>> On Thu, Mar 2, 2023 at 5:04 AM Kevin Traynor >>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have a question about UNH CI periodic runs. I had 2x runs of CI on >>>>>> 21.11-staging on the same commit, a few days apart. >>>>>> >>>>>> The issue I see is that the first test run came back all green, so I >>>>>> assume good and I can push to 21.11 branch. However, the second run >>>>>> comes back with additional tests that showed failures. >>>>>> >>>>>> So I'm wondering why there are additional tests in the second run? and >>>>>> if/how skipped tests are being reported? >>>>>> >>>>>> At least with the fips tests I have seen previously so I don't think >>>>>> they are all newly enabled tests in the days in-between. >>>>>> >>>>>> Details below. >>>>>> >>>>>> thanks, >>>>>> Kevin. >>>>>> >>>>>> Initial test run: >>>>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23476/ >>>>>> >>>>>> Second test run: >>>>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23560/ >>>>>> >>>>>> Additional tests in the second run: >>>>>> Ubuntu 20.04 VM - dpdk_fips_validation (warning, not reported in >>>>>> dashboard?) >>>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_zuc_autotest >>>> (fail) >>>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_snow3g_autotest >>>>>> (fail) >>>>>> Arm Intel XL710-QDA2 4000 Mbps - lpm_autotest, unit_tests_mbuf >>>>>> Arm Broadcom 25000 Mbps - unit_tests_mbuf,nic_single_core_tests >>>>>> Ubuntu 20.04 ARM GCC Cross compile - dpdk_meson_compile >>>>>> Ubuntu 20.04 ARM SVE - lpm_autotest >>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> >