From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4190BA0032 for ; Mon, 22 Nov 2021 14:34:47 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0977C4014E; Mon, 22 Nov 2021 14:34:47 +0100 (CET) Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) by mails.dpdk.org (Postfix) with ESMTP id 680D34003C for ; Mon, 22 Nov 2021 14:34:45 +0100 (CET) Received: by mail-ed1-f51.google.com with SMTP id t5so77320834edd.0 for ; Mon, 22 Nov 2021 05:34:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=24tcPPu5uBTv26UQtmYGVT9U9fIKdRjxhBfy0lzf4fk=; b=hswCfgPC7RvwNtoq0Vn0aDutd0PdVKGy9QPmDn/cofIf/60/YyIf3FJgAab5snqsiT XnOFj3iwVwFGb50mvTxErFc961ABZVXdEjoPiPI/vVnN6Gb+PZX0KgyOBARWzQALw5TZ Jf0YrJUN50p/pGcd9Gi8LAzhZuPQ2pQdoItm8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=24tcPPu5uBTv26UQtmYGVT9U9fIKdRjxhBfy0lzf4fk=; b=DBsOT1bABlnIXLurqDx9LmzoBC1yklyQyCzMwwdLe2aLrAs6xJfLRq/JTBZPaDDYcc PXyk952cOGMmR+WUmI+rtcNrzNIwisPglAY5E61VkCSoWXcHIyxfMLZhO+Ud5eBiHUfb OTpukLBLZgVtkxK0JsF2T0WTcIaBoqAoKZmA8UksMecoQAQu9nOq/rc/bArdh4M2/RHi bzlbyUngcaEi1FPODDLM7GIC9hZNserlAhlb5r1OIAqKmdri66Ws7cpiYXMxBJxYVtTi 6hZvST7Hpe+3jXKzNcpb6QUPq2O+HOCjalMxMiIkuMhcI4Aa46Sh601L/SJdCI/CrLb/ KLxA== X-Gm-Message-State: AOAM530jwb14XWlZUcX0tieZ0N0q33sTGHaY+DyUUMyFxvJs8WgxPUNS 8vHnEhxYSqeBv0yOc65oQU2/VF61EhJs3m+u+bsdmA== X-Google-Smtp-Source: ABdhPJxIWwRfscUUIOHPfIJb7JnEPJrmuFLtiE1dFd6siLjulomj5fXudWaTh02hi59JCWq7Fvppk4d/kDcafQXMHIY= X-Received: by 2002:a05:6402:40d2:: with SMTP id z18mr57217773edb.395.1637588085045; Mon, 22 Nov 2021 05:34:45 -0800 (PST) MIME-Version: 1.0 References: <1928246.j4tpOohVRJ@thomas> In-Reply-To: From: Lincoln Lavoie Date: Mon, 22 Nov 2021 08:34:33 -0500 Message-ID: Subject: Re: [dpdk-dev] [Bug 826] red_autotest random failures To: David Marchand Cc: "Dumitrescu, Cristian" , Thomas Monjalon , "Ajmera, Megha" , "Singh, Jasvinder" , "Liguzinski, WojciechX" , dev , Aaron Conole , "Yigit, Ferruh" , "ci@dpdk.org" , "Zegota, AnnaX" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org On Mon, Nov 22, 2021 at 3:17 AM David Marchand wrote: > On Fri, Nov 19, 2021 at 5:54 PM Dumitrescu, Cristian > wrote: > > On a different point, we should probably tweak our autotests to > differentiate between logical failures and those failures related to > resources not being available, and flag the test result accordingly in the > report. For example, if memory allocation fails, the test should be flagged > as "Not enough resources" instead of simply "Failed". In the first case, > the next step should be fixing the test setup, while in the second case the > next step should be fixing the code. What do people think on this? > > In such case, the test must return TEST_SKIPPED. > > If the purpose of the component / function being tested is to get / create / reserve the resource(s), the failure might be valid. So it can't be applied across the board. But places where the test is checking other functionality, this might at least prevent some failures that are transient (i.e. based on what the test could "get" from the system at that moment in time). > I did a pass for cores count / specific hw requirements, some time ago. > See https://git.dpdk.org/dpdk/commit/?id=e0f4a0ed4237 > > > -- > David Marchand > > -- *Lincoln Lavoie* Principal Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 lylavoie@iol.unh.edu https://www.iol.unh.edu +1-603-674-2755 (m)