From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD53AA0032; Mon, 22 Nov 2021 14:34:47 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2381C410E6; Mon, 22 Nov 2021 14:34:47 +0100 (CET) Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by mails.dpdk.org (Postfix) with ESMTP id 93FE24003C for ; Mon, 22 Nov 2021 14:34:45 +0100 (CET) Received: by mail-ed1-f54.google.com with SMTP id e3so76970180edu.4 for ; Mon, 22 Nov 2021 05:34:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=24tcPPu5uBTv26UQtmYGVT9U9fIKdRjxhBfy0lzf4fk=; b=hswCfgPC7RvwNtoq0Vn0aDutd0PdVKGy9QPmDn/cofIf/60/YyIf3FJgAab5snqsiT XnOFj3iwVwFGb50mvTxErFc961ABZVXdEjoPiPI/vVnN6Gb+PZX0KgyOBARWzQALw5TZ Jf0YrJUN50p/pGcd9Gi8LAzhZuPQ2pQdoItm8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=24tcPPu5uBTv26UQtmYGVT9U9fIKdRjxhBfy0lzf4fk=; b=IQsc6GvlRot8qIdXpeTIqGmhdS64eM+i2h80xFCqyH94nIc5TpOMxKoZLMEjRlWs7a Y2WE3Dvh37zqNe701XJaI9mpcOZvzO/Cy1ATGfInpXyyfD3FMBqimoUZNDhxP03JJLLw xUWmp89+l7uzWzGd/RLDXYHoXH14kbCahhuyRZGVrVY397dVuPIvdDzfJ/UFE84ChIZV JPz0jiEUXW8knmOYuTEi3HNMHZgaZyCMcAZRKviC3I+mLAujPxQ2vA1fMIkOCF5u0vpc fN39te7jXN7Kogm+KhySUucOs0ffybnEEUthJfKin3MdrAZ5iMZBKTkIZnXd8RO1JYf/ cB3A== X-Gm-Message-State: AOAM5335zP7xAG0eypKE4ZmmXw11DBC3RVczNe9hNOrCUM8oQpki5aU1 auCsgI8TTgp0qmg/faP3QK25FxpCS+xZK7iUc4c9JQ== X-Google-Smtp-Source: ABdhPJxIWwRfscUUIOHPfIJb7JnEPJrmuFLtiE1dFd6siLjulomj5fXudWaTh02hi59JCWq7Fvppk4d/kDcafQXMHIY= X-Received: by 2002:a05:6402:40d2:: with SMTP id z18mr57217773edb.395.1637588085045; Mon, 22 Nov 2021 05:34:45 -0800 (PST) MIME-Version: 1.0 References: <1928246.j4tpOohVRJ@thomas> In-Reply-To: From: Lincoln Lavoie Date: Mon, 22 Nov 2021 08:34:33 -0500 Message-ID: Subject: Re: [dpdk-dev] [Bug 826] red_autotest random failures To: David Marchand Cc: "Dumitrescu, Cristian" , Thomas Monjalon , "Ajmera, Megha" , "Singh, Jasvinder" , "Liguzinski, WojciechX" , dev , Aaron Conole , "Yigit, Ferruh" , "ci@dpdk.org" , "Zegota, AnnaX" Content-Type: multipart/alternative; boundary="00000000000019169b05d160b01e" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --00000000000019169b05d160b01e Content-Type: text/plain; charset="UTF-8" On Mon, Nov 22, 2021 at 3:17 AM David Marchand wrote: > On Fri, Nov 19, 2021 at 5:54 PM Dumitrescu, Cristian > wrote: > > On a different point, we should probably tweak our autotests to > differentiate between logical failures and those failures related to > resources not being available, and flag the test result accordingly in the > report. For example, if memory allocation fails, the test should be flagged > as "Not enough resources" instead of simply "Failed". In the first case, > the next step should be fixing the test setup, while in the second case the > next step should be fixing the code. What do people think on this? > > In such case, the test must return TEST_SKIPPED. > > If the purpose of the component / function being tested is to get / create / reserve the resource(s), the failure might be valid. So it can't be applied across the board. But places where the test is checking other functionality, this might at least prevent some failures that are transient (i.e. based on what the test could "get" from the system at that moment in time). > I did a pass for cores count / specific hw requirements, some time ago. > See https://git.dpdk.org/dpdk/commit/?id=e0f4a0ed4237 > > > -- > David Marchand > > -- *Lincoln Lavoie* Principal Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 lylavoie@iol.unh.edu https://www.iol.unh.edu +1-603-674-2755 (m) --00000000000019169b05d160b01e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Mon, Nov 22, 2021 at 3:17 AM David Marchand <= david.marchand@redhat.com&= gt; wrote:
On Fr= i, Nov 19, 2021 at 5:54 PM Dumitrescu, Cristian
<cris= tian.dumitrescu@intel.com> wrote:
> On a different point, we should probably tweak our autotests to differ= entiate between logical failures and those failures related to resources no= t being available, and flag the test result accordingly in the report. For = example, if memory allocation fails, the test should be flagged as "No= t enough resources" instead of simply "Failed". In the first= case, the next step should be fixing the test setup, while in the second c= ase the next step should be fixing the code. What do people think on this?<= br>
In such case, the test must return TEST_SKIPPED.

If the purpose of the component / function being tested is to get / creat= e / reserve the resource(s), the failure might be valid. So it can't be= applied across the board.=C2=A0 But places where the test is checking othe= r functionality, this might at least prevent some failures that are transie= nt (i.e. based on what the test could "get" from the system at th= at moment in time).=C2=A0=C2=A0

=C2=A0
=
I did a pass for cores count / specific hw requirements, some time ago.
See https://git.dpdk.org/dpdk/commit/?id=3De0f4a0= ed4237


--
David Marchand



--
Lincoln Lavoie
Prin= cipal Engineer, Broadband Technologies
21 Madbury Rd., Ste. 100, = Durham, NH 03824
+1-603-674-= 2755 (m)

--00000000000019169b05d160b01e--