From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8B3E1430A1; Sat, 19 Aug 2023 00:33:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 800344323A; Sat, 19 Aug 2023 00:33:56 +0200 (CEST) Received: from mail-oa1-f45.google.com (mail-oa1-f45.google.com [209.85.160.45]) by mails.dpdk.org (Postfix) with ESMTP id D3DB340EDF for ; Sat, 19 Aug 2023 00:33:53 +0200 (CEST) Received: by mail-oa1-f45.google.com with SMTP id 586e51a60fabf-1c8d895a602so821177fac.2 for ; Fri, 18 Aug 2023 15:33:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1692398033; x=1693002833; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=MOQF8q0nWsnYs77OISfBIHP2d/TQ9IwGTCbN4XmGU9Y=; b=jiml0ouHsAIqv2pLh6Jbn+PCoWilRIPfRtD1o1/AxI5usDCqK0q66RQJW1ph/Sibkb tifziWOolKfoLoD80Uv1xHch4imps+ab67ptITZvnZFqe4JkH+pgXTH8x7B7GSzB4bfy zCyLfDlwkyTE9FLMV4PSuju+58C6vMht8gLaY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692398033; x=1693002833; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=MOQF8q0nWsnYs77OISfBIHP2d/TQ9IwGTCbN4XmGU9Y=; b=JBRWcXs5Jc7RZYRPG6STkZP6ie+jh0LWIbSbQsPINnSPlHf8E1Y9SJWylH/tF+4Qi+ lurpkFLPCtViYsyRYFMV0TvLh8rmKdgLZmg+EvHRtoky0rtkvFgOJfiz914Id9PQIEKo JZaaGkwcBHwgKeeZ70cWC0SkvB/bz10FOfl8uiVd8X6c6QyQoykapYIh02UwQ1+UV9AV dvR5GNPVuEIdEWlYYkgVO6ZaaD3PtzVNIzIzg0B3I0DIZFO43POWhpb6TIfFqbiiX9oz BZH9m2zILAfLRqMy0bO11/KGSSSTh2SlJON0sI5GMKnyeXfoM4MdetGfIwIHPHHetOF5 1W0w== X-Gm-Message-State: AOJu0YygWdnRiltbMHtrlOv1zuqvPXM8BYVXmZdHJeVue8BXfFxGgR5h gMJsGZ1Q3biL+1neoiZKCOdnjtAfgRIvXPwHQKVyGw== X-Google-Smtp-Source: AGHT+IEmTmRxcqtYQzyDz1w1qez6PJLIC0tY0EtQukrEL5xUu/Jowgh6vP1UYix7IfVoRBtoVRDJo2CtHBEbKUW+3E4= X-Received: by 2002:a05:6870:1d0:b0:1bb:c946:b80e with SMTP id n16-20020a05687001d000b001bbc946b80emr571330oad.28.1692398033079; Fri, 18 Aug 2023 15:33:53 -0700 (PDT) MIME-Version: 1.0 References: <20230721115125.55137-1-bruce.richardson@intel.com> <20230815151053.996469-1-bruce.richardson@intel.com> <20230815151053.996469-5-bruce.richardson@intel.com> In-Reply-To: From: Patrick Robb Date: Fri, 18 Aug 2023 18:33:42 -0400 Message-ID: Subject: Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix To: David Marchand Cc: Bruce Richardson , dev@dpdk.org, ci@dpdk.org, =?UTF-8?Q?Morten_Br=C3=B8rup?= , Honnappa Nagarahalli , "Ruifeng Wang (Arm Technology China)" , Thomas Monjalon , Aaron Conole , Ferruh Yigit Content-Type: multipart/alternative; boundary="00000000000094daa006033a1f8f" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --00000000000094daa006033a1f8f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 18, 2023 at 3:07=E2=80=AFAM David Marchand wrote: > It is a "recurring" yet not so well known issue. > This unit test fails if any part of the DPDK did not release all > (hugepage backed) memory and associated hugepages before exiting. > > In your case here, there is a virtio-net device that the container > tries to get its hands on because DPDK scans and probes all available > resources by default (and fails to, in this case, but that's not > important). > Triggering this virtio-net probing makes ethdev allocate its shared > memzone for port data, but nothing in ethdev releases the memzone when > exiting. > Fixing this could be tricky... as the current ethdev code is really > vague around which locks protect what (if anything..). > > I think we hit this issue in the past, and avoided it by running the > tests with dynamically linked DPDK binaries (and by doing this, avoid > the net drivers get loaded). > > I can see that you are running the unit tests with a static binary in > the report you sent. > I think the default is shared mode, so I wonder what could be the > reason why UNH builds with static here. > In any case, could you have a try and switch to > -Ddefault_library=3Dshared (or remove forcing to static mode)? > > Thanks for helping debug this David. I think the default library actually is static, but let me know if I misunderstand: https://git.dpdk.org/dpdk/tree/meson.build. If you meson configure on the output of 'meson setup build' it says static is the default. ``` default_options: [ 'buildtype=3Drelease', 'c_std=3Dc11', 'default_library=3Dstatic', 'warning_level=3D2', ], ``` I tried a build forcing default_library=3Dshared, and all fast-tests unit tests failed. I can look further into this if you think shared mode might be the best way forward though. But I am assuming for now if static is indeed the default, that is the way we should build for testing. For your patch, it looks like there are actually more eal flags test failures: http://mails.dpdk.org/archives/test-report/2023-August/444170.htm= l. In any case, for debugging purposes, I will do a run with the eal_flags_file_prefix_autotest also included and send you the meson test detail log on slack. --00000000000094daa006033a1f8f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Fri, Aug 18, 2023 at 3:07=E2=80=AFAM D= avid Marchand <david.marcha= nd@redhat.com> wrote:
It is a "recurring" yet not so well known issue.
This unit test fails if any part of the DPDK did not release all
(hugepage backed) memory and associated hugepages before exiting.

In your case here, there is a virtio-net device that the container
tries to get its hands on because DPDK scans and probes all available
resources by default (and fails to, in this case, but that's not
important).
Triggering this virtio-net probing makes ethdev allocate its shared
memzone for port data, but nothing in ethdev releases the memzone when
exiting.
Fixing this could be tricky... as the current ethdev code is really
vague around which locks protect what (if anything..).

I think we hit this issue in the past, and avoided it by running the
tests with dynamically linked DPDK binaries (and by doing this, avoid
the net drivers get loaded).

I can see that you are running the unit tests with a static binary in
the report you sent.
I think the default is shared mode, so I wonder what could be the
reason why UNH builds with static here.
In any case, could you have a try and switch to
-Ddefault_library=3Dshared (or remove forcing to static mode)?

Thanks for helping debug this David.

=
I think the default library actually is static, but let me know if I = misunderstand:=C2=A0= https://git.dpdk.org/dpdk/tree/meson.build. If you meson configure on t= he output of 'meson setup build' it says static is the default.=C2= =A0

```
=C2=A0 =C2=A0 =C2=A0 =C2=A0 default_options: = [
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'buildtype=3Drelease'= ;,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'c_std=3Dc11',
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'default_library=3Dstatic'= ;,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'warning_level=3D2'= ,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ],
```

=
I tried a build forcing default_library=3Dshared, and all fast-t= ests unit tests failed. I can look further into this if you think shared mo= de might be the best way forward though. But I am assuming for now if stati= c is indeed the default, that is the way we should build for testing.=C2=A0=

For your patch, it looks like there are actually = more eal flags test failures:=C2=A0http://mails.dpdk.org/archives/test-= report/2023-August/444170.html. In any case, for debugging purposes, I = will do a run with the=C2=A0eal_flags_file_prefix_autotest also included an= d send you the meson test detail log on slack.=C2=A0

--00000000000094daa006033a1f8f--