From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A77D1430A0; Sat, 19 Aug 2023 00:33:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 26D75410D7; Sat, 19 Aug 2023 00:33:55 +0200 (CEST) Received: from mail-oa1-f50.google.com (mail-oa1-f50.google.com [209.85.160.50]) by mails.dpdk.org (Postfix) with ESMTP id D5561410D7 for ; Sat, 19 Aug 2023 00:33:53 +0200 (CEST) Received: by mail-oa1-f50.google.com with SMTP id 586e51a60fabf-1c4f47810c6so839912fac.0 for ; Fri, 18 Aug 2023 15:33:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1692398033; x=1693002833; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=MOQF8q0nWsnYs77OISfBIHP2d/TQ9IwGTCbN4XmGU9Y=; b=jiml0ouHsAIqv2pLh6Jbn+PCoWilRIPfRtD1o1/AxI5usDCqK0q66RQJW1ph/Sibkb tifziWOolKfoLoD80Uv1xHch4imps+ab67ptITZvnZFqe4JkH+pgXTH8x7B7GSzB4bfy zCyLfDlwkyTE9FLMV4PSuju+58C6vMht8gLaY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692398033; x=1693002833; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=MOQF8q0nWsnYs77OISfBIHP2d/TQ9IwGTCbN4XmGU9Y=; b=XYZdH0ARHq0ddKPiWLf/50W6g6eFNwVRVxi1qYvNWTEpIhfiemCeKDVWBbZex6WFaZ bYPaEtGApOv2nXe11ZNCUSAPh9vdD1EWTz5BAY7xkE9OscL3CFGAUDaL6Z4t8hX6i9uG UtolfZ6CSK8Yms3zFum3ofWPtv4sESKsIfafwdxK09vg61vzC79axrrHLU3j+HSOnMF4 //El074j7v1V880KfP7awJCFKpNPPLeMGJKoZbWKl9B+2yYdMObxOTsrEYbV3DXKzhi9 o2v0A4wfS+ysCtMe18ZKjcSKT0hGcj711kt8SYHLEcgsqiqMFegclUawEavpz3rCDdF0 /LMg== X-Gm-Message-State: AOJu0YzIY3zIdJWgPjLfMhSa4JgHhqJKXAOa9xIL/WP3FW1eIYInCxH1 ACx6WuX2QLg+UtF7+lTj2EuF+zBffHsgPpbPzXNeXw== X-Google-Smtp-Source: AGHT+IEmTmRxcqtYQzyDz1w1qez6PJLIC0tY0EtQukrEL5xUu/Jowgh6vP1UYix7IfVoRBtoVRDJo2CtHBEbKUW+3E4= X-Received: by 2002:a05:6870:1d0:b0:1bb:c946:b80e with SMTP id n16-20020a05687001d000b001bbc946b80emr571330oad.28.1692398033079; Fri, 18 Aug 2023 15:33:53 -0700 (PDT) MIME-Version: 1.0 References: <20230721115125.55137-1-bruce.richardson@intel.com> <20230815151053.996469-1-bruce.richardson@intel.com> <20230815151053.996469-5-bruce.richardson@intel.com> In-Reply-To: From: Patrick Robb Date: Fri, 18 Aug 2023 18:33:42 -0400 Message-ID: Subject: Re: [PATCH v5 04/10] app/test: build using per-file dependency matrix To: David Marchand Cc: Bruce Richardson , dev@dpdk.org, ci@dpdk.org, =?UTF-8?Q?Morten_Br=C3=B8rup?= , Honnappa Nagarahalli , "Ruifeng Wang (Arm Technology China)" , Thomas Monjalon , Aaron Conole , Ferruh Yigit Content-Type: multipart/alternative; boundary="00000000000094daa006033a1f8f" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --00000000000094daa006033a1f8f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 18, 2023 at 3:07=E2=80=AFAM David Marchand wrote: > It is a "recurring" yet not so well known issue. > This unit test fails if any part of the DPDK did not release all > (hugepage backed) memory and associated hugepages before exiting. > > In your case here, there is a virtio-net device that the container > tries to get its hands on because DPDK scans and probes all available > resources by default (and fails to, in this case, but that's not > important). > Triggering this virtio-net probing makes ethdev allocate its shared > memzone for port data, but nothing in ethdev releases the memzone when > exiting. > Fixing this could be tricky... as the current ethdev code is really > vague around which locks protect what (if anything..). > > I think we hit this issue in the past, and avoided it by running the > tests with dynamically linked DPDK binaries (and by doing this, avoid > the net drivers get loaded). > > I can see that you are running the unit tests with a static binary in > the report you sent. > I think the default is shared mode, so I wonder what could be the > reason why UNH builds with static here. > In any case, could you have a try and switch to > -Ddefault_library=3Dshared (or remove forcing to static mode)? > > Thanks for helping debug this David. I think the default library actually is static, but let me know if I misunderstand: https://git.dpdk.org/dpdk/tree/meson.build. If you meson configure on the output of 'meson setup build' it says static is the default. ``` default_options: [ 'buildtype=3Drelease', 'c_std=3Dc11', 'default_library=3Dstatic', 'warning_level=3D2', ], ``` I tried a build forcing default_library=3Dshared, and all fast-tests unit tests failed. I can look further into this if you think shared mode might be the best way forward though. But I am assuming for now if static is indeed the default, that is the way we should build for testing. For your patch, it looks like there are actually more eal flags test failures: http://mails.dpdk.org/archives/test-report/2023-August/444170.htm= l. In any case, for debugging purposes, I will do a run with the eal_flags_file_prefix_autotest also included and send you the meson test detail log on slack. --00000000000094daa006033a1f8f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Fri, Aug 18, 2023 at 3:07=E2=80=AFAM D= avid Marchand <david.marcha= nd@redhat.com> wrote:
It is a "recurring" yet not so well known issue.
This unit test fails if any part of the DPDK did not release all
(hugepage backed) memory and associated hugepages before exiting.

In your case here, there is a virtio-net device that the container
tries to get its hands on because DPDK scans and probes all available
resources by default (and fails to, in this case, but that's not
important).
Triggering this virtio-net probing makes ethdev allocate its shared
memzone for port data, but nothing in ethdev releases the memzone when
exiting.
Fixing this could be tricky... as the current ethdev code is really
vague around which locks protect what (if anything..).

I think we hit this issue in the past, and avoided it by running the
tests with dynamically linked DPDK binaries (and by doing this, avoid
the net drivers get loaded).

I can see that you are running the unit tests with a static binary in
the report you sent.
I think the default is shared mode, so I wonder what could be the
reason why UNH builds with static here.
In any case, could you have a try and switch to
-Ddefault_library=3Dshared (or remove forcing to static mode)?

Thanks for helping debug this David.

=
I think the default library actually is static, but let me know if I = misunderstand:=C2=A0= https://git.dpdk.org/dpdk/tree/meson.build. If you meson configure on t= he output of 'meson setup build' it says static is the default.=C2= =A0

```
=C2=A0 =C2=A0 =C2=A0 =C2=A0 default_options: = [
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'buildtype=3Drelease'= ;,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'c_std=3Dc11',
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'default_library=3Dstatic'= ;,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 'warning_level=3D2'= ,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ],
```

=
I tried a build forcing default_library=3Dshared, and all fast-t= ests unit tests failed. I can look further into this if you think shared mo= de might be the best way forward though. But I am assuming for now if stati= c is indeed the default, that is the way we should build for testing.=C2=A0=

For your patch, it looks like there are actually = more eal flags test failures:=C2=A0http://mails.dpdk.org/archives/test-= report/2023-August/444170.html. In any case, for debugging purposes, I = will do a run with the=C2=A0eal_flags_file_prefix_autotest also included an= d send you the meson test detail log on slack.=C2=A0

--00000000000094daa006033a1f8f--