From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id D6E9E4240B;
Wed, 18 Jan 2023 11:45:58 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 873DF400D6;
Wed, 18 Jan 2023 11:45:58 +0100 (CET)
Received: from us-smtp-delivery-124.mimecast.com
(us-smtp-delivery-124.mimecast.com [170.10.133.124])
by mails.dpdk.org (Postfix) with ESMTP id DF8BC4003F
for <dev@dpdk.org>; Wed, 18 Jan 2023 11:45:57 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
s=mimecast20190719; t=1674038757;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:content-type:content-type:
in-reply-to:in-reply-to:references:references;
bh=e/3AoApRb9nrm3g1so6OVW4lDasX9mdpVBOwaBynZw8=;
b=YWzUzQ0WfB+EKKa7plwYVDfOvsKrim7jjL1UeaYNurYKLzaUT7RipYCPGIdSSQtao+fc9u
k5EQvkwnMsK0/Q7un9KZVu/uC47MixtF+fv63mkkwePA317s7hIT7eduGu8pHb37TPMqES
wifTUEW8QbZvzkvZM3N09sF0aBqskwk=
Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com
[209.85.215.197]) by relay.mimecast.com with ESMTP with STARTTLS
(version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id
us-mta-182-kIGEg2JIPRuPJCoHEyOFJQ-1; Wed, 18 Jan 2023 05:45:56 -0500
X-MC-Unique: kIGEg2JIPRuPJCoHEyOFJQ-1
Received: by mail-pg1-f197.google.com with SMTP id
d10-20020a631d4a000000b00491da16dc44so15371539pgm.16
for <dev@dpdk.org>; Wed, 18 Jan 2023 02:45:55 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=cc:to:subject:message-id:date:from:in-reply-to:references
:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=e/3AoApRb9nrm3g1so6OVW4lDasX9mdpVBOwaBynZw8=;
b=snXemZFUnw41EnIpQjhQaiWufzpujVlrSKvAR2ESUh+c8bWf1nJgNp3Hgqdfw/vBHf
M1NDZgcGEfaGRnm74BOf+zr3KAqJrd1999pk/LqQK5HxFsx6KeSJqSLNRb0dIZV7FaTB
4h6wir7tLVMHzPsNTSTPox5tUUD0A7CtT8RQZ9dkA1sl+X25szJihk3UfADdTn8KQcbE
b7UnTBsUSoVr2HdEF9sLGdp67TaWRyjcFd520sIMXwk8Yfk6HZmkaQ9iFHxLz5CvY4sD
7M5Avvnh2kjEFWkvecVS35DuL9KW6Wz/1u0AIMYzp46NrulHvo4Y9U8Ip/6YgO9V33/K
npQQ==
X-Gm-Message-State: AFqh2kp/HCjgcCDu7MQ66LL0KaLLbM+VRCdkkG/8FOc2WN6L00cL4YMM
Zl8X/vfR9PkUDBaknOF73IjKhaHFzSrhZfTaUAphkAQbwqosxKdN1Jvvd7zZG2bkL6Pzvuh7QvN
aI4E4aFjeplthEpGbhmU=
X-Received: by 2002:a63:1d4d:0:b0:4ce:20c3:5d95 with SMTP id
d13-20020a631d4d000000b004ce20c35d95mr424874pgm.25.1674038755024;
Wed, 18 Jan 2023 02:45:55 -0800 (PST)
X-Google-Smtp-Source: AMrXdXuwu5ZQpZcO+/CFRKK56N5CZQyIMQVvYfQmt/f9/T6KO4if+YywVvMlaAgNxsjpD6V2Gk2ZBGgR/I6AzeuSpAM=
X-Received: by 2002:a63:1d4d:0:b0:4ce:20c3:5d95 with SMTP id
d13-20020a631d4d000000b004ce20c35d95mr424872pgm.25.1674038754681; Wed, 18 Jan
2023 02:45:54 -0800 (PST)
MIME-Version: 1.0
References: <20230107133953.306784-1-thomas@monjalon.net>
<20230111195345.1275693-1-thomas@monjalon.net>
<CAJFAV8y1sfww_N7WYHgd+mRkn51Hoq1isfNKLUX2uT=MRLgsWA@mail.gmail.com>
<b5995b8a-635e-8562-6481-0db504744296@amd.com>
In-Reply-To: <b5995b8a-635e-8562-6481-0db504744296@amd.com>
From: David Marchand <david.marchand@redhat.com>
Date: Wed, 18 Jan 2023 11:45:43 +0100
Message-ID: <CAJFAV8wBr0cR7g7JiJn9a1GUuccw4WeL3Ck_wpusRpgFEDTaGA@mail.gmail.com>
Subject: Re: [PATCH v4] devtools: parallelize ABI check
To: Ferruh Yigit <ferruh.yigit@amd.com>
Cc: Thomas Monjalon <thomas@monjalon.net>, dev@dpdk.org,
bruce.richardson@intel.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset="UTF-8"
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
On Thu, Jan 12, 2023 at 3:15 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
> >> abidiff $ABIDIFF_OPTIONS $dump $dump2 || {
> >> abiret=$?
> >> - echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'" >&2
> >> - error=1
> >> - echo
> >> + echo "Error: ABI issue reported for abidiff $ABIDIFF_OPTIONS $dump $dump2" >&2
> >> if [ $(($abiret & 3)) -ne 0 ]; then
> >> echo "ABIDIFF_ERROR|ABIDIFF_USAGE_ERROR, this could be a script or environment issue." >&2
> >> fi
> >> @@ -57,8 +55,13 @@ for dump in $(find $refdir -name "*.dump"); do
> >> if [ $(($abiret & 8)) -ne 0 ]; then
> >> echo "ABIDIFF_ABI_INCOMPATIBLE_CHANGE, this change breaks the ABI." >&2
> >> fi
> >> - echo
> >> + return 1
> >> }
> >> -done
> >> +}'
> >> +
> >> +error=
> >> +find $refdir -name "*.dump" |
> >> +xargs -n1 -P0 sh -c 'eval "$diff_func"; run_diff $0' ||
> >> +error=1
> >>
> >> [ -z "$error" ] || [ -n "$warnonly" ]
> >
> > For the record, on my system, calling this script is ~5 times faster:
> > - before
> > real 0m5,447s
> > user 0m4,497s
> > sys 0m0,937s
> >
> > - after
> > real 0m1,202s
> > user 0m10,784s
> > sys 0m2,027s
> >
> >
> >> diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh
> >> index f15a3b9aaf..61f7510ea1 100755
> >> --- a/devtools/gen-abi.sh
> >> +++ b/devtools/gen-abi.sh
> >> @@ -22,5 +22,6 @@ for f in $(find $installdir -name "*.so.*"); do
> >> fi
> >>
> >> libname=$(basename $f)
> >> - abidw --out-file $dumpdir/${libname%.so*}.dump $f
> >> -done
> >> + echo $dumpdir/${libname%.so*}.dump $f
> >> +done |
> >> +xargs -n2 -P0 abidw --out-file
> >> --
> >> 2.39.0
> >>
> >
> > - before
> > real 0m8,237s
> > user 0m7,704s
> > sys 0m0,504s
> >
> > - after
> > real 0m2,517s
> > user 0m14,145s
> > sys 0m0,766s
> >
> >
> > Ferruh, I am seeing quite different numbers for running those scripts
> > (clearly not of the minute order).
> > I switched to testing/building in tmpfs some time ago.
> > It requires a good amount of memory (I empirically allocated 40G), but
> > maybe worth a try for you?
> >
>
> I run 'test-meson-builds.sh' script directly and yes I am getting
> different numbers although there is still improvement, not in scale with
> what you are getting, with v4 I have following:
>
> - before
> real 10m3.248s
> user 39m8.664s
> sys 14m52.870s
>
> - after
> real 7m49.086s
> user 39m59.507s
> sys 15m0.598s
Well, yes, I did not realise which apples you were looking at :-).
The change looks good in any case.
--
David Marchand