From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D6E9E4240B; Wed, 18 Jan 2023 11:45:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 873DF400D6; Wed, 18 Jan 2023 11:45:58 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id DF8BC4003F for ; Wed, 18 Jan 2023 11:45:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674038757; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=e/3AoApRb9nrm3g1so6OVW4lDasX9mdpVBOwaBynZw8=; b=YWzUzQ0WfB+EKKa7plwYVDfOvsKrim7jjL1UeaYNurYKLzaUT7RipYCPGIdSSQtao+fc9u k5EQvkwnMsK0/Q7un9KZVu/uC47MixtF+fv63mkkwePA317s7hIT7eduGu8pHb37TPMqES wifTUEW8QbZvzkvZM3N09sF0aBqskwk= Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-182-kIGEg2JIPRuPJCoHEyOFJQ-1; Wed, 18 Jan 2023 05:45:56 -0500 X-MC-Unique: kIGEg2JIPRuPJCoHEyOFJQ-1 Received: by mail-pg1-f197.google.com with SMTP id d10-20020a631d4a000000b00491da16dc44so15371539pgm.16 for ; Wed, 18 Jan 2023 02:45:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=e/3AoApRb9nrm3g1so6OVW4lDasX9mdpVBOwaBynZw8=; b=snXemZFUnw41EnIpQjhQaiWufzpujVlrSKvAR2ESUh+c8bWf1nJgNp3Hgqdfw/vBHf M1NDZgcGEfaGRnm74BOf+zr3KAqJrd1999pk/LqQK5HxFsx6KeSJqSLNRb0dIZV7FaTB 4h6wir7tLVMHzPsNTSTPox5tUUD0A7CtT8RQZ9dkA1sl+X25szJihk3UfADdTn8KQcbE b7UnTBsUSoVr2HdEF9sLGdp67TaWRyjcFd520sIMXwk8Yfk6HZmkaQ9iFHxLz5CvY4sD 7M5Avvnh2kjEFWkvecVS35DuL9KW6Wz/1u0AIMYzp46NrulHvo4Y9U8Ip/6YgO9V33/K npQQ== X-Gm-Message-State: AFqh2kp/HCjgcCDu7MQ66LL0KaLLbM+VRCdkkG/8FOc2WN6L00cL4YMM Zl8X/vfR9PkUDBaknOF73IjKhaHFzSrhZfTaUAphkAQbwqosxKdN1Jvvd7zZG2bkL6Pzvuh7QvN aI4E4aFjeplthEpGbhmU= X-Received: by 2002:a63:1d4d:0:b0:4ce:20c3:5d95 with SMTP id d13-20020a631d4d000000b004ce20c35d95mr424874pgm.25.1674038755024; Wed, 18 Jan 2023 02:45:55 -0800 (PST) X-Google-Smtp-Source: AMrXdXuwu5ZQpZcO+/CFRKK56N5CZQyIMQVvYfQmt/f9/T6KO4if+YywVvMlaAgNxsjpD6V2Gk2ZBGgR/I6AzeuSpAM= X-Received: by 2002:a63:1d4d:0:b0:4ce:20c3:5d95 with SMTP id d13-20020a631d4d000000b004ce20c35d95mr424872pgm.25.1674038754681; Wed, 18 Jan 2023 02:45:54 -0800 (PST) MIME-Version: 1.0 References: <20230107133953.306784-1-thomas@monjalon.net> <20230111195345.1275693-1-thomas@monjalon.net> In-Reply-To: From: David Marchand Date: Wed, 18 Jan 2023 11:45:43 +0100 Message-ID: Subject: Re: [PATCH v4] devtools: parallelize ABI check To: Ferruh Yigit Cc: Thomas Monjalon , dev@dpdk.org, bruce.richardson@intel.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Jan 12, 2023 at 3:15 PM Ferruh Yigit wrote: > >> abidiff $ABIDIFF_OPTIONS $dump $dump2 || { > >> abiret=$? > >> - echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'" >&2 > >> - error=1 > >> - echo > >> + echo "Error: ABI issue reported for abidiff $ABIDIFF_OPTIONS $dump $dump2" >&2 > >> if [ $(($abiret & 3)) -ne 0 ]; then > >> echo "ABIDIFF_ERROR|ABIDIFF_USAGE_ERROR, this could be a script or environment issue." >&2 > >> fi > >> @@ -57,8 +55,13 @@ for dump in $(find $refdir -name "*.dump"); do > >> if [ $(($abiret & 8)) -ne 0 ]; then > >> echo "ABIDIFF_ABI_INCOMPATIBLE_CHANGE, this change breaks the ABI." >&2 > >> fi > >> - echo > >> + return 1 > >> } > >> -done > >> +}' > >> + > >> +error= > >> +find $refdir -name "*.dump" | > >> +xargs -n1 -P0 sh -c 'eval "$diff_func"; run_diff $0' || > >> +error=1 > >> > >> [ -z "$error" ] || [ -n "$warnonly" ] > > > > For the record, on my system, calling this script is ~5 times faster: > > - before > > real 0m5,447s > > user 0m4,497s > > sys 0m0,937s > > > > - after > > real 0m1,202s > > user 0m10,784s > > sys 0m2,027s > > > > > >> diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh > >> index f15a3b9aaf..61f7510ea1 100755 > >> --- a/devtools/gen-abi.sh > >> +++ b/devtools/gen-abi.sh > >> @@ -22,5 +22,6 @@ for f in $(find $installdir -name "*.so.*"); do > >> fi > >> > >> libname=$(basename $f) > >> - abidw --out-file $dumpdir/${libname%.so*}.dump $f > >> -done > >> + echo $dumpdir/${libname%.so*}.dump $f > >> +done | > >> +xargs -n2 -P0 abidw --out-file > >> -- > >> 2.39.0 > >> > > > > - before > > real 0m8,237s > > user 0m7,704s > > sys 0m0,504s > > > > - after > > real 0m2,517s > > user 0m14,145s > > sys 0m0,766s > > > > > > Ferruh, I am seeing quite different numbers for running those scripts > > (clearly not of the minute order). > > I switched to testing/building in tmpfs some time ago. > > It requires a good amount of memory (I empirically allocated 40G), but > > maybe worth a try for you? > > > > I run 'test-meson-builds.sh' script directly and yes I am getting > different numbers although there is still improvement, not in scale with > what you are getting, with v4 I have following: > > - before > real 10m3.248s > user 39m8.664s > sys 14m52.870s > > - after > real 7m49.086s > user 39m59.507s > sys 15m0.598s Well, yes, I did not realise which apples you were looking at :-). The change looks good in any case. -- David Marchand