From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 956BF423AE; Wed, 11 Jan 2023 15:11:27 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 849D540E25; Wed, 11 Jan 2023 15:11:27 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 8059640A7D for ; Wed, 11 Jan 2023 15:11:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1673446285; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hzoF1RXTgaRd/2sFjKz+9N72U35N5sSu/j9UrZ6pKBU=; b=jP7h1jPQ7qSJzk97twmBX7Cnibdi/5+zKpQezZc7NdY37vb+l6xRhrzicxfzNeED1s89Qp MOaXsIYrOcLMhMKccp7RGPh5Il4pvU8axeF9ph4iVzpFg8fBa/WLKAakCvr0nuv9VGvlaf VGL4gFYxb6pncdo2amiUAegYyO6z0f4= Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-660-cthNxMYCNjmkz0DjJ5WAfg-1; Wed, 11 Jan 2023 09:11:24 -0500 X-MC-Unique: cthNxMYCNjmkz0DjJ5WAfg-1 Received: by mail-pg1-f200.google.com with SMTP id r126-20020a632b84000000b004393806c06eso6707948pgr.4 for ; Wed, 11 Jan 2023 06:11:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=hzoF1RXTgaRd/2sFjKz+9N72U35N5sSu/j9UrZ6pKBU=; b=kfXwZrqEElkg6SSqP82rQyCw93vkmYWuZfCjVCJjtlZjfe+0VTmH/HLMqaeqj9rqjM gEDZMVi1swYYCcv6yqanM4Y+xii+YTStWAfg2E6RgwUas5q6BkJMlwAnJwW5++yo58/f HJUFygpwlJqFw43bjcvaIoFs/Pm0QnsGxSXKy1RKIIYxxWTBVDXXq/YVr+EqLFm5yKer 0Iwe+NxViRqlRiS4TDXCRMgTdI217Nvf0ooYwGYx5fcvcvvwqQt5jyuUKTxF0BdW979W HCChmFuDmHrLOsMmWudvGcvxqktjoKKmFhdD7yzTP4LytjPphs6Z9HH0xwNvAU38pzqQ vOjA== X-Gm-Message-State: AFqh2kpdILXetrSyAA1BJ1oMFIKQeNZy4QDiE7E7L6Z+z2Uhg1dI/APE mKgRHzjcVUnNB7Dy4Bwd/ZTO5GihBqmq+LkHOc5sPMMGh1mlqFWbE9rWUErjoRhpJniXLBNjW+4 a5XAp1SVsqKK1o/jzq2s= X-Received: by 2002:a17:902:efcb:b0:186:971b:b7e5 with SMTP id ja11-20020a170902efcb00b00186971bb7e5mr4190549plb.54.1673446282040; Wed, 11 Jan 2023 06:11:22 -0800 (PST) X-Google-Smtp-Source: AMrXdXsbe1WzsWDYEAbhvkjBSbdXK3+Osctx7fHp9z19ky/0tKmImL5O82C7Cw2s18oLGNJEble31O1fV02IGsIaMv4= X-Received: by 2002:a17:902:efcb:b0:186:971b:b7e5 with SMTP id ja11-20020a170902efcb00b00186971bb7e5mr4190547plb.54.1673446281773; Wed, 11 Jan 2023 06:11:21 -0800 (PST) MIME-Version: 1.0 References: <20230107133953.306784-1-thomas@monjalon.net> <20230111131652.1265058-1-thomas@monjalon.net> In-Reply-To: <20230111131652.1265058-1-thomas@monjalon.net> From: David Marchand Date: Wed, 11 Jan 2023 15:11:10 +0100 Message-ID: Subject: Re: [PATCH v3] devtools: parallelize ABI check To: Thomas Monjalon Cc: dev@dpdk.org, Ferruh Yigit X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Jan 11, 2023 at 2:16 PM Thomas Monjalon wrote: > > Generation and comparison of ABI dumps are done on multiple cores > thanks to xargs -P0. > It can accelerate this long step by 5 in my tests. > > xargs reports a global error if one of the process has an error. > > Running a shell function with xargs requires to export it with -f, > and that is a specific capability of bash. > > Signed-off-by: Thomas Monjalon > Tested-by: Ferruh Yigit > --- > v2: > - find dump2 inside the function > - force bash because of export -f > v3: > - revert to POSIX sh > - use POSIX eval instead of export -f (issues on Ubuntu) > --- > devtools/check-abi.sh | 21 +++++++++++++-------- > devtools/gen-abi.sh | 5 +++-- > 2 files changed, 16 insertions(+), 10 deletions(-) > > diff --git a/devtools/check-abi.sh b/devtools/check-abi.sh > index c583eae2fd..d58c867c60 100755 > --- a/devtools/check-abi.sh > +++ b/devtools/check-abi.sh > @@ -34,19 +34,18 @@ else > ABIDIFF_OPTIONS="$ABIDIFF_OPTIONS --headers-dir2 $incdir2" > fi > > -error= > -for dump in $(find $refdir -name "*.dump"); do > +export diff_func='run_diff() { > + dump=$1 > + newdir=$2 > name=$(basename $dump) > dump2=$(find $newdir -name $name) > if [ -z "$dump2" ] || [ ! -e "$dump2" ]; then > echo "Error: cannot find $name in $newdir" >&2 > - error=1 > - continue > - fi > + return 1 > + fi; > abidiff $ABIDIFF_OPTIONS $dump $dump2 || { > abiret=$? > - echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'" >&2 > - error=1 > + echo "Error: ABI issue reported for abidiff $ABIDIFF_OPTIONS $dump $dump2" >&2 > echo > if [ $(($abiret & 3)) -ne 0 ]; then > echo "ABIDIFF_ERROR|ABIDIFF_USAGE_ERROR, this could be a script or environment issue." >&2 > @@ -58,7 +57,13 @@ for dump in $(find $refdir -name "*.dump"); do > echo "ABIDIFF_ABI_INCOMPATIBLE_CHANGE, this change breaks the ABI." >&2 > fi > echo > + return 1 > } > -done > +}' > + > +error= > +find $refdir -name "*.dump" | > +xargs -n1 -P0 sh -c 'eval "$diff_func"; run_diff $0 '$newdir || > +error=1 Do we need to pass $newdir ? Like, for example, ABIDIFF_OPTIONS seems inherited, right? Otherwise this trick looks to work. -- David Marchand