From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DDD6042D8C; Thu, 29 Jun 2023 15:03:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6A3D540EDB; Thu, 29 Jun 2023 15:03:16 +0200 (CEST) Received: from smtp-relay-internal-0.canonical.com (smtp-relay-internal-0.canonical.com [185.125.188.122]) by mails.dpdk.org (Postfix) with ESMTP id 86AD3406B7 for ; Thu, 29 Jun 2023 15:03:14 +0200 (CEST) Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 389D83F182 for ; Thu, 29 Jun 2023 13:03:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1688043794; bh=t8KazjX25vaNdh1BsrscfzrQinKGhEZWOCqN7BrgMNU=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=DEzC5GmLHEMbXqK/rcry9YpD0RR8lkzDe82ClS2sMDs1RlV64vEGSkx5TJFaMOSQe uSI9ME3B5zQNCnT3XwWK3kofH98rfvI7tyN1DdqEJhqaAy/STnFTDQBal1/XA7UKe2 YQQeHGJRpH8lO2wsoAvBCL1rZLOGdRvnIZQotFnXACjnNTqUBZjWF1dRUtDf30n8ER 3nz7v+dvIoo2Xiy7v6cZ04PnfVc6cAC+U8cGV8KkV5ol6vKXIOeSHMNdeus06FkHSU K5axpadcicT3uj7Vs3fTfC5Br/QTSA9HZRjSo6NYqgdbmO3X8Vlyl6OzakerqPr8di 5eqduOtZcs25A== Received: by mail-pj1-f72.google.com with SMTP id 98e67ed59e1d1-262e619fbd0so381758a91.0 for ; Thu, 29 Jun 2023 06:03:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688043792; x=1690635792; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t8KazjX25vaNdh1BsrscfzrQinKGhEZWOCqN7BrgMNU=; b=lJeKcHEdc2nHIIYmJs8/keK6M7KM5qo6sLHPdH0uZOoKctNZFrrxTAokcH5LxBv5HM Xkq+0GMgjY8f2qwWcvHf0MPXbyuQ1q4rH1Pu3QmdPZ+oyrCMo5vdw/40fhixHJXeGYbQ xcYX/uW5Rc8kfFc5Yq/4+ibNrZBhgR+MI5jrM5K9hwi2n+7mFOjr95aYH3nnwnUuYMw1 iDDsvB4Uk0eKQZ0vYtX1BpiaXiXCT3El5CQszFPPjzkmLriblNJqxa1kFphtn2QJzFor vpaqtyS23zy2e3VuXwb+yPtQ8c/tlQJQ1ytDrM0S3e8ODpesxbz6oT3Xye2t/9Zf4p1r vuoA== X-Gm-Message-State: AC+VfDzTx75vra8FFJ9HzjxL3Cpk+jDaBUhI83w504uVUv9UR2vB7Bqe 22yEmzEDBwuT3Yh4bUJmtBTiEcavZbYIg2dAQp9mReALPudXqkeuUnkFkRTgME51GMGDxJSOXyb KBhkLdq2czYBV68PF5bGXgOLxO8/IxKEwdD0ztNmZxP2UGEg= X-Received: by 2002:a17:90a:1990:b0:260:9ce1:2e82 with SMTP id 16-20020a17090a199000b002609ce12e82mr22326932pji.43.1688043792267; Thu, 29 Jun 2023 06:03:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ65isVVjtIWqyHBovXogau/YkSlk8ZocZ85zW9PMGKF5Okb+Yj2/u0umjKPNlnEFzm/0A9bL9Eqc0t7eQCGW20= X-Received: by 2002:a17:90a:1990:b0:260:9ce1:2e82 with SMTP id 16-20020a17090a199000b002609ce12e82mr22326893pji.43.1688043791655; Thu, 29 Jun 2023 06:03:11 -0700 (PDT) MIME-Version: 1.0 References: <20230629125838.1995751-1-christian.ehrhardt@canonical.com> In-Reply-To: <20230629125838.1995751-1-christian.ehrhardt@canonical.com> From: Christian Ehrhardt Date: Thu, 29 Jun 2023 15:02:45 +0200 Message-ID: Subject: Re: [PATCH] doc: ensure sphinx output is reproducible To: dev Cc: Luca Boccassi Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Jun 29, 2023 at 2:58=E2=80=AFPM = wrote: > > From: Christian Ehrhardt > > By adding -j we build in parallel, to make building on multiprocessor > machines more effective. While that works it does also break > reproducible builds as the order of the sphinx generated searchindex.js > is depending on execution speed of the individual processes. Just FYI (this didn't fit fit well in the commit message) an example of such a fail can be seen at https://salsa.debian.org/paelzer-guest/dpdk/-/jobs/4372883 If you download the artifact, extract dpdk-doc, apply js-beautify for readability and then diff it you'll find it same-content-different-order. Examples of two builds: - https://paste.ubuntu.com/p/VhWYNRv7kN/ - https://paste.ubuntu.com/p/KcQk4Km9xM/ > Signed-off-by: Christian Ehrhardt > --- > buildtools/call-sphinx-build.py | 5 ----- > 1 file changed, 5 deletions(-) > > diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-bui= ld.py > index 39a60d09fa..d8879306de 100755 > --- a/buildtools/call-sphinx-build.py > +++ b/buildtools/call-sphinx-build.py > @@ -15,12 +15,7 @@ > # set the version in environment for sphinx to pick up > os.environ['DPDK_VERSION'] =3D version > > -# for sphinx version >=3D 1.7 add parallelism using "-j auto" > -ver =3D run([sphinx, '--version'], stdout=3DPIPE, > - stderr=3DSTDOUT).stdout.decode().split()[-1] > sphinx_cmd =3D [sphinx] + extra_args > -if Version(ver) >=3D Version('1.7'): > - sphinx_cmd +=3D ['-j', 'auto'] > > # find all the files sphinx will process so we can write them as depende= ncies > srcfiles =3D [] > -- > 2.41.0 > --=20 Christian Ehrhardt Senior Staff Engineer and acting Director, Ubuntu Server Canonical Ltd