From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1DD8FA0C41 for ; Thu, 30 Sep 2021 12:32:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D828B40DDA; Thu, 30 Sep 2021 12:32:46 +0200 (CEST) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by mails.dpdk.org (Postfix) with ESMTP id C8EAF4067E for ; Thu, 30 Sep 2021 12:32:45 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 6145E5C013A; Thu, 30 Sep 2021 06:32:45 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 30 Sep 2021 06:32:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm2; bh= 1/VcZloQVYjqCko0EljzxsVrICrfA/bLGrVJVAJA3NY=; b=nhxJrMn+tsII4ZWM myWjzh+2MDAeHrNkzv+848ds8pZHii38EdXjoTYhWHe5SGaOA3KLUAHae5AgxfZ3 0OTF+WRKCwWIU3AddKVOF6gk3mVPimNzqpf8grBUGtcqfIjRyxZe8mSP61QrY7vl llFenfPMbya035jBYtJgYOitDjK5Xxg8Ht2eQ/pHJTRyh6SX5r5mICLdPlmO8Gba ldwZbZOdS/CLjCHR7gbngMCDNeT+MNv+Hz1TPbc9y+CnoSKiD1AA7t0BiqBn8Rk2 XtxN9qXnRSapHT8Vpum/MIBh07ZG+KUU4ttSaZoNjCqR6XNKFMj3ihgX8W60d4z8 KOosGQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=1/VcZloQVYjqCko0EljzxsVrICrfA/bLGrVJVAJA3 NY=; b=gP8bZPRVACzsGGlkAUbaMWADra35VuYV8YSMzjLpm7sS0ZN7Xy7oJ3Q6K 7m7qkAjwsUQaS2G0K6wA3nPvr4G+fS2kIe1fMAG6mzU+zd6VJBoNpHmKsNCUwA+V c9maFUI6ScpoPKOgGbiH2eVnwFOkFIRfnHTzWdcf5QE7MoiBWdgIvlIc29rF1LJE HIwa6KzW3rQdHjixO2KlfZd96LAEbAEWCUORMVrrhXtSscbaensLoU+BgOH2aV0+ eZ67Rs1m6jD7EPXVFWq5Ap4cCE3CPjORDfn06HvoqWS/9oyJL34JfD7NmOronX/L eq/QbgsO65++qoxeUU6K0NAS2k4iQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudekgedgvdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkjghfggfgtgesthfure dttddtvdenucfhrhhomhepvfhhohhmrghsucfoohhnjhgrlhhonhcuoehthhhomhgrshes mhhonhhjrghlohhnrdhnvghtqeenucggtffrrghtthgvrhhnpedugefgvdefudfftdefge elgffhueekgfffhfeujedtteeutdejueeiiedvffegheenucevlhhushhtvghrufhiiigv pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomhgrshesmhhonhhjrghlohhnrd hnvght X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 30 Sep 2021 06:32:44 -0400 (EDT) From: Thomas Monjalon To: alialnu@nvidia.com Cc: ci@dpdk.org, jerinj@marvell.com, ferruh.yigit@intel.com, david.marchand@redhat.com, juraj.linkes@pantheon.tech Date: Thu, 30 Sep 2021 12:32:43 +0200 Message-ID: <2586128.JSg8es4y6H@thomas> In-Reply-To: <20210921143542.4412-11-alialnu@nvidia.com> References: <20210906154537.1299-1-alialnu@nvidia.com> <20210921143542.4412-1-alialnu@nvidia.com> <20210921143542.4412-11-alialnu@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-ci] [PATCH v2 10/10] tools: skip the IDs we already fetched X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" 21/09/2021 16:35, alialnu@nvidia.com: > From: Ali Alnubani > > Store the IDs we already fetched in a file and don't > run 'callcmd' again for them. We store all IDs. Should we manually remove olds one from time to time? We need an explanation about the strategy, why it is needed. I think it is because filtering by date is not enough. In order to not miss any patch, we should request a date earlier than the previous fetch and skip those already fetched. Where the "earlier date" is defined? > Signed-off-by: Ali Alnubani > --- > tools/poll-pw | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/tools/poll-pw b/tools/poll-pw > index e197a91..dacf34b 100755 > --- a/tools/poll-pw > +++ b/tools/poll-pw > @@ -65,6 +65,11 @@ if [ ! $? -eq 0 ] ; then > exit 1 > fi > > +poll_pw_ids_file=/tmp/poll_pw_${resource_type}_ids > +if [ ! -f "$poll_pw_ids_file" ] ; then > + touch $poll_pw_ids_file > +fi > + > URL="${URL}/${resource_type}/?project=${project}&" > > callcmd () # > @@ -77,7 +82,11 @@ while true ; do > date_now=$(date '+%FT%T') > since=$(date -d $(cat $since_file | tr '\n' ' ') '+%FT%T') > for id in $(curl -s "${URL}since=${since}" | jq '.[].id') ; do > + if grep -q "^${id}$" $poll_pw_ids_file ; then > + continue > + fi > callcmd $id > + echo $id >>$poll_pw_ids_file > done > echo -n $date_now >$since_file > # pause before next check >