From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1CD76A0C45 for ; Tue, 26 Oct 2021 16:07:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E53AC40E0F; Tue, 26 Oct 2021 16:07:11 +0200 (CEST) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by mails.dpdk.org (Postfix) with ESMTP id 9B549407FF for ; Tue, 26 Oct 2021 16:07:10 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id A080F5C03A7; Tue, 26 Oct 2021 10:07:09 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 26 Oct 2021 10:07:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm2; bh= 4eNB43R833r3hVuVGIPg+pZvdNr6RybGK3LZbIQIroo=; b=fY8eruZNt+TKUccX NQ0SvowqBZ+d+L27gzFUzhOP0FyGt1OCoUqHdhcF6MsFofZtY/0VAfkJ/f59geSU WYUakGD5/qxbvG177keyPH7E6zYJsFC+iE8tQuOyfqq4YVfA5q9CL1MSZgmwbwaX SCapYW0DuHcSmolPyn57HtlhZKYlld/f8EF0Dku+sV6vlDJoPxe7rJgJ9rFtYwiy 4XPZcRzXSLyQXWRhoiOYiOGmN5EAUGX4KaKWNMVbyAVszQbyMTMqIst0MVAhpZt2 RGG+GyBz9sQx6pKi8LrONBdRk9yM2Hk83SU9CjC1yl9kylG03ny1yIrjVfexIRVq 4KI3pQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=4eNB43R833r3hVuVGIPg+pZvdNr6RybGK3LZbIQIr oo=; b=kn99+xIJbC57NbzlJQxKgAT/p0hIS16LpwlSQP15hfMksSmoPodImfzvN /7ETJ8AC+youuvRfv3yWiAf5V35SE33dKw+yffNo3Ocv0gMxerZFE+N4Of3OWxVp Sxwb0hzC00eJ8rNYt0T6JoIyDLsz3h/R24b4pxplI72CVYDt1Za/gQsp3QhDUnwW LX0pN1KpKqWVw9iG6gX+ARmdrsBRTujsElJspGXJhstuBZXRcSKVK53hmQlpHgHb /1/b+N4yszZ4EY7/XaJMrC3UETOw9sSfAD6FatPhbm1Jj7AuR2vAzJw8gwcIhn8+ S7DhXqFcEKF0Uifaffst1WwZDC4Qg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrvdefkedggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkfgjfhgggfgtsehtufertddttddvnecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecugg ftrfgrthhtvghrnhepudeggfdvfeduffdtfeeglefghfeukefgfffhueejtdetuedtjeeu ieeivdffgeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homhepthhhohhmrghssehmohhnjhgrlhhonhdrnhgvth X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Oct 2021 10:07:08 -0400 (EDT) From: Thomas Monjalon To: Ali Alnubani Cc: "ci@dpdk.org" , "jerinj@marvell.com" , "ferruh.yigit@intel.com" , "david.marchand@redhat.com" , "juraj.linkes@pantheon.tech" Date: Tue, 26 Oct 2021 16:07:05 +0200 Message-ID: <13296855.Cuyoo0G14J@thomas> In-Reply-To: References: <20210906154537.1299-1-alialnu@nvidia.com> <2024535.ONPTUEBfDP@thomas> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-ci] [PATCH v2 10/10] tools: skip the IDs we already fetched X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" 18/10/2021 10:04, Ali Alnubani: > From: Thomas Monjalon > > 11/10/2021 21:30, Ali Alnubani: > > > From: Thomas Monjalon > > > > 21/09/2021 16:35, alialnu@nvidia.com: > > > > > From: Ali Alnubani > > > > > > > > > > Store the IDs we already fetched in a file and don't > > > > > run 'callcmd' again for them. > > > > > > > > We store all IDs. Should we manually remove olds one from time to time? > > > > > > Do you have a suggestion for when should we clear this file? Maybe each > > time the script starts? > > > > Yes at each start, we can remove the very old entries, like more than 10 > > hours old. > > Wouldn't this require using and depending on a database (at least an SQLite one) to additionally store timestamps to use for queries? > Do you have a simpler way in mind to keep track of the IDs and their timestamps to process later based on how long they have been in the file? Maybe keep only up to a certain number of IDs in this file (up to a 1000 maybe)? Yes we can store each entry as a line starting with timestamp: timestamp ID