From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3CA4CA0C57 for ; Thu, 4 Nov 2021 17:53:29 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 31B2942762; Thu, 4 Nov 2021 17:53:29 +0100 (CET) Received: from AZHDRRW-EX02.NVIDIA.COM (azhdrrw-ex02.nvidia.com [20.64.145.131]) by mails.dpdk.org (Postfix) with ESMTP id CC0A3411C9 for ; Thu, 4 Nov 2021 17:53:27 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.105) by mxs.oss.nvidia.com (10.13.234.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.858.15; Thu, 4 Nov 2021 09:53:26 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LWTsYiwcKYpMvChSeR4i/WSpu+4pyHPie0u8CHFDqqOG8JtJgmv6tC7H+I1YVbc/fNGKhWvvjhC0m9TBusQAAITJZpZr8iTCikM9fGCGB044zNUi9Ko/GlQK6MaT3AF1MFxEbgtyOXCT5BVLkM4CbS03KQk4PtU6EXP41ibFj+a8k0/Ho1mYbJXrR20L826LF5v5ST1QhHRaYy2RW0lN0WUkjsIaKtvkmTepTa1Qo2lRYDSyQxMkpFNybNqp4hmLHBMKzoPEw/t94t0XMfz+/aqhXWnWQwR9bSqIlie6X6WjJc60ZmVHejPr395F4zTiyYVpA1WFhU1uU5p7B2MWPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gfpgwNIE3YHM7iqHbkxcY5SVsUqB5MuAi5hf4m5CXUU=; b=LQDeluZ1CnoknDZqNg6dlIouN1vKMI5zo8n6KQi9ecBb4w5l6kChmZxxOCi0W+rJeMp4xJo9i0fA2Wsdu4DFLxnLfaIx0oCiGiugBRv99Rbk0tR1ArlJeIsdIkiETNFVy86FgWDrHVgEedvqdYDqNZ7Ewx5gut9poz7K8txjhj7VphP43BHZqme9QtrfB4U/dZoW4B58w70RuFJrldkmMQhSkB2cN8vrpPhKRwFcjVEYMTJwqDLVdgrQsVx8475Zx0yoGmXGSRsQqFwDzswAR/cYfMI6wKkMH2Ilod9A69CwglThmzRehEFYSPmrvETI06c0cA9qRw2dDli/LUN1PA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gfpgwNIE3YHM7iqHbkxcY5SVsUqB5MuAi5hf4m5CXUU=; b=pdYP5fFMKSNzBazIuAc0f+SB96sN4Zlt4fKxguwA6/GB5hkaDoP71lsGcNbyyHkFZwghTzQqhwxB/WsiMegv0S7te2wiSQMeVCkyFoFzELPV3msAhmdJaXMm34arBk+9vIfWwbKcdPpENSHBkE7+2b/HOOP3PHlUWXy9BkEygN/o4DZpFv9lyfqXnBoYV6xWwcKGwz6tP9K6RlbaZCaNjypE/fbz3lTdF4MkSnNii7tS+LJxPDCLGFe2WzS/PJdLYyC0SzoUBZ4a+KY4Fj/0hWt61ZOd4L4TrB0Ud7joYlZlNiXUmI3Ort0dmb5aSfbCNPOcepD/EYDLHeTwPd+Mqg== Received: from CH0PR12MB5156.namprd12.prod.outlook.com (2603:10b6:610:bb::17) by CH0PR12MB5105.namprd12.prod.outlook.com (2603:10b6:610:bc::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.15; Thu, 4 Nov 2021 16:53:25 +0000 Received: from CH0PR12MB5156.namprd12.prod.outlook.com ([fe80::c8b:4fa:420d:5e5c]) by CH0PR12MB5156.namprd12.prod.outlook.com ([fe80::c8b:4fa:420d:5e5c%5]) with mapi id 15.20.4649.019; Thu, 4 Nov 2021 16:53:25 +0000 From: Ali Alnubani To: NBU-Contact-Thomas Monjalon CC: "ci@dpdk.org" , "jerinj@marvell.com" , "ferruh.yigit@intel.com" , "david.marchand@redhat.com" , "juraj.linkes@pantheon.tech" Thread-Topic: [PATCH v2 10/10] tools: skip the IDs we already fetched Thread-Index: AQHXteaDE41qb2aGHky+hgvhdsrt2KvOPJfwgADA54CACX/sYIAM/EuAgA5SPqA= Date: Thu, 4 Nov 2021 16:53:25 +0000 Message-ID: References: <20210906154537.1299-1-alialnu@nvidia.com> <2024535.ONPTUEBfDP@thomas> <13296855.Cuyoo0G14J@thomas> In-Reply-To: <13296855.Cuyoo0G14J@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ebda8c77-91af-4581-2f5d-08d99fb39e7e x-ms-traffictypediagnostic: CH0PR12MB5105: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: xuQUd9pJOU6NWlOpKmSr4zqitolI21XyKPfHQlvVXJBS+JGGN9YGQ+A4kw2QLZ+Jwob/C/jRIoAMP5kfe8lklaWd5QupX6HaPfzWlTr4VXwYQFWohwGRJgP/A9lfwG6BOOr+HEsXVf3VG3bLyiM+RKQiGySpw+ih430ATMwZJEA2frFU8rqTGFA7+lywjK4BsA0KmrlbFp1MZxlqHK9y6xK5OB6xsPuaQb7nKXsYOkYjgN5iGvEyVDuiawVRDGqGjXFinDkMmF+gGLapE52/LfbtqCIL39qFIa4r/oB0OnJpOUOQ5oV6rKDg3fGzuTUkS8owOv2e4M4k5VB9wjuC7Zz2MjKmHjMwXeLesTftvvvP6b4GAajNftRoTC9xvDXpGBLmg5uev+MncsjTiNq2MjYI2L51K1gdamLiC08lP5O6+EKhva6bkGUDykcF0ef0/6C1AnGcnvpiW2FesdtXxPB2O/iWKiSGwAsbx+1I0/qMvV16y5xof3iEla+JOsdAVhLCghXaZoFSUI7T6D2RBuliBVeSxrt8U5/tinGNVJoZU1alFn6WU963+6Lfp4Gtlqk/aAGoYbLWFVeN+Jlc03lm4+gFqEBYtui+V8O83AWSEbPJgUfTXy91wOQqUPg1W5NIkkM1dXbMS9gJiikTk4V6Vpktq+Sx+tHUxVUpDzHXV5WExUOUgfXISrxWZklJyscLbnRJKztNevbhbb8peg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH0PR12MB5156.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(316002)(71200400001)(5660300002)(38070700005)(83380400001)(2906002)(107886003)(66446008)(66556008)(64756008)(66946007)(52536014)(9686003)(55016002)(86362001)(33656002)(26005)(38100700002)(6506007)(53546011)(6862004)(8676002)(508600001)(8936002)(122000001)(54906003)(186003)(4326008)(76116006)(7696005)(66476007); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?S6JY7dDfhaKVrg229NHOrqkaXbRoj+Rnn96lWX3lcHQgkT0kRUQTKkdn93JO?= =?us-ascii?Q?H71LS62wj1XPZCaGVtDI+5UVh91PoXTfGlkQd7l/AkPvnO/v4jvBnZ9n8u36?= =?us-ascii?Q?1za0bmtOtvfxpwHlMBd8HjlubLdlPtUZ0nrvkBYmKwOazCZtFNXQNKjT83S4?= =?us-ascii?Q?GDoxeRngufY5sqaKa1+iXxaxCPYtnrjgSQqoy9oo4V6CbSWJknO1O4LVKmjm?= =?us-ascii?Q?cMeUxEiF3c6PzgwKPgDvoFjxLXK4SsGK0B6KmZ/zIlVi9eh8Nhf1c1Gepm0B?= =?us-ascii?Q?xORl8fwmtkCdLmmeSQqPoYD6gTVMfEJHnZ8/iQdugp8h9iSg9bNKkPWPoCqm?= =?us-ascii?Q?qik4Q7p5e76RaPIEDHosps276Gikn1Nn8gyegWFiVorXgtiGNCYJYP4k2ytE?= =?us-ascii?Q?xm48+3Z40+7YGitO4QMknkew6ZEnJs4KBY8NJif6SdAJSUaq/7DbGU+Wjh39?= =?us-ascii?Q?9uZ92RXhzJgs1bSTKLBc1Z9pvm4Cuv3RHm5qcXloVY6NUFRXeq2JQfpvCQnj?= =?us-ascii?Q?b2i/BnZOTnN0Qmtr02Uj4JxkO8K6jeXnHeQJT/G99js/hjXrwcibOIvIUlW1?= =?us-ascii?Q?5+/q8/xLeV+PdTF5+dVx39BI1HMFZHeu5165cbyhiJ6062kGyoEZpiJ6qGFO?= =?us-ascii?Q?iN+0uOC+sh3a8fteoQkE0aib+g7XJnp8XX3K1kfdGZpZ6+TL4/MMXPDciOfG?= =?us-ascii?Q?P1ctTD7J6v+jwXE7mEGwN8u72YKyueTcVe1pBzCUl+Tl+wfbBGEHv34DN6LL?= =?us-ascii?Q?+n8jjz6+b7HASWfo9z3H+ES5K5TcKTcSnqpyZxyNUc5/QSLrSF9l9hexdzXQ?= =?us-ascii?Q?9UPJ9kDgoi71MLRC3tHJMhEkqjHR6wSayH72QT5foi6sRnHDIX2uTd1GANrk?= =?us-ascii?Q?Kr5tLypiCwy96WtAOVdx1BNPVntyIhq/DlqZM55abnnLG+QpM1lDI3B1gB7J?= =?us-ascii?Q?WQga0b0GL7rAzw0KKAt5iJnmuZi7PRjFxjn3SxdVmD/kocvt8TT6zRN9FyAH?= =?us-ascii?Q?bRA3sotowP3+UPzAHjRzjyXx9r0P0pSU0boV1p+87EjfzWG8cpX/lR1llPFm?= =?us-ascii?Q?vUtVQAzSpNsJAYwRe1ZL5wIrGGEQxNgOowcNEPkdjes0sOvTmnqYiHopwP3t?= =?us-ascii?Q?BxMg+vGNqJIT7pDUR93F2ZvHT8Qv6JuGeC8uXaI0mrL6ubhrUlHIaJd2LbFX?= =?us-ascii?Q?jW5q8JzvI9viuX4lwISoaWJpGaf+UkBdLSs2fbugh1bV0Ggip534isemcPsW?= =?us-ascii?Q?tj8XduB2d39YEWClKuh6IGCa62HskAnqWWBqna2L4BBmaMR7G1A7abXrx4tf?= =?us-ascii?Q?3lxBO3hDqwbBdmnUDuXc3k4Cb1Qe8J1LcxCkPGPX1dw5OgDqc2MmAFxD6GVF?= =?us-ascii?Q?UZD6dCOHa+M1xg7NNjNSJuUGJ/Q9Gnun3Xcv5ep9ZTPL5fqVW+ZrIZQGrYGD?= =?us-ascii?Q?xM5gSqNICgfklzakj1qBcXgb0NHw7nVFva60PkHA8vQc0ryCKCWJo2//Cym7?= =?us-ascii?Q?5497rLrCYNweLw94GVZpH3dyScLBb3yUQRCe2zlMKCP7Y0x11mCfG3Ilusfe?= =?us-ascii?Q?dvwLO0eRlII/4MsZFSgEgd2zZ8jg9nWfcxRw6aDkwnybTCrQYfSAYobO+rKn?= =?us-ascii?Q?qAwYWaboivNE84B7cy7yttQ=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH0PR12MB5156.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: ebda8c77-91af-4581-2f5d-08d99fb39e7e X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Nov 2021 16:53:25.5483 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: nDAP0Q9FmdzdwpmC6tDfJIUei2JxMaq+IwXBgszaULewGBGshqixTB9SOC9WPf/FIa6N6k6TcGaV/0AW6tL4rQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5105 Subject: Re: [dpdk-ci] [PATCH v2 10/10] tools: skip the IDs we already fetched X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" > -----Original Message----- > From: Thomas Monjalon > Sent: Tuesday, October 26, 2021 5:07 PM > To: Ali Alnubani > Cc: ci@dpdk.org; jerinj@marvell.com; ferruh.yigit@intel.com; > david.marchand@redhat.com; juraj.linkes@pantheon.tech > Subject: Re: [PATCH v2 10/10] tools: skip the IDs we already fetched >=20 > 18/10/2021 10:04, Ali Alnubani: > > From: Thomas Monjalon > > > 11/10/2021 21:30, Ali Alnubani: > > > > From: Thomas Monjalon > > > > > 21/09/2021 16:35, alialnu@nvidia.com: > > > > > > From: Ali Alnubani > > > > > > > > > > > > Store the IDs we already fetched in a file and don't > > > > > > run 'callcmd' again for them. > > > > > > > > > > We store all IDs. Should we manually remove olds one from time to > time? > > > > > > > > Do you have a suggestion for when should we clear this file? Maybe > each > > > time the script starts? > > > > > > Yes at each start, we can remove the very old entries, like more than= 10 > > > hours old. > > > > Wouldn't this require using and depending on a database (at least an SQ= Lite > one) to additionally store timestamps to use for queries? > > Do you have a simpler way in mind to keep track of the IDs and their > timestamps to process later based on how long they have been in the file? > Maybe keep only up to a certain number of IDs in this file (up to a 1000 > maybe)? >=20 > Yes we can store each entry as a line starting with timestamp: > timestamp ID If the purpose is to not allow the script to grow indefinitely, is it ok to= just cut the file by half once it reaches a specific number of lines (like= 1000)?. Do you agree with this approach? It'd be easier to implement compared to st= oring and processing timestamps.