From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 75CFDA0C4B for ; Mon, 8 Nov 2021 07:29:44 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7159040151; Mon, 8 Nov 2021 07:29:44 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2083.outbound.protection.outlook.com [40.107.236.83]) by mails.dpdk.org (Postfix) with ESMTP id 1DAFF40040 for ; Mon, 8 Nov 2021 07:29:43 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Zfr9pC4uanzfmDZf9SP6TNJxkY2GnC96rC1ZStR95d4kHmneEbPnJKYRzxZlkWIzndLn4Mekrht1uK5FnDOQ+RAUqS3+9kL5hMebtwuPoLLC7q0mH/i8k+wp7Y2P+/st4M1a8xpiONuAt6VRuPWgToZjVYYJOJRLOe2sc+u+5+XBrBQCMDSoXs9fCqbic2ke0LhEFlpuGONcy0EPFfl6SdgYSzPfIIDy9mZVSwj14AG8ZxusiXt10vdq1w97Pe1sPBWswq2x+IjtJPO6ZHszyx71dJp1M5m46SqrZqjkjS/V4+LKTLEYVRu88cig38jkNKLgZOIq8CVgtB1kW6I/qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vn8Pr2evVkg1SQURryYgEQGyv1TWpPbgA5+9BePL1Ks=; b=M9PMe1l+zrkz6bc8QLS+2gH2J2yFViU4lPN8aPOmaUGbczZol3XN9RrtkE6IJ6e/aQ71/eqvPDa7WsQiWu8tWxfTMAmix0bhu4Zf2259VcEB7Ps35EZTsklgoakW6XTC7I6jJ4x6CFy00wb5rX+tFKqx0lCyH55MzN0EcW596PiDWPSQfZLfEkxoa38KKWBe5s2nbQPacI2QqNtao2VtjoMciOoB7s4H1LvYv/LRvpr5ykJSOdKXcljXsZY8eK3LyheE0v+atC37mIz/ByeMv5cbnjBCsn1edc4pjpQr3BEx8Cs5gCinqb/t/ZLK+q1b122A/Hmw19XcymnYAW79Wg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vn8Pr2evVkg1SQURryYgEQGyv1TWpPbgA5+9BePL1Ks=; b=RD23fFez0+lYPECgkTePk7Nlm6P6eTYUWDZ3yrz51S9hzz/ObkxZw4VWFMCxUexplIZ8O7r2E3wsClMPC4UWQCRDryqFlIjZynOQOa1kVuRp9UdLF9mOD8m+2uo13GHo1IOYgAcfv6jBwy7eQqzlOt7xdSHtzoY/d98BtRje4SohzVFYIXAUi1sCmx5nIND5FJIa4r5A8J+7d4oQC09EmkJRqOUDH7xqPMDRZ/R+vCPtXl13N1kq8w0ay6jyB8tJ97GCseiYlMxArH1kID7RTqbO1MvZ64z8I+uJf9cpqa5Q6vkDAnO0EJQ3CWP+itgaD4/Vp78BwD6a337On+w2Tg== Received: from DS7PR03CA0295.namprd03.prod.outlook.com (2603:10b6:5:3ad::30) by MN2PR12MB3392.namprd12.prod.outlook.com (2603:10b6:208:cb::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.15; Mon, 8 Nov 2021 06:29:41 +0000 Received: from DM6NAM11FT012.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3ad:cafe::7c) by DS7PR03CA0295.outlook.office365.com (2603:10b6:5:3ad::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 06:29:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT012.mail.protection.outlook.com (10.13.173.109) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4669.10 via Frontend Transport; Mon, 8 Nov 2021 06:29:40 +0000 Received: from nvidia.com (172.20.187.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 8 Nov 2021 06:29:37 +0000 From: Ali Alnubani To: CC: , , , , Date: Mon, 8 Nov 2021 08:28:15 +0200 Message-ID: <20211108062815.7614-11-alialnu@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211108062815.7614-1-alialnu@nvidia.com> References: <20210906154537.1299-1-alialnu@nvidia.com> <20211108062815.7614-1-alialnu@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.5] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aeee9dae-adcf-4ad4-d7f4-08d9a2812551 X-MS-TrafficTypeDiagnostic: MN2PR12MB3392: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VYwUfKxRvyxuerAdPysAw1GjrUH0LL+eJi2/Pwokr026Or19nLy0SldOB/fIHuPc0A79lGZwl0mc+kV7Xja1T7H3wUMv4CrijDDK0+/H42k53NScnbqSmeERKMaOZ5isn69hyYbv802rQHL0QX5BujZ1ssQQydMpwYqf6WhVgSgigD1+WO9+1nSQkoY1UvfHt9Kg+vKroSbAc/YofbSct44Pp8aN3llxx3qcNEd8PiRstTI9Llt0OqEiVDoBVu4eOPepvxv9CBWi90pilVDZSVlH1H3GvF8N2+OLIaNwheDk3xz1gM0/dlGyDJ2S+RFi5U1e9ba+uI4we1NtBKI9sfLjKiiSjfbSx3Hq3z/Dw3TZpBOW16Nzxp7jdS3NCodVkiIHUMAR/RKxI+RpgU508Xsj2ug66+E0vreNAS9lObSCkqsnFRHGsfCOfI0b3XV7Q7B10q+m7fwXeuKm2maPpDlcu363qnWiG+OdzTUVZfEwlh89h+zAQPCamapHRwAobq95AO8hh5GBSRtHApAux0oiznlWD6y+Y9x0Tfiep9NoSoYnheH05DsDjD+8cLbHd8vW6wQbDUMpicjNe45sG/NhU0YBRihbyuyBBXVVR7f3LsT5n/8gXhuIN6k0OGqRpRFnluMVn5XC22e+yHYnpgjCIz+L7g8dkj9PIktlzHd13BBcJy1RDNrVAOHsI1nvE8VcZYrMwRAV9creYOsX2Q== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(336012)(8936002)(7636003)(1076003)(86362001)(6666004)(26005)(8676002)(36756003)(54906003)(5660300002)(6916009)(70206006)(70586007)(82310400003)(55016002)(36860700001)(426003)(47076005)(2906002)(186003)(2616005)(316002)(16526019)(4326008)(356005)(508600001)(83380400001)(7696005)(6286002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2021 06:29:40.9278 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aeee9dae-adcf-4ad4-d7f4-08d9a2812551 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT012.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3392 Subject: [dpdk-ci] [PATCH v4 10/10] tools: skip the IDs we already fetched X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" To avoid calling 'callcmd' more than once for an ID, which can happen if a Patchwork ID was created between recording 'date_now' and fetching the API, store the IDs we already fetched in a file and don't run 'callcmd' again for any ID that exists in the file while keeping the number of entries less than 1000 in that file. Signed-off-by: Ali Alnubani --- Changes in v4: - Limit the number of lines in poll_pw_ids_file (Suggested by Thomas Monjalon). tools/poll-pw | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/tools/poll-pw b/tools/poll-pw index f4b89c7..dd19016 100755 --- a/tools/poll-pw +++ b/tools/poll-pw @@ -66,6 +66,11 @@ if [ ! $? -eq 0 ] ; then exit 1 fi +poll_pw_ids_file=/tmp/poll_pw_${resource_type}_ids +if [ ! -f "$poll_pw_ids_file" ] ; then + touch $poll_pw_ids_file +fi + URL="${URL}/events/?category=${resource_type}-completed" callcmd () # @@ -84,11 +89,19 @@ while true ; do | jq "try ( .payload.${resource_type}.id )") [ -z "$(echo $ids | tr -d '\n')" ] && break for id in $ids ; do + if grep -q "^${id}$" $poll_pw_ids_file ; then + continue + fi callcmd $id + echo $id >>$poll_pw_ids_file done page=$((page+1)) done echo -n $date_now >$since_file + # keep only up to 1000 entries in poll_pw_ids_file + if [ $(wc -l <$poll_pw_ids_file) -ge 1000 ] ; then + echo "$(tail -500 $poll_pw_ids_file)" >$poll_pw_ids_file + fi # pause before next check sleep $PAUSE_SECONDS done -- 2.25.1