From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A3FF5A0C54 for ; Mon, 6 Sep 2021 17:46:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9A5E5410ED; Mon, 6 Sep 2021 17:46:53 +0200 (CEST) Received: from AZHDRRW-EX01.nvidia.com (azhdrrw-ex01.nvidia.com [20.51.104.162]) by mails.dpdk.org (Postfix) with ESMTP id 87C1E4111D for ; Mon, 6 Sep 2021 17:46:52 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by mxs.oss.nvidia.com (10.13.234.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.858.15; Mon, 6 Sep 2021 08:46:51 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k8DaABmRvHrvvHXr7bLOBkgzv7paI3KAEOpG/l/AAZdRDV2z0PlV6bvvAVwAv/9OdEsQxTFM8ZkpVtXDbiryhCbFz3kIo+HW8vSW3b1JW1FmSr+QMQO6BK9Vl2yU9R681xxVeSK3mfcjVi1laae82G3ZHvl2rKW5TZNFJsRLLtbq4N7UIX/nUNBXq4eeFQAcZiJEaeIijWkRrC9wv2RWvN7JqMppk7tdbcmQVfScPS8DZEfy0SGEorAuSlacqVc3r8geu51ztMArErv/z6oxgRfVuahmp2pNU1HqyShnWiKfFrtVZCrwulcusDYjmvL3yRSoGSvwrrUn7V9ZrPa6qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=/yidV0q4tBWbHGx3UBJDmuFCamTVibgB/bajF/Uc5MM=; b=ISr+LIDP8jfVmjbnmsQUM8gffiiQ8w0bJ4zinLg5h9rTRV3zNM4++KjkyHhWR3EqC0XitMspfzbm/kfnTizMOPdtywgbzdLPzCMNX7qfCD5/k2gGvmlQQvbwhgvez/MsKAuoPL4OIRS+Vjhbcuw5/g/k/4rWYpJ7mgEXIXGr/pWtJVxGlGHCvevmBs8C/VTZA0uGDrZxLL6dsQrFwWBClUJghFG8+ZMmZ/cySI9FPgpSXHFJtDMljU3KIXnaLQrKH/9SXjIGhgQQ6P9uTPTLxcOXD1i8VENa4cSVsfIgu6fYKjWZIa7C0+svYRd990yXB0ZdikLRVcm1znoimj0s/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/yidV0q4tBWbHGx3UBJDmuFCamTVibgB/bajF/Uc5MM=; b=pKMz4XuY/yBjGAMeJ8zCuD+YsI2yYJkgO617Qp3Z1IyCwdxeD7MvBSL0wzGrnRWTC6sALhM2e/x7Y30H6Gwxe8kty+jOJAqo4wltYRo1CI953NxY3ljgmVlCAPtEvSOHbMuKCt1GLrTqhvjC68ZHDsL1cEwlAneuYicdIEJKa0kuYsTcoc2jc2Wb0lXDDMENCdA+fDFbdquTZ2vGN6tLM8w74/yzuHs0LLM4YOQYVhnaKfuj90F+sqyIxASWAylVPUMW7czdAF3rs9npL7fH0THD/zUCXZSW2Z2+CDmD34iBAyNk3tMpiWyeBegDV3K1nPy4EzASycHcRhOMN9Nmzw== Received: from DM5PR21CA0061.namprd21.prod.outlook.com (2603:10b6:3:129::23) by MW3PR12MB4459.namprd12.prod.outlook.com (2603:10b6:303:56::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Mon, 6 Sep 2021 15:46:50 +0000 Received: from DM6NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:3:129:cafe::bf) by DM5PR21CA0061.outlook.office365.com (2603:10b6:3:129::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.3 via Frontend Transport; Mon, 6 Sep 2021 15:46:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by DM6NAM11FT031.mail.protection.outlook.com (10.13.172.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4478.19 via Frontend Transport; Mon, 6 Sep 2021 15:46:50 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 6 Sep 2021 15:46:49 +0000 Received: from nvidia.com (172.20.187.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 6 Sep 2021 15:46:47 +0000 From: Ali Alnubani To: CC: , , , Date: Mon, 6 Sep 2021 18:45:35 +0300 Message-ID: <20210906154537.1299-8-alialnu@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210906154537.1299-1-alialnu@nvidia.com> References: <20210906154537.1299-1-alialnu@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b5448018-e3a2-4943-b264-08d9714d8a9d X-MS-TrafficTypeDiagnostic: MW3PR12MB4459: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-Transport-Forked: True X-MS-Oob-TLC-OOBClassifiers: OLM:1728; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QZyipWwIG3rCtx8FsPslQy+aAtqCbNBZwjM8kTvcIALx8S1wyRguKMeadquF9aIBjb7fKjdrEBGtxFgVnHgl2ZDmXi5owd9eVtlQwcPj3sUbDBAedj4u/3fb5fJwWgvNCsXBeF1bgRc4wQT7uKqEyHHd/wqkUsvIt4FrYWFxg7isW1umr2TaW2LaXUUkmUGm2lY5zzN7bgIc8F4Bi7SM+xQ9rZDOnN47xil/MQe8CkJYGQth5dPEZ0MHTs0XRb6QPT0qbceF2B05QRsr0vfxektWuHulH+6cxs6ANljz/+7otepku5CuoPSyUiv2lUNQwYBHKgTQygWDOmWLyOhzg8MyFQJ7t1HjvRwFi+9hOBWsT0YG+BIAOm+U/BcF4Glt+wamZkLKBg/dCUR85WYbc2u0FSvnVyL2yhbjF3nNzL4BJbvA0rHmKScBYCAJFm7cIXW+mcHn5n1rg+INFJp0dDxoe+/TTrYjH2MSSrGK5uf+b8KBAhyHQWIAdWHVo4Ze4rMnAkVyzLe4y8bQI7V/WW+pdZ/UlkHD6zkPn95agqn8ESHlrtLuA25eEjcIgUMLAh7/zR1UG2bniBMgwWodDor4rrqZa8PSVI/93z8oOi5gldJQgpg4O+UeTDzzyP1x0f/H4gN1CRnW2JU+Rc9gncp+qPTu6ZXStSrquHk8Y4oBSv5nqZMm4rB6dlR1Za0IAOo5pkqMp+5DnWOGJKiwvFqNfZVMbp2Z92+OuLtGOXlKTRY431aak28o27i1FnpGCWYjPbthrJaiU9hdULLSgtvzNLRLmH1j/hUUPGiHCLE= X-Forefront-Antispam-Report: CIP:216.228.112.35; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid04.nvidia.com; CAT:NONE; SFS:(4636009)(136003)(376002)(346002)(39860400002)(396003)(46966006)(36840700001)(6916009)(70586007)(8936002)(7696005)(336012)(2616005)(478600001)(186003)(6666004)(1076003)(55016002)(26005)(36756003)(356005)(16526019)(54906003)(2906002)(86362001)(7636003)(36860700001)(83380400001)(8676002)(4326008)(107886003)(82310400003)(6286002)(70206006)(426003)(316002)(82740400003)(36906005)(47076005)(5660300002); DIR:OUT; SFP:1101; X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2021 15:46:50.0397 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b5448018-e3a2-4943-b264-08d9714d8a9d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.35]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4459 Subject: [dpdk-ci] [PATCH 7/9] tools: filter new patchwork IDs by date X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" API resource IDs are guaranteed to be unique, but they aren't guaranteed to have no gaps, for example, the following series IDs are nonexistent: 16157, 17181, 18235. Filtering by the date since the last check is necessary to later add support for fetching new series IDs in addition to patch IDs. Instead of requiring a file that contains the next patch ID, a file containing the timestamp of the last time the API was fetched is now used. Each time the API is fetched for new patches, the timestamp in the file gets updated, and the script sleeps an amount of time specified by PAUSE_SECONDS before attempting to fetch new resources again. The pause amount between each poll attempt is still 100 seconds. Setting the env variable 'TZ' might be necessary if your timezone doesn't match the server's timezone. The package jq (Command-line JSON processor) is now required by the script. Example usage: $ export TZ="Europe/Paris" $ export MAINTAINERS_FILE_PATH=/path/to/dpdk/MAINTAINERS $ ./tools/poll-pw /path/to/last.txt \ '/path/to/maintainers.py --command set_pw_delegate --type patch $1' Signed-off-by: Ali Alnubani --- tools/poll-pw | 56 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 35 insertions(+), 21 deletions(-) diff --git a/tools/poll-pw b/tools/poll-pw index bdf860a..e104dab 100755 --- a/tools/poll-pw +++ b/tools/poll-pw @@ -1,58 +1,72 @@ -#! /bin/sh -e +#! /bin/sh # SPDX-License-Identifier: BSD-3-Clause # Copyright 2017 6WIND S.A. # Copyright 2018 Mellanox Technologies, Ltd URL=http://patches.dpdk.org/api +PAUSE_SECONDS=100 print_usage () { cat <<- END_OF_HELP - usage: $(basename $0) + usage: $(basename $0) [OPTIONS] Poll patchwork and call command for each patch. - The first patchwork id to be checked is read from the counter file. - The command should use '$1' to be evaluated as patchwork id. - When a patch is found and the command is successful, - then the counter is incremented. + The first date to filter with is read from the specified file. + The command should use '$1' to be evaluated as the patch id. + The date in the specifed file is updated after each pull. END_OF_HELP } +which jq >/dev/null 2>&1 +if [ ! $? -eq 0 ] ; then + echo "The command 'jq' doesn't exist, please install it." + exit 1 +fi + while getopts h arg ; do case $arg in h ) print_usage ; exit 0 ;; ? ) print_usage >&2 ; exit 1 ;; esac done + if [ $# -lt 2 ] ; then - printf 'missing argument\n\n' >&2 + printf 'missing argument(s)\n\n' >&2 print_usage >&2 exit 1 fi shift $(($OPTIND - 1)) -counter=$1 +since_file=$1 shift cmd=$* +if [ ! -f "$since_file" ] ; then + echo "The file '$since_file' doesn't exist." + exit 1 +fi + +date -d "$(cat $since_file | tr '\n' ' ')" >/dev/null 2>&1 +if [ ! $? -eq 0 ] ; then + echo "The file '$since_file' doesn't contain a valid date format." + exit 1 +fi + +URL="${URL}/patches/?" + callcmd () # { eval $cmd } -checkid () # -{ - curl -sfIo /dev/null $URL/patches/$1/ || - curl -sfIo /dev/null $URL/covers/$1/ -} - -pwid=$(cat $counter) +set -e while true ; do - # process all recent patches - while checkid $pwid ; do - callcmd $pwid || break - pwid=$(($pwid + 1)) - echo $pwid >$counter + date_now=$(date '+%FT%T') + since=$(date -d $(cat $since_file | tr '\n' ' ') '+%FT%T') + for id in $(curl -s "${URL}since=${since}" | jq '.[].id') ; do + callcmd $id || break done + echo -n $date_now >$since_file # pause before next check - sleep 100 + sleep $PAUSE_SECONDS done -- 2.25.1