From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <ci-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id C205EA0C4C
for <public@inbox.dpdk.org>; Tue, 21 Sep 2021 16:36:51 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id B387541140;
Tue, 21 Sep 2021 16:36:51 +0200 (CEST)
Received: from AZHDRRW-EX02.NVIDIA.COM (azhdrrw-ex02.nvidia.com
[20.64.145.131]) by mails.dpdk.org (Postfix) with ESMTP id 3250141140
for <ci@dpdk.org>; Tue, 21 Sep 2021 16:36:50 +0200 (CEST)
Received: from NAM02-DM3-obe.outbound.protection.outlook.com (104.47.56.49) by
mxs.oss.nvidia.com (10.13.234.37) with Microsoft SMTP Server
(version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.858.15; Tue, 21 Sep
2021 07:36:49 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=V1e5HdETtWbSXTbhU0aqb2nWlAslSnfKuuLy9QTvmbYkm16l889xn7t/yPOh7NrfbxiUbXE/RB0IzcXkNaq1hEF8/yunClyjHgdNfEPOUU3n+nBW93NdFZdtD1cKKAJOC/EuEihEryiF2F0ePaIXhHrKdsH9AwEPSJSpnf0HnYu/n0K8NtdPx0Zs6r2rW6BI2gpBr3+/TWTlc7O1t+c9pak9Dk/1T7oiDK3wxZeNzE0w+FAPkJ3HG7kA2gB70jNakZDFeLN0Az/ZDx3r1vMFEAvUlGesnRCkn2PVd0IP8JFSkgfEMdMMXsu1Vb71CF6EgP/MurNHpNdpz9m4WspoEg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
bh=Vg55Ed01JWXkcRBEPBi/n+XXDA8Mgmqc9iN9W2n6H2c=;
b=Khly0HS8ybDV6m+EWtccwi6xepCwxo3p90cSjvu0/IgvI4Fs5GYe3U2Olv7aVyqwBytqmdspY773bgF29A5h+4Pd/PWSdi1fCmtWqSTJ8VWf/3IPUlP8ebqsjE3U7JjvbPEEwVHzI1GiDfBDV30ezaHzMZ3CGEy4uPd21eYdxSenBySfsBjWsxJhO96BrvCT4JVBum1kHtUsyJrLICbhVk33EZbko4m48pxF+nuCOsmpCTAsFNL/cd2MpoKcjzQwSqOq5JnRH5cg7MVy1sEFsvnaqSzgOK+/kHpmBvGhpag6xxFrwo8KqOgh5Knn/krJZBrLY1e8BgF2/CVNDUf1vA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
216.228.112.32) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com;
dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com;
dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;
s=selector2;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=Vg55Ed01JWXkcRBEPBi/n+XXDA8Mgmqc9iN9W2n6H2c=;
b=lIWEK3P9ObfgreSpTy5shpYi9t60Cl6ZEsnl6Hi2rKeO3w5cIlUpIzMbAlT/P4NZv9u5AYO/f25zIbXZmhdWwGdyLKINPZm4K5NaoWVCZph4jYDLm4JDycFFozHf+HChKC2Uk3ZsE7wUkebZN12WsteaOLgeXKqCt7PGLnb2/Hy+yyABU5ZCdZOFU1Kf7ihtOHBMosrD2UkEFaQpJAuo1WMeYp9jgRczn/zdCfnv1qzer5FfFSHTSIjPdrFSUzSxDQ1ND8QL8E9wiyN9pZOeVbUAGCpQHsAygWPlGOccUqDKH0T8P0NmCY2+gVKGkOAQ8yGywiekzLTv3uv2g5LDHw==
Received: from DM5PR15CA0056.namprd15.prod.outlook.com (2603:10b6:3:ae::18) by
BN8PR12MB3489.namprd12.prod.outlook.com (2603:10b6:408:44::21) with
Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.4523.16; Tue, 21 Sep 2021 14:36:46 +0000
Received: from DM6NAM11FT036.eop-nam11.prod.protection.outlook.com
(2603:10b6:3:ae:cafe::f) by DM5PR15CA0056.outlook.office365.com
(2603:10b6:3:ae::18) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13 via Frontend
Transport; Tue, 21 Sep 2021 14:36:46 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32)
smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed)
header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
216.228.112.32 as permitted sender) receiver=protection.outlook.com;
client-ip=216.228.112.32; helo=mail.nvidia.com;
Received: from mail.nvidia.com (216.228.112.32) by
DM6NAM11FT036.mail.protection.outlook.com (10.13.172.64) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id
15.20.4523.14 via Frontend Transport; Tue, 21 Sep 2021 14:36:46 +0000
Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL109.nvidia.com
(172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 21 Sep
2021 07:36:45 -0700
Received: from nvidia.com (172.20.187.5) by DRHQMAIL107.nvidia.com
(10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 21 Sep
2021 14:36:42 +0000
From: <alialnu@oss.nvidia.com>
To: <ci@dpdk.org>
CC: <thomas@monjalon.net>, <jerinj@marvell.com>, <ferruh.yigit@intel.com>,
<david.marchand@redhat.com>, <juraj.linkes@pantheon.tech>
Date: Tue, 21 Sep 2021 17:35:39 +0300
Message-ID: <20210921143542.4412-8-alialnu@nvidia.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210921143542.4412-1-alialnu@nvidia.com>
References: <20210906154537.1299-1-alialnu@nvidia.com>
<20210921143542.4412-1-alialnu@nvidia.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [172.20.187.5]
X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To
DRHQMAIL107.nvidia.com (10.27.9.16)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: e8965081-d76c-492b-a5ea-08d97d0d3d26
X-MS-TrafficTypeDiagnostic: BN8PR12MB3489:
X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr
X-Microsoft-Antispam-PRVS: <BN8PR12MB34891856FC8338537663ED76DAA19@BN8PR12MB3489.namprd12.prod.outlook.com>
X-MS-Exchange-Transport-Forked: True
X-MS-Oob-TLC-OOBClassifiers: OLM:1728;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: spRuxLkIOJHApCEQ4p9sEB38DFfSTvogph6dv43g6rXP8Uth9MSR/txgwHw7RQhq9u1TI6eNTKniDC5FOkrnQ82rpRlJedfpQZyJbM1w3tToSjjLuLRhsLVAuQIl1/PVs8Hry6gdxkRQBQso18vm/sLEI6h6zq4hdTu4IjJXORItAdl/H5zXM8jvJL8eGCpMIPykRbOS1P+JjUDSuUvLUicMIya7hhLwnWJfkfQ6DLhP/RwzZtF6nlyV0GT8gpxi1AAOYouSBrmHODlbQUQdZqbj+wv4J4nG7/1uCaGfL5Q8CEcgx9h7SUGlD63wsU8VQE47FqYavcDptxycdQ0VHeHuJ1M54Sy1aWuWefHpUHuQCV7wdSdDo2Thu13c7trwzkDCDHtfftC+G1ICf6sMku2ZnHGq5hd7E3oQJK6j21PzEOUUmonDGBoE6fCkCY2rkL7y1OS7rHL9Le1rdrSWDnGYhnpHOnzWf3LH1lYsWX9EceoBM9dYcTqnsOxCws1DPAEB27dKRDCl2nWuInqXabtB2qK52tq9XI8PJmI4kytfn9W3mhdbcgWvLUMOxohnplU37kRsphbHnkyZsS0zSVz4Y6NYWD14EUAuc1+avs6W2UpB99GV952efBzz/kF3Pyam4HoKJsam4jRcCxdO4vRexeH+/KNsLhyNfrJ3WVBWLm6nAGJiTJrGhowTVrSsis9xgLUVyZILg6FHM0W/GSWYPQ2CUCkVBKzAQ7hIv0+l4+Uo0oO9RXQfakxjffTpsThp8JGWMXB0fmZX5aLZqnnjeHuOiDW0cKKCr3kFzO0=
X-Forefront-Antispam-Report: CIP:216.228.112.32; CTRY:US; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid01.nvidia.com; CAT:NONE;
SFS:(4636009)(46966006)(36840700001)(508600001)(356005)(6916009)(186003)(83380400001)(2616005)(54906003)(1076003)(55016002)(7636003)(6286002)(2906002)(36756003)(7696005)(8676002)(316002)(107886003)(8936002)(426003)(26005)(70206006)(36860700001)(5660300002)(16526019)(86362001)(4326008)(82310400003)(336012)(47076005)(2876002)(70586007)(6666004);
DIR:OUT; SFP:1101;
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2021 14:36:46.2586 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e8965081-d76c-492b-a5ea-08d97d0d3d26
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.32];
Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT036.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB3489
Subject: [dpdk-ci] [PATCH v2 07/10] tools: filter new patchwork IDs by date
X-BeenThere: ci@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK CI discussions <ci.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/ci>,
<mailto:ci-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/ci/>
List-Post: <mailto:ci@dpdk.org>
List-Help: <mailto:ci-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/ci>,
<mailto:ci-request@dpdk.org?subject=subscribe>
Errors-To: ci-bounces@dpdk.org
Sender: "ci" <ci-bounces@dpdk.org>
From: Ali Alnubani <alialnu@nvidia.com>
API resource IDs are guaranteed to be unique, but they aren't guaranteed
to have no gaps, for example, the following series IDs are
nonexistent: 16157, 17181, 18235.
Filtering by the date since the last check is necessary to later
add support for fetching new series IDs in addition to patch IDs.
Instead of requiring a file that contains the next patch ID,
a file containing the timestamp of the last time the API was fetched
is now used.
Each time the API is fetched for new patches, the timestamp
in the file gets updated, and the script sleeps an amount of time specified
by PAUSE_SECONDS before attempting to fetch new resources again.
The pause amount between each poll attempt is still 100 seconds.
Setting the env variable 'TZ' might be necessary if your timezone
doesn't match the server's timezone.
The package jq (Command-line JSON processor) is now required by the
script.
Example usage:
$ export TZ="Europe/Paris"
$ export MAINTAINERS_FILE_PATH=/path/to/dpdk/MAINTAINERS
$ ./tools/poll-pw /path/to/last.txt \
'/path/to/pw_maintainers_cli.py --type patch set_pw_delegate $1'
Signed-off-by: Ali Alnubani <alialnu@nvidia.com>
---
Changes in v2:
- Removed an unnecessary 'break' statement in the for loop.
tools/poll-pw | 56 ++++++++++++++++++++++++++++++++-------------------
1 file changed, 35 insertions(+), 21 deletions(-)
diff --git a/tools/poll-pw b/tools/poll-pw
index bdf860a..6ac8a61 100755
--- a/tools/poll-pw
+++ b/tools/poll-pw
@@ -1,58 +1,72 @@
-#! /bin/sh -e
+#! /bin/sh
# SPDX-License-Identifier: BSD-3-Clause
# Copyright 2017 6WIND S.A.
# Copyright 2018 Mellanox Technologies, Ltd
URL=http://patches.dpdk.org/api
+PAUSE_SECONDS=100
print_usage () {
cat <<- END_OF_HELP
- usage: $(basename $0) <counter> <command>
+ usage: $(basename $0) [OPTIONS] </path/to/last.txt> <command>
Poll patchwork and call command for each patch.
- The first patchwork id to be checked is read from the counter file.
- The command should use '$1' to be evaluated as patchwork id.
- When a patch is found and the command is successful,
- then the counter is incremented.
+ The first date to filter with is read from the specified file.
+ The command should use '$1' to be evaluated as the patch id.
+ The date in the specifed file is updated after each pull.
END_OF_HELP
}
+which jq >/dev/null 2>&1
+if [ ! $? -eq 0 ] ; then
+ echo "The command 'jq' doesn't exist, please install it."
+ exit 1
+fi
+
while getopts h arg ; do
case $arg in
h ) print_usage ; exit 0 ;;
? ) print_usage >&2 ; exit 1 ;;
esac
done
+
if [ $# -lt 2 ] ; then
- printf 'missing argument\n\n' >&2
+ printf 'missing argument(s)\n\n' >&2
print_usage >&2
exit 1
fi
shift $(($OPTIND - 1))
-counter=$1
+since_file=$1
shift
cmd=$*
+if [ ! -f "$since_file" ] ; then
+ echo "The file '$since_file' doesn't exist."
+ exit 1
+fi
+
+date -d "$(cat $since_file | tr '\n' ' ')" >/dev/null 2>&1
+if [ ! $? -eq 0 ] ; then
+ echo "The file '$since_file' doesn't contain a valid date format."
+ exit 1
+fi
+
+URL="${URL}/patches/?"
+
callcmd () # <patchwork id>
{
eval $cmd
}
-checkid () # <patchwork id>
-{
- curl -sfIo /dev/null $URL/patches/$1/ ||
- curl -sfIo /dev/null $URL/covers/$1/
-}
-
-pwid=$(cat $counter)
+set -e
while true ; do
- # process all recent patches
- while checkid $pwid ; do
- callcmd $pwid || break
- pwid=$(($pwid + 1))
- echo $pwid >$counter
+ date_now=$(date '+%FT%T')
+ since=$(date -d $(cat $since_file | tr '\n' ' ') '+%FT%T')
+ for id in $(curl -s "${URL}since=${since}" | jq '.[].id') ; do
+ callcmd $id
done
+ echo -n $date_now >$since_file
# pause before next check
- sleep 100
+ sleep $PAUSE_SECONDS
done
--
2.25.1