From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 90BEB433F9; Tue, 28 Nov 2023 22:02:03 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 32FAD42E22; Tue, 28 Nov 2023 22:02:03 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 1D0F042E0F for ; Tue, 28 Nov 2023 22:02:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1701205321; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2oEy9z1NvA0f7c4BKQBVKUr2NXhFvmmso1rIyeV3L7Y=; b=bdWRCh9iHpEc9LAMONczN06pNU517Opn/cGLF0mIQCMgGHFUMwb1f6ei3fJBJzXJCj5Y7l W1I44rJtW5zzqX8n2esMmdvDiBG7IHdxO/47/cJdYZ9BKS1j6NhrI20VEGNng11jaYmTMu c04D5SicnO/dhrDMj1yjafkldZWTi6w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-606-vY9N4Sk2PgmISIPb67LGQA-1; Tue, 28 Nov 2023 16:01:57 -0500 X-MC-Unique: vY9N4Sk2PgmISIPb67LGQA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 94AF280C348; Tue, 28 Nov 2023 21:01:57 +0000 (UTC) Received: from RHTPC1VM0NT.lan (unknown [10.22.32.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3B19240C6E83; Tue, 28 Nov 2023 21:01:57 +0000 (UTC) From: Aaron Conole To: ci@dpdk.org Cc: Michael Santana , David Marchand , Thomas Monjalon , Patrick Robb , Dumitru Ceara Subject: [PATCH v3 2/3] recheck: Add a recheck parser for patchwork comments Date: Tue, 28 Nov 2023 16:01:53 -0500 Message-ID: <20231128210155.1388785-3-aconole@redhat.com> In-Reply-To: <20231128210155.1388785-1-aconole@redhat.com> References: <20231128210155.1388785-1-aconole@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Add a recheck parsing tool that will allow for labs to build a recheck workflow based on specific recheck labels and projects, with an associated state machine and querying interface. The output of the recheck parsing tool is json and can be fed to jq or other json parsing utilities for better field support. Acked-by: Michael Santana Signed-off-by: Aaron Conole --- RECHECK.rst | 81 ++++++++++++++++++++++++++++++++++++++ pw_mon | 59 +++++++++++++++++++++++++++- recheck_tool | 100 +++++++++++++++++++++++++++++++++++++++++++++++ series_db_lib.sh | 72 +++++++++++++++++++++++++++++++++- 4 files changed, 310 insertions(+), 2 deletions(-) create mode 100644 RECHECK.rst create mode 100755 recheck_tool diff --git a/RECHECK.rst b/RECHECK.rst new file mode 100644 index 0000000..080f397 --- /dev/null +++ b/RECHECK.rst @@ -0,0 +1,81 @@ +.. + Licensed under the terms of the GNU General Public License as published + by the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. You may obtain a copy of the + license at + + https://www.gnu.org/licenses/old-licenses/gpl-2.0.html + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + +============================ +Recheck Requests +============================ + +Various testing labs performing CI testing on new patch series send +their results to the appropriate mailing lists for a pw-ci monitor +to collect the reports and publish them to patchwork. On each +patch in the series, the results will appear with test category +contexts corresponding to the various test types which are run. +Examples include github-, travis-ci, etc. + +If a reported failure on a series seems suspicious to the patch submitter +or maintainer, then there may be an interest in requesting a retest on the +series for the failing label(s) in order to verify the failure is not +spurious or a false positive. This retest demonstrates to the submitter or +maintainer that the failure can be reliably reproduced. Unfortunately, at +present, the best way to accomplish this is to reach out to lab maintainers +via email or Slack. This is not ideal for developers in need of quick test +results. + +Going forward, CI testing labs have the option to implement a request for +retest of their respective test labels on patchwork via emails sent to the +developer mailing list. This is accomplished using the the `recheck_tool` +to advance the retest check state through various points. + +An example might look like:: + + $ export PROJ=foo + $ export INST=patches.foo.com + $ export FILTER=foo-ci + $ export FOO_TOKEN=abcd1234!@$% + $ ./pw_mon --pw-project=$PROJ --pw-instance=$INST \ + --add-filter-recheck=$FILTER + + $ for recheck in $(./recheck_tool --pw-project=$PROJ --pw-instance=$INST \ + --filter=$FILTER --state=0); do + echo "Asking for retest $recheck" + SERIES=$(echo $recheck | jq -rc '.series_id') + SHA=$(echo $recheck | jq -rc '.sha') + MSG=$(echo $recheck | jq -rc '.message_id') + ./foo_ci_restart --pw-project=$PROJ --pw-instance=$INST \ + --series-id=$SERIES --sha=$SHA --token=$FOO_TOKEN + ./recheck_tool --pw-project=$PROJ --pw-instance=$INST --msgid=$MSG \ + --series-id=$SERIES --filter=$FILTER \ + --update --state=0 --new-state=1 + done + +From the users' perspective, in order to request a retest on your patch +series, send an email reply to one of your series’s patch or cover letter +emails with email content of the format used below:: + + Recheck-request: + +The valid delimiter is a comma optionally followed by a space: “,” “, “ + +Valid examples:: + + Recheck-request: foo, bar, baz + + Recheck-request: foo,bar,baz + + Recheck-request: foo,bar,baz, + +Individual projects will have their own policies around who may request a +recheck, how many recheck requests may be sent, etc. This should be +documented by the individual projects. diff --git a/pw_mon b/pw_mon index e1c1eab..bbe9837 100755 --- a/pw_mon +++ b/pw_mon @@ -1,6 +1,6 @@ #!/bin/sh # SPDX-Identifier: gpl-2.0-or-later -# Copyright (C) 2018, Red Hat, Inc. +# Copyright (C) 2018-2023 Red Hat, Inc. # # Monitors a project on a patchwork instance for new series submissions # Records the submissions in the series database (and emits them on the @@ -44,6 +44,7 @@ if [ "$1" != "" ]; then fi fi +recheck_filter="" while [ "$1" != "" ]; do if echo "$1" | grep -q -s -E ^--pw-project= ; then @@ -64,8 +65,13 @@ while [ "$1" != "" ]; do echo "" echo "Options:" echo " --pw-credentials=u:p Sets user / password for web client" + echo " --add-filter-recheck=filter Adds a filter to flag that a recheck needs to be done" echo "" exit 0 + elif echo "$1" | grep -q -s -E ^--add-filter-recheck=; then + filter_str=$(echo "$1" | sed s/--add-filter-recheck=//) + recheck_filter="$filter_str $recheck_filter" + shift else echo "Unknown option: '$1'" echo "Rerun with --help for details" @@ -190,7 +196,58 @@ function check_superseded_series() { done } +function run_recheck() { + local recheck_list=$(echo "$7" | sed -e 's/^Recheck-request: // ' -e 's/,/ /g') + + for filter in $recheck_filter; do + for check in $recheck_list; do + if [ "$filter" == "$check" ]; then + insert_recheck_request_if_needed "$1" "$3" "$8" "$check" "$2" "$9" + fi + done + done +} + +function check_patch_for_retest_request() { + local patch_url="$1" + + local patch_json=$(curl -s "$userpw" "$patch_url") + local patch_comments_url=$(echo "$patch_json" | jq -rc '.comments') + local patch_id=$(echo "$patch_json" | jq -rc '.id') + if [ "Xnull" != "X$patch_comments_url" ]; then + local comments_json=$(curl -s "$userpw" "$patch_comments_url") + + local seq_end=$(echo "$comments_json" | jq -rc 'length') + if [ "$seq_end" -a $seq_end -gt 0 ]; then + seq_end=$((seq_end-1)) + for comment_id in $(seq 0 $seq_end); do + local recheck_requested=$(echo "$comments_json" | jq -rc ".[$comment_id].content" | grep "^Recheck-request: ") + if [ "X$recheck_requested" != "X" ]; then + local msgid=$(echo "$comments_json" | jq -rc ".[$comment_id].msgid") + run_recheck "$pw_instance" "$series_id" "$project" "$url" "$repo" "$branchname" "$recheck_requested" "$msgid" "$patch_id" + fi + done + fi + fi +} + +function check_series_needs_retest() { + local pw_instance="$1" + + series_get_active_branches "$pw_instance" | while IFS=\| read -r series_id project url repo branchname; do + local patch_comments_url=$(curl -s "$userpw" "$url" | jq -rc '.patches[] | .url') + + for patch in $patch_comments_url; do + check_patch_for_retest_request $patch + done + done +} + check_undownloaded_series "$pw_instance" "$pw_project" check_completed_series "$pw_instance" "$pw_project" check_new_series "$pw_instance" "$pw_project" check_superseded_series "$pw_instance" + +# check for retest requests after a series is still passing all the +# checks above +check_series_needs_retest "$pw_instance" diff --git a/recheck_tool b/recheck_tool new file mode 100755 index 0000000..98031a9 --- /dev/null +++ b/recheck_tool @@ -0,0 +1,99 @@ +#!/bin/sh +# SPDX-Identifier: gpl-2.0-or-later +# Copyright (C) 2023 Red Hat, Inc. +# +# Licensed under the terms of the GNU General Public License as published +# by the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. You may obtain a copy of the +# license at +# +# https://www.gnu.org/licenses/old-licenses/gpl-2.0.html +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +mode="select" + +while [ "$1" != "" ]; do + if echo "$1" | grep -q -s -E ^--help ; then + echo "recheck / retest state machine script" + echo "" + echo "$0:" + echo " --pw-project=: Patchwork project." + echo " --pw-instance=: Patchwork instance." + echo " --filter=: Job / request for recheck." + echo " --state=<0..>: Resync state ID." + echo " --msgid=: Message ID to select." + echo " --update: Set tool in update mode" + echo " --new-state=<0..>: New state ID to set" + echo " --series-id=<..>: Series ID" + echo "" + echo "Will spit out a parsable json for each db line when selecting" + exit 0 + elif echo "$1" | grep -q -s -E ^--pw-project= ; then + pw_project=$(echo "$1" | sed s/^--pw-project=//) + elif echo "$1" | grep -q -s -E ^--pw-instance= ; then + pw_instance=$(echo "$1" | sed s/^--pw-instance=//) + elif echo "$1" | grep -q -s -E ^--filter= ; then + filter=$(echo "$1" | sed s/^--filter=//) + elif echo "$1" | grep -q -s -E ^--state= ; then + recheck_state=$(echo "$1" | sed s/^--state=//) + elif echo "$1" | grep -q -s -E ^--msgid= ; then + message_id=$(echo "$1" | sed s/^--msgid=//) + elif echo "$1" | grep -q -s -E ^--update ; then + mode="update" + elif echo "$1" | grep -q -s -E ^--new-state= ; then + new_recheck_state=$(echo "$1" | sed s/^--new-state=//) + elif echo "$1" | grep -q -s -E ^--series-id= ; then + series_id=$(echo "$1" | sed s/^--series-id=//) + else + echo "Unknown option: '$1'" + echo "Rerun with --help for details" + exit 1 + fi + shift +done + +source $(dirname $0)/series_db_lib.sh + +if [ "$mode" == "select" ]; then + echo -n "{\"rechecks\":[" + printed="" + for request in $(get_recheck_requests_by_project "$pw_instance" \ + "$pw_project" \ + "$recheck_state" \ + "$filter"); do + message_id=$(echo $request | cut -d\| -f1) + series_id=$(echo $request | cut -d\| -f2) + patch_id=$(echo $request | cut -d\| -f3) + + sha=$(get_sha_for_series_id_and_patch "$series_id" "$patch_id" \ + "$pw_instance") + echo -n "$printed{\"pw_instance\":\"$pw_instance\",\"series_id\":$series_id,\"patch_id\":$patch_id,\"sha\":\"$sha\",\"msg_id\":\"$message_id\",\"state\":\"$recheck_state\",\"requested\":\"$filter\"}" + printed="," + done + echo "]}" +elif [ "$mode" == "update" ]; then + if [ "X$new_recheck_state" == "X" -o "X$series_id" == "X" ]; then + echo "Need to set a series-id and a new recheck state when updating." + exit 1 + fi + + request=$(get_recheck_request "$pw_instance" "$pw_project" "$message_id" \ + "$filter" "$series_id" "$recheck_state") + if [ "X$request" == "X" ]; then + echo "{\"result\":\"notfound\"}" + exit 0 + fi + + set_recheck_request_state "$pw_instance" "$pw_project" "$message_id" \ + "$filter" "$series_id" "$new_recheck_state" + + echo "{\"result\":\"executed\",\"recheck\":{\"pw_instance\": \"$pw_instance\", \"series_id\":$series_id, \"msg_id\":\"$message_id\", \"state\":\"$new_recheck_state\", \"requested\": \"$filter\"}}" +else + echo "Uknown state: $mode" + exit 1 +fi diff --git a/series_db_lib.sh b/series_db_lib.sh index 6c2d98e..e03a02e 100644 --- a/series_db_lib.sh +++ b/series_db_lib.sh @@ -1,6 +1,6 @@ #!/bin/sh # SPDX-Identifier: gpl-2.0-or-later -# Copyright (C) 2018,2019 Red Hat, Inc. +# Copyright (C) 2018-2023 Red Hat, Inc. # # Licensed under the terms of the GNU General Public License as published # by the Free Software Foundation; either version 2 of the License, or @@ -114,6 +114,22 @@ EOF run_db_command "INSERT INTO series_schema_version(id) values (7);" fi + run_db_command "select * from series_schema_version;" | egrep '^8$' > /dev/null 2>&1 + if [ $? -eq 1 ]; then + sqlite3 ${HOME}/.series-db </dev/null 2>&1; then + echo "INSERT INTO recheck_requests (recheck_message_id, recheck_requested_by, recheck_series, recheck_patch, patchwork_instance, patchwork_project, recheck_sync) values (\"$recheck_msgid\", \"$recheck_requested_by\", \"$recheck_series\", $recheck_patch, \"$recheck_instance\", \"$recheck_project\", 0);" | series_db_execute + fi +} + +function get_recheck_request() { + local recheck_instance="$1" + local recheck_project="$2" + local recheck_msgid="$3" + local recheck_requested_by="$4" + local recheck_series="$5" + local recheck_state="$6" + + echo "select * from recheck_requests where patchwork_instance=\"$recheck_instance\" and patchwork_project=\"$recheck_project\" and recheck_requested_by=\"$recheck_requested_by\" and recheck_series=\"$recheck_series\" and recheck_message_id=\"$recheck_msgid\" and recheck_sync=$recheck_state;" | series_db_execute +} + +function set_recheck_request_state() { + local recheck_instance="$1" + local recheck_project="$2" + local recheck_msgid="$3" + local recheck_requested_by="$4" + local recheck_series="$5" + local recheck_state="$6" + + echo "UPDATE recheck_requests set recheck_sync=$recheck_state where patchwork_instance=\"$recheck_instance\" and patchwork_project=\"$recheck_project\" and recheck_requested_by=\"$recheck_requested_by\" and recheck_series=\"$recheck_series\";" | series_db_execute +} -- 2.41.0