Re: [RFC v2 2/3] recheck: Add a recheck parser for patchwork comments

[Aaron Conole <aconole@redhat.com>]

Dumitru Ceara <dceara@redhat.com> writes:

> On 11/7/23 21:31, Aaron Conole wrote:
>> Add a recheck parsing tool that will allow for labs to build a
>> recheck workflow based on specific recheck labels and projects,
>> with an associated state machine and querying interface.
>> 
>> The output of the recheck parsing tool is json and can be fed to
>> jq or other json parsing utilities for better field support.
>> 
>> Signed-off-by: Aaron Conole <aconole@redhat.com>
>> ---
>
> Awesome, OVN really needs this kind of recheck capability in patchwork.
> One comment inline.
>
>>  pw_mon           |  58 ++++++++++++++++++++++++++-
>>  recheck_tool     | 100 +++++++++++++++++++++++++++++++++++++++++++++++
>>  series_db_lib.sh |  64 +++++++++++++++++++++++++++++-
>>  3 files changed, 220 insertions(+), 2 deletions(-)
>>  create mode 100755 recheck_tool
>> 
>> diff --git a/pw_mon b/pw_mon
>> index da4b9a9..29a9cb4 100755
>> --- a/pw_mon
>> +++ b/pw_mon
>> @@ -1,6 +1,6 @@
>>  #!/bin/sh
>>  # SPDX-Identifier: gpl-2.0-or-later
>> -# Copyright (C) 2018, Red Hat, Inc.
>> +# Copyright (C) 2018-2023 Red Hat, Inc.
>>  #
>>  # Monitors a project on a patchwork instance for new series submissions
>>  # Records the submissions in the series database (and emits them on the
>> @@ -44,6 +44,7 @@ if [ "$1" != "" ]; then
>>      fi
>>  fi
>>  
>> +recheck_filter=""
>>  
>>  while [ "$1" != "" ]; do
>>      if echo "$1" | grep -q -s -E ^--pw-project= ; then
>> @@ -67,6 +68,10 @@ while [ "$1" != "" ]; do
>>          echo "    --add-filter-recheck=filter	Adds a filter to flag that a recheck needs to be done"
>>          echo ""
>>          exit 0
>> +    elif echo "$1" | grep -q -s -E ^--add-filter-recheck=; then
>> +        filter_str=$(echo "$1" | sed s/--add-filter-recheck=//)
>> +        recheck_filter="$filter_str $recheck_filter"
>> +        shift
>>      else
>>          echo "Unknown option: '$1'"
>>          echo "Rerun with --help for details"
>> @@ -191,7 +196,58 @@ function check_superseded_series() {
>>      done
>>  }
>>  
>> +function run_recheck() {
>> +    local recheck_list=$(echo "$7" | sed -e 's/^Recheck-request: // ' -e 's/,/ /g')
>> +
>> +    for filter in $recheck_filter; do
>> +        for check in $recheck_list; do
>> +            if [ "$filter" == "$check" ]; then
>> +                insert_recheck_request_if_needed "$1" "$3" "$8" "$check" "$2" "$9"
>> +            fi
>> +        done
>> +    done
>> +}
>> +
>> +function check_patch_for_retest_request() {
>> +    local patch_url="$1"
>> +
>> +    local patch_json=$(curl -s "$userpw" "$patch_url")
>> +    local patch_comments_url=$(echo "$patch_json" | jq -rc '.comments')
>> +    local patch_id=$(echo "$patch_json" | jq -rc '.id')
>> +    if [ "Xnull" != "X$patch_comments_url" ]; then
>> +        local comments_json=$(curl -s "$userpw" "$patch_comments_url")
>> +
>> +        local seq_end=$(echo "$comments_json" | jq -rc 'length')
>> +        if [ "$seq_end" -a $seq_end -gt 0 ]; then
>> +            seq_end=$((seq_end-1))
>> +            for comment_id in $(seq 0 $seq_end); do
>> +                local recheck_requested=$(echo "$comments_json" | jq -rc ".[$comment_id].content" | grep "^Recheck-request: ")
>
> This is "user facing", does the expected format of the comment need to
> be documented somewhere, e.g., in the README.rst file?

We definitely **should** add this to some documentation.  For now, it is
just knowledge we roll around in our heads.  OTOH, the recheck was
discussed at:

https://mails.dpdk.org/archives/ci/2021-May/001189.html

So we did argue a bit about the formatting, etc.

For OVN project (for example), we probably would want two different test
names - one for the ovn-kubernetes workflow, and one for the build and
test workflow.  And that is something we can hash out and at least
announce (with possibly some documentation in the OVN tree).

> On that note, can we add a reference to some of the documentation in the
> email the bot generates as reply to the patch in the series?

That's a good idea.  I need to spend some time to write the up a
comprehensive documentation of the robot anyway.

> Thanks,
> Dumitru
>
>> +                if [ "X$recheck_requested" != "X" ]; then
>> +                    local msgid=$(echo "$comments_json" | jq -rc ".[$comment_id].msgid")
>> +                    run_recheck "$pw_instance" "$series_id" "$project" "$url" "$repo" "$branchname" "$recheck_requested" "$msgid" "$patch_id"
>> +                fi
>> +            done
>> +        fi
>> +    fi
>> +}
>> +
>> +function check_series_needs_retest() {
>> +    local pw_instance="$1"
>> +
>> +    series_get_active_branches "$pw_instance" | while IFS=\| read -r series_id project url repo branchname; do
>> +        local patch_comments_url=$(curl -s "$userpw" "$url" | jq -rc '.patches[] | .url')
>> +
>> +        for patch in $patch_comments_url; do
>> +            check_patch_for_retest_request $patch
>> +        done
>> +    done
>> +}
>> +
>>  check_undownloaded_series "$pw_instance" "$pw_project"
>>  check_completed_series "$pw_instance" "$pw_project"
>>  check_new_series "$pw_instance" "$pw_project"
>>  check_superseded_series "$pw_instance"
>> +
>> +# check for retest requests after a series is still passing all the
>> +# checks above
>> +check_series_needs_retest "$pw_instance"
>> diff --git a/recheck_tool b/recheck_tool
>> new file mode 100755
>> index 0000000..98031a9
>> --- /dev/null
>> +++ b/recheck_tool
>> @@ -0,0 +1,100 @@
>> +#!/bin/sh
>> +# SPDX-Identifier: gpl-2.0-or-later
>> +# Copyright (C) 2023 Red Hat, Inc.
>> +#
>> +# Licensed under the terms of the GNU General Public License as published
>> +# by the Free Software Foundation; either version 2 of the License, or
>> +# (at your option) any later version.  You may obtain a copy of the
>> +# license at
>> +#
>> +#    https://www.gnu.org/licenses/old-licenses/gpl-2.0.html
>> +#
>> +# Unless required by applicable law or agreed to in writing, software
>> +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
>> +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
>> +# License for the specific language governing permissions and limitations
>> +# under the License.
>> +
>> +mode="select"
>> +
>> +while [ "$1" != "" ]; do
>> +    if echo "$1" | grep -q -s -E ^--help ; then
>> +        echo "recheck / retest state machine script"
>> +        echo ""
>> +        echo "$0:"
>> +        echo " --pw-project=<proj>:	Patchwork project."
>> +        echo " --pw-instance=<inst>:	Patchwork instance."
>> +        echo " --filter=<str>:		Job / request for recheck."
>> +        echo " --state=<0..>:		Resync state ID."
>> +        echo " --msgid=<msgid>:		Message ID to select."
>> +        echo " --update:		Set tool in update mode"
>> +        echo " --new-state=<0..>:	New state ID to set"
>> +        echo " --series-id=<..>:	Series ID"
>> +        echo ""
>> +        echo "Will spit out a parsable json for each db line when selecting"
>> +        exit 0
>> +    elif echo "$1" | grep -q -s -E ^--pw-project= ; then
>> +        pw_project=$(echo "$1" | sed s/^--pw-project=//)
>> +    elif echo "$1" | grep -q -s -E ^--pw-instance= ; then
>> +        pw_instance=$(echo "$1" | sed s/^--pw-instance=//)
>> +    elif echo "$1" | grep -q -s -E ^--filter= ; then
>> +        filter=$(echo "$1" | sed s/^--filter=//)
>> +    elif echo "$1" | grep -q -s -E ^--state= ; then
>> +        recheck_state=$(echo "$1" | sed s/^--state=//)
>> +    elif echo "$1" | grep -q -s -E ^--msgid= ; then
>> +        message_id=$(echo "$1" | sed s/^--msgid=//)
>> +    elif echo "$1" | grep -q -s -E ^--update ; then
>> +        mode="update"
>> +    elif echo "$1" | grep -q -s -E ^--new-state= ; then
>> +        new_recheck_state=$(echo "$1" | sed s/^--new-state=//)
>> +    elif echo "$1" | grep -q -s -E ^--series-id= ; then
>> +        series_id=$(echo "$1" | sed s/^--series-id=//)
>> +    else
>> +        echo "Unknown option: '$1'"
>> +        echo "Rerun with --help for details"
>> +        exit 1
>> +    fi
>> +    shift
>> +done
>> +
>> +source $(dirname $0)/series_db_lib.sh
>> +
>> +if [ "$mode" == "select" ]; then
>> +    echo -n "{\"rechecks\":["
>> +    printed=""
>> +    for request in $(get_recheck_requests_by_project "$pw_instance" \
>> +                                                     "$pw_project" \
>> +                                                     "$recheck_state" \
>> +                                                     "$filter"); do
>> +        message_id=$(echo $request | cut -d\| -f1)
>> +        series_id=$(echo $request | cut -d\| -f2)
>> +        patch_id=$(echo $request | cut -d\| -f3)
>> +
>> +        sha=$(get_sha_for_series_id_and_patch "$series_id" "$patch_id" \
>> +                                              "$pw_instance")
>> +        echo -n "$printed{\"pw_instance\":\"$pw_instance\",\"series_id\":$series_id,\"patch_id\":$patch_id,\"sha\":\"$sha\",\"msg_id\":\"$message_id\",\"state\":\"$recheck_state\",\"requested\":\"$filter\"}"
>> +        printed=","
>> +    done
>> +    echo "]}"
>> +elif [ "$mode" == "update" ]; then
>> +    if [ "X$new_recheck_state" == "X" -o "X$series_id" == "X" ]; then
>> +        echo "Need to set a series-id and a new recheck state when updating."
>> +        exit 1
>> +    fi
>> +
>> +    request=$(get_recheck_request "$pw_instance" "$pw_project" "$message_id" \
>> +                                  "$filter" "$series_id" "$recheck_state")
>> +    if [ "X$request" == "X" ]; then
>> +        echo "{\"result\":\"notfound\"}"
>> +        exit 0
>> +    fi
>> +
>> +    set_recheck_request_state "$pw_instance" "$pw_project" "$message_id" \
>> +                              "$filter" "$series_id" "$new_recheck_state"
>> +
>> +    echo "{\"result\":\"executed\",\"recheck\":{\"pw_instance\": \"$pw_instance\", \"series_id\":$series_id, \"msg_id\":\"$message_id\", \"state\":\"$new_recheck_state\", \"requested\": \"$filter\"}}"
>> +else
>> +    echo "Unknown state: $mode"
>> +    exit 1
>> +fi
>> +
>> diff --git a/series_db_lib.sh b/series_db_lib.sh
>> index 6c2d98e..ca33c1f 100644
>> --- a/series_db_lib.sh
>> +++ b/series_db_lib.sh
>> @@ -1,6 +1,6 @@
>>  #!/bin/sh
>>  # SPDX-Identifier: gpl-2.0-or-later
>> -# Copyright (C) 2018,2019 Red Hat, Inc.
>> +# Copyright (C) 2018-2023 Red Hat, Inc.
>>  #
>>  # Licensed under the terms of the GNU General Public License as published
>>  # by the Free Software Foundation; either version 2 of the License, or
>> @@ -114,6 +114,22 @@ EOF
>>          run_db_command "INSERT INTO series_schema_version(id) values (7);"
>>      fi
>>  
>> +    run_db_command "select * from series_schema_version;" | egrep '^8$' > /dev/null 2>&1
>> +    if [ $? -eq 1 ]; then
>> +        sqlite3 ${HOME}/.series-db <<EOF
>> +CREATE TABLE recheck_requests (
>> +recheck_id INTEGER,
>> +recheck_message_id STRING,
>> +recheck_requested_by STRING,
>> +recheck_series STRING,
>> +recheck_patch INTEGER,
>> +patchwork_instance STRING,
>> +patchwork_project STRING,
>> +recheck_sync INTEGER
>> +);
>> +EOF
>> +        run_db_command "INSERT INTO series_schema_version(id) values (8);"
>> +    fi
>>  }
>>  
>>  function series_db_exists() {
>> @@ -390,3 +406,49 @@ function get_patch_id_by_series_id_and_sha() {
>>  
>>      echo "select patch_id from git_builds where patchwork_instance=\"$instance\" and series_id=$series_id and sha=\"$sha\";" | series_db_execute
>>  }
>> +
>> +function get_recheck_requests_by_project() {
>> +    local recheck_instance="$1"
>> +    local recheck_project="$2"
>> +    local recheck_state="$3"
>> +    local recheck_requested_by="$4"
>> +
>> +    series_db_exists
>> +
>> +    echo "select recheck_message_id,recheck_series,recheck_patch from recheck_requests where patchwork_instance=\"$recheck_instance\" and patchwork_project=\"$recheck_project\" and recheck_sync=$recheck_state and recheck_requested_by=\"$recheck_requested_by\";" | series_db_execute
>> +}
>> +
>> +function insert_recheck_request_if_needed() {
>> +    local recheck_instance="$1"
>> +    local recheck_project="$2"
>> +    local recheck_msgid="$3"
>> +    local recheck_requested_by="$4"
>> +    local recheck_series="$5"
>> +    local recheck_patch="$6"
>> +
>> +    if ! echo "select * from recheck_requests where recheck_message_id=\"$recheck_msgid\";" | series_db_execute | grep $recheck_msgid >/dev/null 2>&1; then
>> +        echo "INSERT INTO recheck_requests (recheck_message_id, recheck_requested_by, recheck_series, recheck_patch, patchwork_instance, patchwork_project, recheck_sync) values (\"$recheck_msgid\", \"$recheck_requested_by\", \"$recheck_series\", $recheck_patch, \"$recheck_instance\", \"$recheck_project\", 0);" | series_db_execute
>> +    fi
>> +}
>> +
>> +function get_recheck_request() {
>> +    local recheck_instance="$1"
>> +    local recheck_project="$2"
>> +    local recheck_msgid="$3"
>> +    local recheck_requested_by="$4"
>> +    local recheck_series="$5"
>> +    local recheck_state="$6"
>> +
>> +    echo "select * from recheck_requests where patchwork_instance=\"$recheck_instance\" and patchwork_project=\"$recheck_project\" and recheck_requested_by=\"$recheck_requested_by\" and recheck_series=\"$recheck_series\" and recheck_message_id=\"$recheck_msgid\" and recheck_sync=$recheck_state;" | series_db_execute
>> +}
>> +
>> +function set_recheck_request_state() {
>> +    local recheck_instance="$1"
>> +    local recheck_project="$2"
>> +    local recheck_msgid="$3"
>> +    local recheck_requested_by="$4"
>> +    local recheck_series="$5"
>> +    local recheck_state="$6"
>> +
>> +    echo "UPDATE recheck_requests set recheck_sync=$recheck_state where patchwork_instance=\"$recheck_instance\" and patchwork_project=\"$recheck_project\" and recheck_requested_by=\"$recheck_requested_by\" and recheck_series=\"$recheck_series\";" | series_db_execute
>> +}