From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8FB8C43358; Fri, 17 Nov 2023 20:31:19 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 708BD40285; Fri, 17 Nov 2023 20:31:19 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 5ACE84021E for ; Fri, 17 Nov 2023 20:31:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700249476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2zpUb/AEd36Doqnlv2nOJNxul0iivJJpANX5+xtONw8=; b=hYWJQFsJ7PybhCGIgKYPBuJFDSrKKhwvyZLaXxLxrYawPE4BZSkQjHccMqtb1czcrFzDkE zim461qM718t96vKuZZJloLyQ1AplLGlCbU10rK2UFOjvxr1/huAuvwr7m3smfVW1iOkyj /VVNA8EmNvg9+NL0HygGYClA3+Am4v0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-167-cjMIIbaXOmWhsxVDjVCi8Q-1; Fri, 17 Nov 2023 14:31:14 -0500 X-MC-Unique: cjMIIbaXOmWhsxVDjVCi8Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 343CF811000; Fri, 17 Nov 2023 19:31:14 +0000 (UTC) Received: from RHTPC1VM0NT (unknown [10.22.34.37]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D34FD40C6EB9; Fri, 17 Nov 2023 19:31:13 +0000 (UTC) From: Aaron Conole To: Michael Santana Cc: ci@dpdk.org, David Marchand , Thomas Monjalon , Patrick Robb , Dumitru Ceara Subject: Re: [RFC v2 3/3] github: add a tool for restarting checks References: <20231107203158.1261199-1-aconole@redhat.com> <20231107203158.1261199-4-aconole@redhat.com> Date: Fri, 17 Nov 2023 14:31:08 -0500 In-Reply-To: (Michael Santana's message of "Fri, 17 Nov 2023 11:47:50 -0500") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Michael Santana writes: > On Tue, Nov 7, 2023 at 3:32=E2=80=AFPM Aaron Conole = wrote: >> >> The recheck framework can track specific recheck labels and track a >> state for them. Add a tool that will restart github workflow runs >> and mark the git build table to check for a repoll. This will allow >> existing monitor tools to re-poll the workflow and report rerun >> results. >> >> Signed-off-by: Aaron Conole > Acked-by: Michael Santana Thanks. >> --- >> github_restart | 141 +++++++++++++++++++++++++++++++++++++++++++++++ >> series_db_lib.sh | 16 ++++++ >> 2 files changed, 157 insertions(+) >> create mode 100755 github_restart >> >> diff --git a/github_restart b/github_restart >> new file mode 100755 >> index 0000000..5ba9677 >> --- /dev/null >> +++ b/github_restart >> @@ -0,0 +1,141 @@ >> +#!/bin/bash >> +# SPDX-Identifier: gpl-2.0-or-later >> +# Copyright (C) 2023, Red Hat, Inc. >> +# >> +# Restarts a github job run. This can be used in conjunction with >> +# the recheck requests to provide the ability for a user to restart >> +# a test - in case the workflow is suspected of having a spurious run. >> +# >> +# Licensed under the terms of the GNU General Public License as publish= ed >> +# by the Free Software Foundation; either version 2 of the License, or >> +# (at your option) any later version. You may obtain a copy of the >> +# license at >> +# >> +# https://www.gnu.org/licenses/old-licenses/gpl-2.0.html >> +# >> +# Unless required by applicable law or agreed to in writing, software >> +# distributed under the License is distributed on an "AS IS" BASIS, WIT= HOUT >> +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See = the >> +# License for the specific language governing permissions and limitatio= ns >> +# under the License. >> + >> +wait_start=3D"no" >> + >> +while [ "$1" !=3D "" ]; do >> + if echo "$1" | grep -q -s -E ^--pw-project=3D ; then >> + pw_project=3D$(echo "$1" | sed s/^--pw-project=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--pw-instance=3D ; then >> + pw_instance=3D$(echo "$1" | sed s/^--pw-instance=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--pw-credentials=3D ; then >> + pw_credential=3D$(echo "$1" | sed s/^--pw-credentials=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--series-id=3D ; then >> + series_id=3D$(echo "$1" | sed s/^--series-id=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--wait-start ; then >> + wait_start=3D"yes" >> + if echo "$1" | grep -q -s -E ^--wait-start=3D ; then >> + wait_start=3D$(echo "$1" | sed s/^--wait-start=3D//) >> + fi >> + shift >> + elif echo "$1" | grep -q -s -E ^--github-token=3D ; then >> + github_token=3D$(echo "$1" | sed s/^--github-token=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--repository=3D ; then >> + reponame=3D$(echo "$1" | sed s/^--repository=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--run-id=3D ; then >> + runid=3D$(echo "$1" | sed s/^--repository=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--sha=3D ; then >> + sha=3D$(echo "$1" | sed s/^--sha=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--workflow=3D ; then >> + workflow=3D$(echo "$1" | sed s/^--workflow=3D//) >> + shift >> + elif echo "$1" | grep -q -s -E ^--help ; then >> + echo "github restarting script" >> + echo "$0: args" >> + echo " --pw-project=3D Project name" >> + echo " --pw-instance=3D URL for pw" >> + echo " --series-id=3Did Series ID for reference" >> + echo " --github-token=3Dtoken Token for github" >> + echo " --wait-start[=3Dyes|no] Wait for the remote to= start" >> + echo " --repository=3Drepo Repository name (ex fo= o/bar)" >> + echo " --run-id=3Did run ID" >> + echo " --workflow=3Dworkflow Workflow name" >> + echo " --sha=3Dcommit Commit sha" >> + echo "" >> + exit 0 >> + else >> + echo "Unknown option: '$1'" >&2 >> + echo "Rerun with --help for details" >&2 >> + exit 1 >> + fi >> +done > This entire block makes me feel like maybe we should switch to python :) Agreed. We are on shell scripts for the time being, but a good effort would be to re-do things in python and use better frameworks for everything (for example, passing data between scripts with json rather than the sqlite output strings directly). >> + >> +source $(dirname $0)/series_db_lib.sh >> + >> +if [ "X$wait_start" !=3D "Xno" -a "X$wait_start" !=3D "Xyes" ]; then >> + echo "Unrecognized '$wait_start' argument to --wait-start=3D" >&2 >> + echo "valid values are 'yes' or 'no'." >&2 >> + exit 1 >> +fi >> + >> +if [ "X$github_token" =3D=3D "X" ]; then >> + echo "Please set a github API token." >&2 >> + echo "Use --help for more info." >&2 >> + exit 1 >> +fi >> + >> +if [ "X$reponame" =3D=3D "X" ]; then >> + echo "Please set a repository (ie: '--repository=3Downer/repo')." >= &2 >> + echo "Use --help for more info." >&2 >> + exit 1 >> +fi >> + >> +AUTH=3D"Authorization: token ${github_token}" >> +APP=3D"Accept: application/vnd.github.v3+json" >> + >> +if [ "X$runid" =3D=3D "X" ]; then >> + >> + # lookup the runs based on the shasum >> + >> + if [ "X$sha" =3D=3D "X" ]; then >> + echo "Need a runid or shasum to key off." >&2 >> + echo "See --help for more details." >&2 >> + exit 1 >> + fi >> + >> + comma=3D"" >> + for job in $(curl -s -S -H "${AUTH}" -H "${APP}" \ >> + >> "https://api.github.com/repos/${reponame}/actions/runs?head_sha=3D${sha}= " >> | \ >> + jq -rc '.workflow_runs[].id') >> + do >> + runid=3D"${comma}${job}" >> + comma=3D"," >> + done >> +fi >> + >> +echo -n "{\"results\":[" >> +comma=3D"" >> +for job in $(echo "$runid" | sed 's/,/ /'); do >> + result=3D$(curl -s -X POST -L -S -H "${AUTH}" -H "${APP}" \ >> + "https://api.github.com/repos/${reponame}/actions/run= s/$job/rerun") > Okay! >> + msg=3D$(echo "$result" | jq -rc '.message') >> + >> + echo -n "$comma{\"run\":$job,\"result\":" >> + if [ "X$msg" =3D=3D "Xnull" ]; then >> + echo -n "\"sent\"" >> + if [ "X$series_id" !=3D "X" ]; then >> + echo -n ",\"gap_sync\":\"reset\"" >> + set_unsynced_for_series "$series_id" "$pw_instance" "gap_sy= nc" >> + fi >> + else >> + echo -n "\"err\",\"error\":\"$msg\"" >> + fi >> + echo -n "}" >> +done >> +echo "]}" >> diff --git a/series_db_lib.sh b/series_db_lib.sh >> index ca33c1f..3f052ad 100644 >> --- a/series_db_lib.sh >> +++ b/series_db_lib.sh >> @@ -382,6 +382,14 @@ function set_synced_for_series() { >> echo "update git_builds set gap_sync=3D1, obs_sync=3D1 where patchw= ork_instance=3D\"$instance\" and series_id=3D$series_id;" | series_db_execu= te >> } >> >> +function set_unsynced_for_series() { >> + local series_id=3D"$1" >> + local instance=3D"$2" >> + local ci_instance=3D"$3" >> + >> + echo "update git_builds set $ci_instance=3D0 where patchwork_instan= ce=3D\"$instance\" and series_id=3D$series_id;" | series_db_execute >> +} >> + >> function insert_commit() { >> local series_id=3D"$1" >> local patch_id=3D"$2" >> @@ -407,6 +415,14 @@ function get_patch_id_by_series_id_and_sha() { >> echo "select patch_id from git_builds where patchwork_instance=3D\"= $instance\" and series_id=3D$series_id and sha=3D\"$sha\";" | series_db_exe= cute >> } >> >> +function get_sha_for_series_id_and_patch() { > Is this function being used? Yes - but it should have appeared in 2/3 rather than here in 3/3. I'll fix it before applying things and I plan to send a PATCH series out with those changes as well. >> + local series_id=3D"$1" >> + local patch_id=3D"$2" >> + local instance=3D"$3" >> + >> + echo "select sha from git_builds where patchwork_instance=3D\"$inst= ance\" and series_id=3D\"$series_id\" and patch_id=3D\"$patch_id\"" | serie= s_db_execute >> +} >> + >> function get_recheck_requests_by_project() { >> local recheck_instance=3D"$1" >> local recheck_project=3D"$2" >> -- >> 2.41.0 >>