Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial backports automation #21

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Initial backports automation #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
87f5634
Initial backports automation
squidadm Mar 14, 2023
f86264d
Specify repository for PR updates
squidadm Nov 2, 2023
de73471
Merge branch 'main' into add-backports
yadij Nov 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions maintenance/code-backports.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#!/bin/sh

# silence debug output from VCS
beQuiet="--quiet"
#beQuiet=""
verpath="$1"
version=`basename "$verpath" | sed s/squid-//`
vnext=$(( $version + 1 ))
srcbranch=v$vnext

cd $verpath

# Dev branch does not receive backports
test -f .BASE && exit 0

# clean the current workspace
gitCleanWorkspace ()
{
git clean $beQuiet -xdf --exclude="\.BASE"
git checkout $beQuiet -- .
}

if ! test -d .git ; then
echo "ERROR: missing git repository"
exit 1;
fi
if ! test -d ../squid-$vnext ; then
echo "ERROR: missing git repository to backport from"
exit 1;
fi
if test -e ../squid-$vnext/.BASE ; then
srcbranch=`cat ../squid-$vnext/.BASE`
fi
Comment on lines +13 to +33
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this convoluted .BASE hack and related manipulation, I suggest making srcbranch a script parameter with master (or the main official repository branch name if you want to compute that to avoid hard-coding master) as its default value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do you propose that script parameter comes from? The script running this script only has the "/path/to/repo" and "vN" branch name available. The requirement to use "master" for development prohibits using git branch to determine if there is a newer vN branch existing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exact answer depends on how this script is used in its environment. If I were to design this, and I wanted the script to run unattended by default, then

  • If I were looking for a quick 80% solution, then I would probably hard-code (in the script where this new script is started from) the names of the first and the last vN branch. There are just a few of them, and the list does not change often... For example, today, those hard-coded names would probably be v5 and v6 since it is very unlikely that we will automatically backport something from v4 and earlier branches. The script will then iterate all versions between those two hard-coded points plus master.
  • If I were looking for a comprehensive solution, then I would probably compute the list of all vN branches (plus the primary branch) from the list of all official branches in GitHub squid-cache/squid repository.

However, I am not implying that something like the above is the best solution, or even that this PR should become hostage to changing the environment to make something like that possible. I am just answering your question, with the rather limited information that I have about the existing environment and, more importantly, about environmental changes you are willing to adopt. Ideally, the whole environment should be designed with certain maintenance automation goals/principles in mind, and I doubt we share those today.


abortAndExit ()
{
git $1 --abort 2>&1 >/dev/null
exit 1
}

abortAndContinue ()
{
git $1 --abort 2>&1 >/dev/null || true
}

# Prepare backports branch for updates
git checkout $beQuiet v$version-next-backports 2>/dev/null &&
gitCleanWorkspace &&
git fetch $beQuiet --all &&
git pull $beQuiet origin v$version-next-backports &&
git rebase origin/v$version 2>/dev/null &&
git push -u origin +v$version-next-backports || abortAndExit rebase

# Find backports to attempt:
prlist=`gh pr list -L 1 --repo squid-cache/squid --state closed --label backport-to-v$version | wc -l`
if test "$prlist" -ne 0; then
gh pr list --repo squid-cache/squid --state closed --label backport-to-v$version | while read prnum text; do
# find a commit in $srcbranch with " (#$prnum)" subject suffix
git log --oneline github/$srcbranch --grep=" (#$prnum)\$" | while read sha rest; do
msg=`git cherry-pick $beQuiet $sha 2>&1 || abortAndContinue cherry-pick`
if ! test -z "$msg" ; then
echo "$msg" | grep -E "^error:"
unlabel=`echo "$msg" | grep -E "The.previous.cherry-pick.is.now.empty"`
if ! test -z "$unlabel" ; then
gh pr edit --repo squid-cache/squid $prnum --remove-label backport-to-v$version
gh pr comment --repo squid-cache/squid $prnum --body "queued for backport to v$version"
fi
fi
gitCleanWorkspace
done
done
git push $beQuiet 2>&1 >/dev/null || true

# Create a PR to merge (if needed)
if test `git diff github/$srcbranch v$version-next-backports 2>/dev/null | wc -l` -ne 0 ; then
git push $beQuiet -u origin +v$version-next-backports >/dev/null || exit 1
# skip if there is an existing PR already open awaiting merge
prlist=`gh pr list -L 1 --repo squid-cache/squid --head v$version-next-backports | wc -l`
if test "$prlist" -eq 0 ; then
gh pr create --repo squid-cache/squid --base v$version --title "v$version Next Backports" --body "" || true
fi
fi

fi
34 changes: 20 additions & 14 deletions maintenance/code-maintenance.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ gitCleanWorkspace ()
git checkout $beQuiet -- .
}

abortAndExit ()
{
git "$1" --abort 2>&1 >/dev/null
exit 1
}

runMaintenanceScript ()
{
script=$1
Expand All @@ -29,18 +35,18 @@ runMaintenanceScript ()

if test -x "$script" ; then

git checkout $beQuiet $forkPoint 2>/dev/null &&
git checkout $beQuiet $branch 2>/dev/null &&
git checkout $beQuiet $forkPoint 2>&1 >/dev/null &&
git checkout $beQuiet $branch 2>&1 >/dev/null &&
gitCleanWorkspace &&
git rebase github/$forkPoint 2>/dev/null || ( git rebase --abort 2>/dev/null ; exit 1 )
git rebase github/$forkPoint 2>&1 >/dev/null || abortAndExit rebase

gitCleanWorkspace &&
$script >/dev/null || exit 1

# only update modified files. Ignore deleted, added, etc.
git status 2>&1 | grep "modified:" | while read a b; do git add $b; done
git commit $beQuiet --all -m "$prtitle" || true
git push $beQuiet -f --set-upstream origin +$branch || exit 1
git status 2>&1 | grep "modified:" | while read a b; do git add $beQuiet $b >/dev/null; done
git commit $beQuiet --all -m "$prtitle" >/dev/null || true
git push $beQuiet -f --set-upstream origin +$branch >/dev/null || exit 1
gitCleanWorkspace

if test `git diff $beQuiet $forkPoint $branch 2>/dev/null | wc -l` -ne 0 ; then
Expand All @@ -56,7 +62,6 @@ runMaintenanceScript ()
fi
}


(
cd $1
version=`basename "$1" | sed s/squid-//`
Expand All @@ -71,20 +76,21 @@ runMaintenanceScript ()
exit 1;
fi

git fetch $beQuiet --all
git fetch $beQuiet --all >/dev/null
gitCleanWorkspace

git pull $beQuiet --all
# Update version branch to match github
git checkout $beQuiet $forkPoint >/dev/null

git pull $beQuiet --all >/dev/null
gitCleanWorkspace

# Update version branch to match github
git checkout $beQuiet $forkPoint
# ON CONFLICTS: abort
git rebase github/$forkPoint || ( git rebase --abort ; exit 1 )
git rebase github/$forkPoint >/dev/null || abortAndExit rebase
gitCleanWorkspace

git push $beQuiet -f --set-upstream origin +$forkPoint 2>&1
git push $beQuiet --tags github
git push $beQuiet -f --set-upstream origin +$forkPoint 2>&1 >/dev/null
git push $beQuiet --tags github 2>&1 >/dev/null
gitCleanWorkspace

runMaintenanceScript ./bootstrap.sh v$version-bootstrap "Bootstrapped"
Expand Down