Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: improve things around auto-apply of fixes #36349

Closed
wants to merge 12 commits into from
Closed

ci: improve things around auto-apply of fixes #36349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
97bfb3a
Only apply fixes for PRs
tobiasdiez Sep 28, 2023
75324e2
Install git as well
tobiasdiez Sep 28, 2023
0e7c0a4
reorder commands
tobiasdiez Sep 28, 2023
43194f8
First setup git repo, then apply fixes
tobiasdiez Sep 28, 2023
01678e1
Merge branch 'develop' into fix_get_ci_fixes-1
tobiasdiez Oct 10, 2023
7b12774
run also on forks, and change docbuild, conda and linter accordingly
tobiasdiez Oct 13, 2023
b5ee735
add safe directory git config
tobiasdiez Oct 13, 2023
e61d557
manually install git
tobiasdiez Oct 13, 2023
c729118
add --allow-unrelated-histories
tobiasdiez Oct 13, 2023
66cf6e1
Merge branch 'develop' into fix_get_ci_fixes-1
tobiasdiez Oct 15, 2023
da5d53d
Merge branch 'develop' into fix_get_ci_fixes-1
tobiasdiez Oct 21, 2023
20fb5c3
Merge branch 'develop' into fix_get_ci_fixes-1
tobiasdiez Nov 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ci/merge-fixes.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ else
git tag -f test_head
$GH pr checkout -b pr-$a $a
git checkout -q test_head
if git merge --no-edit --squash -q pr-$a; then
if git merge --no-edit --squash --allow-unrelated-histories -q pr-$a; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if one has to use this flag, typically something's wrong already

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The checkout action results in a detached head for PRs. You can call that "wrong" but its by design.

echo "::endgroup::"
if git commit -q -m "Merge https://github.com/$REPO/pull/$a" -a --no-allow-empty; then
echo "Merged #$a"
Expand Down
63 changes: 23 additions & 40 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,43 +27,39 @@ concurrency:
cancel-in-progress: true

jobs:
get_ci_fixes:
build:
runs-on: ubuntu-latest
container: ghcr.io/sagemath/sage/sage-${{ github.event.inputs.platform || 'ubuntu-focal-standard' }}-with-targets:${{ github.event.inputs.docker_tag || 'dev'}}
steps:
- name: Install git (macos)
if: runner.os == 'macOS'
run: |
brew update
brew install git

- name: Install git (linux)
if: runner.os == 'Linux'
run: |
apt update
apt install -y git
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, one cannot assume that on Linux, apt is the package manger.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it's working, so I call it good enough...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the workflow is run manually, the container can be selected by the user. As you know.
"That it's working" is because you didn't test it on a system that does not use apt.


- name: Checkout
id: checkout
uses: actions/checkout@v4

- name: Install GH CLI
uses: dev-hanz-ops/[email protected]
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
with:
gh-cli-version: 2.32.0

- name: Merge CI fixes from sagemath/sage
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
run: |
git config --global --add safe.directory "$GITHUB_WORKSPACE"
.ci/merge-fixes.sh
env:
GH_TOKEN: ${{ github.token }}
- name: Store CI fixes in upstream artifact
run: |
mkdir -p upstream
if git format-patch --stdout test_base > ci_fixes.patch; then
cp ci_fixes.patch upstream/
fi
- uses: actions/upload-artifact@v3
with:
path: upstream
name: upstream

build:
runs-on: ubuntu-latest
container: ghcr.io/sagemath/sage/sage-${{ github.event.inputs.platform || 'ubuntu-focal-standard' }}-with-targets:${{ github.event.inputs.docker_tag || 'dev'}}
needs: [get_ci_fixes]
steps:
- name: Checkout
id: checkout
uses: actions/checkout@v4

- name: Update system packages
id: prepare
run: |
export PATH="build/bin:$PATH"
eval $(sage-print-system-package-command auto update)
eval $(sage-print-system-package-command auto --spkg --yes --no-install-recommends install git)

- name: Add prebuilt tree as a worktree
id: worktree
Expand All @@ -72,19 +68,6 @@ jobs:
git config --global --add safe.directory $(pwd)
.ci/retrofit-worktree.sh worktree-image /sage

- name: Download upstream artifact
uses: actions/download-artifact@v3
with:
path: upstream
name: upstream

- name: Apply CI fixes from sagemath/sage
# After applying the fixes, make sure all changes are marked as uncommitted changes.
run: |
if [ -r upstream/ci_fixes.patch ]; then
(cd worktree-image && git commit -q -m "current changes" --allow-empty -a && git am; git reset --quiet old; git add -N .) < upstream/ci_fixes.patch
fi

- name: Incremental build
id: incremental
run: |
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci-conda.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
- uses: actions/checkout@v4

- name: Merge CI fixes from sagemath/sage
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
run: |
.ci/merge-fixes.sh
env:
Expand Down
58 changes: 24 additions & 34 deletions .github/workflows/doc-build-pdf.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,43 +21,46 @@ concurrency:
cancel-in-progress: true

jobs:
get_ci_fixes:
build-docs-pdf:
runs-on: ubuntu-latest
container: ghcr.io/sagemath/sage/sage-${{ github.event.inputs.platform || 'ubuntu-focal-standard' }}-with-targets:${{ github.event.inputs.docker_tag || 'dev'}}
steps:
- name: Install git (macos)
if: runner.os == 'macOS'
tobiasdiez marked this conversation as resolved.
Show resolved Hide resolved
run: |
brew update
brew install git

- name: Install git (linux)
if: runner.os == 'Linux'
run: |
apt update
apt install -y git

- name: Checkout
id: checkout
uses: actions/checkout@v4

- name: Install GH CLI
uses: dev-hanz-ops/[email protected]
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
with:
gh-cli-version: 2.32.0

- name: Merge CI fixes from sagemath/sage
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
run: |
git config --global --add safe.directory "$GITHUB_WORKSPACE"
.ci/merge-fixes.sh
env:
GH_TOKEN: ${{ github.token }}
- name: Store CI fixes in upstream artifact
run: |
mkdir -p upstream
if git format-patch --stdout test_base > ci_fixes.patch; then
cp ci_fixes.patch upstream/
fi
- uses: actions/upload-artifact@v3
with:
path: upstream
name: upstream

build-docs-pdf:
runs-on: ubuntu-latest
container: ghcr.io/sagemath/sage/sage-${{ github.event.inputs.platform || 'ubuntu-focal-standard' }}-with-targets:${{ github.event.inputs.docker_tag || 'dev'}}
needs: [get_ci_fixes]
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Update system packages
run: |
export PATH="build/bin:$PATH"
eval $(sage-print-system-package-command auto update)
eval $(sage-print-system-package-command auto --yes --no-install-recommends install zip)
eval $(sage-print-system-package-command auto --spkg --yes --no-install-recommends install git texlive)

eval $(sage-print-system-package-command auto --spkg --yes --no-install-recommends install texlive)

- name: Add prebuilt tree as a worktree
id: worktree
Expand All @@ -67,19 +70,6 @@ jobs:
git config --global user.name "Build & Test workflow"
.ci/retrofit-worktree.sh worktree-image /sage

- name: Download upstream artifact
uses: actions/download-artifact@v3
with:
path: upstream
name: upstream

- name: Apply CI fixes from sagemath/sage
# After applying the fixes, make sure all changes are marked as uncommitted changes.
run: |
if [ -r upstream/ci_fixes.patch ]; then
(cd worktree-image && git commit -q -m "current changes" --allow-empty -a && git am; git reset --quiet old; git add -N .) < upstream/ci_fixes.patch
fi

- name: Incremental build
id: incremental
run: |
Expand Down
52 changes: 15 additions & 37 deletions .github/workflows/doc-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,39 +16,30 @@ concurrency:
cancel-in-progress: true

jobs:
get_ci_fixes:
build-docs:
runs-on: ubuntu-latest
container: ghcr.io/sagemath/sage/sage-ubuntu-focal-standard-with-targets:dev
steps:
- name: Update system packages
run: |
apt-get update && apt-get install -y git zip

- name: Checkout
id: checkout
uses: actions/checkout@v4

- name: Install GH CLI
uses: dev-hanz-ops/[email protected]
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
with:
gh-cli-version: 2.32.0

- name: Merge CI fixes from sagemath/sage
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
run: |
git config --global --add safe.directory "$GITHUB_WORKSPACE"
.ci/merge-fixes.sh
env:
GH_TOKEN: ${{ github.token }}
- name: Store CI fixes in upstream artifact
run: |
mkdir -p upstream
if git format-patch --stdout test_base > ci_fixes.patch; then
cp ci_fixes.patch upstream/
fi
- uses: actions/upload-artifact@v3
with:
path: upstream
name: upstream

build-docs:
runs-on: ubuntu-latest
container: ghcr.io/sagemath/sage/sage-ubuntu-focal-standard-with-targets:dev
needs: [get_ci_fixes]
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Update system packages
run: |
apt-get update && apt-get install -y git zip

- name: Add prebuilt tree as a worktree
id: worktree
Expand All @@ -74,19 +65,6 @@ jobs:
(echo ".buildinfo"; echo '*.inv'; echo '.git*'; echo '*.svg'; echo '*.pdf'; echo '*.png'; echo 'searchindex.js') > .gitignore; \
git add -A && git commit --quiet -m "old")

- name: Download upstream artifact
uses: actions/download-artifact@v3
with:
path: upstream
name: upstream

- name: Apply CI fixes from sagemath/sage
# After applying the fixes, make sure all changes are marked as uncommitted changes.
run: |
if [ -r upstream/ci_fixes.patch ]; then
(cd worktree-image && git commit -q -m "current changes" --allow-empty -a && git am; git reset --quiet old; git add -N .) < upstream/ci_fixes.patch
fi

- name: Incremental build
id: incremental
run: |
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
uses: actions/checkout@v4

- name: Merge CI fixes from sagemath/sage
if: ${{ github.event_name == 'pull_request' || github.event.repository.fork }}
run: |
.ci/merge-fixes.sh
env:
Expand Down
Loading