-
-
Notifications
You must be signed in to change notification settings - Fork 454
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH Actions: Create daily integration branch #33222
Comments
comment:1
Volker, would you be willing to share your existing scripts that probably already implement something like step 2 above? It does not have to be pretty. |
comment:2
Scripts are part of https://github.com/sagemath/git-trac-command:
|
comment:3
Thanks for the pointer |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Dependencies: #30933 |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Commit: |
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
|
This comment has been minimized.
This comment has been minimized.
Author: Matthias Koeppe |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
comment:69
Replying to @tobiasdiez:
The build & test can take long. If it is prebuilt on GH Actions and I can download it for inspection, that's a benefit.
I don't like |
comment:70
Replying to @tobiasdiez:
Yes, but they don't help much. The login action could replace these 5 lines:
And the other one is not useful |
comment:71
Replying to @mkoeppe:
Okay, makes sense. Thanks for the explanation. |
comment:72
Replying to @mkoeppe:
But the push happens before the final "Build & test integration head", so it neither contains the test results nor the build, or I'm missing something? |
comment:73
Yes, that's true as it is in the current branch - but that final step can be extended to also build; just not done yet |
comment:74
Replying to @mkoeppe:
Yes, the |
comment:75
Because of the lines that follow |
comment:76
I guess it's getting late here for me, but your messages get more and more cryptic. There are a lot of ifs here... |
comment:77
Sorry, I meant the last |
comment:78
Replying to @mkoeppe:
wow, I never knew about It should be in our developer manual. |
comment:79
Replying to @dimpase:
Yes, that's the one (should have directly provided a link, sry). It's quite powerful and usually works really well. Don't know how Matthias can work on the github workflows without quickly checking locally if everything goes well. |
comment:80
Replying to @mkoeppe:
Even after sleep I don't understand your comment. The second if doesn't do anything in my opinion since the Github token is always there and we need to login to ghcr as you said earlier. |
comment:81
Replying to @mkoeppe:
Thinking about this again: would it be an idea to run the integration workflow as follows:
Not sure how long the last step takes, but step 4 should finish in 30-60 mins (depending on how much packages are touched in the to-be-merged tickets) and step 5 about 1h. So this should all fit very nicely in one workflow. So the main advantage would be that you don't have to rebuild all packages from scratch during the integration workflow. This is under the assumption that the packages needed for the pdf build of the documentation don't influence the tests, because otherwise the test-baseline could be wrong (but is probably still good enough). |
comment:82
Just FYI, I'm planning on adding a CI step that runs for each ticket at #33294. Sufficiently different from this ticket that both have merits, just a heads up. |
comment:83
Replying to @tobiasdiez:
With a lot of patience! |
comment:84
Replying to @tobiasdiez:
these are system packages, not sage packages |
comment:86
Mainly to better understand the process behind the scenes, why is an additional "integration" branch necessary? Can one not simply push to "develop" (in case there are no errors)? I guess this comes down to the question "what does the release manager has to do in the background that demands a manual merge into the develop branch". |
I'm guessing this is not needed |
We create a GH Actions workflow, run daily (intended for running on
sagetrac-mirror
) as a cron job (https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule).
It can also be triggered manually via
workflow_dispatch
.Example run at
https://github.com/mkoeppe/sage/runs/5060070979
Merge mergeable Trac tickets from the current milestone, ordered by priority/deps, using
git releasemgr merge-all --milestone=current
Run using tox/docker on a stable platform (
ubuntu-focal-recommended
). (The new package factorrecommended
is likestandard
, but with texlive etc. added, as needed fordoc-pdf
). We setSAGE_DOCTEST_RANDOM_SEED=0
to reduce random failures.Only testsuite failures that do not already fail in the baseline are considered failures.
Docker images are pushed to
ghcr.io
, see https://github.com/mkoeppe?tab=packages&q=recommendedRelated issues and PRs:
Refinements:
Provide it with Trac credentials so it can set tickets back to
needs_work
on merge failures (using git releasemgr merge-all: Put tickets with merge conflict to needs_work WITH useful info git-trac-command#57)--> Trac account requested
Run
git log --first-parent
on files with merge conflictProvide it with GitHub credentials so it can push a branch
$RELEASE_TAG+integration-$DATE
, if it passes build + doctests + doc-pdf.Perhaps
git bisect --first-parent
for passing; set first failing ticket back toneeds_work
Another workflow for integration testing of all tickets in the current milestone with
--patchbot-status=Spkg
and--status=needs_review positive_review
with the full portability CI.Depends on #33233
Depends on #31529
Depends on #33103
Depends on #33277
CC: @vbraun @dimpase
Component: distribution
Author: Matthias Koeppe
Branch/Commit: u/mkoeppe/gh_actions__create_daily_integration_branch @
37c76a6
Issue created by migration from https://trac.sagemath.org/ticket/33222
The text was updated successfully, but these errors were encountered: