Refactoring of the CI. #3676
Merged
Refactoring of the CI. #3676
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The previous CI was strangely relying on workflows to split the CI and have it run in parallel. A lot of the work was made several times: e.g. installing rust stable + nightly, installing packages, starting our builder image etc. Overall it was done x6 times. It was strongly overkill considering some of the steps run in (<1s). After this refactoring we run in two "jobs" with a different set of steps. - Lint runs clippy + all of the smaller jobs (cargo fmt, cargo deny, checking the license) - Test runs the unit test. In addition, we stop relying on the builder image and the associated confusion. For instance, the rust stable installed in the CI workflow was actually not used. The quickwit-builder image one was taken because of the rust-toolchain.toml. After the PR, we run on stock ubuntu-latest directly. We then rely on rustup to install the right version of cargo/rust as define in the rust-toolchain. Updating the rust version does not require any extra changes. cargo deny / cargo nextest need to be build unfortunately. That step is however cached.
|
After the PR is merged, we need to edit the settings of the main branch. |
fulmicoton
commented
Jul 24, 2023
| if: matrix.task.name == 'cargo nextest' | ||
| uses: taiki-e/install-action@nextest | ||
| - name: ${{ matrix.task.name }} | ||
| if: steps.modified-files.outputs.qw-src == 'true' && steps.modified-files.outputs.ui-src == 'false' |
There was a problem hiding this comment.
that was broken.
guilload
approved these changes
Jul 24, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The previous CI was strangely relying on workflows to split the CI and have it run in parallel.
A lot of the work was made several times:
e.g. installing rust stable + nightly, installing packages, starting our builder image etc.
Overall it was done x6 times. It was strongly overkill considering some of the steps run in (<1s).
After this refactoring we run in two "jobs" with a different set of steps.
In addition, we stop relying on the builder image and the associated confusion. For instance, the rust stable installed in the CI workflow was actually not used. The quickwit-builder image one was taken because of the rust-toolchain.toml.
After the PR, we run on stock ubuntu-latest directly. We then rely on rustup to install the right version of cargo/rust as define in the rust-toolchain.
Updating the rust version does not require any extra changes.
cargo deny / cargo nextest need to be build unfortunately. That step is however cached.