Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test build pr outside container #487

Merged
merged 2 commits into from
Mar 25, 2024
Merged

Conversation

Wauplin
Copy link
Contributor

@Wauplin Wauplin commented Mar 25, 2024

This PR adds a new options custom_container to build PR in the CI in a custom container. By default, workflow runs inside "huggingface/transformers-doc-builder" which is the container needed for transformers and that has all dependencies like torch, tensorflow, etc. However, it takes quite some time (>2min) to initialize the container in the CI given its size. In theory, most libraries don't need this container so we should be able to bypass this step and run directly on ubuntu:latest.

This PR is related to huggingface/huggingface_hub#2140 in which I tested the idea.


As discussed on slack (internal), plan is:

  1. we merge Test build pr outside container #487 in its current state
  2. we test to disable custom_container (i.e. custom_container: "") on several repos, including huggingface_hub (already done), datasets, diffusers, hub-docs, etc.
  3. for the repos that needs the custom container, we set custom_container: "huggingface/transformers-doc-builder" in their CI. This should not change anything since it is the default value for now.
  4. Once we fixed the repos in which we think transformers image is needed, we make a new PR on doc-builder to change the default value of custom_container to ""

This way we should be able to make the transition between "all of them use transformers image" to "most of them use a bare ubuntu" without a breaking change in the process.

@mishig25
Copy link
Contributor

lgtm, why it is still in draft mode ?

@Wauplin Wauplin marked this pull request as ready for review March 25, 2024 16:45
@Wauplin Wauplin requested a review from mishig25 March 25, 2024 16:45
@Wauplin
Copy link
Contributor Author

Wauplin commented Mar 25, 2024

lgtm, why it is still in draft mode ?

Lol, you're too fast. Just wanted to update description first 😄 Should be ready now :)

Copy link
Contributor

@mishig25 mishig25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! please feel free to merge whenever

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants