-
-
Notifications
You must be signed in to change notification settings - Fork 21.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI: Update mymindstorm/setup-emsdk
to v14, should fix cache folder conflicts
#87575
Conversation
mymindstorm/setup-emsdk
to v14, should fix cache folder conflicts
Oh, that's awesome! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this doesn't help. They simply added the workflow name as a prefix to the old naming scheme. But our CI has one entry point workflow. So both builds still have the same name for the cache:
🔗 GHA-3.1.39-linux-x64
The second-to-run build still throws an error, but for some reason it's just not caught as a problem anymore:
Failed to save: Unable to reserve cache with key 🔗 GHA-3.1.39-linux-x64, another job may be creating this cache. More details: Cache already exists. Scope: refs/pull/87575/merge, Key: 🔗 GHA-3.1.39-linux-x64, Version: bb287b8d239dae54b2b69d026bd1284e06aff2037346bb5491e7f80dd3ba6b57
So it seems we still need to do something manually to resolve this. Seems like they've added cache-key
to the configuration options. We should populate it with something fitting. See here.
8aaac86
to
04586fe
Compare
.github/workflows/web_builds.yml
Outdated
with: | ||
version: ${{env.EM_VERSION}} | ||
actions-cache-folder: ${{env.EM_CACHE_FOLDER}} | ||
cache-key: ${{ matrix.cache-name }}-emsdk |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops, I forgot this variable only contains a prefix, and we actually add more to it when creating the actual cache name. So this works, but it's still not unique enough for multiple runs:
This should match our main cache naming scheme.
cache-key: ${{ matrix.cache-name }}-emsdk | |
cache-key: ${{ matrix.cache-name }}-${{env.GODOT_BASE_BRANCH}}-${{github.ref}}-${{github.sha}}-emsdk |
Sorry!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, changed.
Overall, I wonder why caching is so convoluted. What we'd want typically is to just cache based on the Emscripten version, and that's it. But somehow it keeps trying to recreate the cache each time it's using it, instead of just using the version that's already there?
IMO, if the cache worked as I think it should, we'd download Emscripten 3.1.39 once, and never again. All builds on all Godot branches would pull from the same cache and never need to write to it concurrently, unless we happen to bump the Emscripten version to all branches at once.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm should the -emsdk
part come earlier in the cache key? IIRC substrings are removed from right to left to find the closest cache hit. It should maybe be emsdk-${{ matrix.cache-name }}-${{env.GODOT_BASE_BRANCH}}-${{github.ref}}-${{github.sha}}
instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point. I guess if it's added at the end, we can accidentally download it when trying to find cache for scons itself.
It's not that substrings are removed, we do it manually with restore-keys
(and setup-emsdk
doesn't, AFAICT). But we do indeed risk a collision because the first part is the same. So yeah, should probably add it somewhere earlier, or at the very start. Good catch!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, I wonder why caching is so convoluted. What we'd want typically is to just cache based on the Emscripten version, and that's it. But somehow it keeps trying to recreate the cache each time it's using it, instead of just using the version that's already there?
IMO, if the cache worked as I think it should, we'd download Emscripten 3.1.39 once, and never again. All builds on all Godot branches would pull from the same cache and never need to write to it concurrently, unless we happen to bump the Emscripten version to all branches at once.
Yes, something feels wrong about it, but I'll need to look closer into it. I also noticed that before trying to restore the cache it looks for a pre-existing folder with it, as a manual step. This suggests we may be able to cache it manually however we want. But if the override attempt happens in actions/cache
, then I'm not sure how to counteract that.
04586fe
to
97110e9
Compare
…nflicts https://github.com/mymindstorm/setup-emsdk/releases/tag/v14 Co-authored-by: Yuri Sizov <[email protected]>
97110e9
to
35ef0b3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Thanks! |
Cherry-picked for 3.6. |
Cherry-picked for 4.2.2. |
https://github.com/mymindstorm/setup-emsdk/releases/tag/v14