fix: use correct path to toolcache #1494

KnisterPeter · 2022-12-07T13:30:30Z

The toolcache on GitHub Actions need to be in
/opt/hostedtoolcache. This is the case for all
environment variables set by act, but it's not the case for the volume mounted into the container.

The toolcache on GitHub Actions need to be in /opt/hostedtoolcache. This is the case for all environment variables set by act, but it's not the case for the volume mounted into the container.

github-actions · 2022-12-07T13:32:54Z

🦙 MegaLinter status: ✅ SUCCESS

Descriptor	Linter	Files	Errors	Elapsed time
✅ EDITORCONFIG	editorconfig-checker	1	0	0.02s
✅ REPOSITORY	gitleaks	yes	no	2.27s
✅ REPOSITORY	git_diff	yes	no	0.17s
✅ REPOSITORY	grype	yes	no	7.48s
✅ REPOSITORY	secretlint	yes	no	1.07s
✅ REPOSITORY	trivy-sbom	yes	no	0.6s
✅ REPOSITORY	trufflehog	yes	no	14.89s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by

mergify · 2022-12-07T13:42:38Z

@KnisterPeter this pull request has failed checks 🛠

catthehacker · 2022-12-07T15:46:38Z

It fails because you can't have a mount over data and in my images I keep node in /opt/hostedtoolcache

KnisterPeter · 2022-12-20T14:48:51Z

We have /opt/hostedtoolcache hardcoded in a few places in the act codebase. Not being able to mount a volume to that means that the toolcache is unusable and broken.
Tools will be reinstalled all the time.

@nektos/act-maintainers Would it be okay for your to remove the hardcoded paths and replace with with a configurable option (while still defaulting to the hardcoded value).

ChristopherHX · 2022-12-20T17:00:50Z

The failing test will be altered in #1503 to use a different docker image. This change should be sufficient for this change to pass all tests.

Would it be okay for your to remove the hardcoded paths and replace with with a configurable option (while still defaulting to the hardcoded value).

This option would need a --container- prefix, because it should only apply to the docker environment. I'm not shure how such an option would solve a problem, this PR wouldn't solve.

KnisterPeter · 2022-12-20T17:05:12Z

Cool. I think this PR is obsolete then and I can close it.

ChristopherHX · 2022-12-20T17:07:56Z

I think this PR is obsolete then and I can close it.

Not shure what make you believe that this PR is obsolete, my mentioned change only resolve the not passing tests not anything else

A workaround would be use
--container-options "-v toolcache:/opt/hostedtoolcache" to bypass the testsuite.

KnisterPeter · 2022-12-20T17:10:54Z

I thought your changes will resolve the more general toolcache issue.
I'll keep this PR open then until #1503 is merged and see if I can continue on this.

github-actions · 2023-01-20T00:11:07Z

PR is stale and will be closed in 14 days unless there is new activity

codecov · 2023-04-17T12:34:13Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5a80a04) 61.56% compared to head (f029fdc) 61.53%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1494      +/-   ##
==========================================
- Coverage   61.56%   61.53%   -0.04%     
==========================================
  Files          53       53              
  Lines        9002     9002              
==========================================
- Hits         5542     5539       -3     
- Misses       3020     3022       +2     
- Partials      440      441       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ChristopherHX · 2023-04-17T12:43:05Z

While the tests are now passing. We would still have to move the tools of the default docker images out of the mount point.

Otherwise these docker images end up with an empty tool_cache and the preinstalled node etc. are away.

KnisterPeter · 2023-04-28T05:55:34Z

@ZauberNerd I think we can close that, can't we?

github-actions · 2023-10-26T00:09:24Z

PR is stale and will be closed in 14 days unless there is new activity

lawndoc · 2024-01-29T20:36:24Z

While the tests are now passing. We would still have to move the tools of the default docker images out of the mount point.

Otherwise these docker images end up with an empty tool_cache and the preinstalled node etc. are away.

Why do you need the mount point to be empty? Doesn't docker initialize new named volumes using the contents of the image? Or am I misunderstanding the issue?

ChristopherHX · 2024-01-29T21:43:10Z

@lawndoc Thanks to remind me about this, yes I should update the default image and get this merged.

Doesn't docker initialize new named volumes using the contents of the image? Or am I misunderstanding the issue?

the toolcache volume is long term persisting data. This volume already exists for all used act atleast once via docker backend without the data on that location of the image.

We have a default image with /opt/hostedtoolcache/.../node/bin in PATH, so the most likly already existing volume is empty then you see error node not in PATH.

Even setup-node needs node in PATH or the workflow fails.

The solution would be to not store any required files inside that location.

Using the small images of the image survey will also lead to an empty toolcache folder

ChristopherHX · 2024-01-29T21:55:14Z

The large images also have files on these location, but not the critical node binary so that is fine.

ChristopherHX · 2024-01-29T22:13:52Z

This PR should make the location free of files catthehacker/docker_images#121

I believe the toolcache mounting side effects are currently not that small without changing the image. actions/runner uses /__t for some reasons, but the official setup-python action doesn't work using that path.

@lawndoc If you have a better idea let me know.

I don't feel comfortable to have luck with getting the correct content in /opt/hostedtoolcache for builtin tools like node.

My experience with named volumes in docker was, that they are always empty unless the Dockerfile uses the VOLUME keyword. (Could be a wrong knownledge base from my side)

ChristopherHX · 2024-01-30T21:40:30Z

@lawndoc You are completely right the toolcache is initialized with the content of the container dir if it didn't exist yet...

However if it was initialized non empty (even a empty volume is initialized by docker via a copy from image) it throws node not found error, if the wrong node tool is present in the toolcache volume.

I'm merging this now, default image is patched and confirmed it reduces the problems caused by divergent tool-cache.

You as a potential gitea actions user, might need to wait longer for the next sync.

mergify · 2024-01-30T21:55:39Z

KnisterPeter this pull request has failed checks 🛠

I want to delete these messages...

ChristopherHX · 2024-01-30T22:32:08Z

Wow good job docker:

2024-01-30T22:06:17.5231148Z     runner_test.go:203: 
2024-01-30T22:06:17.5231557Z         	Error Trace:	/home/runner/work/act/act/pkg/runner/runner_test.go:203
2024-01-30T22:06:17.5232114Z         	            				/home/runner/work/act/act/pkg/runner/runner_test.go:323
2024-01-30T22:06:17.5234123Z         	Error:      	Expected nil, but got: &fmt.wrapError{msg:"failed to create container: 'Error response from daemon: failed to mkdir /var/lib/docker/volumes/act-toolcache/_data/node: mkdir /var/lib/docker/volumes/act-toolcache/_data/node: file exists'", err:errdefs.errSystem{error:(*errors.withStack)(0xc0003fa828)}}
2024-01-30T22:06:17.5234335Z         	Test:       	TestRunEvent/shells/pwsh
2024-01-30T22:06:17.5234722Z         	Messages:   	/home/runner/work/act/act/pkg/runner/testdata/shells/pwsh

looks like the images with data in /opt/hostedtoolcache are affected by concurrency bugs.

Good to make this folder empty with only file permission applied to the volume

lawndoc · 2024-02-01T14:12:12Z

Thank you for your work on this! It's greatly appreciated!

The toolcache on GitHub Actions need to be in /opt/hostedtoolcache. This is the case for all environment variables set by act, but it's not the case for the volume mounted into the container. Co-authored-by: Björn Brauer <[email protected]> Co-authored-by: ChristopherHX <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

fix: use correct path to toolcache

eaf4187

The toolcache on GitHub Actions need to be in /opt/hostedtoolcache. This is the case for all environment variables set by act, but it's not the case for the volume mounted into the container.

KnisterPeter requested a review from a team as a code owner December 7, 2022 13:30

pull-request-size bot added the size/XS label Dec 7, 2022

KnisterPeter self-assigned this Dec 7, 2022

mergify bot added the needs-work Extra attention is needed label Dec 7, 2022

KnisterPeter marked this pull request as draft December 7, 2022 14:56

KnisterPeter closed this Dec 20, 2022

KnisterPeter deleted the fix-toolcache branch December 20, 2022 17:05

KnisterPeter restored the fix-toolcache branch December 20, 2022 17:08

KnisterPeter reopened this Dec 20, 2022

github-actions bot added the stale label Jan 20, 2023

github-actions bot closed this Feb 4, 2023

catthehacker reopened this Feb 4, 2023

catthehacker removed the stale label Feb 4, 2023

Merge branch 'master' into fix-toolcache

f57c26f

mergify bot removed the needs-work Extra attention is needed label Apr 17, 2023

github-actions bot added the stale label Oct 26, 2023

github-actions bot closed this Nov 9, 2023

lawndoc mentioned this pull request Jan 27, 2024

Tool cache is not in effect #1987

Closed

ChristopherHX reopened this Jan 29, 2024

Merge branch 'master' into fix-toolcache

23932e3

ChristopherHX removed the stale label Jan 29, 2024

ChristopherHX mentioned this pull request Jan 30, 2024

fix: EACCESS error regarding /opt/hostedtoolcache #2179

Closed

ChristopherHX marked this pull request as ready for review January 30, 2024 21:40

ChristopherHX approved these changes Jan 30, 2024

View reviewed changes

Merge branch 'master' into fix-toolcache

f029fdc

mergify bot added the needs-work Extra attention is needed label Jan 30, 2024

nektos deleted a comment from mergify bot Jan 30, 2024

mergify bot removed the needs-work Extra attention is needed label Jan 30, 2024

mergify bot merged commit 054caec into master Jan 30, 2024
10 checks passed

mergify bot deleted the fix-toolcache branch January 30, 2024 22:43

AndesKrrrrrrrrrrr mentioned this pull request Feb 15, 2024

chore: Socket logic consolidation #2180

Closed

lng2020 mentioned this pull request Sep 18, 2024

Env RUNNER_TOOL_CACHE not passed to runner go-gitea/gitea#32055

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: use correct path to toolcache #1494

fix: use correct path to toolcache #1494

KnisterPeter commented Dec 7, 2022

github-actions bot commented Dec 7, 2022 •

edited

Loading

mergify bot commented Dec 7, 2022

catthehacker commented Dec 7, 2022

KnisterPeter commented Dec 20, 2022

ChristopherHX commented Dec 20, 2022 •

edited

Loading

KnisterPeter commented Dec 20, 2022

ChristopherHX commented Dec 20, 2022

KnisterPeter commented Dec 20, 2022

github-actions bot commented Jan 20, 2023

codecov bot commented Apr 17, 2023 •

edited

Loading

ChristopherHX commented Apr 17, 2023

KnisterPeter commented Apr 28, 2023

github-actions bot commented Oct 26, 2023

lawndoc commented Jan 29, 2024 •

edited

Loading

ChristopherHX commented Jan 29, 2024 •

edited

Loading

ChristopherHX commented Jan 29, 2024

ChristopherHX commented Jan 29, 2024

ChristopherHX commented Jan 30, 2024

mergify bot commented Jan 30, 2024 •

edited by ChristopherHX

Loading

ChristopherHX commented Jan 30, 2024

lawndoc commented Feb 1, 2024

fix: use correct path to toolcache #1494

fix: use correct path to toolcache #1494

Conversation

KnisterPeter commented Dec 7, 2022

github-actions bot commented Dec 7, 2022 • edited Loading

🦙 MegaLinter status: ✅ SUCCESS

mergify bot commented Dec 7, 2022

catthehacker commented Dec 7, 2022

KnisterPeter commented Dec 20, 2022

ChristopherHX commented Dec 20, 2022 • edited Loading

KnisterPeter commented Dec 20, 2022

ChristopherHX commented Dec 20, 2022

KnisterPeter commented Dec 20, 2022

github-actions bot commented Jan 20, 2023

codecov bot commented Apr 17, 2023 • edited Loading

Codecov Report

ChristopherHX commented Apr 17, 2023

KnisterPeter commented Apr 28, 2023

github-actions bot commented Oct 26, 2023

lawndoc commented Jan 29, 2024 • edited Loading

ChristopherHX commented Jan 29, 2024 • edited Loading

ChristopherHX commented Jan 29, 2024

ChristopherHX commented Jan 29, 2024

ChristopherHX commented Jan 30, 2024

mergify bot commented Jan 30, 2024 • edited by ChristopherHX Loading

ChristopherHX commented Jan 30, 2024

lawndoc commented Feb 1, 2024

github-actions bot commented Dec 7, 2022 •

edited

Loading

ChristopherHX commented Dec 20, 2022 •

edited

Loading

codecov bot commented Apr 17, 2023 •

edited

Loading

lawndoc commented Jan 29, 2024 •

edited

Loading

ChristopherHX commented Jan 29, 2024 •

edited

Loading

mergify bot commented Jan 30, 2024 •

edited by ChristopherHX

Loading