Pixi location change breaks self-hosted runner #66

humphd · 2024-01-17T19:23:07Z

Today I updated our main branch to use pixi v0.12.0 and setup-pixi to v0.4.1. After this, I continued work on another branch, which was behind main, and my CI jobs fail as follows:

Prepare all required actions
Getting action download info
Download action repository 'prefix-dev/[email protected]' (SHA:ccc5c07ed948b849df4b[3](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7560629057/job/20587085983#step:4:3)578d785f3afd7[4](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7560629057/job/20587085983#step:4:4)65c67)
Run ./.github/actions/pixi-setup
Run prefix-dev/[email protected]
Downloading Pixi
Restoring pixi cache
/home/github/work/_temp/bin/pixi install --locked --manifest-path pixi.toml
  × the project location seems to be change from `/home/runner/work/
  │ DeepStructure/DeepStructure/.pixi` to `/home/github/work/DeepStructure/
  │ DeepStructure/.pixi`, this is not allowed.
  │ Please remove the `.pixi` folder and run again

Error: The process '/home/github/work/_temp/bin/pixi' failed with exit code 1

We are using a self-hosted runner for our CI/CD jobs. I had a similar issue in the past with cache on self-hosted runners when upgrading (#25), but this seems slightly different.

The text was updated successfully, but these errors were encountered:

humphd · 2024-01-17T19:26:10Z

NOTE: rebasing the other branch didn't help this. Do I need to manually intervene on the self-hosted runner?

wolfv · 2024-01-17T19:30:16Z

Maybe you could change the cache key so that the cache is busted?

humphd · 2024-01-17T19:33:14Z

My action for setting this up looks like this:

        - name: Setup pixi
          uses: prefix-dev/[email protected]
          with:
              pixi-version: v0.12.0
              # Don't use $HOME/.pixi to work around self-runner env issues
              # https://github.com/prefix-dev/setup-pixi/issues/25
              pixi-bin-path: ${{ runner.temp }}/bin/pixi
              post-cleanup: true
              cache: true

I'm not sure how I'd change the cache key. Should I disable cache for now?

humphd · 2024-01-17T20:46:12Z

I've disabled cache for now, and my CI is passing again. It would be good to understand the proper upgrade path in future, since I've hit a bug like this twice. Perhaps I'm doing something wrong.

pavelzw · 2024-01-17T21:55:06Z

The cache key should incorporate the pixi environment path... strange

pavelzw · 2024-01-17T22:01:05Z

Can you try rerunning it with debug logging enabled? If I remember correctly it then tells you how the cache key is constructed https://github.com/prefix-dev/setup-pixi/blob/main/.github/assets/enable-debug-logging-dark.png?raw=true

pavelzw · 2024-01-18T17:25:18Z

#67 now also incorporates the pixi version in the cache. Not sure what exactly the problem in your case is (the sha should already have incorporated the change from /home/runner to /home/github).

humphd · 2024-01-18T23:56:13Z

Thanks for the fast turn around on this. I tried using v0.4.2, and all but one job now passes (cache worked in many, but fails in one still):

Getting action download info
Download action repository 'prefix-dev/[email protected]' (SHA:b0a2ad1b7[3](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20637919464?pr=272#step:4:3)611895cf1e818b[4](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20637919464?pr=272#step:4:4)2b7f59ba976daad)
Run ./.github/actions/pixi-setup
Run prefix-dev/[email protected]
Downloading Pixi
  Pixi installed to /home/github/work/_temp/bin/pixi
Restoring pixi cache
  Cache Size: ~201 MB (211274843 B)
  /usr/bin/tar -xf /home/github/work/_temp/3fb8640d-88ea-4a2b-a0a5-c064c970cc18/cache.tzst -P -C /home/github/work/DeepStructure/DeepStructure --use-compress-program unzstd
  Received 211274843 of 211274843 (100.0%), 201.3 MBs/sec
  Cache restored successfully
  Restored cache with key `pixi-linux-64-97462a084456cffae39c68b8ef5f49c9e4b931462e0b7875a0dcfe0a04dcdf[65](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20637919464?pr=272#step:4:67)`
/home/github/work/_temp/bin/pixi install --locked --manifest-path pixi.toml
  × the project location seems to be change from `/home/runner/work/
  │ DeepStructure/DeepStructure/.pixi` to `/home/github/work/DeepStructure/
  │ DeepStructure/.pixi`, this is not allowed.
  │ Please remove the `.pixi` folder and run again

Error: The process '/home/github/work/_temp/bin/pixi' failed with exit code 1

I notice warnings from other jobs:

Failed to save: Unable to reserve cache with key pixi-linux-64-97462a084456cffae39c68b8ef5f49c9e4b931462e0b7875a0dcfe0a04dcdf65, another job may be creating this cache. More details: Cache already exists. Scope: refs/pull/272/merge, Key: pixi-linux-64-97462a084456cffae39c68b8ef5f49c9e4b931462e0b7875a0dcfe0a04dcdf65, Version: 342ecd975041fe92a374ced072af88d17c89e3fd816ae855bfa15784b510c1c0

I enabled debug info:

##[debug]Evaluating condition for step: 'Setup pixi'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Setup pixi
Prepare all required actions
Getting action download info
Download action repository 'prefix-dev/[email protected]' (SHA:b0a2ad[1](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20638080622#step:4:1)b73611895cf1e818b4[2](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20638080622#step:4:2)b7f59ba976daad)
##[debug]Download 'https://api.github.com/repos/prefix-dev/setup-pixi/tarball/b0a2ad1b73611895cf1e818b42b7f59ba976daad' to '/home/github/work/_actions/_temp_1f14d1a3-9b49-4414-806c-931ba3b8a31b/65dc7da7-973c-43ec-9276-54521ae45b96.tar.gz'
##[debug]Unwrap 'prefix-dev-setup-pixi-b0a2ad1' to '/home/github/work/_actions/prefix-dev/setup-pixi/v0.4.2'
##[debug]Archive '/home/github/work/_actions/_temp_1f14d1a3-9b49-4414-806c-931ba3b8a31b/65dc7da7-973c-43ec-9276-54521ae45b96.tar.gz' has been unzipped into '/home/github/work/_actions/prefix-dev/setup-pixi/v0.4.2'.
##[debug]action.yml for action: '/home/github/work/_actions/prefix-dev/setup-pixi/v0.4.2/action.yml'.
##[debug]Register post job cleanup for action: /./.github/actions/pixi-setup
##[debug]Loading inputs
##[debug]Loading env
Run ./.github/actions/pixi-setup
##[debug]Evaluating condition for step: 'run'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: run
##[debug]Register post job cleanup for action: prefix-dev/[email protected]
##[debug]Loading inputs
##[debug]Evaluating: format('{0}/bin/pixi', runner.temp)
##[debug]Evaluating format:
##[debug]..Evaluating String:
##[debug]..=> '{0}/bin/pixi'
##[debug]..Evaluating Index:
##[debug]....Evaluating runner:
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'temp'
##[debug]..=> '/home/github/work/_temp'
##[debug]=> '/home/github/work/_temp/bin/pixi'
##[debug]Result: '/home/github/work/_temp/bin/pixi'
##[debug]Loading env
Run prefix-dev/[email protected]
##[debug]Inputs: {"pixiVersion":"v0.12.0","cache":true,"pixiBinPath":"/home/github/work/_temp/bin/pixi","postCleanup":true}
##[debug]lockFileAvailable: true
##[debug]Inferred options: {"pixiSource":{"version":"v0.12.0"},"logLevel":"vv","manifestPath":"pixi.toml","pixiLockFile":"pixi.lock","runInstall":true,"frozen":false,"locked":true,"cache":{"cacheKeyPrefix":"pixi-","cacheWrite":true},"pixiBinPath":"/home/github/work/_temp/bin/pixi","postCleanup":true}
##[debug]process.env.HOME: /mnt/github
##[debug]os.homedir(): /mnt/github
::group::Downloading Pixi
Downloading Pixi
##[debug]Skipping pixi login.
::group::Restoring pixi cache
Restoring pixi cache
##[debug]Executing: /home/github/work/_temp/bin/pixi install --locked --manifest-path pixi.toml -vv
/home/github/work/_temp/bin/pixi install --locked --manifest-path pixi.toml -vv
  × the project location seems to be change from `/home/runner/work/
  │ DeepStructure/DeepStructure/.pixi` to `/home/github/work/DeepStructure/
  │ DeepStructure/.pixi`, this is not allowed.
  │ Please remove the `.pixi` folder and run again

/home/github/work/_actions/prefix-dev/setup-pixi/v0.4.2/dist/index.js:4506
            error2 = new Error(`The process '${this.toolPath}' failed with exit code ${this.processExitCode}`);
                     ^

Error: The process '/home/github/work/_temp/bin/pixi' failed with exit code 1
    at _ExecState._setResult (/home/github/work/_actions/prefix-dev/setup-pixi/v0.4.2/dist/index.js:4506:22)
    at _ExecState.CheckComplete (/home/github/work/_actions/prefix-dev/setup-pixi/v0.4.2/dist/index.js:4492:16)
    at ChildProcess.<anonymous> (/home/github/work/_actions/prefix-dev/setup-pixi/v0.4.2/dist/index.js:4[3](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20638080622#step:4:3)95:21)
    at ChildProcess.emit (node:events:51[4](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20638080622#step:4:4):28)
    at maybeClose (node:internal/child_process:110[5](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20638080622#step:4:5):1[6](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20638080622#step:4:6))
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Node.js v20.[8](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7577333934/job/20638080622#step:4:8).1
##[debug]Node Action run completed with exit code 1
##[debug]Finished: run
##[debug]Evaluating condition for step: 'run'
##[debug]Evaluating: (success() && (inputs.run-setup == 'true'))
##[debug]Evaluating And:
##[debug]..Evaluating success:
##[debug]..=> false
##[debug]=> false
##[debug]Expanded: (false && (inputs['run-setup'] == 'true'))
##[debug]Result: false
##[debug]Finishing: Setup pixi

pavelzw · 2024-01-19T04:58:52Z

Thanks! Found the error:
I was generating the cache key from the lockfile location (this implies where the env is)
In your case, the location was "pixiLockFile":"pixi.lock" instead of the absolute path. Thus we know nothing about the environment path and the cache keys are the same even if they shouldn't be 😅

pavelzw · 2024-01-19T05:38:45Z

I think the easiest way to fix this is to add the CWD to the cache key as well.

pavelzw · 2024-01-19T05:58:12Z

Fixed in [email protected]. Thanks for the help @humphd 🎉

humphd · 2024-01-19T13:33:04Z

Confirming that v0.4.3 has fixed things in my end, and I'm now able to re-enable cache and have it work. Well done, and thanks for all the help.

pavelzw mentioned this issue Jan 18, 2024

feat: use named environments (only "default" for now) prefix-dev/pixi#674

Merged

pavelzw mentioned this issue Jan 19, 2024

Add cwd sha to cache key #68

Merged

pavelzw closed this as completed in #68 Jan 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pixi location change breaks self-hosted runner #66

Pixi location change breaks self-hosted runner #66

humphd commented Jan 17, 2024

humphd commented Jan 17, 2024

wolfv commented Jan 17, 2024

humphd commented Jan 17, 2024

humphd commented Jan 17, 2024

pavelzw commented Jan 17, 2024

pavelzw commented Jan 17, 2024

pavelzw commented Jan 18, 2024

humphd commented Jan 18, 2024

pavelzw commented Jan 19, 2024

pavelzw commented Jan 19, 2024

pavelzw commented Jan 19, 2024

humphd commented Jan 19, 2024

Pixi location change breaks self-hosted runner #66

Pixi location change breaks self-hosted runner #66

Comments

humphd commented Jan 17, 2024

humphd commented Jan 17, 2024

wolfv commented Jan 17, 2024

humphd commented Jan 17, 2024

humphd commented Jan 17, 2024

pavelzw commented Jan 17, 2024

pavelzw commented Jan 17, 2024

pavelzw commented Jan 18, 2024

humphd commented Jan 18, 2024

pavelzw commented Jan 19, 2024

pavelzw commented Jan 19, 2024

pavelzw commented Jan 19, 2024

humphd commented Jan 19, 2024