Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to extract archive on windows #940

Open
Krande opened this issue Jun 21, 2024 · 11 comments · May be fixed by #1138
Open

Failed to extract archive on windows #940

Krande opened this issue Jun 21, 2024 · 11 comments · May be fixed by #1138

Comments

@Krande
Copy link

Krande commented Jun 21, 2024

When using pixi and rattler-build v0.18 pixi run dca -c https://repo.prefix.dev/code-aster -c conda-forge on this repository, I get

 ╭─ Running build for recipe: code-aster-17.0.21-py_nompi_release_h_200
 │
 │ ╭─ Fetching source code
 │ │ Downloaded file from https://gitlab.com/krande/src/-/archive/17.0.21.3/src-17.0.21.3.tar.gz
 │ │ Validated SHA256 values of the downloaded file!
 │ │ Extracting tar file to cache: "C:\\Work\\code\\condapackaging\\output\\src_cache\\src-17_0_21_3_9da7ea83.tar.gz"
 │ │
 │ ╰─────────────────── (took 29 seconds)
 │
 ╰─────────────────── (took 29 seconds)
 × error Error building package: Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\src-17_0_21_3_9da7ea83\.tmpWso4Cf\src-17.0.21.3\waf`
Error:   × Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\src-17_0_21_3_9da7ea83\.tmpWso4Cf\src-17.0.21.3\waf`

Let me know if you need any additional information from me?

Best Regards
Kristoffer
@wolfv
Copy link
Member

wolfv commented Jun 21, 2024

Interesting - does the archive contain symlinks?

@Krande
Copy link
Author

Krande commented Jun 21, 2024 •
edited
Loading

yeah, I think there are (at least) a few symlink files at the root of the archive; waf, waf_std, waf_mpi, waf_debug

image

@wolfv
Copy link
Member

wolfv commented Jun 21, 2024

Hmm symlinks are tricky on Windows. You can enable developer mode to allow symlinks on Windows without elevated permissions. I don't know how conda-build (or other archive extraction programs) deal with this.

Do you know if this used to work in conda-build?

@wolfv
Copy link
Member

wolfv commented Jun 21, 2024

https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development

@Krande
Copy link
Author

Krande commented Jun 21, 2024

These files arent used on windows. So they are only relevant for linux. So I guess they're treated as simple binary files?

@wolfv
Copy link
Member

wolfv commented Jun 21, 2024

Could you try to extract this archive with tar or conda-build on Windows? I am just wondering what they do...

Potentially we could warn (and ignore) symlinks when we cannot create them (instead of failing).

@Krande
Copy link
Author

Krande commented Jun 21, 2024 •
edited
Loading

It works without issues when I run conda mambabuild .

Source cache directory is: C:\Work\Miniforge3\envs\mamba-build\conda-bld\src_cache
Found source in cache: code-aster_17.0.21.3_win_9da7ea830c.tar.gz
Extracting download
source tree in: C:\Work\Miniforge3\envs\mamba-build\conda-bld\code-aster_1718963599220\work

And when I do

tar -xvzf /c/Work/Miniforge3/envs/mamba-build/conda-bld/src_cache/code-aster_17.0.21.3_win_9da7ea830c.tar.gz -C /c/Work/

it also works fine. Note that I had to change my directory mapping string when using tar

Update:

Tar does give me some warnings though:

tar: src-17.0.21.3/waf: Cannot create symlink to 'waf_std': No such file or directory
src-17.0.21.3/waf.engine
src-17.0.21.3/waf.main
src-17.0.21.3/waf_debug
tar: src-17.0.21.3/waf_debug: Cannot create symlink to 'waf_variant': No such file or directory
src-17.0.21.3/waf_mpi
tar: src-17.0.21.3/waf_mpi: Cannot create symlink to 'waf_variant': No such file or directory
src-17.0.21.3/waf_std
tar: src-17.0.21.3/waf_std: Cannot create symlink to 'waf_variant': No such file or directory
src-17.0.21.3/waf_variant
src-17.0.21.3/waftools/
src-17.0.21.3/waftools/clangdb.py
src-17.0.21.3/waftools/ext_aster.py
src-17.0.21.3/waftools/mathematics.py
src-17.0.21.3/waftools/med_cfg.py
src-17.0.21.3/waftools/metis.py
src-17.0.21.3/waftools/msvc_lib.py
src-17.0.21.3/waftools/mumps.py
src-17.0.21.3/waftools/official_platforms.py
src-17.0.21.3/waftools/parallel.py
src-17.0.21.3/waftools/parmetis.py
src-17.0.21.3/waftools/petsc.py
src-17.0.21.3/waftools/python_cfg.py
src-17.0.21.3/waftools/runtest.py
src-17.0.21.3/waftools/scotch.py
src-17.0.21.3/waftools/wafutils.py
src-17.0.21.3/wscript
tar: Exiting with failure status due to previous errors

@wolfv
Copy link
Member

wolfv commented Jun 21, 2024

OK, so it exits with a failure status, but still extracts everything except for the symlinks. I think we can follow that behavior.

@wolfv
Copy link
Member

wolfv commented Jun 21, 2024

Meaning warn, but still proceed.

@Krande
Copy link
Author

Krande commented Jun 21, 2024

Yeah, it seems like it extracts all files. So if that's a possibility then that sounds great!

@Krande
Copy link
Author

Krande commented Oct 26, 2024

Hey @wolfv, I think I just ran into another one of these tar extraction with symlink on windows issues.

 │ │ ╰─────────────────── (took 0 seconds)
 │ │ Downloaded file from https://github.com/thelfer/tfel/archive/refs/tags/TFEL-4.2.1.tar.gz
 │ │ Validated SHA256 values of the downloaded file!
 │ │ Extracting tar file to cache: "C:\\Work\\code\\condapackaging\\output\\src_cache\\TFEL-4_2_1_14f27257.tar.gz"
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 ╰─────────────────── (took 6 seconds)
 × error Error building package: Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\TFEL-4_2_1_14f27257\.tmp97hhn7\tfel-TFEL-4.2.1\README`
Error:   × Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\TFEL-4_2_1_14f27257\.tmp97hhn7\tfel-TFEL-4.2.1\README`

When I try to manually extract using tar on windows I it extracts seemingly everything but ends with
tar: Error exit delayed from previous errors.

I figured I could take a look, and I did manage to pass the extraction with a slight modification of "extract.rs".
See main...Krande:rattler-build:fix/skip-symlink-extraction.

I'm making a PR also with the suggested fix where we can continue the discussion.

@Krande Krande linked a pull request Oct 26, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: add tolerant tar extraction on windows to solve failing on symlinks Krande/rattler-build
2 participants
@wolfv @Krande