Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[runtime_env] Passing NFS path to runtime_env raises error due to ValueError: ZIP does not support timestamps before 1980 #46379

Closed
ericl opened this issue Jul 2, 2024 · 4 comments · Fixed by #46634
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core core-runtime-env Issues related to Ray environment dependencies P1 Issue that should be fixed within a few weeks

Comments

@ericl
Copy link
Contributor

ericl commented Jul 2, 2024

What happened + What you expected to happen

To ray.init(runtime_env={"working_dir": X}), pass a path which has timestamps of zero (e.g., 1970). This can happen for NFS mounts often.

Ray will raise RuntimeEnvSetupError in:

ray/_private/runtime_env/packaging.py:531, in create_package(directory, target_path, include_parent_dir, excludes, logger)
    530 logger.info(f"Creating a file package for local directory '{directory}'.")
--> 531 _zip_directory(
    532     directory,
    533     excludes,
    534     target_path,
    535     include_parent_dir=include_parent_dir,
    536     logger=logger,
    537 )
...
File /usr/lib/python3.10/zipfile.py:364, in ZipInfo.__init__(self, filename, date_time)
    363 if date_time[0] < 1980:
--> 364     raise ValueError('ZIP does not support timestamps before 1980')
    366 # Standard values:
ValueError: ZIP does not support timestamps before 1980

Versions / Dependencies

Latest Ray

Reproduction script

This could probably be reproduced locally by manually setting the timestamps of a dir to 1970.

Issue Severity

High: It blocks me from completing my task.

@ericl ericl added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) core-runtime-env Issues related to Ray environment dependencies labels Jul 2, 2024
@ericl
Copy link
Contributor Author

ericl commented Jul 2, 2024

cc @edoakes

@anyscalesam anyscalesam added the core Issues that should be addressed in Ray Core label Jul 8, 2024
@jjyao jjyao added P1 Issue that should be fixed within a few weeks and removed triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Jul 8, 2024
@pcmoritz
Copy link
Contributor

@ericl Do you want to submit a patch? If yes I can review / merge it :)

@ericl
Copy link
Contributor Author

ericl commented Jul 11, 2024

Hey @pcmoritz, I don't have the bandwidth to work on Ray patches at the moment, but if it helps I can report that multiple users have run into this issue in the last few weeks.

@richardliaw
Copy link
Contributor

seems like a 1-line change (strict_timestamps=False, for everywhere we use ZipFiles)?

@jjyao can someone on core team submit a quick PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core core-runtime-env Issues related to Ray environment dependencies P1 Issue that should be fixed within a few weeks
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants