Improve symlink support on Windows

[bdice]

@slowglow mentioned in #214:

I am here because of this error, which I encountered while doing the tutorial:
"Error: signac cannot create linked views on Windows, because symbolic links are not supported by the platform."
The second part of this error message seems to be factually incorrect. As @bdice pointed when opening this issue, "only administrators have this ability by default". It is possible for non-admin users to create symlinks on Windows 10 since at least Dec 2016 (https://blogs.windows.com/windowsdeveloper/2016/12/02/symlinks-windows-10/). In order to enable non-admin users to create symlinks "Developer mode" should be enabled (as described here: https://www.howtogeek.com/howto/16226/complete-guide-to-symbolic-links-symlinks-on-windows-or-linux/)
On my machine developer mode is enabled, and I can confirm that symlinks are created with the mklink command. However, signac view doesn't work and returns the above mentioned error.

I proposed that it would be fine to enable this feature if we are able to test it effectively and fail safely on Windows systems that don't support symlinks. Our continuous integration may not have the appropriate privileges to create symlinks, so the tests need to be optional (depending on whether the system is capable of running the tests).

As a short term solution, @slowglow proposes to change the error message to something like "Error: signac cannot create linked views. Symbolic links on Windows are currently not supported by signac."

[vyasr]

I agree with your proposed solution and caveats. We need to be able to test effectively and fail gracefully.

[mikemhenry]

We either need to add windows to our CI or consider windows unsupported. I'm going to add the 2.0 milestone to this issue since we need to decide formally if windows will be supported in 2.x. Since we don't have windows in our CI I'm tempted to close this issue unless someone wants to add windows to the CI + take on the increased maintenance workload. Another solution is to point users to the WSL which now is officially supported by Microsoft.

[mikemhenry]

I was wrong! We do test on Windows, so we should support this.

[bdice]

@mikemhenry This issue just affects specific features that rely on symlinks, primarily linked views. Windows does support symlinks but it depends on system settings and user privileges. To resolve this issue, we would need to:

1. Try to create a symlink from Python on a Windows system with symlink support enabled. If this fails, we can't do anything and should close the issue as "wontfix."
2. Replace our current approach of warning/failing immediately on Windows with an attempt to create symlinks (try/except), where the exception message should indicate some kind of helpful info if the user is on Windows.
3. Try to enable tests on CI. If we can't enable symlinks (may require admin privileges) then we need to update the skip condition to only skip on "Windows and symlinks not enabled" to ensure that the tests do run locally on Windows if permissions allow it.

This follows my proposal from above:

I proposed that it would be fine to enable this feature if we are able to test it effectively and fail safely on Windows systems that don't support symlinks. Our continuous integration may not have the appropriate privileges to create symlinks, so the tests need to be optional (depending on whether the system is capable of running the tests).

[slowglow]

It seems to me that providing access to the data on the file system (hidden behind the job id) through the human-readable linked views is one of the core features.

I might be missing something, but as an alternative, have you considered replacing symlinks with simple shortcuts on windows? This will increase the maintenance load though. But may solve the problem of testing symlinks on the CI platform.

Another valid proposal - using WSL - removes the problem from the maintainers, but on the user side is even more involved than enabling symlinks. Users, who take this approach will have many other reasons to do it, not just signac. Also, as much as it's fair to point the users to WSL (as a solution to the problem of accessing linked views on Windows), this does not make the package "Windows-compatible".

[bdice]

@slowglow Thanks for your feedback. Most (all?) of our code contributors use Mac/Linux/WSL, so linked views on Windows have not been requested until now, but we’d be happy to have the feature improvement! Would you like to help contribute a pull request to help solve this? If you’d like to have some guidance or discuss in real time with the project team, we have regular developer meetings and office hours, as well as a Slack workspace. See here for the calendar & links to join: https://docs.signac.io/en/latest/community.html

[slowglow]

@bdice
Thanks for the invitation. I'd be glad to contribute, when I get more comfortable with the package. Currently, I am barely past the tutorial and it's is not yet part of my workflow.

I agree with your proposed steps 1-3 for the development of the v2.0.0 milestone. This would enable the same codebase on different platforms.

In the future, linked views implementation as Windows shortcuts to the job id directories may be considered after careful discussion of advantages and disadvantages.

Expected advantages of such approach are that:

• ability to create shortcuts it does not depend on system settings and user privileges.
• user-friendliness, as Windows users are more comfortable with shortcuts than the symlinks.

Expected disadvantages are that:

• in order to keep signac working in the same way across platforms, low-level access to shortcuts and symlinks has to be abstracted. This may be too much of a development effort. (Someone who's familiar with the code base should call the shots.)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[slowglow]

There may be a very simple solution: instead of a blank refusal to create a symlink on Windows, try and catch the OSError, then issue a helpful error message.

Here's what I did:

pip install signac

After installing follow the tutorial and try to execute the following line:

project.create_linked_view(prefix="projects/tutorial/view")

It will return the hardcoded error message for windows platforms:

"signac cannot create linked views on Windows, because symbolic links are not supported by the platform."

This is not entirely true, so let's comment out the error first.

1. Go to
   ..\python-3.8.9.amd64\lib\site-packages\signac\contrib\linked_view.py"
   find the create_linked_view function and comment out the following lines:

# Windows does not support the creation of symbolic links.
if sys.platform == "win32":
    raise OSError(
        "signac cannot create linked views on Windows, because "
        "symbolic links are not supported by the platform."
    )

2. Now try again to create a linked view:

project.create_linked_view(prefix="projects/tutorial/view")

Most likely you will get the error:

OSError: [WinError 1314] A required privilege is not held by the client: '..\\..\\..\\workspace\\5a456c131b0c5897804a4af8e77df5aa' -> 'projects/tutorial/view\\p\\10.0\\job'

The means that windows was not set up to enable the creation of symlinks.

3. Set up windows to enable creation of symlinks (You need administrative rights)

Settings → Privacy & Security → For Developers → Use developer features: Select “Developer mode”.

If you use Python >=3.8 that's all there is to it. For Python < 3.8, a few more steps are required.
It gets more involved if your Windows is Home edition.
Read here: https://www.scivision.dev/windows-symbolic-link-permission-enable

If the user Windows system is setup correctly for the creation of symlinks, just commenting out of the relevant portion of the code should be sufficient to solve this issue.

[bdice]

@slowglow Thanks for the analysis. Please consider filing a PR if you would like this change to be released in the future.

[slowglow]

Done. Please review, comment and merge.