Tracking issue for std::fs::canonicalize

[alexcrichton]

This is a tracking issue for the unstable fs_canonicalize feature in the standard library. There are a number of reasons it is not currently stable:

• The implementation on Unix isn't technically memory safe.
• The implementation on Windows returns some quite odd paths which aren't always accepted by all tools.
• The guarantees about what "canonical" means haven't been well documented and audited to see what it means to do across platforms.

Stabilizing this should probably also consider C++'s implementation as well!

[nagisa]

The implementation on Windows returns some quite odd paths which aren't always accepted by all tools.

Sounds like issue in the tools rather than pathes or canonicalisation algorithm.

[alexcrichton]

Nominating for stabilization or deprecation in 1.5, I don't think this is likely to change any time soon so we should just make a decision. I believe the memory unsafety on Unix has been resolved in #27930, Windows returning weird things is fine, and I'm ok just documenting what syscalls this is calling.

[retep998]

This API is getting the canonical path for something, aka the one true path for that something. There may be multiple valid paths to get to something, due to symbolic links and such, but there is only one canonical path. Also, returning a verbatim path on Windows is the right thing to do, because the path is a valid NT path, and with the \\?\ prefix it is also a valid win32 path, but if the \\?\ prefix is removed, then it might no longer be a valid win32 path due to win32 having a few more restrictions on paths than NT.

However I do wish that there was a way to get the canonical path of a file directly from a File and not just a Path.

[cuviper]

@retep998 There's no such thing as a canonical path for a File in a unix-style filesystem. Once you're down to a file descriptor, there's no telling what Path corresponds without combing through the filesystem for a matching inode number. Even then, with hard links there may be multiple paths reaching the same underlying file, none more canonical than any other. Or an open file may have no path at all if it was unlinked.

EDIT: I guess Linux does have the original path symlinks in /proc/self/fd/, and BSDs can mount fdesc for something similar. But I doubt Rust should try to expose this.

[retep998]

Oh, that's a good point about hard links. I shall investigate!

[alexcrichton]

This issue is now entering its cycle-long FCP for stabilization in 1.5

[Diggsey]

The windows version of this function currently only works for files: it will fail for directories. This differs significantly from the unix implementation, which uses realpath.

This should work for both files and directories on all platforms. It should also be documented that it will only work for paths that exist.

[retep998]

Here are findings:

GetFullPathNameW does not deal with symbolic links at all whatsoever. Thus it's only use for us is to get the current directory for a specific drive letter when we're normalizing drive relative paths, when we eventually one day add a normalization method. Since it is bound by MAX_PATH and doesn't hit the filesystem we can implement all the rest of the logic ourselves.

GetFinalPathNameByHandleW does handle all manner of reparse points, but it does not handle hard links. Therefore the documentation should clearly state this behavior.

If you want to know whether two paths point to the same file and account for hard links, you should use GetFileInformationByHandleEx with FileIdInfo, which provides a unique identifier for each file. Files which are hard links to each other will have the same identifier.

[retep998]

Also, would we want to expose canonicalize on File itself since that is effectively how the implementation works on Windows?

[alexcrichton]

Thanks for the findings and quick fixes @retep998 and @Diggsey!

I'm not currently aware of a great canonicalize implementation for File on Unix, so we may want to hold off on that for now. If it's not easy to do on Linux then the best way forward would be a Windows-specific extension trait for File, but that may not be worth it quite yet.

[retep998]

Also a question to decide on. If we have C:\foo\bar.txt for example, where bar.txt is a symbolic link to somewhere else, should canonicalize(r"C:\foo\bar.txt") return the canonical path to the symbolic link bar.txt or to the final thing that bar.txt points to?

[Diggsey]

Definitely the latter: canonicalization should (at least by default) resolve all symbolic links. To do otherwise, and resolve some but not the last one, is just asking for trouble.

[alexcrichton]

We probably shouldn't try to invent too many Rust-specific semantics for this kind of function, instead it may be worth taking a look at what Unix and Windows print out and then go from there.

[Diggsey]

Windows will always follow symlinks unless the application indicates otherwise by passing in a special flag to CreateFile. I assume linux does something similar.

[alexcrichton]

The libs team discussed this during triage today and the decision was to stabilize.

[dpc]

Docs say this is still unstable: https://doc.rust-lang.org/stable/std/fs/fn.canonicalize.html

[nagisa]

These are stable docs. Afair stable is still 1.4.
On Nov 14, 2015 8:01 PM, "Dawid Ciężarkiewicz" [email protected]
wrote:

Docs say this is still unstable:
https://doc.rust-lang.org/stable/std/fs/fn.canonicalize.html

—
Reply to this email directly or view it on GitHub
#27706 (comment).

[steveklabnik]

Yes, @nagisa is correct. They'll be stable in the next release.

[dpc]

My mistake, I was unaware how this process works. I thought if something is stable, it's already in stable release.

[steveklabnik]

Check out http://blog.rust-lang.org/2014/10/30/Stability.html, which explains the train model for release processes. Everything lands on nightly, then makes its way through beta and then finally into a release. We're on step 2 of that process for this particular feature.