-
Notifications
You must be signed in to change notification settings - Fork 697
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Try using smaller package ids on Windows (see #3430) #3431
Conversation
On Windows we have serious problems with path lengths. Windows imposes a maximum path length of 260 chars, and even if we can use the windows long path APIs ourselves, we cannot guarantee that ghc, gcc, ld, ar, etc etc all do so too. So our only choice is to limit the lengths of the paths, and the only real way to do that is to limit the size of the 'InstalledPackageId's that we generate. We do this by truncating the package names and versions and also by truncating the hash sizes. Truncating the package names and versions is technically ok because they are just included for human convenience, the full source package id is included in the hash. Truncating the hash size is disappointing but also technically ok. We rely on the hash primarily for collision avoidance not for any securty properties (at least for now).
This will require a little bit of testing on windows from someone. |
truncateHash (HashValue h) = HashValue (BS.take 20 h) | ||
|
||
-- Truncate a string, with a visual indication that it is truncated. | ||
truncateStr n s | length s <= n = s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor nitpick: length (truncateStr 0 "x") /= 0
:-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right it will not work for 0, since it has to have a visual indication of the truncation. It's only a local function and we're using it with >0 values so... erm... so nitpick be gone! :-)
What kinds of tests need to be done? Are automatic tests on AppVeyor not enough? |
@dcoutts BTW, I've enabled integration-tests2 on AppVeyor and they seem to fail on Windows: https://ci.appveyor.com/project/23Skidoo/cabal/build/%23830%20%28master%29 Can you look into it? |
@23Skidoo so that failure was unrelated, so still need some test case so we can check if this fixes things. |
@23Skidoo so borrowing @hvr's windows box, I can reproduce a failure with Failure:
becomes the successful
That long path goes from length 264 to 198. So we could add an integration test with "a-totally-outrageous-and-unreasonably-long-package-name-0.0.0.1" |
So actually it's a bit hard to add an integration test for this yet since it needs to install into the store, and currently that is only done for packages from hackage. The plan is to support installing to the store also for local tarball packages, so once we have that support then we can more easily make a test like this. |
Otherwise LGTM. |
Ok. |
@23Skidoo was this ever backported to the 1.24 branch? |
@hvr Nope. |
Backport #3431 to the 1.24 branch
On Windows we have serious problems with path lengths (see #3430).
Windows imposes a maximum path length of 260 chars, and even if we
can use the windows long path APIs ourselves, we cannot guarantee
that ghc, gcc, ld, ar, etc etc all do so too.
So our only choice is to limit the lengths of the paths, and the only
real way to do that is to limit the size of the 'InstalledPackageId's
that we generate. We do this by truncating the package names and
versions and also by truncating the hash sizes.
Truncating the package names and versions is technically ok because they
are just included for human convenience, the full source package id is
included in the hash.
Truncating the hash size is disappointing but also technically ok. We
rely on the hash primarily for collision avoidance not for any securty
properties (at least for now).