Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Broken on illumos? #63

Open
whorfin opened this issue Apr 22, 2021 · 4 comments
Open

Broken on illumos? #63

whorfin opened this issue Apr 22, 2021 · 4 comments

Comments

@whorfin
Copy link

whorfin commented Apr 22, 2021

Trying to track down some caching problems with navidrome navidrome/navidrome#1048
and found that pulling down master and running "go test" fails:

$ go version
go version go1.16.3 illumos/amd64
$ uname -a
SunOS pergamum 5.11 omnios-r151030-5bd7739fe4 i86pc i386 i86pc illumos
$ git clone https://github.com/karrick/godirwalk.git
Cloning into 'godirwalk'...
remote: Enumerating objects: 1141, done.
remote: Counting objects: 100% (30/30), done.
remote: Compressing objects: 100% (21/21), done.
Receiving objects:  99% (1130/1141)remote: Total 1141 (delta 13), reused 21 (delta 9), pack-reused 1111
Receiving objects: 100% (1141/1141), 255.23 KiB | 4.12 MiB/s, done.
Resolving deltas: 100% (617/617), done.
$ pushd godirwalk/
~/navidrome/godirwalk ~/navidrome
$ go test
--- FAIL: TestReadDirents (0.00s)
    --- FAIL: TestReadDirents/without_symlinks (0.00s)
        readdir_test.go:14: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
    --- FAIL: TestReadDirents/with_symlinks (0.00s)
        readdir_test.go:51: GOT: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestScanner (0.00s)
    --- FAIL: TestScanner/collect_names (0.00s)
        scandir_test.go:22: GOT: "aaaaaa\x03" (extra)
        scandir_test.go:22: WANT: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" (missing)
        scandir_test.go:22: GOT: "symlin\x03" (extra)
        scandir_test.go:22: WANT: "symlinks" (missing)
    --- FAIL: TestScanner/collect_dirents (0.00s)
        scandir_test.go:35: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkCompatibleWithFilepathWalk (0.00s)
    --- FAIL: TestWalkCompatibleWithFilepathWalk/test_root (0.00s)
        walk_test.go:79: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkSkipThis (0.00s)
    --- FAIL: TestWalkSkipThis/SkipThis (0.00s)
        walk_test.go:154: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkFollowSymbolicLinks (0.00s)
    walk_test.go:196: GOT: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestErrorCallback (0.00s)
    --- FAIL: TestErrorCallback/halt (0.00s)
        walk_test.go:239: unexpected error callback for /tmp/godirwalk-378882479/d0/symlinks: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory
    --- FAIL: TestErrorCallback/skipnode (0.00s)
        walk_test.go:271: unexpected error callback for /tmp/godirwalk-378882479/d0/symlinks: lstat /tmp/godirwalk-378882479/d0/symlinks/nothin: no such file or directory
--- FAIL: TestPostChildrenCallback (0.00s)
    walk_test.go:299: GOT: lstat /tmp/godirwalk-378882479/d0/aaaaaa: no such file or directory; WANT: []
FAIL
drwx------
drwxrwxr-x /d0
-rwxrwxr-x /d0/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
drwxrwxr-x /d0/d1
-rwxrwxr-x /d0/d1/f2
-rwxrwxr-x /d0/f1
drwxrwxr-x /d0/skips
drwxrwxr-x /d0/skips/d2
-rwxrwxr-x /d0/skips/d2/f3
-rwxrwxr-x /d0/skips/d2/skip
-rwxrwxr-x /d0/skips/d2/z1
drwxrwxr-x /d0/skips/d3
-rwxrwxr-x /d0/skips/d3/f4
drwxrwxr-x /d0/skips/d3/skip
-rwxrwxr-x /d0/skips/d3/skip/f5
-rwxrwxr-x /d0/skips/d3/z2
drwxrwxr-x /d0/symlinks
drwxrwxr-x /d0/symlinks/d4
Lrwxrwxrwx /d0/symlinks/d4/toSD1 -> ../toD1
Lrwxrwxrwx /d0/symlinks/d4/toSF1 -> ../toF1
Lrwxrwxrwx /d0/symlinks/nothing -> ../f0
Lrwxrwxrwx /d0/symlinks/toAbs -> /tmp/godirwalk-378882479/d0/f1
Lrwxrwxrwx /d0/symlinks/toD1 -> ../d1
Lrwxrwxrwx /d0/symlinks/toF1 -> ../f1
exit status 1
FAIL    github.com/karrick/godirwalk    0.023s

That weird truncation of "aaaaaa" is exactly what we're seeing with navidrome
@whorfin
Copy link
Author

whorfin commented Apr 22, 2021

Relevant to #31

@whorfin
Copy link
Author

whorfin commented Apr 22, 2021

In answer to the "if the tests fail please provide the results of go test -v"

$ go test -v
=== RUN   TestDirent
=== RUN   TestDirent/file
=== RUN   TestDirent/directory
=== RUN   TestDirent/symlink
=== RUN   TestDirent/symlink/to_file
=== RUN   TestDirent/symlink/to_directory
--- PASS: TestDirent (0.00s)
    --- PASS: TestDirent/file (0.00s)
    --- PASS: TestDirent/directory (0.00s)
    --- PASS: TestDirent/symlink (0.00s)
        --- PASS: TestDirent/symlink/to_file (0.00s)
        --- PASS: TestDirent/symlink/to_directory (0.00s)
=== RUN   TestReadDirents
=== RUN   TestReadDirents/without_symlinks
    readdir_test.go:14: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
=== RUN   TestReadDirents/with_symlinks
    readdir_test.go:51: GOT: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestReadDirents (0.00s)
    --- FAIL: TestReadDirents/without_symlinks (0.00s)
    --- FAIL: TestReadDirents/with_symlinks (0.00s)
=== RUN   TestReadDirnames
--- PASS: TestReadDirnames (0.00s)
=== RUN   TestScanner
=== RUN   TestScanner/collect_names
    scandir_test.go:22: GOT: "aaaaaa\x03" (extra)
    scandir_test.go:22: WANT: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" (missing)
    scandir_test.go:22: GOT: "symlin\x03" (extra)
    scandir_test.go:22: WANT: "symlinks" (missing)
=== RUN   TestScanner/collect_dirents
    scandir_test.go:35: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
=== RUN   TestScanner/symlink_to_directory
--- FAIL: TestScanner (0.00s)
    --- FAIL: TestScanner/collect_names (0.00s)
    --- FAIL: TestScanner/collect_dirents (0.00s)
    --- PASS: TestScanner/symlink_to_directory (0.00s)
=== RUN   TestWalkCompatibleWithFilepathWalk
=== RUN   TestWalkCompatibleWithFilepathWalk/test_root
    walk_test.go:79: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
=== RUN   TestWalkCompatibleWithFilepathWalk/ignore_skips
--- FAIL: TestWalkCompatibleWithFilepathWalk (0.00s)
    --- FAIL: TestWalkCompatibleWithFilepathWalk/test_root (0.00s)
    --- PASS: TestWalkCompatibleWithFilepathWalk/ignore_skips (0.00s)
=== RUN   TestWalkSkipDir
=== RUN   TestWalkSkipDir/skip_file_at_root
=== RUN   TestWalkSkipDir/skip_dir_at_root
=== RUN   TestWalkSkipDir/skip_nodes_under_root
=== RUN   TestWalkSkipDir/SkipDirOnSymlink
--- PASS: TestWalkSkipDir (0.00s)
    --- PASS: TestWalkSkipDir/skip_file_at_root (0.00s)
    --- PASS: TestWalkSkipDir/skip_dir_at_root (0.00s)
    --- PASS: TestWalkSkipDir/skip_nodes_under_root (0.00s)
    --- PASS: TestWalkSkipDir/SkipDirOnSymlink (0.00s)
=== RUN   TestWalkSkipThis
=== RUN   TestWalkSkipThis/SkipThis
    walk_test.go:154: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestWalkSkipThis (0.00s)
    --- FAIL: TestWalkSkipThis/SkipThis (0.00s)
=== RUN   TestWalkFollowSymbolicLinks
    walk_test.go:196: GOT: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory; WANT: []
--- FAIL: TestWalkFollowSymbolicLinks (0.00s)
=== RUN   TestErrorCallback
=== RUN   TestErrorCallback/halt
    walk_test.go:239: unexpected error callback for /tmp/godirwalk-286333511/d0/symlinks: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory
=== RUN   TestErrorCallback/skipnode
    walk_test.go:271: unexpected error callback for /tmp/godirwalk-286333511/d0/symlinks: lstat /tmp/godirwalk-286333511/d0/symlinks/nothin: no such file or directory
--- FAIL: TestErrorCallback (0.00s)
    --- FAIL: TestErrorCallback/halt (0.00s)
    --- FAIL: TestErrorCallback/skipnode (0.00s)
=== RUN   TestPostChildrenCallback
    walk_test.go:299: GOT: lstat /tmp/godirwalk-286333511/d0/aaaaaa: no such file or directory; WANT: []
--- FAIL: TestPostChildrenCallback (0.00s)
FAIL
drwx------
drwxrwxr-x /d0
-rwxrwxr-x /d0/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
drwxrwxr-x /d0/d1
-rwxrwxr-x /d0/d1/f2
-rwxrwxr-x /d0/f1
drwxrwxr-x /d0/skips
drwxrwxr-x /d0/skips/d2
-rwxrwxr-x /d0/skips/d2/f3
-rwxrwxr-x /d0/skips/d2/skip
-rwxrwxr-x /d0/skips/d2/z1
drwxrwxr-x /d0/skips/d3
-rwxrwxr-x /d0/skips/d3/f4
drwxrwxr-x /d0/skips/d3/skip
-rwxrwxr-x /d0/skips/d3/skip/f5
-rwxrwxr-x /d0/skips/d3/z2
drwxrwxr-x /d0/symlinks
drwxrwxr-x /d0/symlinks/d4
Lrwxrwxrwx /d0/symlinks/d4/toSD1 -> ../toD1
Lrwxrwxrwx /d0/symlinks/d4/toSF1 -> ../toF1
Lrwxrwxrwx /d0/symlinks/nothing -> ../f0
Lrwxrwxrwx /d0/symlinks/toAbs -> /tmp/godirwalk-286333511/d0/f1
Lrwxrwxrwx /d0/symlinks/toD1 -> ../d1
Lrwxrwxrwx /d0/symlinks/toF1 -> ../f1
exit status 1
FAIL    github.com/karrick/godirwalk    0.026s

@karrick
Copy link
Owner

karrick commented May 4, 2022

I wonder whether Illumos' file system file name limits are related? https://illumos.org/man/2/intro

@whorfin
Copy link
Author

whorfin commented May 4, 2022

I am not sure which limits you are referring to
When godirwalk.Walk() was replaced with filepath.WalkDir() in navidrome , all of the unwanted filename truncation disappeared and cross-platform functionality was enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@karrick @whorfin