Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gcloud storage rsync output is messy #1790

Open
jsoref opened this issue Jun 19, 2024 · 0 comments
Open

gcloud storage rsync output is messy #1790

jsoref opened this issue Jun 19, 2024 · 0 comments

Comments

@jsoref
Copy link
Contributor

jsoref commented Jun 19, 2024

Windows 11 Version 23H2 (OS Build 22631.3737)
Your current Google Cloud CLI version is: 480.0.0

PS F:\test> gcloud storage rsync   --no-clobber --ignore-symlinks --recursive  gs://sample-bucket-name/test-with-colons/ .
WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file:__. to 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file$1__.At gs://sample-bucket-name/test-with-colons/**, worker process 16680 thread 15056 listed 1...
WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming 375b0fab3be7fc9544fbaa6c2b97da43ff6d2112.t-with-colons_** to 375b0fab3be7fc9544fbaa6c2b97da43ff6d2112.t-with-colons_$2$2WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file:__. to 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file$1__.WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming .\hello:cruel:world:how:are:you to .\hello$1cruel$1world$1how$1are$1youCopying gs://sample-bucket-name/test-with-colons/hello:cruel:world:how:are:you to file://.\hello$1cruel$1world$1how$1are$1you
  Completed files 1/1 | 5.0B/5.0B

Problems

There are missing line breaks

Ideally, the output would be closer to:

PS F:\test> gcloud storage rsync   --no-clobber --ignore-symlinks --recursive  gs://sample-bucket-name/test-with-colons/ .
WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file:__. to 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file$1__.
At gs://sample-bucket-name/test-with-colons/**, worker process 16680 thread 15056 listed 1...
WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming 375b0fab3be7fc9544fbaa6c2b97da43ff6d2112.t-with-colons_** to 375b0fab3be7fc9544fbaa6c2b97da43ff6d2112.t-with-colons_$2$2
WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file:__. to 3fec0319fc0cfde4a47aa40b533691f14e03e4a1.file$1__.
WARNING: The following characters are invalid in Windows file and directory names: /:*?"<>|
Renaming .\hello:cruel:world:how:are:you to .\hello$1cruel$1world$1how$1are$1you
Copying gs://sample-bucket-name/test-with-colons/hello:cruel:world:how:are:you to file://.\hello$1cruel$1world$1how$1are$1you
  Completed files 1/1 | 5.0B/5.0B

The choice of replacement characters is unfortunate

macOS has picked some replacement characters -- which means that people who use these tools on macOS to touch a FAT volume will already have things encoded in a certain way -- specifically, : is encoded as U+F022, not $1.

The output is more or less unreadable

What is the :__ stuff -- where did it come from? Where is it going? When I look on disk, I can't find it. Testing with an NTFS volume doesn't show any streams in the file, so it doesn't appear to be a stream.

Where is the .t-with-colons_$2$2 ? I can't find it.

At most, I'd expect one warning per file.

I'd also expect to get 0 warnings for a basic case of gcloud storage rsync --no-clobber --ignore-symlinks --recursive gs://sample-bucket-name/test-without-colons .

But even in that case (and using NTFS as the destination volume), I'm getting 3 WARNINGS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant