Cannot use on Apple Silicon due to incompatible binary dependencies in upstream packages

[jimmcslim]

Environment details

• OS: macOS Monterey 12.6.1 (running on an Apple M1 Pro)
• Node.js version: 18.16.0
• npm version: 9.5.1
• @google-cloud/storage version: 6.10.1

Steps to reproduce

1. Attempt to use this library on an Apple Silicon machine using a version of Node.js compiled for the ARM architecture.
2. Observe a runtime error from dyld Missing symbol.

@google-cloud/storage is dependent upon hash-stream-validation, which in turn is dependent upon node-fast-crc32c. However the binary dependencies of node-fast-crc32c are NOT compatible with the M1/ARM architecture.

There is this PR for node-fast-crc32c which would resolve this issue, however it is long stalled.

Similarly there is this issue on hash-stream-validation requesting that this library also attempt to solve the problem... acknowledging that the upstream dependency appears to no longer be developed... however this issue also appears to have stalled.

I'm reminded of this XKCD sketch - I certainly hope Google might consider reimplementing the functionality of these packages into your code base, and also consider including Apple Silicon in your CI/CD pipeline to ensure a positive developer experience.

Thanks!

[ddelgrosso1]

@jimmcslim thank you for raising this issue. As you noted this issue lies in our dependency tree. We have been in the process of an effort to try and reduce our usage of 3rd party libraries to avoid things like this (and reduce overall library size). While I'm not sure there is much we can do about the current lack of progress in the PRs you linked, we will certainly investigate if more compatible alternatives exist as this seems to be the quickest / easiest path.

Unless there is enough signal from the community I would guess taking it upon ourselves to write an alternative would be a much larger / longer term undertaking. Is this issue or similar binary incompatibilities affecting your usage of any other Google Cloud SDKs?

Editing my above comments (probably shouldn't answer before coffee...). hash-stream-validation was removed in this PR: #1951. Are you sure you are seeing this issue with v6.10.1?

Edit 2: The above change was released with v6.0.0 https://github.com/googleapis/nodejs-storage/releases/tag/v6.0.0

[jimmcslim]

Thanks for the response. I thought I had upgraded across the board to 6.10.1 but perhaps I still have dependencies on the older version as well, I will investigate again and see what I can do.

[ddelgrosso1]

@jimmcslim were you able to identify if you still had an old version hanging around?

[ddelgrosso1]

Going to close this. As noted above hash-stream-validation has been removed in v6.x. If there are still issues please feel free to reopen with additional information.