Feat: adding support for sharded precomputed

[xgui3783]

This is a PR adding support for sharded precomputed

This PR should (hopefully) support both read/write from local/http, much like the existing FileAccessor and HttpAccessor.

I will be polishing the PR by:

• adding test coverage
• update usage example

in the coming days.

I would like already to gauge if:

• neuroglancer-scripts is interested to have the ability to read sharded precomputed data source
• feedbacks on code

[codecov]

Codecov Report

Attention: 19 lines in your changes are missing coverage. Please review.

Comparison is base (a6bce3f) 92.61% compared to head (7f1d4b1) 93.96%.

Files	Patch %	Lines
src/neuroglancer_scripts/sharded_file_accessor.py	98.23%	3 Missing and 2 partials ⚠️
src/neuroglancer_scripts/scripts/scale_stats.py	55.55%	3 Missing and 1 partial ⚠️
src/neuroglancer_scripts/accessor.py	90.90%	1 Missing and 2 partials ⚠️
src/neuroglancer_scripts/sharded_http_accessor.py	95.77%	2 Missing and 1 partial ⚠️
src/neuroglancer_scripts/sharded_base.py	99.17%	1 Missing and 1 partial ⚠️
src/neuroglancer_scripts/volume_reader.py	81.81%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #35      +/-   ##
==========================================
+ Coverage   92.61%   93.96%   +1.34%     
==========================================
  Files          25       28       +3     
  Lines        1490     2138     +648     
  Branches      219      308      +89     
==========================================
+ Hits         1380     2009     +629     
- Misses         63       75      +12     
- Partials       47       54       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[xgui3783]

@ylep I believe the PR is mostly ready. Can you take a look when you get a chance?

[ylep]

First, I want to thank you a lot @xgui3783 for adding this very significant feature to neuroglancer-scripts! I had been aware of the existence of that sharded storage format, but saw that it was a complex feature to implement so I did not find the time to do so myself.

Next, I apologize for taking so long to review the code. Actually, this is a large amount of code so I have not been able to review it as deeply as I would like. Most of my comments are rather superficial unfortunately; but on the other hand I have no doubt that you have structured the code in a very good way.

My comments are in-line with the code, I'd just like to add one general comment: the new files that you introduced, and those that you contributed to substantially, should have a short copyright header added / updated, in the style of:
https://github.com/HumanBrainProject/neuroglancer-scripts/blob/53e8042411cd6c52d80159f9f8a486e09c877ed1/src/neuroglancer_scripts/chunk_encoding.py#L1C1-L4C74

In particular, the files that contain third-party code should have the copyright and licence of that code included.

With these few minor changes addressed, I will be happy to approve and merge this PR, and make a new release of neuroglancer-scripts!

[xgui3783]

@ylep I have added an additional commit, which hopefully address most if not all of your concerns.

can you take another look when you get a chance?

[ylep]

@xgui3783 Huge thanks for implementing this feature, it looks to me in a very good state now!

This feature alone is probably worth a 1.2.0 release, don't you think?

[xgui3783]

Thanks @ylep for the quick response. I would fully support a 1.2.0 release.