Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept subgroup-based matches.h5 files #28

Open
jatentaki opened this issue Oct 5, 2020 · 2 comments
Open

Accept subgroup-based matches.h5 files #28

jatentaki opened this issue Oct 5, 2020 · 2 comments

Comments

@jatentaki
Copy link

To my best knowledge, you currently only accept matches.h5 files with keys in form of {filename1}-{filename2}. I would request that support for {filename1}/{filename2} is added, which is interpreted by HDF as subgroups and has the advantage that hashing is performed in stages (first filename1 then filename2). This has no impact on IMW submissions which only have thousands of file pairs, but single-stage hashing starts choking at > ~300k keys. It would be a quality of life improvement, because I could release my code with a simple script which is both compatible with IMW submissions and scales to large scenes (currently it's mutually exclusive).

@ducha-aiki
Copy link
Collaborator

ducha-aiki commented Oct 5, 2020

Could you do PR for that, please? :)

@etrulls
Copy link
Collaborator

etrulls commented Oct 5, 2020

I asked Michal to make an issue so we don't forget about it, but let's wait until we know if there are significant changes for the 2021 version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants