Support reference bundles. #1713
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cmnbroad
force-pushed
the
cn_ref_bundles
branch
from
11db598
to
94875b9
Compare
cmnbroad
force-pushed
the
cn_ref_bundles
branch
from
94875b9
to
b0a5f1c
Compare
cmnbroad
force-pushed
the
cn_ref_bundles
branch
from
b0a5f1c
to
ae17645
Compare
lbergelson
approved these changes
Sep 17, 2024
There was a problem hiding this comment.
@cmnbroad This looks good to me. I have a few style questions and typos, feel free to do what you want with those though.
| @@ -110,4 +117,47 @@ public HaploidReferenceDecoder getHaploidReferenceDecoder( | |||
| return (HaploidReferenceDecoder) resolveForDecoding(inputBundle).getDecoder(inputBundle, HaploidReferenceDecoderOptions); | |||
| } | |||
|
|
|||
| /** | |||
| * Create q reference bundle given only a fasta path, including an index and a dictionary | |||
| @@ -110,4 +117,47 @@ public HaploidReferenceDecoder getHaploidReferenceDecoder( | |||
| return (HaploidReferenceDecoder) resolveForDecoding(inputBundle).getDecoder(inputBundle, HaploidReferenceDecoderOptions); | |||
| } | |||
|
|
|||
| /** | |||
| * Create q reference bundle given only a fasta path, including an index and a dictionary | |||
There was a problem hiding this comment.
Suggested change
| * Create q reference bundle given only a fasta path, including an index and a dictionary | |
| * Create a reference bundle given only a fasta path, including an index and a dictionary |
| @@ -64,6 +65,21 @@ public FastaSequenceFile(final Path path, final boolean truncateNamesAtWhitespac | |||
| this.in = new FastLineReader(IOUtil.openFileForReading(path)); | |||
| } | |||
|
|
|||
| /** | |||
| * Constructs a FastaSequenceFile that reads from the specified fasta and dictionary file. Makes no | |||
| * assumptions that the fata and dict file are in the same directory. | |||
There was a problem hiding this comment.
Suggested change
| * assumptions that the fata and dict file are in the same directory. | |
| * assumptions that the fasta and dict file are in the same directory. |
| * Open the given indexed fasta sequence file. Throw an exception if the file cannot be opened. | ||
| * | ||
| * @param path The file to open. | ||
| * @param dictPath the dictionar path (may be null) |
There was a problem hiding this comment.
Suggested change
| * @param dictPath the dictionar path (may be null) | |
| * @param dictPath the dictionary path (may be null) |
| if (IOUtil.isBlockCompressed(path.toPath(), true)) { | ||
| throw new SAMException("Indexed block-compressed FASTA file cannot be handled: " + path); | ||
| } | ||
| this.channel = Files.newByteChannel(path.toPath()); |
There was a problem hiding this comment.
I wonder if we should add getChannel methods to IOPath. We have getStream ones.
| if (!truncateNamesAtWhitespace) { | ||
| throw new RuntimeException("preferIndexed option requires truncateNamesAtWhitespace"); | ||
| } | ||
| final Optional<BundleResource> indexPathResource = referenceBundle.get(BundleResourceType.CT_REFERENCE_INDEX); |
There was a problem hiding this comment.
These chains of optional if-present-get are kind of gross. What do you think of flatmap chains instead? I think they're a little more readable, at least once you know what you're looking at.
IOPath indexPath = referenceBundle.get(BundleResourceType.CT_REFERENCE_INDEX)
.flatMap(BundleResource::getIOPath)
.orElse(null);
if (preferIndexed && indexPath != null !Files.exists(indexPath.toPath())) {
throw new RuntimeException(String.format("FASTA index file %s does not exist", indexPath));
}
| final IOPath remoteIndex = new HtsPath( | ||
| Files.copy( | ||
| indexFile.toPath(), | ||
| nioFastaDir.resolve(indexFile.getBaseName().get() + indexFile.getExtension().get())).toUri().toString()); |
There was a problem hiding this comment.
I think you meant indexDir here?
There was a problem hiding this comment.
These copy blocks seem like they could be extracted into a function, it might help avoid this copy paste sorts of errors.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The main feature implemented here is that a bundle can be used in place of a reference, without executing any code paths that assume that the individual resources are located in the same parent directory (there are a LOT of such code paths, since most of the existing code base makes these assumptions). There are a number of tests added that exist purely to catch any regressions where such assumptions are reintroduced, at lease in the code paths where bundles are enabled.