GRIDSS AnnotateInsertedSequence error with ~2.8 million SV events: htsjdk.samtools.util.RuntimeIOException: Stream closed

[scwatts]

Hi, I've run into an unusual error in GRIDSS. I've been regularly running GRIDSS on human tumor/normal paired samples where reads are aligned to the GRCh38 reference.

One recent sample with ~2.8 million SV events (~4.5 million VCF entries) reproducibly fails right at the end of the AnnotateInsertedSequence stage in GRIDSS call step, and gives the following error:

ERROR   2023-10-24 15:11:34     ExternalProcessHelper   External process still alive: "bwa mem -K 10000000 -L 0,0 -t 16 GRCh38_full_analysis_set_plus_decoy_hla.fa -"
Exception in thread "ExternalProcessStreamingAligner" htsjdk.samtools.util.RuntimeIOException: java.io.IOException: Stream closed
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:70)
        at htsjdk.samtools.SAMTextReader.advanceLine(SAMTextReader.java:221)
        at htsjdk.samtools.SAMTextReader.access$800(SAMTextReader.java:37)
        at htsjdk.samtools.SAMTextReader$RecordIterator.next(SAMTextReader.java:257)
        at htsjdk.samtools.SAMTextReader$RecordIterator.next(SAMTextReader.java:228)
        at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:574)
        at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:553)
        at au.edu.wehi.idsv.alignment.ExternalProcessStreamingAligner.readAllAlignments(ExternalProcessStreamingAligner.java:139)
        at au.edu.wehi.idsv.alignment.ExternalProcessStreamingAligner.lambda$ensureAligner$2(ExternalProcessStreamingAligner.java:77)
        at java.base/java.lang.Thread.run(Thread.java:833)

Reducing the number of events in the input VCF and rerunning the AnnotateInsertedSequence stage allows it to complete succesfully (I've repeated this a dozen or so times with random downsampling, sweeping across a range of different event counts). There is a threshold of around ~2 million SV events where the error starts occuring again.

The input VCF is well-formed and BWA works as expected when testing that. I'm seeing this occur within a Nextflow pipeline and none of the other several hundred samples have encountered this error, so I don't suspect it's a configuration issue.

Do you have any thoughts or suggestions on how to go about resolving this? I can send you whatever commands and logs that might help - I didn't want to dump everything in the initial GH issue. Thanks!