Skip to content

Incremental Parallel Reduction

johnmcclean-aol edited this page Jun 11, 2015 · 1 revision

Parallel Reduction

FutureStreams can be converted to JDK Streams via the stream() method.

EagerFutureStream.react(()->loadData(),()->loadData())
                .map(this::process)
                .stream()
                .parallel()
                .reduce(this::combiner); 

Incremental Parallel Reduction

LazyFutureStream.react(dataStream)
                .map(this::process)
                .withParallelReduction(new ParallelReductionConfig(300,true)).
                .reduce(this::combiner); 

LazyFutureStream can perform reduction operations in parallel, as data streams in asynchronously. In the example above they will be performed in batches of 300. Both non-terminal async Stream operations and incremental reduction batches can be performed concurrently.

Clone this wiki locally