NullPointerException during count aggregation

[dguy]

Caused by: java.lang.NullPointerException
at io.confluent.ksql.planner.plan.AggregateNode.lambda$aggregateReKey$1(AggregateNode.java:272)
at org.apache.kafka.streams.kstream.internals.KStreamImpl$2.apply(KStreamImpl.java:157)
at org.apache.kafka.streams.kstream.internals.KStreamImpl$2.apply(KStreamImpl.java:154)
at org.apache.kafka.streams.kstream.internals.KStreamMap$KStreamMapProcessor.process(KStreamMap.java:41)
at org.apache.kafka.streams.processor.internals.ProcessorNode$1.run(ProcessorNode.java:46)
at

[hjafarpour]

@dguy can you include the query that causes this.

[dguy]

@hjafarpour i can't as it came from a private customer, so is confidential

[k1th]

@hjafarpour
anonymized. (you have access though to our private channel)
[2017-12-12 09:37:37,604] INFO Streaming query ’select YEARMONTH_CUSTOMER_KEY, count(*) from TABLE_REKEYED_VIA_STREAM where rowkey = ‘2017-11_SOMEID’ group by YEARMONTH_CUSTOMER_KEY;' (io.confluent.ksql.rest.server.resources.streaming.StreamedQueryResource:71)

[k1th]

The NPE happens, because value is null here:
newKey.append(String.valueOf(value.getColumns().get(index)));
that is simply a record with and key and no value...

[deinspanjer]

@k1th regarding your last comment, are you saying this is an expected behavior?
I just started searching for references to the NPE I'm getting when trying to do a query similar to the following:
SELECT s.id, MIN(s.time) AS start_time, MAX(e.time) AS end_time FROM start_stream s LEFT JOIN end_tbl e ON s.id = e.id GROUP BY s.id;

[k1th]

@deinspanjer nope. this is not expected. This should fail gracefully imho.

[miguno]

Another practical example where this issue will cause problems:

Imagine you want to combine Kafka Streams applications with KSQL. This issue would, for example, prevent you from (1) running Kafka Streams application that is outputting a KTable to Kafka, and then (2) reading this output into a KSQL query.

[PoojaKarande]

I face issue when performing SUM, MIN or MAX aggregation on the values of a Map<varchar, double>.

ksql> SELECT sum(data['some_key']), id FROM raw_event WINDOW TUMBLING (SIZE 20 SECONDS) GROUP BY id;
ServerError:java.lang.NullPointerException

Count aggregation works fine!