Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filtering WINDOWS results in NULL WINDOWS #2989

Closed
patrickFuerst opened this issue Jun 17, 2019 · 2 comments
Closed

Filtering WINDOWS results in NULL WINDOWS #2989

patrickFuerst opened this issue Jun 17, 2019 · 2 comments

Comments

@patrickFuerst
Copy link

I'm not sure if this is actually wanted behaviour, also because KSQL CLI is not showing the NULL messages but the kafka-console-consumer does.

What I'm doing is filtering out all windows but the "oldest" one. The result is one message containing the aggregated values for the "oldest" window and a lot of NULL messages for the rest of the windows. I guess emitting these NULL messages is not really wanted.

This can be reproduced with the following steps:

In KSQL:

CREATE STREAM test (location VARCHAR, parts INT ) WITH (VALUE_FORMAT='json', KAFKA_TOPIC='test');

CREATE STREAM test_keyed AS SELECT * FROM test PARTITION BY LOCATION;

CREATE TABLE test_windowed AS SELECT
    location,
    SUM(parts) AS total
FROM
    test_keyed
WINDOW HOPPING (SIZE 10 MINUTES, ADVANCE BY 2 SECONDS)
GROUP BY location
HAVING
MAX(rowtime) < WINDOWEND()
AND WINDOWEND() < MAX(rowtime)+2*1000

Emits one message and a lot of NULLs:

./kafka-console-consumer --bootstrap-server localhost:9092 --topic TEST_WINDOWED

Send test message:

./kafka-console-producer --broker-list localhost:9092 --topic test

with { "location" : "site1" , "parts": 10 }

@big-andy-coates
Copy link
Contributor

Hi @patrickFuerst

Looks like you're running into this issue with HAVING clauses: #3558

Please vote for this issue, (by adding a thumbs up vote), if you want to see it fixed.

@big-andy-coates
Copy link
Contributor

In the interest of keeping our issue backlog tidy I'm going to close this issue. Please feel free to reopen with more details if the answer does not fully answer your question of if you have a follow up question. Thanks for using KSQL!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants