Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windowed Tables should use [delete, compact] for their Sink Topics #5554

Closed
agavra opened this issue Jun 4, 2020 · 0 comments · Fixed by #5743
Closed

Windowed Tables should use [delete, compact] for their Sink Topics #5554

agavra opened this issue Jun 4, 2020 · 0 comments · Fixed by #5743
Assignees
Labels
bug P0 Denotes must-have for a given milestone
Milestone

Comments

@agavra
Copy link
Contributor

agavra commented Jun 4, 2020

Describe the bug

A windowed table such as the one below is not created with the compacted topic cleanup policy.

CREATE TABLE bar 
  AS SELECT count(*) as count, id 
  FROM foo 
  WINDOW SESSION (1 hour) 
  GROUP BY id;

To Reproduce

Create the stream above based on:

CREATE STREAM bar AS SELECT count(*) as count, id FROM foo WINDOW SESSION (1 hour) GROUP BY id;

List the configs of the topic:

(13:11:06) confluent-5.5.1-SNAPSHOT ➤ ./bin/kafka-configs --topic BAR --bootstrap-server localhost:29092 --describe --all
All configs for topic BAR are:
...
{DEFAULT_CONFIG:log.cleanup.policy=delete}
...

Expected behavior

The topic should use cleanup.policy=[compact,delete]

Actual behaviour

The topic uses cleanup.policy=[delete]

Additional context

There are two workarounds:

  1. we can create the topic before issuing the create statement
  2. we can alter the topic using kafka-configs to change the cleanup policy:
--alter --add-config cleanup.policy=[compact,delete]

It seems like this was called out by @rodesai here: #1042 (comment) but it was not addressed

@agavra agavra added needs-triage P0 Denotes must-have for a given milestone labels Jun 4, 2020
@apurvam apurvam added this to the 0.11.0 milestone Jun 10, 2020
@apurvam apurvam added the bug label Jun 10, 2020
@agavra agavra self-assigned this Jun 23, 2020
@vcrfxia vcrfxia self-assigned this Jun 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug P0 Denotes must-have for a given milestone
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants