Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not start & initialize Kafka streams on Sandbox validations #8150

Open
spena opened this issue Sep 17, 2021 · 0 comments
Open

Do not start & initialize Kafka streams on Sandbox validations #8150

spena opened this issue Sep 17, 2021 · 0 comments

Comments

@spena
Copy link
Member

spena commented Sep 17, 2021

This code issue is found by #8130

Sandboxed objects were implemented in KSQL to validate statements when some calls to Kafka were needed. The validation should not execute any real call (i.e. delete topics, create, etc). During a validation, the full execution path is tested. For instance, in the case of persistent queries, the validation will check a query is removed from the query registry, new queries are being added to the registry, etc. To do that, the validation process creates a sandbox for the query registry to prevent real queries are indeed removed, stopped.

However, Kafka streams are not sandboxed, and thus a new stream is executed and stopped during the validation. But old queries aren't stopped, so it caused an issue during CREATE OR REPLACE when the new query wanted to access the same materialized store.

If old queries are sandboxed to prevent stopping them, I think we should also sandbox new queries to prevent starting them and stopping them. This requires some refactoring in the code to sandbox the new persistent query metadata and implement no-op methods that deal with kafka streams.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants