Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ensure only deserializable cmds are written to command topic #5645

Conversation

big-andy-coates
Copy link
Contributor

@big-andy-coates big-andy-coates commented Jun 18, 2020

Description

fixes: #5643

Ensures all commands written to the command topic can be deserialized before writing them.

A non-deserializable command causes the command runner thread to die.
Even restarting the server won't help as the server will stop when it hits the non-deserializable command again.

This adds some level of protection.

(I know I'd previously been against this, but ... meh... maybe I was wrong ;) )

Submitting a statement to the server which the server finds it can't deserialize now results in an error and the command is, importantly, not added to the command queue:

ksql> CREATE TABLE OUTPUT as SELECT 1 as k, count(1) AS ID FROM INPUT group by 1;
Did not write the command to the command topic as it could not be deserialized. This is a bug! Please raise a Github issue containing the series of commands you ran to get to this point.

Testing done

usual

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

fixes: confluentinc#5643

Ensures all commands written to the command topic can be deserialized before writing them.

A non-deserializable command causes the command runner thread to die.
Even restarting the server won't help as the server will stop when it hits the non-deserializable command again.

This adds some level of protection.
Copy link
Member

@stevenpyzhang stevenpyzhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, I think this change makes sense.

@stevenpyzhang
Copy link
Member

@big-andy-coates could you change the target branch to 6.0.x, this would be a worthwhile fix to include in that version.

Comment on lines +73 to +74
* command will kill the command runner thread, this is a safety net to ensure commands written to
* the command topic can be deserialzied.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this isn't strictly true if the version reading is different than the version writing; but I'm not opposed to the extra safety net 😛

@big-andy-coates big-andy-coates merged commit c27c85a into confluentinc:master Jun 19, 2020
@big-andy-coates big-andy-coates deleted the enque_only_deserializable_commands branch June 19, 2020 17:49
agavra pushed a commit that referenced this pull request Jun 19, 2020
)

* fix: ensure only deserializable cmds are written to command topic

fixes: #5643

Ensures all commands written to the command topic can be deserialized before writing them.

A non-deserializable command causes the command runner thread to die.
Even restarting the server won't help as the server will stop when it hits the non-deserializable command again.

This adds some level of protection.


Co-authored-by: Andy Coates <[email protected]>
Copy link
Contributor

@rodesai rodesai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just adding my +1 that I think this makes sense

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GROUP BY with no non-aggregate columns crashes CommandRunner thread
4 participants