Some Exceptions Render ksqlDB Unusable

[agavra]

Describe the bug

Some bugs (e.g. #5882) make it so that the command topic is unusable. I'm not entirely certain how the command passed validation, but as soon as it is on the command topic it is no longer possible to start the ksqlDB server or submit any commands. The only option is to delete the command topic and recover it without the "bad" command.

To Reproduce

Repro using steps in #5882

Expected behavior

The command either never makes it onto the command topic, or we have some way to remediate that doesn't involve deleting the command topic.

Actual behavior

ksqlDB server becomes unusable.

Additional context

Original problem was observed on 5.3, it might be that this is no longer as bad of an issue but we should verify this.

[agavra]

It looks like the case for #5882 does not make it onto the command topic, so it might not necessarily be possible for this case to come up in normal operation. The only time I can imagine this happening is if upgrading/downgrading ksqlDB versions causes an incompatible message to be on the command topic that causes the runner to crash (most issues won't). We can probably downgrade this from p0 cc @apurvam

[apurvam]

Thanks @agavra . Should we add a awaiting-info or some other label and drop from the milestone? It seems we can't actually reproduce this behavior right now.

[agavra]

I'm happy with that