Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: remove ErrorEntity and throw on connector error instead #8998

Merged
merged 4 commits into from
Apr 8, 2022

Conversation

vcrfxia
Copy link
Contributor

@vcrfxia vcrfxia commented Apr 7, 2022

Description

This PR is stacked on #8983. Only the commits starting with chore: remove ErrorEntity and throw on connector error instead need to be reviewed.

Currently, when connector requests encounter errors they return a 200 response code with an ErrorEntity object in the response. This is problematic because:

  • the 200 response code is confusing, given that the request did not succeed
  • in the case when multiple ksql statements are issued as a single request, the 200 response code from a failed connector statement causes subsequent statements to be executed even though they shouldn't be.

This PR updates the connect executors to return an error response (resulting in a non-200 response code) when requests fail.

Testing done

Unit + integration + manual.

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@vcrfxia vcrfxia force-pushed the remove-connect-error-entity branch from ed580e4 to 38ece5a Compare April 7, 2022 22:48
@vcrfxia vcrfxia changed the title Remove connect error entity chore: remove ErrorEntity and throw on connector error instead Apr 7, 2022
@vcrfxia vcrfxia changed the title chore: remove ErrorEntity and throw on connector error instead fix: remove ErrorEntity and throw on connector error instead Apr 7, 2022
@@ -100,8 +98,7 @@
SHOW_COLUMNS(ShowColumns.class, ListSourceExecutor::columns),
EXPLAIN(Explain.class, ExplainExecutor::execute),
DESCRIBE_FUNCTION(DescribeFunction.class, DescribeFunctionExecutor::execute),
DESCRIBE_CONNECTOR(DescribeConnector.class,
new DescribeConnectorExecutor(new DefaultConnectServerErrors())::execute),
DESCRIBE_CONNECTOR(DescribeConnector.class, StatementValidator.NO_VALIDATION),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed this validation because it was a no-op -- validation only fails if an error is thrown, and no error was ever thrown from DescribeConnectorExecutor::validate since it returned a 200 response with an error entity on error, instead of throwing.

It doesn't make sense to validate this request because there's nothing to validate (validating it amounts to issuing the request to Connect, which is what execution does). The same logic applies to DropConnector and ListConnectors, both of which are already not being validated.

See #8999 for more on validating connector statements.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, DescribeConnectorExecutor::execute never throws an exception indeed.

I guess we should do the same for LIST_VARIABLES and DESCRIBE_FUNCTION -- can do it in a follow-up

@vcrfxia vcrfxia marked this pull request as ready for review April 8, 2022 00:05
@vcrfxia vcrfxia requested a review from a team as a code owner April 8, 2022 00:05
.status(statusResponse.httpCode())
.entity(new KsqlErrorMessage(
Errors.toErrorCode(statusResponse.httpCode()),
"Failed to query connector status: " + statusResponse.error().get()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: move errorMsg to a final var for consistency and readability -- applies to all KsqlRestExceptions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm interesting. I think most places in the codebase inline the error message directly rather than having a separate line for final String errorMsg = ...; but I do see a few places where we opt for the latter and are inconsistent as a result. For relatively short error messages such as this one, I don't personally think it's harder to read when the error message is inlined but I don't feel too strongly. I'll make the change.

@@ -54,8 +59,7 @@ public static StatementExecutorResponse execute(
final List<String> errors = validate(createConnector, client);
if (!errors.isEmpty()) {
final String errorMessage = "Validation error: " + String.join("\n", errors);
return StatementExecutorResponse.handled(Optional.of(new ErrorEntity(
statement.getStatementText(), errorMessage)));
throw new KsqlException(errorMessage);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming client connection failed here thus KsqlException instead of KsqlRestException?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite -- KsqlException means it's a user error, which is necessarily the case if there's a validation error but not necessarily the case if other errors are encountered.

The final exception type returned from the engine is always KsqlRestException. KsqlException is automatically wrapped in KsqlRestException here:

} catch (final KsqlRestException e) {
LOG.info("Processed unsuccessfully: " + request + ", reason: ", e);
throw e;
} catch (final KsqlStatementException e) {
LOG.info("Processed unsuccessfully: " + request + ", reason: ", e);
return Errors.badStatement(e.getRawMessage(), e.getSqlStatement());
} catch (final KsqlException e) {
LOG.info("Processed unsuccessfully: " + request + ", reason: ", e);
return errorHandler.generateResponse(e, Errors.badRequest(e));
} catch (final Exception e) {
LOG.info("Processed unsuccessfully: " + request + ", reason: ", e);
return errorHandler.generateResponse(
e, Errors.serverErrorForStatement(e, request.getKsql()));
}

For the non-validation connector errors, we don't know whether it's a user error or not without inspecting the error itself, which is why this PR simply returns a KsqlRestException directly rather than trying to figure out what inner error to throw in those cases.

Copy link
Contributor

@Gerrrr Gerrrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks good!

@vcrfxia vcrfxia force-pushed the remove-connect-error-entity branch from 730ebc8 to ea6780e Compare April 8, 2022 17:24
@vcrfxia vcrfxia merged commit 6e55521 into confluentinc:master Apr 8, 2022
@vcrfxia vcrfxia deleted the remove-connect-error-entity branch April 8, 2022 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants