Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring to use schema type for comparison. #614

Merged

Conversation

hjafarpour
Copy link
Contributor

This PR will unify schema checking and uses schema type in all places that we compare schemas.

@hjafarpour hjafarpour self-assigned this Jan 11, 2018
Copy link
Contributor

@rodesai rodesai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@@ -137,13 +137,13 @@ private Schema getKsqlType(final String sqlType) {
}

private String getKsqlTypeInJson(final Schema schemaType) {
if (schemaType == Schema.INT64_SCHEMA) {
if (schemaType.type() == Schema.Type.INT64) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use switch statement here instead a bunch of IFs? To make a code style consistent with what has been done earlier in the PR.

Copy link
Contributor

@dguy dguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have lots of code that is doing switch or if on the schema.type() - this is a bit of a code smell IMO. Can we do this in a more OO way? i.e, maybe a registry or similar where we get an object for a schema.type(), where the object has methods on it like enforceFieldType etc?

@@ -168,8 +168,8 @@ public void testSimpleAggregateLogicalPlan() throws Exception {
Assert.assertTrue(aggregateNode.getGroupByExpressions().size() == 1);
Assert.assertTrue(aggregateNode.getGroupByExpressions().get(0).toString().equalsIgnoreCase("TEST1.COL0"));
Assert.assertTrue(aggregateNode.getRequiredColumnList().size() == 2);
Assert.assertTrue(aggregateNode.getSchema().fields().get(1).schema() == Schema.FLOAT64_SCHEMA);
Assert.assertTrue(aggregateNode.getSchema().fields().get(2).schema() == Schema.INT64_SCHEMA);
Assert.assertTrue(aggregateNode.getSchema().fields().get(1).schema().type() == Schema.Type.FLOAT64);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we change these to use assertThat(..)

Assert.assertTrue(projectedSchemaKStream.getSchema().field("COL0").schema() == Schema.INT64_SCHEMA);
Assert.assertTrue(projectedSchemaKStream.getSchema().field("COL2").schema() == Schema.STRING_SCHEMA);
Assert.assertTrue(projectedSchemaKStream.getSchema().field("COL3").schema() == Schema.FLOAT64_SCHEMA);
Assert.assertTrue(projectedSchemaKStream.getSchema().field("COL0").schema().type() == Schema.Type.INT64);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Assert.assertTrue(exprType0 == Schema.BOOLEAN_SCHEMA);
Assert.assertTrue(exprType1 == Schema.BOOLEAN_SCHEMA);
Assert.assertTrue(exprType2 == Schema.BOOLEAN_SCHEMA);
Assert.assertTrue(exprType0.type() == Schema.Type.BOOLEAN);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link

@bluemonk3y bluemonk3y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remodelling as a enum would simplify the code and mean you can drop the switch and if statements. see comments below.

} else {
throw new KsqlFunctionException(
"Invalid cast operation: Cannot cast " + exprStr + " to Integer.");
switch (schema.type()) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to push down this logic into Schema.TYPE enums? - i.e. implement castToIntegerString() on each enum? You also get comparitors and a bunch of other stuff for free.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no, we are using Schema class from connect so no control on it!

} else {
throw new KsqlException("Type is not supported: " + schema);

switch (schema.type()) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could also be push down do an enum on Schema.INT32.enforceInteger() etc - then you dont need this class

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no, we are using Schema class from connect so we cannot change it!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hjafarpour - I still dont see more OO modelling in the latest commit...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I forgot to submit my comments yesterday. @bluemonk3y as I mentioned above we use Schema class from Connect package so we don't have control on the class implementation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can create our own class to do this - i.e., one that wraps the Schema

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 - the code with have in here needs a serious refactor

Copy link
Contributor

@dguy dguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - we should follow up with a refactor to remove the switch and if blocks

@hjafarpour hjafarpour merged commit 32519dd into confluentinc:master Jan 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants