Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fixed nondeterminism in UdfIndex #7719

Merged

Conversation

Sullivan-Patrick
Copy link
Contributor

@Sullivan-Patrick Sullivan-Patrick commented Jun 23, 2021

Description

When working on LEAST and GREATEST functions, I noticed there was some nondeterminism inside the UdfIndex class, specifically with the order field, which was intended to give comparably equal functions a final ordering based in the way they were added inside the Udf's class. This wasn't working in practice, and was leading to equally comparable methods that matched a query being selected arbitrarily. For example, when calling LEAST(INT, LONG), several UDFs could be selected: LEAST(LONG, LONG...), LEAST(DOUBLE, DOUBLE...), LEAST(DECIMAL, DECIMAL).

To fix this, UdfIndex.Node.compare() was changed to no longer consider order. As this change implies we are no longer able to select a matching function if multiple exist, getFunction now throws a KsqlException if there is more than one equally comparable method that would match the query.

A minor change that was made on top of this is adding to UdfIndex.Node.compare() the preference of picking methods with fewer generics as a final comparison.

BREAKING CHANGE: Existing queries that relied on vague implicit casting will not be started after an upgrade, and new queries that rely on vague implicit casting will be rejected. For example, foo(INT, INT) will not be able to resolve against two underlying function signatures of foo(BIGINT, BIGINT) and foo(DOUBLE, DOUBLE). Calling a function whose only parameter is variadic with an explicit null will also result in the call being rejected as vague.

Testing done

Added to UDFIndexTest negative tests which ensure that queries which would rely on vague implicit casting fail properly.

Some tests that relied on vague implicit casting may need to be removed, this is worth discussion.

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@Sullivan-Patrick Sullivan-Patrick requested a review from a team as a code owner June 23, 2021 15:59
@ghost
Copy link

ghost commented Jun 23, 2021

@confluentinc It looks like @Sullivan-Patrick just signed our Contributor License Agreement. 👍

Always at your service,

clabot

Copy link
Contributor

@vcrfxia vcrfxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Sullivan-Patrick ! LGTM besides the failing unit test and updating the "breaking change" description as we discussed offline. Great stuff.

Copy link
Member

@JimGalasyn JimGalasyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@vcrfxia vcrfxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Sullivan-Patrick . This is great! One minor comment inline.

Also a suggestion to improve the "breaking change" message in your PR description (which is also great, btw): let's clarify what it means for "Functions that relied on vague implicit casting" to "no longer work." To "no longer work" means that any new queries will be rejected, and any existing queries will not be started after an upgrade.

LGTM otherwise!

Copy link
Contributor

@vcrfxia vcrfxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Sullivan-Patrick ! 🎉

@vcrfxia vcrfxia merged commit cd1a988 into confluentinc:master Jun 28, 2021
@Sullivan-Patrick Sullivan-Patrick deleted the change-udfindex-behavior branch July 1, 2021 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants