-
Notifications
You must be signed in to change notification settings - Fork 533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Spark 3.5 support #514
Conversation
@@ -59,7 +59,7 @@ private[sql] case class StatefulHyperloglogPlus( | |||
|
|||
override def dataType: DataType = BinaryType | |||
|
|||
override def aggBufferSchema: StructType = StructType.fromAttributes(aggBufferAttributes) | |||
override def aggBufferSchema: StructType = DataTypeUtils.fromAttributes(aggBufferAttributes) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
StructType.fromAttributes
has been refactored into DataTypeUtils.fromAttributes
Reference: apache/spark@8ff6b7a
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good, it would be great if we could approve and merge this so that we could use it with V3.5
@eycho-am Would you mind to take a look on this, thanks again! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for this change. Will merge if the tests pass.
Issue #, if available:
#507
Description of changes:
mvn clean package
run successfully on my local machine. Test result is as followed:By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.