Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BIGTOP-4084. Upgrade Tez to 0.10.3. #1246

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

iwasakims
Copy link
Member

https://issues.apache.org/jira/browse/BIGTOP-4084

This PR bumps the verison of Tez to 0.10.3. We can remove most of the patch needed for Tez 0.10.2.

  • TEZ-4492, TEZ-4493, TEZ-4520 are contained in Tez 0.10.3.
  • We do not need TEZ-4319 since snappy-java-1.1.8.2 (supporting aarch64) is pulled as transitive dependency of hadoop now.

@iwasakims
Copy link
Member Author

I got protobuf related error on smoke-tests of Tez. I'm looking into the cause.

java.lang.IllegalAccessError: tried to access field com.google.protobuf.AbstractMessage.memoizedSize from class org.apache.tez.dag.api.records.DAGProtos$ConfigurationProto,

@iwasakims
Copy link
Member Author

While hadoop uses shaded protobuf now, Hadoop 3.3.6 retains dependency on protobuf-java-2.5.0 for compatibility and the protobuf-java-2.5.0.jar is in the classpath. Since TEZ-4363 upgraded the protobuf to 3.21., Tez must use its own protobuf-java-3.21.1.jar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant