[HUDI-3588] Remove hudi-common and hudi-hadoop-mr jars in Presto Docker image #4997
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nsivabalan
approved these changes
Mar 11, 2022
There was a problem hiding this comment.
LGTM.
I assume from a presto standpoint, we will never go back to not using hudi-presto-bundle and hence the proposed fixed is ok for me.
But lets get it stamped with @codope once.
5 tasks
codope
approved these changes
Mar 16, 2022
vingov
pushed a commit
to vingov/hudi
that referenced
this pull request
Apr 3, 2022
stayrascal
pushed a commit
to stayrascal/hudi
that referenced
this pull request
Apr 12, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the pull request
The latest master of Presto relies on hudi-presto-bundle, while current Presto releases rely on hudi-common and hudi-hadoop-mr 0.9.0, which are pulled in plugin/hive-hadoop2/ in the Docker setup, making it hard to test the latest changes in Hudi due to class conflict.
As a temporary workaround, this PR removes
hudi-common-0.9.0.jarandhudi-hadoop-mr-0.9.0.jarinplugin/hive-hadoop2/to avoid conflicts with latest hudi-presto-bundle.jar, which is copied from the build. This PR also upgrades Presto to 0.271, so that the new presto docker image is pushed toapachehudi/hudi-hadoop_2.8.4-prestobase_0.271:latest. In this way, if there is any issue with the changes in this PR, we can roll this back without any side effect.Brief change log
hudi-common-0.9.0.jarandhudi-hadoop-mr-0.9.0.jarwithinplugin/hive-hadoop2/indocker/hoodie/hadoop/prestobase/Dockerfileapachehudi/hudi-hadoop_2.8.4-prestobase_0.271:latestVerify this pull request
This pull request is already covered by existing integration tests.
Committer checklist
Has a corresponding JIRA in PR title & commit
Commit message is descriptive of the change
CI is green
Necessary doc changes done or have another open PR
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.