Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump iceberg to 0.14 #4738

Merged
merged 7 commits into from
Jul 20, 2022
Merged

Bump iceberg to 0.14 #4738

merged 7 commits into from
Jul 20, 2022

Conversation

snazy
Copy link
Member

@snazy snazy commented Jul 19, 2022

No description provided.

@snazy snazy requested a review from dimas-b July 19, 2022 17:12
@snazy snazy self-assigned this Jul 19, 2022
@snazy snazy changed the title Bump iceberg 0.14 Bump iceberg to 0.14 Jul 19, 2022
@codecov
Copy link

codecov bot commented Jul 19, 2022

Codecov Report

Merging #4738 (868c883) into main (52de336) will decrease coverage by 1.03%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #4738      +/-   ##
============================================
- Coverage     85.63%   84.60%   -1.04%     
+ Complexity     3604     3533      -71     
============================================
  Files           415      415              
  Lines         16974    16974              
  Branches       1453     1453              
============================================
- Hits          14536    14361     -175     
- Misses         1939     2120     +181     
+ Partials        499      493       -6     
Flag Coverage Δ
java 84.82% <100.00%> (-1.16%) ⬇️
javascript 81.98% <ø> (ø)
python 82.86% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...va/org/apache/iceberg/nessie/NessieExtCatalog.java 89.65% <ø> (-0.35%) ⬇️
...a/org/apache/iceberg/nessie/NessieViewCatalog.java 61.42% <100.00%> (+0.55%) ⬆️
.../org/projectnessie/server/authz/CelAuthorizer.java 0.00% <0.00%> (-100.00%) ⬇️
...rojectnessie/server/rest/RegisterObjectMapper.java 0.00% <0.00%> (-100.00%) ⬇️
...sie/server/filters/SinglePageAppRoutingFilter.java 0.00% <0.00%> (-100.00%) ⬇️
...ctnessie/server/authn/NessieHttpAuthenticator.java 0.00% <0.00%> (-94.12%) ⬇️
...jectnessie/server/authz/CelBatchAccessChecker.java 0.00% <0.00%> (-91.08%) ⬇️
...essie/server/authz/CompiledAuthorizationRules.java 0.00% <0.00%> (-82.61%) ⬇️
...essie/versioned/BackendLimitExceededException.java 0.00% <0.00%> (-50.00%) ⬇️
...essie/services/rest/ValidationExceptionMapper.java 12.50% <0.00%> (-37.50%) ⬇️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 52de336...868c883. Read the comment docs.

Copy link
Member

@dimas-b dimas-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, one minor comment

Comment on lines 108 to 109
// TODO only used in a single test
// tableCatalog.refresh();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah - that refresh() functionality is gone now (on NessieCatalog since apache/iceberg#4491).
Updated the comment, but leaving it as a todo as a reminder.

/cc @nastra

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeah the refresh() shouldn't be necessary anymore. There was an issue back then that required calling this explicitly but I fixed this with the refactoring to the NessieClient wrapper in Iceberg

@@ -82,7 +82,8 @@ public void initialize(String inputName, Map<String, String> options) {
x -> x.replace(NessieUtil.NESSIE_CONFIG_PREFIX, "");

this.api =
createNessieClientBuilder(options.get(NessieUtil.CONFIG_CLIENT_BUILDER_IMPL))
createNessieClientBuilder(
options.get(NessieUtil.NESSIE_CONFIG_PREFIX + "client-builder-impl"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should we use the existing NessieConfigConstants.CONF_NESSIE_CLIENT_BUILDER_IMPL?

@@ -69,8 +73,10 @@ dependencies {
testImplementation(project(":nessie-spark-extensions-base")) { testJarCapability() }
testImplementation(project(":nessie-spark-extensions"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Is it good idea to rename nessie-spark-extensions to nessie-spark-3.1-extensions (just to have a similar syntax as spark-3.2 module)? Now reader has to figure out which version of spark is nessie-spark-extensions

@@ -104,11 +105,15 @@ public void setConf(Configuration conf) {
}

public void refresh() throws NessieNotFoundException {
tableCatalog.refresh();
// TODO This NessieExtCatalog is a *hack* to get some Iceberg view support into Nessie.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the whole method can be just removed

@snazy snazy merged commit 76036f4 into projectnessie:main Jul 20, 2022
@snazy snazy deleted the bump-iceberg-0.14 branch July 20, 2022 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants