[HUDI-2662] Downloads from Nexus Pentaho repo taking too long #3901

codope · 2021-11-01T13:09:31Z

What is the purpose of the pull request

https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release is unavailable to serve any requests. We are removing the url from pom as we confirmed that all jars are available in other nexus repository urls.

Brief change log

(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request

(Please pick either of the following options)

This pull request is a trivial rework / code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

Has a corresponding JIRA in PR title & commit
Commit message is descriptive of the change
CI is green
Necessary doc changes done or have another open PR
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

hudi-bot · 2021-11-01T13:13:40Z

CI report:

4d86b0e0bb3c4db66b5c098d4777590dd1c24128 Azure: FAILURE
d84e6eb Azure: PENDING

Bot commands

@hudi-bot supports the following commands:

@hudi-bot run travis re-run the last Travis build
@hudi-bot run azure re-run the last Azure build

nsivabalan · 2021-11-01T18:02:28Z

I inspected every jar thats getting delayed with pentaho. I could confirm that we can find every jar of interest in other nexus domains.


https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/flink/force-shading/1.12.1/force-shading-1.12.1-sources.jar

https://repo.maven.apache.org/maven2/org/apache/flink/force-shading/1.12.1/

https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/flink/flink-shaded-guava/18.0-12.0/flink-shaded-guava-18.0-12.0-sources.jar

https://repo.maven.apache.org/maven2/org/apache/flink/flink-shaded-guava/18.0-12.0/


Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/flink/flink-shaded-asm-7/7.1-12.0/flink-shaded-asm-7-7.1-12.0-sources.jar

https://repo.maven.apache.org/maven2/org/apache/flink/flink-shaded-asm-7/7.1-12.0/


Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/flink/force-shading/1.13.1/force-shading-1.13.1-sources.jar

https://repo.maven.apache.org/maven2/org/apache/flink/force-shading/1.13.1/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23-sources.jar
https://repo.maven.apache.org/maven2/tomcat/jasper-runtime/5.5.23/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/xmlenc/xmlenc/0.52/xmlenc-0.52-sources.jar
https://repo.maven.apache.org/maven2/xmlenc/xmlenc/0.52/


Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/stax/stax-api/1.0.1/stax-api-1.0.1-sources.jar
https://repo.maven.apache.org/maven2/stax/stax-api/1.0.1/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0-sources.jar
https://repo.maven.apache.org/maven2/commons-beanutils/commons-beanutils-core/1.8.0/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/hudi/hudi-hadoop-sparkworker-docker/0.10.0-SNAPSHOT/hudi-hadoop-sparkworker-docker-0.10.0-SNAPSHOT-sources.jar


Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/curator/apache-curator/2.7.1/apache-curator-2.7.1-sources.jar
https://repo.maven.apache.org/maven2/org/apache/curator/apache-curator/2.7.1/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/velocity/velocity/1.5/velocity-1.5-sources.jar
https://repo.maven.apache.org/maven2/org/apache/velocity/velocity/1.5/


Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/javax/servlet/jsp-api/2.0/jsp-api-2.0-sources.jar
https://repo.maven.apache.org/maven2/javax/servlet/jsp-api/2.0/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23-sources.jar
https://repo.maven.apache.org/maven2/tomcat/jasper-compiler/5.5.23/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3-sources.jar
https://repo.maven.apache.org/maven2/org/apache/thrift/libfb303/0.9.3/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9-sources.jar
https://repo.maven.apache.org/maven2/com/google/code/findbugs/jsr305/1.3.9/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/io/confluent/kafka-schema-registry-client/5.3.4/kafka-schema-registry-client-5.3.4-sources.jar
https://packages.confluent.io/maven/io/confluent/kafka-schema-registry-client/5.3.4/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/io/confluent/common-utils/5.3.4/common-utils-5.3.4-sources.jar
https://packages.confluent.io/maven/io/confluent/common-utils/5.3.4/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/io/confluent/common-config/5.3.4/common-config-5.3.4-sources.jar
https://packages.confluent.io/maven/io/confluent/common-config/5.3.4/

Downloading from pentaho.org: https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/io/confluent/kafka-avro-serializer/5.3.4/kafka-avro-serializer-5.3.4-sources.jar
https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.3.4/

Once the CI succeeds, we should be good to go ahead and land this in.

nsivabalan · 2021-11-01T18:46:50Z

@hudi-bot azure run

nsivabalan · 2021-11-01T20:10:58Z

@hudi-bot azure run

...i-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java

hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java

nsivabalan · 2021-11-01T22:10:40Z

@hudi-bot azure run

vinothchandar · 2021-11-02T01:46:58Z

hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java

@@ -165,6 +165,10 @@ private void initIfNeeded() {
          logRecords.put(entry.getKey(), entry.getValue());
        }
      }
+    } else {


@nsivabalan @codope why are these changes needed again?

a test was failing in CI and so I fixed it in this patch. Did not want to land this patch with failing CI. I thought of putting up a separate patch, but the other one will depend on this patch since tests might time out.

nsivabalan · 2021-11-02T02:22:42Z

I removed the commented out lines in root pom in here. sorry about that.

[MINOR] Downloads from Nexus Pentaho repo taking too long

2215ea5

Fixing test failure

ca4dfec

vinothchandar requested changes Nov 1, 2021

View reviewed changes

...i-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java Outdated Show resolved Hide resolved

hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java Outdated Show resolved Hide resolved

Adding log statements to triage a test failure:

d84e6eb

nsivabalan force-pushed the ignore-pentaho-repo branch from 4d86b0e to d84e6eb Compare November 1, 2021 22:08

nsivabalan changed the title ~~[MINOR] Downloads from Nexus Pentaho repo taking too long~~ [HUDI-2662] Downloads from Nexus Pentaho repo taking too long Nov 1, 2021

nsivabalan merged commit 5302b9a into apache:master Nov 1, 2021

vinothchandar reviewed Nov 2, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HUDI-2662] Downloads from Nexus Pentaho repo taking too long #3901

[HUDI-2662] Downloads from Nexus Pentaho repo taking too long #3901

codope commented Nov 1, 2021 •

edited by nsivabalan

Loading

hudi-bot commented Nov 1, 2021 •

edited

Loading

nsivabalan commented Nov 1, 2021 •

edited

Loading

nsivabalan commented Nov 1, 2021

nsivabalan commented Nov 1, 2021

nsivabalan commented Nov 1, 2021

vinothchandar Nov 2, 2021

nsivabalan Nov 2, 2021

nsivabalan commented Nov 2, 2021

[HUDI-2662] Downloads from Nexus Pentaho repo taking too long #3901

[HUDI-2662] Downloads from Nexus Pentaho repo taking too long #3901

Conversation

codope commented Nov 1, 2021 • edited by nsivabalan Loading

What is the purpose of the pull request

Brief change log

Verify this pull request

Committer checklist

hudi-bot commented Nov 1, 2021 • edited Loading

CI report:

nsivabalan commented Nov 1, 2021 • edited Loading

nsivabalan commented Nov 1, 2021

nsivabalan commented Nov 1, 2021

nsivabalan commented Nov 1, 2021

vinothchandar Nov 2, 2021

Choose a reason for hiding this comment

nsivabalan Nov 2, 2021

Choose a reason for hiding this comment

nsivabalan commented Nov 2, 2021

codope commented Nov 1, 2021 •

edited by nsivabalan

Loading

hudi-bot commented Nov 1, 2021 •

edited

Loading

nsivabalan commented Nov 1, 2021 •

edited

Loading