Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert Google HTTP Client upgrade #1980

Merged
merged 3 commits into from
Sep 12, 2019
Merged

Conversation

chanseokoh
Copy link
Member

@chanseokoh chanseokoh commented Sep 12, 2019

Fixes the regression of slow network operations introduced at Jib 1.5.0. It is supposed fix multiple issues described in #1970, #1946, #1917, and the Jib users group.


This PR reverts #1882 that upgraded the Google HTTP Client library. I started with git revert, so the Java source file changes are not manual. (I just retained the most recent code comments.)

About dependencies, I just downgraded Google HTTP Client and left others like Guava and the recent google-auth-library-oauth2-http. No particular reason.

@loosebazooka let's take a close look at the changes. And why is it that jib-maven-plugin/build.gradle doesn't have the HTTP Client dependency and some others? jib-gradle-plugin and jib-plugins-common have it.

I'll run integration test locally.

@loosebazooka
Copy link
Member

@loosebazooka let's take a close look at the changes. And why is it that jib-maven-plugin/build.gradle doesn't have the HTTP Client dependency and some others? jib-gradle-plugin and jib-plugins-common have it.

honestly, I don't know, but it looks like jib-gradle-plugin doesn't need it. It must've happened during a rebase, but we can remove it and build still works.

but jib-gradle-plugin also has no explicit dependency on guava or databind, so we can remove those too

implementation("com.google.http-client:google-http-client-apache-v2:${dependencyVersions.GOOGLE_HTTP_CLIENT_APACHE_V2}") {
exclude group: "org.apache.httpcomponents", module: "httpclient"
}
implementation "com.google.http-client:google-http-client:${dependencyVersions.GOOGLE_HTTP_CLIENT}"
implementation("com.google.auth:google-auth-library-oauth2-http:${dependencyVersions.GOOGLE_AUTH_LIBRARY_OAUTH2_HTTP}") {
exclude group: "org.apache.httpcomponents", module: "httpclient"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this still create a conflict?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realized it pulls in google-http-client and upgrading 1.27.0 to 1.30.1, so I think I should exclude google-http-client.

+--- com.google.auth:google-auth-library-oauth2-http:0.16.2 
|    +--- com.google.auth:google-auth-library-credentials:0.16.2
|    +--- com.google.http-client:google-http-client:1.30.1 (*)
|    +--- com.google.http-client:google-http-client-jackson2:1.30.1
|    |    +--- com.google.http-client:google-http-client:1.30.1 (*)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if that'll break at runtime then?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/googleapis/google-cloud-java/blob/v0.80.0/google-cloud-clients/pom.xml recommends

guava.version 26.0 (down from 28.0)
http-client.version 1.27.0 (from 1.31.0)
google.auth.version 0.12.0 (from 0.16.2)

I'll first trying excluding google-http-client from google-auth-library-oauth2-http rather than downgrading it. Will do some test if ADC works on GCB.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ADC on GCB works at least.

exclude group: "org.apache.httpcomponents", module: "httpclient"
}
implementation "org.apache.httpcomponents:httpclient:${dependencyVersions.APACHE_HTTP_CLIENT_OVERRIDE}"
implementation "com.google.http-client:google-http-client:${dependencyVersions.GOOGLE_HTTP_CLIENT}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can remove this, and guava, and jackson

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can there be any possibility that removing these is OK with compilation but causes trouble when the user actually uses our plugin by defining Jib in their builds, like not pulling in required dependencies at their run-time?

Copy link
Member

@loosebazooka loosebazooka Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those dependencies are inherited as runtime implementation from the parent modules.

Copy link
Member

@loosebazooka loosebazooka Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the reason dependencies still need to be jib-plugins-common is because that library doesn't bring in jib-core as a "sourceProject" (which is defined by us as included). This might be easier to explain in person though since it's all proprietary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

although now that I look at it again, it's pretty dumb, jib-plugins-common shouldn't have to redefine those dependencies. There must be a better way, I'll fix that later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And as of now, jib-plugins-common defines google-http-client, guava, and jackson-databind but not google-auth-library-oauth2-http. Why is that?

Copy link
Member

@loosebazooka loosebazooka Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's because it is defined as a regular project dependency (the gradle) way, it doesn't need to include google-auth.... because it doesn't use it.

a dependency defined as implementation on a project is exposed to dependency project as runtime. So they can't use it as a compile time dependency unless they redefine it. The way sourceProject works is that it takes anything defined as implementation and makes it available as implementation on the dependent project as well.

Given that this is confusing, it might be an area to improve the build.

@chanseokoh
Copy link
Member Author

chanseokoh commented Sep 12, 2019

I think this works. ./gradlew :jib-core:dependencies (and such) shows we use 1.27.0. The oauth2 library (pulled in when we added the ADC support) depends on google-http-client, but ADC on GCB works in my test.

@chanseokoh chanseokoh requested a review from a team September 12, 2019 19:15
Copy link
Member

@loosebazooka loosebazooka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants