Fix remote spec handling and hash calculation #3440

johngray · 2019-07-24T13:45:51Z

PR checklist

Read the contribution guidelines.
Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

Current implementataion of maven plugin has the feature of skipping generation if no changes was detected in schema file. While the skipping itself is applied only when the skipIfSpecIsUnchanged option is present in the configuration file - hash calculation is calculated every time:

File storedInputSpecHashFile = getHashFile(inputSpecFile);
ByteSource inputSpecByteSource =
    inputSpecFile.exists()
        ? Files.asByteSource(inputSpecFile)
        : CharSource.wrap(ClasspathHelper.loadFileFromClasspath(inputSpecFile.toString().replaceAll("\\\\","/")))
                        .asByteSource(Charsets.UTF_8);
String  inputSpecHash =inputSpecByteSource.hash(Hashing.sha256()).toString();

This makes impossible to use remote spec files in pom configuration. FileNotFoundException is thrown at the end of generation phase despite that all classes generated fine. Generator itself are working great with remote specs.

My proposition in this PR is to fix this issue and allow the generator to work with remote spec files:

URL inputSpecRemoteUrl = inputSpecRemoteUrl();
File inputSpecTempFile = inputSpecFile;
if (inputSpecRemoteUrl != null) {
    inputSpecTempFile = File.createTempFile("openapi-spec", ".tmp");
    ReadableByteChannel readableByteChannel = Channels.newChannel(inputSpecRemoteUrl.openStream());
    
    FileOutputStream fileOutputStream = new FileOutputStream(inputSpecTempFile);
    FileChannel fileChannel = fileOutputStream.getChannel();

    fileChannel.transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
}

ByteSource inputSpecByteSource =
    inputSpecTempFile.exists()
        ? Files.asByteSource(inputSpecTempFile)
        : CharSource.wrap(ClasspathHelper.loadFileFromClasspath(inputSpecTempFile.toString().replaceAll("\\\\","/"))).asByteSource(Charsets.UTF_8);

return inputSpecByteSource.hash(Hashing.sha256()).toString();

To fix #1792

wing328 · 2019-07-30T14:05:41Z

Thanks for the PR but your commit (as shown in the Commits tab) is not linked to your Github account, which means this PR won't count as your contribution in https://github.com/OpenAPITools/openapi-generator/graphs/contributors.

Let me know if you need help fixing it.

Ref: https://github.com/OpenAPITools/openapi-generator/wiki/FAQ#how-can-i-update-commits-that-are-not-linked-to-my-github-account

wing328 · 2019-07-30T14:06:44Z

If you've time, please also add docstring to the new functions you created.

johngray · 2019-07-31T07:34:48Z

@wing328 Hi. I fixed the commit author and added docstrings

sergey-v-sedov · 2019-08-06T08:59:00Z

@wing328 Hi, William! Please, approve this pool request asap, I'm interesting in it too. Thanks!

This reverts commit 2a2eefe.

wing328 · 2019-09-04T11:03:20Z

It's causing build failure: https://travis-ci.org/OpenAPITools/openapi-generator/builds/580608572

[ERROR] /home/travis/build/OpenAPITools/openapi-generator/modules/openapi-generator-maven-plugin/examples/swagger.yaml [0:0]: unexpected error in Open-API generation
java.lang.IllegalArgumentException: URI is not absolute
    at java.net.URI.toURL (URI.java:1088)
    at org.openapitools.codegen.plugin.CodeGenMojo.inputSpecRemoteUrl (CodeGenMojo.java:793)
    at org.openapitools.codegen.plugin.CodeGenMojo.getHashFile (CodeGenMojo.java:810)
    at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:731)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:134)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:309)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR] 
java.lang.IllegalArgumentException: URI is not absolute
    at java.net.URI.toURL (URI.java:1088)
    at org.openapitools.codegen.plugin.CodeGenMojo.inputSpecRemoteUrl (CodeGenMojo.java:793)
    at org.openapitools.codegen.plugin.CodeGenMojo.getHashFile (CodeGenMojo.java:810)
    at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:731)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:134)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:309)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

I've reverted the change via 03edb64 for the time being:

(cherry picked from commit 2a2eefe)

…hecking file updates. Following up OpenAPITools#3440, supporting auth.

…ation) (#3826) * This patch fixes the bug that we cannot access to remote files when checking file updates. Following up #3440, supporting auth. * 1792 fix remote spec handling and hash calculation (#3440) (cherry picked from commit 2a2eefe) * fix detecting remote file / local file logic while finding the hash file, taking care of IllegalArgumentException for local files. * add testcase

…ols#3440)" This reverts commit 2a2eefe.

… hash calculation) (OpenAPITools#3826) * This patch fixes the bug that we cannot access to remote files when checking file updates. Following up OpenAPITools#3440, supporting auth. * 1792 fix remote spec handling and hash calculation (OpenAPITools#3440) (cherry picked from commit 2a2eefe) * fix detecting remote file / local file logic while finding the hash file, taking care of IllegalArgumentException for local files. * add testcase

wing328 · 2019-09-12T14:00:03Z

@johngray thanks for the PR, which has been included in the v4.1.2 release: https://twitter.com/oas_generator/status/1172068944355528705

* First version of Nim Client * Add some codes * Add some codes * Add some codes * Add some codes * Add some codes * First version of Nim Client * Add some codes * Add some codes * [Dart] Fix README template and update testing doco (#3809) * [Dart] Fix README template and update testing doco - deleted redundant shell script - fixed and updated README template - updated test package and moved to a dev_dependency - removed old unused dev_dependency packages - updated testing documentation in petstore sample * Remove references to dart-flutter-petstore.sh * Fix typos * Fix typo * Support custom git repository (#3757) * add gitHost param to GeneratorSettings and related * parameterize gitHost in READMEs * parameterize gitHost in go.mod * parameterize gitHost in git_push * update petstore samples * run ./bin/utils/export_docs_generators.sh * run meta-codehen.sh * Revert "run meta-codehen.sh" This reverts commit d6d579f. * Revert "run ./bin/utils/export_docs_generators.sh" This reverts commit 1b81538. * Revert "update petstore samples" This reverts commit f513add. * run ensure-up-to-date * Add links to article and video (#3820) * Better Go code format (#3819) * better varible naming * better comments * better code format for go experimental client * better comment, update samples * Add some codes * Add some codes * Add some codes * Add gRPC Protobuf schema generator (#3818) * add grpc protobuf generator * update doc * add new doc * add windows batch, comment out root proto * 1792 fix remote spec handling and hash calculation (#3440) * fixed bug where nullApi.java would be generated. Instead, generated DefaultApi.java to match the default path /{pathParam} (#3821) * Revert "1792 fix remote spec handling and hash calculation (#3440)" This reverts commit 2a2eefe. * Add nickmeinhold to Dart technical committee (#3830) * Bug #2845 typescript angular inheritance (#3812) * issue #2845: enable 'supportsMultipleInheritance' on typescript angular client codegen - note I reran ./bin/openapi3/typescript-angular-petstore-all.sh and no changes occurred. this suggests to me that the petstore.yaml sample should be improved to make use of the anyOf / allOf / oneOf keywords, in order to better show the effects of changes on generated code. * issue #2845: run ./bin/openapi3/typescript-angular-petstore-all.sh * run `mvn clean package && ./bin/typescript-angular-petstore-all.sh` * revert extranous files * fix warnings in csharp-netcore client (#3831) * Add missing files to the form request (#3834) * [client][go] avoid duplicated reflect imports (#3847) * Following up for #3440 (1792 fix remote spec handling and hash calculation) (#3826) * This patch fixes the bug that we cannot access to remote files when checking file updates. Following up #3440, supporting auth. * 1792 fix remote spec handling and hash calculation (#3440) (cherry picked from commit 2a2eefe) * fix detecting remote file / local file logic while finding the hash file, taking care of IllegalArgumentException for local files. * add testcase * Add a link (#3850) * Add Element AI to the list (#3856) * maven-plugin-plugin 3.6.0 (#3854) * [Java][okhttp-gson] fix failure to deserialize floats (#3846) * fixed bug where nullApi.java would be generated. Instead, generated DefaultApi.java to match the default path /{pathParam} * fix to bug #3157 * update samples * Adds Http Info To Dart Api (#3851) * [C++][Pistache] Add missing setter for arrays (#3837) * [C++][Pistache] Add missing setter for arrays Fixes #3769 * [C++][Pistache] Update Petstore sample * typescript-inversify: improve check for required parameters, support multiple media types (#3849) * [typescript-inversify] Allow falsy parameters A required parameter to an api method must not be `null` or `undefined`. It can be any other falsy value, e.g. `""`, `0` or `false` though. This change makes sure an error is only thrown in the former case and not in the latter. * [typescript-inversify] Handle multiple media types The Accept and Content-Type HTTP headers can contain a list of media types. Previously all but the first media type in the api definition were ignored. Now the headers are properly generated. * [typescript-inversify] Fix http client interface The api service methods allow the `body` parameter to be optional. The parameter is then passed to an `IHttpClient`. So it needs to be optional there as well. Also fixed the sample implementation `HttpClient`. Fixes #3618. * [typescript-inversify] Regenerate Petstore sample * [typescript-inversify] Use more explicit null check This does not change the semantic of the generated code, but makes it more explicit. Co-Authored-By: Esteban Gehring <[email protected]> * [typescript-angular] allow empty string basePath (#3489) * [typescript-angular] Fixing #2731 - empty string basePath * typescript-angular: refactor base path configuration * typescript-angular: refactor base path configuration * Fix/r/serialization fix and minor 3xx resp fix (#3817) * fix(qlik): fix for minor serialization bug * fix(r): add petsore generated classes * fix(r): indendation fixes * typescript-axios: Fix baseoptions (#3866) * Fixed missing baseOptions of typescript-axios. The typescript-axios template was missing the baseOptions setting when building an API Configuration. Set it. * update sample. * re-generate typescript axios samples * Rename gRPC generator to "protobuf-schema" (#3864) * rename grpc generator to protobuf-schema * update doc * Prepare v4.1.2 release (#3873) * update samples * update date * fix version in readme * BugFix #2053 Spring Boot fails to parse LocalDate query parameter (#3860) Adds the format annotation so that Spring is able to serialize OpenApi date/date-time format into LocalDate/OffsetDateTime. * update doc, samples (#3875) * update stable release * Update the batch for Windows * Add a test snippet * Update ensure-up-to-date * Add Nim to README.md * Ran ensure-up-to-date to pass CircleCI tests

1792 fix remote spec handling and hash calculation

e9e6043

fujigon mentioned this pull request Sep 4, 2019

Fix remote spec handling and hash calculation #3826

Merged

4 tasks

wing328 added Issue: Bug OpenAPI Generator Maven Plugin labels Sep 4, 2019

wing328 added this to the 4.1.2 milestone Sep 4, 2019

wing328 merged commit 2a2eefe into OpenAPITools:master Sep 4, 2019

wing328 mentioned this pull request Sep 4, 2019

[BUG][Java][spring][maven] FileNotFoundException when spec file is present in another jar #1792

Closed

wing328 changed the title ~~1792 fix remote spec handling and hash calculation~~ Fix remote spec handling and hash calculation Sep 4, 2019

wing328 added a commit that referenced this pull request Sep 4, 2019

Revert "1792 fix remote spec handling and hash calculation (#3440)"

03edb64

This reverts commit 2a2eefe.

fujigon pushed a commit to fujigon/openapi-generator that referenced this pull request Sep 6, 2019

1792 fix remote spec handling and hash calculation (OpenAPITools#3440)

acacd35

(cherry picked from commit 2a2eefe)

fujigon added a commit to fujigon/openapi-generator that referenced this pull request Sep 6, 2019

This patch fixes the bug that we cannot access to remote files when c…

042e176

…hecking file updates. Following up OpenAPITools#3440, supporting auth.

RockyMM mentioned this pull request Sep 9, 2019

[BUG] Maven-plugin - Build fails if inputSpecs are a file from a remote URL #3841

Closed

hokamoto pushed a commit to hokamoto/openapi-generator that referenced this pull request Sep 12, 2019

1792 fix remote spec handling and hash calculation (OpenAPITools#3440)

bdfed92

hokamoto pushed a commit to hokamoto/openapi-generator that referenced this pull request Sep 12, 2019

Revert "1792 fix remote spec handling and hash calculation (OpenAPITo…

99eb680

…ols#3440)" This reverts commit 2a2eefe.

BitcoinOutput mentioned this pull request Nov 29, 2023

[Snyk] Security upgrade karma from 0.13.22 to 5.0.0 BitcoinOutput/openapi-generator#275

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix remote spec handling and hash calculation #3440

Fix remote spec handling and hash calculation #3440

johngray commented Jul 24, 2019 •

edited by wing328

Loading

wing328 commented Jul 30, 2019

wing328 commented Jul 30, 2019

johngray commented Jul 31, 2019

sergey-v-sedov commented Aug 6, 2019

wing328 commented Sep 4, 2019

wing328 commented Sep 12, 2019

Fix remote spec handling and hash calculation #3440

Fix remote spec handling and hash calculation #3440

Conversation

johngray commented Jul 24, 2019 • edited by wing328 Loading

PR checklist

Description of the PR

wing328 commented Jul 30, 2019

wing328 commented Jul 30, 2019

johngray commented Jul 31, 2019

sergey-v-sedov commented Aug 6, 2019

wing328 commented Sep 4, 2019

wing328 commented Sep 12, 2019

johngray commented Jul 24, 2019 •

edited by wing328

Loading