Fix remote spec handling and hash calculation #3826

fujigon · 2019-09-04T05:30:16Z

PR checklist

Read the contribution guidelines.
Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh, ./bin/openapi3/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\. If contributing template-only or documentation-only changes which will change sample output, be sure to build the project first.
Filed the PR against the correct branch: master, 4.1.x, 5.0.x. Default: master.
Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

This is a following up PR of #3440 to fix the #1792 bug, supporting auth.
Before review & merging this PR, please have a look at and merge #3440.

We have #1792 bug since 3.x, but we had workround at that time.
However the workaround doesn't work well for 4.x, and it still prevents us from upgrading to 4.x.

(details of the change, additional tests that have been done, reference to the issue for tracking, etc)

fujigon · 2019-09-04T05:31:45Z

@wing328 Could you kindly have a look at #3440 and this PR? Thanks!

wing328 · 2019-09-04T06:44:21Z

@fujigon thanks for the PR. Can you please PM me via Slack (https://join.slack.com/t/openapi-generator/shared_invite/enQtNzAyNDMyOTU0OTE1LTY5ZDBiNDI5NzI5ZjQ1Y2E5OWVjMjZkYzY1ZGM2MWQ4YWFjMzcyNDY5MGI4NjQxNDBiMTlmZTc5NjY2ZTQ5MGM) when you've time?

wing328 · 2019-09-04T10:46:49Z

~~#3440 has been merged into master~~

#3440 has been reverted as it's causing failure in CI build: https://travis-ci.org/OpenAPITools/openapi-generator/builds/580608572

fujigon · 2019-09-05T02:35:29Z

cherry picked the #3440
trying to figure out the problem of #3440

fujigon · 2019-09-05T05:00:59Z

The problem is occurred when mvn clean compile -f modules/openapi-generator-maven-plugin/examples/java-client.xml

[ERROR] /home/travis/build/OpenAPITools/openapi-generator/modules/openapi-generator-maven-plugin/examples/swagger.yaml [0:0]: unexpected error in Open-API generation
java.lang.IllegalArgumentException: URI is not absolute

wing328 · 2019-09-05T05:09:10Z

Yup, that's the test we added to the CI earlier to catch potential issues with the Maven plugin.

fujigon · 2019-09-05T06:08:18Z

I managed to reproduce the error
Environments

ubuntu 1804 as Windows Subsystem for Linux on Windows 10
openjdk 1.8.0_222 (not occurred in Amazon Correto 8)
mvn 3.5.4
commit be07ca6

to reproduce

mvn clean install
mvn clean compile -f modules/openapi-generator-maven-plugin/examples/java-client.xml

fujigon · 2019-09-05T07:08:45Z

...penapi-generator-maven-plugin/src/main/java/org/openapitools/codegen/plugin/CodeGenMojo.java

-        } catch (URISyntaxException e) {
-            return null;
-        } catch (MalformedURLException e) {
+        } catch (URISyntaxException | MalformedURLException | IllegalArgumentException e) {


This method is to get the URL if remote otherwise null.
when inputSpec is local file and relative path, IllegalArgumentException is thrown.
In this case, we can return null because this method is expected to return URL only when the inputSpec is remote.

(cherry picked from commit 2a2eefe)

…hecking file updates. Following up OpenAPITools#3440, supporting auth.

…ile, taking care of IllegalArgumentException for local files.

wing328 · 2019-09-06T06:59:00Z

I did a test locally with the following command

mvn clean compile -f modules/openapi-generator-maven-plugin/examples/java-client.xml

and replace the inputSpec with https://raw.githubusercontent.com/OpenAPITools/openapi-generator/master/modules/openapi-generator/src/test/resources/2_0/petstore.yaml in java-client.xml

but got the following errors:

[INFO] writing file /Users/williamcheng/Code/openapi-generator/modules/openapi-generator-maven-plugin/examples/target/generated-sources/openapi/src/main/java/org/openapitools/client/auth/OAuthFlow.java
[INFO] writing file /Users/williamcheng/Code/openapi-generator/modules/openapi-generator-maven-plugin/examples/target/generated-sources/openapi/.openapi-generator-ignore
[INFO] writing file /Users/williamcheng/Code/openapi-generator/modules/openapi-generator-maven-plugin/examples/target/generated-sources/openapi/.openapi-generator/VERSION
[ERROR] /Users/williamcheng/Code/openapi-generator/https:/raw.githubusercontent.com/OpenAPITools/openapi-generator/master/modules/openapi-generator/src/test/resources/2_0/petstore.yaml [0:0]: unexpected error in Open-API generation
java.lang.RuntimeException: Could not find https:/raw.githubusercontent.com/OpenAPITools/openapi-generator/master/modules/openapi-generator/src/test/resources/2_0/petstore.yaml on the classpath
    at io.swagger.v3.parser.util.ClasspathHelper.loadFileFromClasspath (ClasspathHelper.java:31)
    at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:726)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR] 
java.lang.RuntimeException: Could not find https:/raw.githubusercontent.com/OpenAPITools/openapi-generator/master/modules/openapi-generator/src/test/resources/2_0/petstore.yaml on the classpath
    at io.swagger.v3.parser.util.ClasspathHelper.loadFileFromClasspath (ClasspathHelper.java:31)
    at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:726)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

fujigon · 2019-09-06T07:28:19Z

modules/openapi-generator-maven-plugin/examples/java-client.xml

@@ -39,6 +40,36 @@
                            <library>jersey2</library>
                        </configuration>
                    </execution>
+                    <execution>


I added the testcase you mentioned #3826 (comment).
But I couldn't reproduce the error in my environment.

mvn clean compile -f modules/openapi-generator-maven-plugin/examples/java-client.xml [INFO] Scanning for projects... [INFO] [INFO] ------------------< org.openapitools:sample-project >------------------- [INFO] Building sample-project 1.0-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ sample-project --- [INFO] Deleting /mnt/c/publicgithub/openapi-generator/modules/openapi-generator-maven-plugin/examples/target [INFO] [INFO] --- openapi-generator-maven-plugin:4.1.2-SNAPSHOT:generate (default) @ sample-project --- [WARNING] Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated. [INFO] OpenAPI Generator: java (client) [INFO] Generator 'java' is considered stable. ...... ...... [INFO] --- openapi-generator-maven-plugin:4.1.2-SNAPSHOT:generate (remote) @ sample-project --- [WARNING] Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated. [INFO] OpenAPI Generator: java (client) [INFO] Generator 'java' is considered stable. ...... ...... [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ sample-project --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /mnt/c/publicgithub/openapi-generator/modules/openapi-generator-maven-plugin/examples/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.6.1:compile (default-compile) @ sample-project --- [INFO] Changes detected - recompiling the module! [WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! [INFO] Compiling 46 source files to /mnt/c/publicgithub/openapi-generator/modules/openapi-generator-maven-plugin/examples/target/classes [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 49.616 s [INFO] Finished at: 2019-09-06T16:24:45+09:00 [INFO] ------------------------------------------------------------------------

Can you check your error will be reproduced with my latest commit including testcase?

mvn clean install mvn clean compile -f modules/openapi-generator-maven-plugin/examples/java-client.xml

Also waiting for how Shippable CI will judge the testcase.

I pull the latest again and no longer seeing the issue. Must be the setup in my end.

Thanks for adding the test case.

I'll merge after all CI tests pass.

… hash calculation) (OpenAPITools#3826) * This patch fixes the bug that we cannot access to remote files when checking file updates. Following up OpenAPITools#3440, supporting auth. * 1792 fix remote spec handling and hash calculation (OpenAPITools#3440) (cherry picked from commit 2a2eefe) * fix detecting remote file / local file logic while finding the hash file, taking care of IllegalArgumentException for local files. * add testcase

wing328 · 2019-09-12T13:55:32Z

@fujigon thanks for the PR, which has been included in the v4.1.2 release: https://twitter.com/oas_generator/status/1172068944355528705

* First version of Nim Client * Add some codes * Add some codes * Add some codes * Add some codes * Add some codes * First version of Nim Client * Add some codes * Add some codes * [Dart] Fix README template and update testing doco (#3809) * [Dart] Fix README template and update testing doco - deleted redundant shell script - fixed and updated README template - updated test package and moved to a dev_dependency - removed old unused dev_dependency packages - updated testing documentation in petstore sample * Remove references to dart-flutter-petstore.sh * Fix typos * Fix typo * Support custom git repository (#3757) * add gitHost param to GeneratorSettings and related * parameterize gitHost in READMEs * parameterize gitHost in go.mod * parameterize gitHost in git_push * update petstore samples * run ./bin/utils/export_docs_generators.sh * run meta-codehen.sh * Revert "run meta-codehen.sh" This reverts commit d6d579f. * Revert "run ./bin/utils/export_docs_generators.sh" This reverts commit 1b81538. * Revert "update petstore samples" This reverts commit f513add. * run ensure-up-to-date * Add links to article and video (#3820) * Better Go code format (#3819) * better varible naming * better comments * better code format for go experimental client * better comment, update samples * Add some codes * Add some codes * Add some codes * Add gRPC Protobuf schema generator (#3818) * add grpc protobuf generator * update doc * add new doc * add windows batch, comment out root proto * 1792 fix remote spec handling and hash calculation (#3440) * fixed bug where nullApi.java would be generated. Instead, generated DefaultApi.java to match the default path /{pathParam} (#3821) * Revert "1792 fix remote spec handling and hash calculation (#3440)" This reverts commit 2a2eefe. * Add nickmeinhold to Dart technical committee (#3830) * Bug #2845 typescript angular inheritance (#3812) * issue #2845: enable 'supportsMultipleInheritance' on typescript angular client codegen - note I reran ./bin/openapi3/typescript-angular-petstore-all.sh and no changes occurred. this suggests to me that the petstore.yaml sample should be improved to make use of the anyOf / allOf / oneOf keywords, in order to better show the effects of changes on generated code. * issue #2845: run ./bin/openapi3/typescript-angular-petstore-all.sh * run `mvn clean package && ./bin/typescript-angular-petstore-all.sh` * revert extranous files * fix warnings in csharp-netcore client (#3831) * Add missing files to the form request (#3834) * [client][go] avoid duplicated reflect imports (#3847) * Following up for #3440 (1792 fix remote spec handling and hash calculation) (#3826) * This patch fixes the bug that we cannot access to remote files when checking file updates. Following up #3440, supporting auth. * 1792 fix remote spec handling and hash calculation (#3440) (cherry picked from commit 2a2eefe) * fix detecting remote file / local file logic while finding the hash file, taking care of IllegalArgumentException for local files. * add testcase * Add a link (#3850) * Add Element AI to the list (#3856) * maven-plugin-plugin 3.6.0 (#3854) * [Java][okhttp-gson] fix failure to deserialize floats (#3846) * fixed bug where nullApi.java would be generated. Instead, generated DefaultApi.java to match the default path /{pathParam} * fix to bug #3157 * update samples * Adds Http Info To Dart Api (#3851) * [C++][Pistache] Add missing setter for arrays (#3837) * [C++][Pistache] Add missing setter for arrays Fixes #3769 * [C++][Pistache] Update Petstore sample * typescript-inversify: improve check for required parameters, support multiple media types (#3849) * [typescript-inversify] Allow falsy parameters A required parameter to an api method must not be `null` or `undefined`. It can be any other falsy value, e.g. `""`, `0` or `false` though. This change makes sure an error is only thrown in the former case and not in the latter. * [typescript-inversify] Handle multiple media types The Accept and Content-Type HTTP headers can contain a list of media types. Previously all but the first media type in the api definition were ignored. Now the headers are properly generated. * [typescript-inversify] Fix http client interface The api service methods allow the `body` parameter to be optional. The parameter is then passed to an `IHttpClient`. So it needs to be optional there as well. Also fixed the sample implementation `HttpClient`. Fixes #3618. * [typescript-inversify] Regenerate Petstore sample * [typescript-inversify] Use more explicit null check This does not change the semantic of the generated code, but makes it more explicit. Co-Authored-By: Esteban Gehring <esteban.gehring@gmail.com> * [typescript-angular] allow empty string basePath (#3489) * [typescript-angular] Fixing #2731 - empty string basePath * typescript-angular: refactor base path configuration * typescript-angular: refactor base path configuration * Fix/r/serialization fix and minor 3xx resp fix (#3817) * fix(qlik): fix for minor serialization bug * fix(r): add petsore generated classes * fix(r): indendation fixes * typescript-axios: Fix baseoptions (#3866) * Fixed missing baseOptions of typescript-axios. The typescript-axios template was missing the baseOptions setting when building an API Configuration. Set it. * update sample. * re-generate typescript axios samples * Rename gRPC generator to "protobuf-schema" (#3864) * rename grpc generator to protobuf-schema * update doc * Prepare v4.1.2 release (#3873) * update samples * update date * fix version in readme * BugFix #2053 Spring Boot fails to parse LocalDate query parameter (#3860) Adds the format annotation so that Spring is able to serialize OpenApi date/date-time format into LocalDate/OffsetDateTime. * update doc, samples (#3875) * update stable release * Update the batch for Windows * Add a test snippet * Update ensure-up-to-date * Add Nim to README.md * Ran ensure-up-to-date to pass CircleCI tests

fujigon force-pushed the fix_issue1792-followup branch from e9e54c8 to ee24676 Compare September 4, 2019 05:43

fujigon force-pushed the fix_issue1792-followup branch from ee24676 to 57908eb Compare September 4, 2019 08:36

fujigon force-pushed the fix_issue1792-followup branch from 57908eb to 47cca89 Compare September 5, 2019 02:34

fujigon force-pushed the fix_issue1792-followup branch 3 times, most recently from cd28d1a to f151daf Compare September 5, 2019 07:05

fujigon commented Sep 5, 2019

View reviewed changes

johngray and others added 3 commits September 6, 2019 13:01

1792 fix remote spec handling and hash calculation (OpenAPITools#3440)

acacd35

(cherry picked from commit 2a2eefe)

This patch fixes the bug that we cannot access to remote files when c…

042e176

…hecking file updates. Following up OpenAPITools#3440, supporting auth.

fix detecting remote file / local file logic while finding the hash f…

a1ecb15

…ile, taking care of IllegalArgumentException for local files.

fujigon force-pushed the fix_issue1792-followup branch from f151daf to a1ecb15 Compare September 6, 2019 04:01

add testcase

6ab750f

fujigon commented Sep 6, 2019

View reviewed changes

wing328 merged commit dd152ee into OpenAPITools:master Sep 6, 2019

wing328 added Enhancement: General OpenAPI Generator Maven Plugin labels Sep 6, 2019

wing328 added this to the 4.1.2 milestone Sep 6, 2019

fujigon deleted the fix_issue1792-followup branch September 6, 2019 08:56

wing328 changed the title ~~Following up for #3440 (1792 fix remote spec handling and hash calculation)~~ Fix remote spec handling and hash calculation Sep 10, 2019

wing328 mentioned this pull request Sep 14, 2019

[WIP] add support to remote files #2373

Closed

3 tasks

Gama11 mentioned this pull request Mar 10, 2021

[REQ] [Java] [Maven] InputSpec download from url #2241

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix remote spec handling and hash calculation #3826

Fix remote spec handling and hash calculation #3826

fujigon commented Sep 4, 2019 •

edited

Loading

fujigon commented Sep 4, 2019

wing328 commented Sep 4, 2019

wing328 commented Sep 4, 2019 •

edited

Loading

fujigon commented Sep 5, 2019

fujigon commented Sep 5, 2019

wing328 commented Sep 5, 2019

fujigon commented Sep 5, 2019

fujigon Sep 5, 2019

wing328 commented Sep 6, 2019

fujigon Sep 6, 2019

fujigon Sep 6, 2019

fujigon Sep 6, 2019 •

edited

Loading

wing328 Sep 6, 2019

wing328 commented Sep 12, 2019

Fix remote spec handling and hash calculation #3826

Fix remote spec handling and hash calculation #3826

Conversation

fujigon commented Sep 4, 2019 • edited Loading

PR checklist

Description of the PR

fujigon commented Sep 4, 2019

wing328 commented Sep 4, 2019

wing328 commented Sep 4, 2019 • edited Loading

fujigon commented Sep 5, 2019

fujigon commented Sep 5, 2019

wing328 commented Sep 5, 2019

fujigon commented Sep 5, 2019

fujigon Sep 5, 2019

Choose a reason for hiding this comment

wing328 commented Sep 6, 2019

fujigon Sep 6, 2019

Choose a reason for hiding this comment

fujigon Sep 6, 2019

Choose a reason for hiding this comment

fujigon Sep 6, 2019 • edited Loading

Choose a reason for hiding this comment

wing328 Sep 6, 2019

Choose a reason for hiding this comment

wing328 commented Sep 12, 2019

fujigon commented Sep 4, 2019 •

edited

Loading

wing328 commented Sep 4, 2019 •

edited

Loading

fujigon Sep 6, 2019 •

edited

Loading