Allow user to override the Code Point Limit required by SnakeYaml

[AmateurECE]

With this patch, a user may set the system property 'maxYamlCodePoints' in order to override the default 3MiB limit configured in the org.yaml.snakeyaml package by default. This limit was implemented to prevent certain Denial-of-Service (DOS) attacks, but users should be given the opportunity to override this value for valid configurations which exceed the limit, such as the Redfish OpenAPI specification (developed by DMTF), which weighs in at 4.9MiB.

This patch was tested to work with openapi-generator-cli v6.3.0.

This PR closes #1871

Signed-off-by: Ethan D. Twardy ethan.twardy@gmail.com

[AmateurECE]

I believe this PR also fixes #1857.

[frantuma]

Thanks!

[ShubhamShekhar1996]

Doesn't work for me though! Updated the dependency and Tried below command:-
mvn clean install -DmaxYamlCodePoints="999999999"
Still getting same error:-
Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points

Can you guide me where I may be wrong?

[AmateurECE]

Hmm...which tool are you using, and can you provide the command line that generates the exception? I was using openapi-generator-cli. It's possible there's another code path that results in construction of a ScannerImpl without checking the system properties.

[ShubhamShekhar1996]

I am using Git bash to run my builds .... I think its not taking the "maxYamlCodePoints"

[AmateurECE]

I'm sorry, I meant which tool is responsible for the error? Are you using swagger-parser-cli to do validation, or are you using openapi-generator-cli to generate client/server code for an API? Can you provide the command that you are running in Git Bash?

[ShubhamShekhar1996]

I'm sorry, I meant which tool is responsible for the error? Are you using swagger-parser-cli to do validation, or are you using openapi-generator-cli to generate client/server code for an API? Can you provide the command that you are running in Git Bash?

I am using "swagger-parser-cli". Below is the command:-

mvn clean install -DmaxYamlCodePoints="99"

[AmateurECE]

Ah, I see. This system property is meant to be interpreted at runtime, not at build time. So, if you're running swagger-parser-cli, I suggest setting the system property in that invocation, like so:

$ java -DmaxYamlCodePoints=999999 -jar ~/.m2/repository/[...]/swagger-parser-cli-<version>.jar <other arguments>

[ShubhamShekhar1996]

Tried below command :-
java -DmaxYamlCodePoints=999999 -jar swagger-parser-v3-2.1.10.jar
But it gives below error though :-
no main manifest attribute, in swagger-parser-v3-2.1.10.jar

I searched but I could not find swagger-parser-cli-.jar in my local m2 repo.Only thing I could find was
.....m2\repository\io\swagger\parser\v3\swagger-parser-v3\2.1.10\wagger-parser-v3-2.1.10.jar

Below is the dependency I am using in my pom.xml:-

io.swagger.parser.v3
swagger-parser
2.1.11

Also is there any way to use it during build (mvn install)?

[ShubhamShekhar1996]

Any suggestions?

[elmehi]

@AmateurECE , i'm running into difficulty with a command of the form:
java -DmaxYamlCodePoints=999999 -jar ./generator/openapi-generator-cli-6.4.0.jar <other arguments>

error: Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points.

any ideas what's wrong?

[AmateurECE]

I believe openapi-generator is still using version 2.1.6 of swagger-parser. This PR was only merged as recently as 2.1.11. So I expect that when they upgrade the version in their Maven configuration, this issue will be resolved. You can build it from source and manually change the version of swagger-parser as a workaround.

[AmateurECE]

Any suggestions?

@ShubhamShekhar1996 The system property just needs to be set whenever swagger-parser is run, so as long as you can set the system property while you're running the swagger maven plugin, it should work for you. I expect your command above did not work because you don't have the library available in your local maven repository. You will have to run mvn install.

[AmateurECE]

@samarth Gupta, the example you provided appears to be setting the `maxYamlCodePoints` value as an *environment variable*, but this mechanism is triggered by a *JVM system property*. Environment variables cannot be used to change this value. See https://stackoverflow.com/a/7055010

…

On Tue, Jul 25, 2023 at 6:35 AM Samarth Gupta ***@***.***> wrote: I am still getting limit exceed error even when I set the required env variable. Below is sample ParseOptions options = new ParseOptions(); options.setResolve(true); options.setResolveFully(true); System.out.println(System.getenv("maxYamlCodePoints")); String openApiSpec = FileUtils.readFileToString(new File("openapi.yaml")); OpenAPI openAPI = new OpenAPIParser().readContents(openApiSpec, emptyList(), options).getOpenAPI(); error Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points. have set maxYamlCodePoints to 9999999999 — Reply to this email directly, view it on GitHub <#1872 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEU5Q3HPLNITKLRCHNQZHQTXR5SLFANCNFSM6AAAAAAT3CCQLA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[svghadi]

Java system properties can be set using _JAVA_OPTIONS env var. Below worked for me

export _JAVA_OPTIONS=-DmaxYamlCodePoints=99999999