Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid schemas found via hypothesis-jsonschema #1142

Closed
Zac-HD opened this issue Jul 10, 2020 · 2 comments · Fixed by #1776
Closed

Invalid schemas found via hypothesis-jsonschema #1142

Zac-HD opened this issue Jul 10, 2020 · 2 comments · Fixed by #1776

Comments

@Zac-HD
Copy link

Zac-HD commented Jul 10, 2020

Hi there! I've been using the schemastore to provide test inputs for my hypothesis-jsonschema library, which generates valid instances for jsonschemas. In the process, I've bumped into some pretty strange use-cases, and discovered that a few of the schemas here aren't actually valid!

  • The Release Drafter configuration file schema contains "required": [] in an object subschema, but an empty list is not valid for this keyword
  • The Cirrus CI configuration files and Bamboo Specs schemas contain references to definitions which don't actually exist in the schema
  • The TypeScript Lint configuration file contains a reference to an items key which is a list of schemas, and while valid for the items keyword this is an invalid reference target
  • and several others have invalid $schema keys, which emit a warning when validated with the Python jsonschema library using jsonschema.validators.validator_for(schema).check_schema(schema) - this might be a useful check to add to CI?

I hope reporting this is helpful - it's not especially detailed information, but I had to add some extra logging code to get this far and don't have any advantage for further debugging or a fix.

@madskristensen
Copy link
Contributor

That's super cool. Currently, we use the tv4 module to test for json schema validity, but it's starting to show its age.

@Zac-HD
Copy link
Author

Zac-HD commented Jul 11, 2020

You might also want to try to resolve each reference and check the result is valid; doing that for schemas without any recursive references is how I caught three that otherwise seemed fine :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants