add tests that verify actual behavior of generated code #1156
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This creates a new category of end-to-end unit tests which run the generator and then import and execute pieces of the generated code. Unlike the "golden record"-based tests, these don't make assertions about the exact implementation of the generated code, but only its overall behavior.
Also, unlike the existing end-to-end tests, these are self-contained: the spec data (and, optionally, any custom config data) is contained in the unit test code, not in separate files. Each test class uses a small spec designed only for the assertions it will make.
I've added some initial tests of basic model class behavior and docstrings. There aren't currently any tests of endpoint behavior, but I think those could be pretty easily implemented using a mock HTTP client.
The intended benefits are:
These aren't intended to completely replace the "golden record"-based tests; those can still be useful to catch regressions in the generator that affect the overall structure of the generated client package. However, I think probably quite a few of the current end-to-end tests could eventually be migrated to this system and might be more maintainable that way.