Added test environment using pytest #5
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Added a small fix: rename tests/{test_structured_data_aginst_parsed_reference_files.py => test_structured_data_against_parsed_reference_files.py} |
|
Rebased to resolve merge conflicts. |
|
Great job @ogenstad ! |
|
Thanks. :) Sure I'll take a look! |
|
@GGabriele, I've rebased to fix the doc strings. Let me know if you need anything else. |
Sandeepsr
pushed a commit
to Sandeepsr/ntc-templates
that referenced
this pull request
Jan 11, 2018
Rename arista_eos_show_interfaces_transceiver to arista_eos_show_inte…
Yakuza-UA
added a commit
to Yakuza-UA/ntc-templates
that referenced
this pull request
May 10, 2020
thomasblass
pushed a commit
to thomasblass/ntc-templates
that referenced
this pull request
Oct 25, 2020
jmcgill298
pushed a commit
to jmcgill298/ntc-templates
that referenced
this pull request
Oct 27, 2021
Change interfaces to use Continue.Record
jvanderaa
pushed a commit
that referenced
this pull request
Nov 10, 2021
itdependsnetworks
pushed a commit
to itdependsnetworks/ntc-templates
that referenced
this pull request
Dec 26, 2022
Add extreme tierra
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As I mentioned in the chat I created a new testing environment with more thorough tests which tests more scenarios than the current tests. It's also quite a lot quicker and it's easier to see what is wrong when the tests fail.
I made some changes to tox.ini, since this isn't currently used by Travis it shouldn't impact anything else.
To test how this works you can just try this in the ntc-template root:
Currently there are a few issues which these tests pick up that the current tests didn't. Such as the "show interfaces status" template which was fixed yesterday and the "show ip ospf neighbor" template.
Also there are some duplicate lines in one of the reference files. Other parsed files are not in the same order as the actual data. In order to fix this the tests could sort the lists before comparing, or the parsed data could be changed.
I also made some changes to tox.ini to make it easier to run the Ansible tests without caring about the environment. You can just run tox and it will setup the requirements. However currently the Ansible tests are run after the pytest tests and since the pytest tests fail due to the incorrect data in the templates and parsed files the Ansible test is never run.
It this PR is accepted and it's desired to keep the Ansible tests I would propose that the code for that is changed so that it also uses pytest. Then the Ansible tests would be run even if the other tests fail.
Some tests are a bit redundant now, this is mostly to make it easier for contributors to spot what the actual error is. Another benefit of this is that it will be easier for contributors to run their own tests before they submit a PR.