Ensure non-mocked message publishing during tests fails #5634
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
To try this out, you can just remove a usage of |
AdamWill
force-pushed
the
tests-unsafe-publish-fail
branch
2 times, most recently
from
8ca3af1 to
99e7c1b
Compare
In working on fedora-infra#5630 I found that what happens when you don't properly mock out message publish attempts in tests isn't great. In CI, the test will hang for several minutes, then eventually proceed. Obviously we don't want to slow the tests down, and there is a 30 minute time limit on the test, so if you do this more than twice or so, the tests will time out. It's not very clear why the tests are timing out and failing - you have to guess that it's message publishing causing the problem. In bcd, the test doesn't hang, it just silently proceeds (I guess the message gets published to the rabbitmq instance in the bcd environment, I haven't checked). So you will not notice that you got it wrong until you submit a PR. Neither of those behaviors is great, so instead, let's mock out the 'real' message publishing function in fedora_messaging in a fixture that's automatically used by every test. We replace it with a mock that raises an exception with a hopefully- useful message. We have to mock _twisted_publish, not publish, because mock_sends also mocks _twisted_publish; if we mocked publish, we would break mock_sends. This way mock_sends gets to override us and work OK. Unfortunately this usually causes a database rollback error so you have to scroll back through some confusing errors to find the 'real' problem, but it's still an improvement, I think. Fixes: fedora-infra#5633 Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamWill
force-pushed
the
tests-unsafe-publish-fail
branch
from
99e7c1b to
d066f40
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In working on #5630 I found that what happens when you don't properly mock out message publish attempts in tests isn't great. In CI, the test will hang for several minutes, then eventually proceed. Obviously we don't want to slow the tests down, and there is a 30 minute time limit on the test, so if you do this more than twice or so, the tests will time out. It's not very clear why the tests are timing out and failing - you have to guess that it's message publishing causing the problem.
In bcd, the test doesn't hang, it just silently proceeds (I guess the message gets published to the rabbitmq instance in the bcd environment, I haven't checked). So you will not notice that you got it wrong until you submit a PR.
Neither of those behaviors is great, so instead, let's mock out the 'real' message publishing function in fedora_messaging in a fixture that's automatically used by every test. We replace it with a mock that raises an exception with a hopefully- useful message. We have to mock _twisted_publish, not publish, because mock_sends also mocks _twisted_publish; if we mocked publish, we would break mock_sends. This way mock_sends gets to override us and work OK.
Unfortunately this usually causes a database rollback error so you have to scroll back through some confusing errors to find the 'real' problem, but it's still an improvement, I think.