-
Notifications
You must be signed in to change notification settings - Fork 567
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tests which fail in multiprocessing contexts #1018
Merged
Merged
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
84108d8
Parallel Demonstrator Tests
alexrudy be4ee65
Mark tests as XFail
alexrudy b13601f
Parallel Demonstrator Tests
alexrudy f4752c0
Mark tests as XFail
alexrudy a08d359
Applied PR feedback
MSeal 6a0f8c8
Merge remote-tracking branch 'rudy/fix-zmq-context-global' into fix-zmq
MSeal 55bb0db
Removed duplicate tests from merge
MSeal cc0ac44
Merge branch 'master' of github.com:jupyter/nbconvert into fix-zmq
MSeal 5c0848f
Added additional timeout delay to test
MSeal b88f3e9
Set additional timeout on the correct test field
MSeal a7a92b1
Added ability to turn off slow tests
MSeal ae011e7
Attempt to get travis passing slow tests
MSeal a443d03
Fixing pytest conf issues with temp directories
MSeal d1a3a6d
Moved conftest into nbconvert path
MSeal 4d3e992
Removed unecessary travis command
MSeal 48beaa4
Another attempt to get Travis
MSeal baea777
Simplified the test execution
MSeal f81e1a2
Yet another travis fix attempt
MSeal 1434a2a
Adding much higher timeouts to failing test
MSeal a818745
Attempt #billion to fix travis
MSeal bcd5157
Travis only-failure debug
MSeal f74f2e1
Added latest jupyter_client to travis for test run
MSeal c3ef4b7
Removed extra debug lines
MSeal 9bbcecb
Resolve conflict with master
MSeal 70d47b7
Removed jupyter_client master install from travis
MSeal File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
84 changes: 84 additions & 0 deletions
84
nbconvert/preprocessors/tests/files/Parallel Execute B.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"# Ensure notebooks can execute in parallel\n", | ||
"\n", | ||
"This notebook uses a file system based \"lock\" to assert that two instances of the notebook kernel will run in parallel. Each instance writes to a file in a temporary directory, and then tries to read the other file from\n", | ||
"the temporary directory, so that running them in sequence will fail, but running them in parallel will succed.\n", | ||
"\n", | ||
"Two notebooks are launched, each which sets the `this_notebook` variable. One notebook is set to `this_notebook = 'A'` and the other `this_notebook = 'B'`." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"import os\n", | ||
"import os.path\n", | ||
"import tempfile\n", | ||
"import time" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"# the variable this_notebook is injectected in a cell above by the test framework.\n", | ||
"this_notebook = 'B'\n", | ||
"other_notebook = 'A'\n", | ||
"directory = os.environ['NBEXECUTE_TEST_PARALLEL_TMPDIR']\n", | ||
"with open(os.path.join(directory, 'test_file_{}.txt'.format(this_notebook)), 'w') as f:\n", | ||
" f.write('Hello from {}'.format(this_notebook))" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"start = time.time()\n", | ||
"timeout = 5\n", | ||
"end = start + timeout\n", | ||
"target_file = os.path.join(directory, 'test_file_{}.txt'.format(other_notebook))\n", | ||
"while time.time() < end:\n", | ||
" time.sleep(0.1)\n", | ||
" if os.path.exists(target_file):\n", | ||
" with open(target_file, 'r') as f:\n", | ||
" text = f.read()\n", | ||
" if text == 'Hello from {}'.format(other_notebook):\n", | ||
" break\n", | ||
"else:\n", | ||
" assert False, \"Timed out – didn't get a message from {}\".format(other_notebook)" | ||
] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.6.7" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 2 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"import time" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"time.sleep(0.01)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": "Python 3", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.7.0" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 2 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are capfd and tmpdir pytest fixtures? Otherwise, where are these being populated when it comes time to test them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes they are provided by pytest: https://docs.pytest.org/en/latest/tmpdir.html#the-tmpdir-fixture https://docs.pytest.org/en/latest/capture.html#accessing-captured-output-from-a-test-function