-
Notifications
You must be signed in to change notification settings - Fork 232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Virtualized or containerized environments might not report number of CPUs correctly #9
Comments
Based on the tests in #10 it seems that |
i presume the only way to do this correct is to get travis use the lxc-proc filesystem for containers (which provides a corrected view of proc) |
But not all of their environments use lxc. |
does it also report it wrong on top of a kvm/osx? |
On KVM yes, you get the same results as with LXC. No idea what about OSX. |
I think that providing a hook will workfor now. We can document that under Travis your conftest.py should look like: def pytest__xdist_auto_workers(cpu_count):
if os.environ.get('TRAVIS') == 'true':
return 2
return cpu_count Not sure about the hook name though. |
pytest-vw contains a pretty good list of known CI services. I think it would be nice for |
Signed-off-by: Julien Nicoulaud <julien.nicoulaud@gmail.com>
…/pytest-xdist#9 Signed-off-by: Julien Nicoulaud <julien.nicoulaud@gmail.com> # Veuillez saisir le message de validation pour vos modifications. Les lignes # commençant par '#' seront conservées ; vous pouvez les supprimer vous-même # si vous le souhaitez. Un message vide abandonne la validation. # # Date : Sun Mar 19 13:49:16 2017 +0100 # # Sur la branche master # Votre branche et 'origin/master' ont divergé, # et ont 1 et 1 commits différents chacune respectivement. # (utilisez "git pull" pour fusionner la branche distante dans la vôtre) # # Modifications qui seront validées : # modifié : tox.ini #
…/pytest-xdist#9 Signed-off-by: Julien Nicoulaud <julien.nicoulaud@gmail.com>
Fixed by #309. |
as pytest-dev/pytest-xdist#9 seems fixed
as pytest-dev/pytest-xdist#9 seems fixed
as pytest-dev/pytest-xdist#9 seems fixed
This problem is mainly present itself in CI environments like the ones Travis provides.
When you use
-n auto
xdist will create more workers than what's available for the container/VM since that is what's written in the /proc/ filesystem whichmultiprocessing.cpu_count()
reads the number of CPUs from.A possible solution would be to use psutil which might be able to read the /proc/ filesystem more correctly but I'm not 100% sure about it.
As a workaround I'd like to suggest adding:
to the code to prevent spawning too many processes and slowing down test runs.
A better workaround would be to provide a hook to allow to reinterpret what auto means and return a different number.
The text was updated successfully, but these errors were encountered: