Run Tests throws ModuleNotFoundError

[FelixBenning]

Issue Type: Bug

Run Tests throws ModuleNotFoundError even though pytest via commandline works and Debug Tests works fine

============================= test session starts =============================
platform win32 -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: d:\code\tf-optimizer-stochastic-runge-kutta, configfile: pyproject.toml
collected 0 items / 1 error

=================================== ERRORS ====================================
____________________ ERROR collecting tests/test_basics.py ____________________
ImportError while importing test module 'd:\code\tf-optimizer-stochastic-runge-kutta\tests\test_basics.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Python39\lib\importlib\__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests\test_basics.py:3: in <module>
    from sgd_g2 import SGD_G2
E   ModuleNotFoundError: No module named 'sgd_g2'
------------------------------- Captured stderr -------------------------------
2021-06-12 09:07:52.260040: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
- generated xml file: C:\Users\felix\AppData\Local\Temp\tmp-14424m9PpSJZ4GgIn.xml -
=========================== short test summary info ===========================
ERROR tests/test_basics.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 3.25s ===============================

The import also works fine in the terminal:

(env) PS D:\code\tf-optimizer-stochastic-runge-kutta> python
Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from sgd_g2 import SGD_G2
2021-06-12 09:14:26.844470: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
>>> 

The relevant package is -e git+git@github.com:FelixBenning/tf-optimizer-stochastic-runge-kutta.git@4908ff6b212a2a3b911ccda5bce995b94a548b6b#egg=SGD_G2 see

pip freeze

(env) PS D:\code\tf-optimizer-stochastic-runge-kutta> pip freeze
absl-py==0.12.0
appdirs==1.4.4
astunparse==1.6.3
atomicwrites==1.4.0
attrs==21.2.0
black==21.6b0
cachetools==4.2.2
certifi==2021.5.30
chardet==4.0.0
click==8.0.1
colorama==0.4.4
flatbuffers==1.12
gast==0.4.0
google-auth==1.31.0
google-auth-oauthlib==0.4.4
google-pasta==0.2.0
grpcio==1.34.1
h5py==3.1.0
idna==2.10
iniconfig==1.1.1
keras-nightly==2.5.0.dev2021032900
Keras-Preprocessing==1.1.2
Markdown==3.3.4
my-optimizer==0.1
mypy-extensions==0.4.3
numpy==1.19.5
oauthlib==3.1.1
opt-einsum==3.3.0
packaging==20.9
pathspec==0.8.1
pluggy==0.13.1
protobuf==3.17.3
py==1.10.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing==2.4.7
pytest==6.2.4
regex==2021.4.4
requests==2.25.1
requests-oauthlib==1.3.0
rsa==4.7.2
-e git+git@github.com:FelixBenning/tf-optimizer-stochastic-runge-kutta.git@4908ff6b212a2a3b911ccda5bce995b94a548b6b#egg=SGD_G2
six==1.15.0
tensorboard==2.5.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.5.0
tensorflow-estimator==2.5.0
termcolor==1.1.0
toml==0.10.2
typing-extensions==3.7.4.3
urllib3==1.26.5
Werkzeug==2.0.1
wrapt==1.12.1

Extension version: 2021.5.926500501
VS Code version: Code 1.57.0 (b4c1bd0a9b03c749ea011b06c6d2676c8091a70c, 2021-06-09T17:18:42.354Z)
OS version: Windows_NT x64 10.0.19042
Restricted Mode: No

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (8 x 3408)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on rasterization: enabled skia_renderer: enabled_on video_decode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	15.91GB (8.51GB free)
Process Argv	--crash-reporter-id 74736abc-a148-4775-8281-3e328798370c
Screen Reader	no
VM	0%

[kimadeline]

Hi @FelixBenning, thank you for reaching out.

Seems like the Run Tests command is being run in a terminal that's not activated, and as such doesn't have access to the tf-optimizer-stochastic-runge-kutta package. This should be addressed by #11039, feel free to upvote it to help us prioritize it!

Can you try starting VS Code from a terminal with the environment already activated?

Thank you!

[FelixBenning]

@kimadeline I assume you mean I should do something like this?

felix@hp-pavilion MINGW64 /d/code
$ cd tf-optimizer-stochastic-runge-kutta/

felix@hp-pavilion MINGW64 /d/code/tf-optimizer-stochastic-runge-kutta (main)
$ ls
README.md         data_deepobs/           obs_bench.py      setup.py
SGD_G2.egg-info/  env/                    pyproject.toml    sgd_g2.py
__pycache__/      my_optimizer.egg-info/  requirements.txt  tests/

felix@hp-pavilion MINGW64 /d/code/tf-optimizer-stochastic-runge-kutta (main)
$ . ./env/Scripts/activate
(env)
felix@hp-pavilion MINGW64 /d/code/tf-optimizer-stochastic-runge-kutta (main)
$ code .

Result is the same

python c:\Users\felix\.vscode\extensions\ms-python.python-2021.5.926500501\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir d:\code\tf-optimizer-stochastic-runge-kutta -s --cache-clear tests
============================= test session starts =============================
platform win32 -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: d:\code\tf-optimizer-stochastic-runge-kutta, configfile: pyproject.toml
collected 0 items / 1 error

=================================== ERRORS ====================================
____________________ ERROR collecting tests/test_basics.py ____________________
ImportError while importing test module 'd:\code\tf-optimizer-stochastic-runge-kutta\tests\test_basics.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Python39\lib\importlib\__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests\test_basics.py:3: in <module>
    from sgd_g2 import SGD_G2
E   ModuleNotFoundError: No module named 'sgd_g2'
------------------------------- Captured stderr -------------------------------
2021-06-14 22:01:36.421350: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
- generated xml file: C:\Users\felix\AppData\Local\Temp\tmp-11940GZ7TebMvWcdw.xml -
=========================== short test summary info ===========================
ERROR tests/test_basics.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 3.47s ===============================

I also tried closing all other instances of vscode and restarting again

[kimadeline]

Yep that was it, although clearly it didn't work 🤔

What happens if you run the call that the extension does directly in the terminal?
python c:\Users\felix\.vscode\extensions\ms-python.python-2021.5.926500501\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir d:\code\tf-optimizer-stochastic-runge-kutta -s --cache-clear tests

And what is the result of that same call, but without the extension's adapter script: python -m pytest --rootdir d:\code\tf-optimizer-stochastic-runge-kutta -s --cache-clear --collect-only tests?

[FelixBenning]

I am not sure if that is the right thing - since test discovery also works in vscode

(env)
felix@hp-pavilion MINGW64 /d/code/tf-optimizer-stochastic-runge-kutta (main)
$ python c:\Users\felix\.vscode\extensions\ms-python.python-2021.5.926500501\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir d:\code\tf-optimizer-stochastic-runge-kutta -s --cache-clear tests
C:\Python39\python.exe: can't open file 'C:\Usersfelix.vscodeextensionsms-python
.python-2021.5.926500501pythonFilestesting_toolsrun_adapter.py': [Errno 2] No su
ch file or directory
(env)
felix@hp-pavilion MINGW64 /d/code/tf-optimizer-stochastic-runge-kutta (main)
$ powershell

\ was not liked by git-bash so I switched to powershell

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
PS D:\code\tf-optimizer-stochastic-runge-kutta> .\env\Scripts\Activate.ps1
(env) PS D:\code\tf-optimizer-stochastic-runge-kutta> python c:\Users\felix\.vscode\extensions\ms-python.python-2021.5.926500501\pythonFiles\testing_tools\run_adapter.py discover pytest -
- --rootdir d:\code\tf-optimizer-stochastic-runge-kutta -s --cache-clear tests
[{"rootid": ".", "root": "D:\\code\\tf-optimizer-stochastic-runge-kutta", "parents": [{"id": "./tests", "kind": "folder", "name": "tests", "parentid": ".", "relpath": ".\\tests"}, {"id":
"./tests/test_basics.py", "kind": "file", "name": "test_basics.py", "parentid": "./tests", "relpath": ".\\tests\\test_basics.py"}], "tests": [{"id": "./tests/test_basics.py::test_optimiza
tion", "name": "test_optimization", "source": ".\\tests\\test_basics.py:7", "markers": [], "parentid": "./tests/test_basics.py"}]}]
(env) PS D:\code\tf-optimizer-stochastic-runge-kutta> code .
(env) PS D:\code\tf-optimizer-stochastic-runge-kutta>

(env) PS D:\code\tf-optimizer-stochastic-runge-kutta> python -m pytest --rootdir d:\code\tf-optimizer-stochastic-runge-kutta -s --cache-clear --collect-only tests
================================================================================== test session starts ===================================================================================
platform win32 -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: d:\code\tf-optimizer-stochastic-runge-kutta, configfile: pyproject.toml
collecting ... 2021-06-14 23:30:09.284322: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
collected 1 item

<Module tests/test_basics.py>
  <Function test_optimization>

=============================================================================== 1 test collected in 2.95s ================================================================================

[kimadeline]

Oh I see, thank you for the clarification!

Could you run the command that appears in the Python output channel when you run the tests, and followed by the same command without the extension's adapter? The first one should be something along the lines of python c:\Users\felix\.vscode\extensions\ms-python.python-2021.5.926500501\pythonFiles\testing_tools\run_adapter.py pytest --rootdir d:\code\tf-optimizer-stochastic-runge-kutta tests, and the second one would be python -m pytest --rootdir d:\code\tf-optimizer-stochastic-runge-kutta tests.

I also have a couple of follow-up questions:

• Does it only happen with the sgd_g2 package?
• Do you have a sample repo I could use to try and replicate the issue on my end?

Thanks!

[Shourai]

Hey @FelixBenning,
I have been encountering the same issue and this solution: #14579 (comment) solved it.

Adding an __init__.py in the test suite folder also seemed to "fix" this particular issue for me without requiring a downgrade. You really shouldn't need it there for the test suite to work though thus the cautious use of the word fix.

Even though that shouldn't be needed as running pytest from the terminal works fine.
Seems it's a pythonpath issue with vscode.

Probably related to: #14570 and #16274

[FelixBenning]

@Shourai That workaround with the __init__.py fixed it - thank you

[FelixBenning]

Closing as a duplicate of #14579

[FelixBenning]

Reopening as I noticed that all of the referenced issues and PR are closed

[kimadeline]

Your issue isn't related to #14570 nor #16274, since it isn't using pyvsc-run-isolated.py. Sounds like it's more a case of #15062, which, incidentally, has a similar workaround.

Feel free to upvote #15062 to help us prioritize it! In the meantime, I will close it as a duplicate of #15062.