Debugger did not start

[kagesenshi]

Environment data

• PTVSD version: 2020.3.71113
• OS and version: Fedora 30
• Python version (& distribution if applicable, e.g. Anaconda): python3-3.7.6-2.fc30.x86_64
• Using VS Code or Visual Studio: VS Code

Actual behavior

Debugger toolbar appears for a while, and then disappear. Noticed issue started after 2020.3.0 update, before that debugger functioning fine. Both vscode and vscode-insider exhibit the same problem, in multiple computers, all running stock Fedora 30 / 31 installations.

Expected behavior

Debugger starts.

Steps to reproduce:

1. Start debugger.

Additional information

No visible error log appear, so not quite sure what additional info to provide. Attached is some logs which was generated from following command (ref: #87 ).

/usr/bin/python3 \
    /home/izhar/.vscode-insiders/extensions/ms-python.python-2020.3.71113/pythonFiles/lib/python/debugpy/wheels/debugpy/adapter/__main__.py \
   
[debugpy.adapter-26256.log](https://github.com/microsoft/debugpy/files/4425434/debugpy.adapter-26256.log)
 --host localhost --port 5678 --log-dir /tmp/log

launch.json contents:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: DQMT",
            "type": "python",
            "request": "launch",
            "pythonPath": "${workspaceFolder}/bin/python",
            "module": "morpfw.cli",
            "console": "internalConsole",
            "redirectOutput": false,
            "debugServer": 5678,
            "internalConsoleOptions": "openOnSessionStart",
            "serverReadyAction": {
                "pattern": "Listening on (.+)",
                "uriFormat": "%s",
                "action": "openExternally"
            },
            "args": [
                "-s",
                "${workspaceFolder}/dev/dqmt/settings.yml",
                "start"
            ],
        }
    ]
}

Also a screencast (do tell me if it couldn't be played)

Screencast from 02-04-20 21:13:44.zip

[kagesenshi]

eh sorry .. should this be filed in vscode-python or here?

[karthiknadig]

Can you uninstall the extension and re-install? See if you get this version. That has a couple of fixes.

[kagesenshi]

updated to 2020.3.71659, still same problem.

[int19h]

This bit in your debug configuration:

"debugServer": 5678

means that VSCode expects the debug adapter to already be running on port 5678. Unless you're starting it manually in the terminal, it won't work. Removing that line should fix the problem.

And this feature is really meant as a debugging facility for developers working on the adapter, so it's not a supported scenario in regular use. I'm curious as to where it came from - do we have a sample config like that somewhere in our docs?

[kagesenshi]

@int19h

the debugserver configuration was added as per instruction in #87 to extract out the attached log. In normal runs, and in that screencast, "debugServer" and "redirectOutput" are not set.

[int19h]

It looks like the link to the log mistakenly ended up in the middle of a code block, so GitHub didn't treat it as a proper attachment. It hasn't GC'd it yet, though - hopefully this link should keep it up now:

debugpy.adapter-26256.log

[int19h]

Was it the only log produced? I'd expect to also see debugpy.launcher*.log at least - that one would be more interesting in this case.

Also, can you try it with "console":"integratedTerminal", and see if that makes any difference? If not, what does the terminal output look like?

[kagesenshi]

Sorry .. didnt notice that ended up in code block .. i think i forgot to attach debug.launcher too..

Did a new run with "integratedTerminal" .. attached is the screencast, and debug logs. In the screencast, I ran it twice, the first run with debugServer enabled, and the next run with debugServer disabled.

same symptoms .. with debugServer enabled I get a timeout, with it disabled, toolbar simply pops a while and then dissappear without any useful error message.

Screencast from 04-04-20 08:24:51.zip

debug.log.zip

[kagesenshi]

Hi

I tried to isolate the problem, it seems to be happening when using ./bin/python generated by buildout (http://www.buildout.org/en/latest/) build system, but its working if its merely virtualenv or system python.

Attached is a project directory with example. Run build.sh to setup the environment and generate ./bin/python, and then try launching the debugger.

issueisolate.zip

[int19h]

These helped a lot, thank you!

Judging by the screencast, that custom python binary doesn't really behave like a real Python interpreter. In particular, it doesn't implement the ability to do something like:

python /foo/bar/

where /foo/bar/ is a directory that contains __main__.py. The debugger uses this extensively - you can see the command line that it generates in the launcher log:

['/home/izhar/Devel/dqmt_deploy/bin/python', '/home/izhar/.vscode-insiders/extensions/ms-python.python-2020.3.71659/pythonFiles/lib/python/debugpy/wheels/debugpy', '--connect', '36213', '--adapter-access-token', '408aeafe7f20486c0f16e64a27b27d942fc9e34c7d0a162feb3eb6bf1054c330', '-m', 'morpfw.cli', '-s', '/home/izhar/Devel/dqmt_deploy/dev/dqmt/settings.yml', 'start']             

i.e.:

.../bin/python .../wheels/debugpy ... -m morpfw.cli ...

(note that .../wheels/debugpy is a directory!)

Looking at the isolated repro, and specifically the source code for the python stub that buildout generates - it's basically bare minimal effort to support the most basic interpreter arguments (and even then doesn't quite handle them right - e.g. python foo.py -m bar is broken!):

#!"c:\python\3.8-64\python.exe"

import sys

sys.path[0:0] = [
  'c:\\temp\\issueisolate\\testdebug',
  ]


_interactive = True
if len(sys.argv) > 1:
    _options, _args = __import__("getopt").getopt(sys.argv[1:], 'ic:m:')
    _interactive = False
    for (_opt, _val) in _options:
        if _opt == '-i':
            _interactive = True
        elif _opt == '-c':
            exec(_val)
        elif _opt == '-m':
            sys.argv[1:] = _args
            _args = []
            __import__("runpy").run_module(
                 _val, {}, "__main__", alter_sys=True)

    if _args:
        sys.argv[:] = _args
        __file__ = _args[0]
        del _options, _args
        with open(__file__) as __file__f:
            exec(compile(__file__f.read(), __file__, "exec"))

if _interactive:
    del _interactive
    __import__("code").interact(banner="", local=globals())

So this is similar to the situation with spark-submit - it's just not a drop-in compatible Python interpreter, and so it can't be used in "pythonPath". To the extent it worked in earlier versions, it was likely because we weren't using that particular feature of Python - but it was accidental, not deliberate. And I'm not even sure this will work correctly even if we get past the issue with directory arguments, given various other problems with the stub script.

So I think this is best treated as a bug on Buildout; at least if their python stubs are intended to substitute for the real interpreter. If not, then we can look into supporting Buildout specifically - if possible! - but that would be a new feature rather than a bug fix.

[kagesenshi]

Thank you for the explanation! that's useful!.. now that i know where the integration issue is, i'll see what i can do as a workaround.

I'll see if upstream buildout can make the option available on the generated ./bin/python

Thanks!