Debugging Code Running in Host Application

[Anti-Distinctlyminty]

Environment data

• debugpy version: 1.5.1
• OS and version: Windows 10 Pro 21H1
• Python version (& distribution if applicable, e.g. Anaconda): 3.7.4 [MSC v.1928 64 bit (AMD64)]
• Using VS Code or Visual Studio: vscode

Actual behavior

Running debugpy.listen("localhost", 0)) starts another instance of the host application (Houdini in this case)

Expected behavior

It should attach the debugger an not launch the host application.

Steps to reproduce:

1. Download Houdini 19.0.455
2. pip install debugpy
3. Launch Houdini
4. Open Houdini and go to Windows > Python Shell
5. Add the location of debugpy to sys.path
6. Run import debugpy;debugpy.listen(("localhost", 0))
7. Another instance of Houdini will launch

[gramster]

I'm not familiar with Houdini, but it sounds like it has its own Python shell? And you are trying to debug that shell?

[Anti-Distinctlyminty]

Hi Graham,
Houdini does indeed have it's own python shell, and ships with it's own interpreter. I contacted support and they hypothesised that debugpy is using sys.executable, which is set to the executable of the host application - Houdini. I do not wish to speak out of turn, but I think the error may be with Houdini, as the docs state:

sys.executable
A string giving the absolute path of the executable binary for the Python interpreter

I know that several apps (Houdini, Unreal Editor) seem to use sys.executable in this manner.

Can you see anyway of circumventing this? I actually tried to set sys.executable back to the python interpreter shipped with Houdini, but the problem persisted. I'm trying to debug a library so it's tough to untangle this and see where it's happening.

[fabioz]

In this case, you need to call something as:

debugpy.configure(python=r"c:\path\to\python.exe")

As it is now, debugpy needs to launch a server to support the connection of multiple processes -- we actually do have a feature request to just use the debugger directly without this intermediate process for such cases: #532 -- but unfortunately this still isn't implemented.

Setting of sys.executable didn't work because the initial value is stored at import time (https://github.com/microsoft/debugpy/blob/main/src/debugpy/server/api.py#L28) -- given that you can configure it with debugpy.configure(python=r"c:\path\to\python.exe"), this shouldn't be an issue.

I'm closing the issue because this should solve it for you (if it doesn't, feel free to comment to reopen the issue).

[Anti-Distinctlyminty]

@fabioz I'm now trying to debug code and the host application must use Python 2.7.15 and it now refuses to connect.
I'm currently trying what was mentioned here, but no luck. In vscode I always get

Timed out waiting for debug server to connect

launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "127.0.0.1",
                "port": 5678
            }
        }
    ]
}

Here's the code I'm running in the embedded python interpreter

from __future__ import print_function

import debugpy
debugpy.log_to("C:\\Users\\Luke\\AppData\\Local\\Temp\\Debugpy")

# Configuring to use the python.exe that is embedded with the application...
# debugpy.configure(python="C:\\Program Files\\Side Effects Software\\Houdini 18.5.351\\python27\\python.exe")
# results in a time out

# As suggested by #532#issuecomment-772370118
# I tried to use the same python version, but this also timed out 
debugpy.configure(python="C:\Python27\python.exe")

address = debugpy.listen(address=("0.0.0.0", 5678))

print("Waiting for client on address{}".format(address))
debugpy.wait_for_client()

debugpy.server-17384.log
debugpy.adapter-19564.log

I am at a loss as to what to do.

[int19h]

Do you know how the host application is running this script? If it gets imported as a module, then the import lock is held for the entire duration of the script, meaning that background threads trying to import modules will deadlock; I suspect that this is what's happening here.

[Anti-Distinctlyminty]

The code above is located in debug\__init__.py. So in the application's built in Python terminal I'm just doing

import debug

[int19h]

That might explain the problem. Instead of having all this code run on module import, try putting it into a function, and then call that function: import debug; debug.main()

[Anti-Distinctlyminty]

That seems to successfully attach now, but the behaviour is strange. I tried this...

from __future__ import print_function
import debugpy

def main():
    debugpy.log_to("C:\\Users\\Luke\\AppData\\Local\\Temp\\Debugpy")
    debugpy.configure(python="C:\Python27\python.exe")
    address = debugpy.listen(address=("0.0.0.0", 5678))

    print("Waiting for client on address{}".format(address))
    debugpy.wait_for_client()

    debugpy.breakpoint() # <-- This doesn't get triggered

And break points I've set are not triggered (see attached video)

breakpoints.mp4

[int19h]

A breakpoint on pass won't be hit, since it's not an executable statement (i.e. at runtime there's literally nothing corresponding to it). breakpoint() should work, though.

Depending on where your script is located, though, it might be treated as part of the standard library - this would be the case if you placed it in your "python27" directory (or any subdirectory, such as Lib or site-packages). I would recommend putting it elsewhere and adjusting sys.path as needed to make it importable, but you can also set "justMyCode": false in launch.json.

[Anti-Distinctlyminty]

The module location is already in sys.path albeit not in the most clean form, something like

//192.168.x.x/path/to/PLUGINS/../PLUGINS/python2.7libs

Should a UNC path of this form work ok?

[int19h]

Ohh, I'm not sure about that one actually. If you print __file__ inside your module, what does it say? If it's still a UNC path, it might be throwing the logic determining whether it's your code or library code off.

[Anti-Distinctlyminty]

Looks to be correct:

//192.168.x.x/path/to/PLUGINS/../PLUGINS/python2.7libs\debug\__init__.py

(Just to state the obvious - this is a redacted representation)

[Anti-Distinctlyminty]

Ok, some human error going on here. I somehow had breakpoints set in files where a drive was mapped, and I was looking at files that were loaded via the UNC path in vscode.

I'm still not sure why debugpy.breakpoint() isn't triggering though. But now I've got to hit the sack. Thank you @int19h