debug adapter fails with Python 3.12/homebrew/MacOS arm64 due to 3.12 regression, works with 3.11/homebrew (VSC-1339)

[mhaberler]

OS

MacOS

Operating System version

Sonoma 14.4.1 (23E224)

Visual Studio Code version

Version: 1.88.0 (Universal) Commit: 5c3e652f63e798a5ac2f31ffd0d863669328dc4c

ESP-IDF version

5.2.1

Python version

3.12.2_1 (broken), 3.11.9 (works), both homebrew

Doctor command output

doctor output for 3.11 and 3.12
doctor-3.12.txt
doctor-3.11.txt
esp_idf_vsc_ext.log

Extension

esp_idf_vsc_ext.log attached above (for failed 3.12 debug attempt towards end)

Description

when debugging using a homebrew Python 3.12 install:

• debugger stops at app_main()
• set a breakpoint a few lines further down
• click continue button
• debugger does NOT stop at line with breakpoint, yellow caret does not move

Explorer->Debug Console shows:

Debug adapter -> Extension: DEBUG_ADAPTER_STARTED
Debug adapter -> Extension: DEBUG_ADAPTER_READY2CONNECT
Debug Adapter initialized
2
can't create new thread at interpreter shutdown <----------------- this is the cause

when debugging using a homebrew Python 3.11 install:

• debugger stops at app_main()
• set a breakpoint a few lines further down
• click continue button
• debugger DOES stop at line with breakpoint, yellow caret DOES move

Explorer->Debug Console does NOT show the above error message!

the cause is very likely the 3.12 regression described here:
https://stackoverflow.com/questions/77631477/runtimeerror-cant-create-new-thread-at-interpreter-shutdown-python-3-12

This is at program startup - debugger stops at app_main():

this is after setting the breakpoint:

Debug Message

Debug adapter -> Extension: DEBUG_ADAPTER_STARTED
Debug adapter -> Extension: DEBUG_ADAPTER_READY2CONNECT
Debug Adapter initialized
2
can't create new thread at interpreter shutdown  <----------------- this is the cause

Other Steps to Reproduce

I hit an attachment size limit above and cannot add more text

The third image above shows the behaviour after hitting the continue icon
NB the yellow caret did NOT move, see the 'can't create new thread at interpreter shutdown' error message in the Debug console

Summary:

• on MacOS/homebrew it is very easy to get an accidential Python upgrade making 3.12 the default
• using this extension, and using Python 3.12 for ESP-IDF triggers the abovementioned regression

the following sequence worked for me

• deleting the extension and wipe .vscode/extensions/espressif.esp-idf-extension-1.7.1
• switch to Python 3.11 by adding to .zprofile: export PATH=$(brew --prefix python@3.11)/libexec/bin:$PATH
• re-run the ESP-IDF install.sh
• reinstall the extension: code --install-extension esp-idf-extension.vsix
• start vscode
• run "ESP-IDF: Install ESP-IDF Extension Python packages" from Command Palette
• Select Debug, click green launch triangle
• debugging works correctly
• no more can't create new thread at interpreter shutdown error message in Debug Console

Recommendation:

• either look for a fixed threading package for 3.12 and ship it / refer to it
• OR make very clear 3.12 currently does not work.

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[mhaberler]

after some more trial and error, the following seems a less painful path which works for me:

• Configure ESP-IDF Extension -> Use existing setup -> Clean IDF setups
• Search ESP-IDF in your system
• reuse IDF_PATH and IDF_TOOLS_PATH directories
• Select Python version -> Provide Python executable path
• explicitly set to a 3.11 or lower version, in my case /opt/homebrew/opt/python@3.11/libexec/bin/python worked fine
• Install
• confirm Pick Workspace ...
• Command Palette -> ESP-IDF: Install ESP-IDF Extension Python packages

[brianignacio5]

We are working on migrating to a new debug adapter that is more stable that our current python one in this PR:

#1131

You are welcome to test it with the PR vsix installer if you like.

[mhaberler]

I did in fact use the #1131 vsix version but forgot to mention this in the current issue report

meaining: this issue applies to #1131 as well

since the underlying issue is present on Windows as well (and hence likely linux too) debugging using this extension is likely broken for all platforms as long as your debug adapter permits 3.12 and imports threading.py which the #1131 vsix also does

and since the #1131 vsix uses threading it evades me how this code is supposed to fix a problem introduced by permitting a combination of Python3.12 and threading.py ?

iow: the #1131 extension PR cannot possibly fix the issue reported here

what will fix it is avoiding either Python3.12 and/or `threading``

the tricky part is: esp-idf installs and works just fine with Python3.12 - it ONLY breaks this extension at runtime leading users to believe all prerequisites are fullfilled

this really should be made an install time error until 3.12 or later and/or threading.py is fixed

[brianignacio5]

The thing is, the new debug adapter doesn't use Python. If you are is because probably you are still using .vscode/launch.json configuration for older debug adapter. Please test the new one with update launch.json

 {
   "configurations": [
     {
       "type": "gdbtarget",
       "request": "attach",
       "name": "Eclipse CDT Remote",
       "program": "${workspaceFolder}/build/${command:espIdf.getProjectName}.elf",
       "initCommands": [
         "set remote hardware-watchpoint-limit 2",
         "mon reset halt",
         "maintenance flush register-cache",
         "thb app_main",
       ],
       "gdb": "${command:espIdf.getXtensaGdb}",
       "target": {
         "port": "3333"
       },
     }
   ]
 }

[mhaberler]

well using your above launch.json fragment finally worked with the new debug adapter
debugger behaviour looks entirely reasonable now

so debug_adapter.py ff seem to be dead code now to start with, threading error message is gone

for others to follow:

• I made homebrew Python3.12 the default Python iinterpreter again on MacOS M1/arm64

• made sure 3.12 is started by default using zsh

• removed .espressif/python_env/*

• re-ran ./install.sh all

• made sure idf.py works at the command line level - in my case by rebuilding hello_world

• start VSCode

• reset any user and workspace settings of the espressif esp-idf extension to defaults

• uninstall the espressif extenstion

• quit VSCode

• install the Bugfix/debugging docs #1131 vsix extension: code --install-extension /Users/mah/Downloads/esp-idf-extension.vsix

• start VSCode

• make sure the ESP-IDF version from step 4 above is used

• confirm Pick Workspace ...

• Command Palette -> ESP-IDF: Install ESP-IDF Extension Python packages

• Full clean

• Command Palette -> ESP-IDF: Set Espressif Device Target

• build, flash, monitor

• click 'Run and Debug', open .vscode/launch.json

• replace by above launch.json snippet

• Run and Debug -> Eclipse CDT Remote

and debugging starts to work reasonably well.

let me understand what the best practice is for debugging:

• for a new project, I add the incantations from your launch.json snippet
• I add .vscode/launch.jsonto the git repo
• was this it?

I noticed at least on MacOS openocd remains running even when exiting VSCode, which interferes with the next debugging session, leading to a message

[OpenOCD]
❌ Open On-Chip Debugger v0.12.0-esp32-20230921 (2023-09-21-13:27)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
debug_level: 2
adapter speed: 20000 kHz
WARNING: boards/esp32-wrover.cfg is deprecated, and may be removed in a future release.
         If your board is ESP32-WROVER-KIT, use board/esp32-wrover-kit-1.8v.cfg instead.
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
Error: couldn't bind tcl to socket on port 6666: Address already in use   <-----

but it seems to do no harm as the debugger comes up nevertheless

[mhaberler]

above I wrote:

I noticed at least on MacOS openocd remains running even when exiting VSCode, which interferes with the next debugging session, leading to a message

correction: it does harm.

scenario:

• connect an esp32s3, build, flash debug, exit VSCode
• connect an esp32c3 to a second USB port, start VSCode, switch to esp32c3 target, build and try to debug
• since openocd still running bound to the esp32s3 port, debug session happily starts on wrong CPU

this needs fixing as well - hard to figure a leftover openocd in the background is at fault

[brianignacio5]

For a new project, you can create a new project using the ESP-IDF: Show Examples or ESP-IDF: Project Wizard command to create a new project from example with .vscode files, or just add .vscode files to currently opened directory with ESP-IDF: Add vscode configuration folder command.

With regards to openOCD, how are you running openOCD, are you using the extension openOCD Server ? Are you stopping the debugger before closing the vscode window or just closing the window ? Any output related to openOCD not closing properly ?

I ask because I have tested on macOS and openOCD seems to close normally after I stop the debug session.

[mhaberler]

just leaving VSCode during a debug session leaves openocd running - this could have been the cause
exiting (I assume what is now the red diagonal unplug icon) terminates openocd properly
it could be this was with the previous extension version, where you had the red square icon for terminating the session - if the debugging session went haywire that icon eventually got you out of the debug session but tended to leave an openocd in the background

will have a closer eye on that exactly when it happens

[mhaberler]

oh I see, + Add .vscode subdirectory files now adds the Eclipse CDT Remote config to launch.json

if I wipe the .vscode directory and restart VSCode, it creates .vscode/settings.json as per target selection:

{
    "idf.adapterTargetName": "esp32c3"
}

for debugging to work, I need a .vscode/settings.json (example M5stack stamp-c3u = esp32c3) like so:

{
    "idf.adapterTargetName": "esp32c3",
    "idf.port": "/dev/cu.usbmodem2131301",
    "idf.flashType": "UART",
    "idf.openOcdConfigs": [
        "board/esp32c3-builtin.cfg"     // <---- add this, then things work
    ]
}

it is unclear to me how I get from the former to the latter other than editing, is this the proper way to do it?

I guess what I'm trying to do is an idiots recipe for going from hello_world to a working debug session given a piece of hardware

next hop: extend towards the Multiple Build Configurations Example for working with different targets

then: add Arduino as component without bringing down the house, and have debugging still work.

[mhaberler]

AHA. launch.json with Eclipse CDT Remote config is only added by + Add .vscode subdirectory files if launch.json does not exist yet

[brianignacio5]

You can check the tutorial documentation should provide most of extension functionality.

[mhaberler]

sorry, after spending a disordinate amount of time trying to work with this extension I give up.

I have decided to stay with Platformio for the easy cases and command line debug for hard cases.

[brianignacio5]

If it not to much to ask, could you describe which issues you faced with this extension and what cases you think we could improve ?