Warning on autocomplete messes up the terminal

[JelleZijlstra]

Bug report

Bug description:

1. Open ./python.exe
2. import sys; code = sys._getframe(1).f_code
3. Type code.co_ and hit tab
4. This:

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-119205: Fix autocompletion bug in new repl #119229
• [3.13] gh-119205: Fix autocompletion bug in new repl (GH-119229) #119407

[eugenetriguba]

@JelleZijlstra Are you handling this or do you mind if I take a look into this? 🙂

[JelleZijlstra]

I'm not, feel free to take a look!

[eugenetriguba]

fwiw, this looks to be an issue with any output:

>>> import tarfile
>>> t = tarfile.TarInfo()
>>> t.tarfile.T<string>:1: DeprecationWarning: The undocumented "tarfile" attribute of TarInfo objects is deprecated and will be removed in Python 3.16

I ran into a similar issue when bringing up pdb:

 % ./python.exe
Python 3.14.0a0 (heads/main:16b46ebd2b, May 20 2024, 08:24:14) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.> /Users/eugene/src/cpython/Lib/rlcompleter.py(76)complete()
                                                                   -> breakpoint()
                                                                                  (Pdb) l

[jima80525]

Just found that this happens on mac, but not ubuntu. New repl doesn't have a windows version with autocomplete.

[koxudaxi]

mac uses /usr/lib/libedit.3.dylib for readline
But, ubuntu uses libreadline.so.8 of GNU.

dyld_info Modules/readline.cpython-314d-darwin.so 
Modules/readline.cpython-314d-darwin.so [arm64]:
    -platform:
        platform     minOS      sdk
           macOS     14.4      14.4   
    -segments:
        load-offset   segment section        sect-size  seg-size perm
        0x00000000    __TEXT                                32KB r.x
        0x00004110             __text             9476
        0x00006614             __stubs             948
        0x000069C8             __stub_helper       972
        0x00006D94             __cstring          1194
        0x00007240             __const            3296
        0x00007F20             __unwind_info       224
        0x00008000    __DATA_CONST                          16KB rw.
        0x00008000             __got               224
        0x0000C000    __DATA                                16KB rw.
        0x0000C000             __la_symbol_ptr     632
        0x0000C278             __data              984
        0x0000C650             __bss                32
    -dependents:
        attributes     load path
                       /usr/lib/libedit.3.dylib
                       /usr/lib/libSystem.B.dylib

[koxudaxi]

I re-build with GNU readline lib. But, The bug reproduce :(

brew install readline
export LDFLAGS="-L/opt/homebrew/opt/readline/lib"
export CPPFLAGS="-I/opt/homebrew/opt/readline/include"
./configure --with-pydebug --with-readline=readline
...

dyld_info Modules/readline.cpython-314d-darwin.so 
Modules/readline.cpython-314d-darwin.so [arm64]:
    -platform:
        platform     minOS      sdk
           macOS     14.4      14.4   
    -segments:
        load-offset   segment section        sect-size  seg-size perm
        0x00000000    __TEXT                                32KB r.x
        0x00003E10             __text             9928
        0x000064D8             __stubs            1008
        0x000068C8             __stub_helper      1032
        0x00006CD0             __cstring          1214
        0x00007190             __const            3456
        0x00007F10             __unwind_info       232
        0x00008000    __DATA_CONST                          16KB rw.
        0x00008000             __got               240
        0x0000C000    __DATA                                16KB rw.
        0x0000C000             __la_symbol_ptr     672
        0x0000C2A0             __data             1016
        0x0000C698             __bss                40
    -dependents:
        attributes     load path
                       /opt/homebrew/opt/readline/lib/libreadline.8.dylib
                       /usr/lib/libSystem.B.dylib

[eugenetriguba]

I tried this with pypy and wasn't able to get this output with the latest version:

~ % pypy
Python 3.10.14 (75b3de9d9035, Apr 21 2024, 10:55:50)
[PyPy 7.3.16 with GCC Apple LLVM 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys; code = sys._getframe(1).f_code
code.co_argcount         code.co_filename         code.co_kwonlyargcount   code.co_name             code.co_stacksize
code.co_cellvars         code.co_firstlineno      code.co_lines()          code.co_names            code.co_varnames
code.co_code             code.co_flags            code.co_linetable        code.co_nlocals
code.co_consts           code.co_freevars         code.co_lnotab           code.co_posonlyargcount
>>>> code.co_

But I'm not sure pypy has that deprecation warning in this codepath so I'll check that.

[danielhollas]

Hmm, it seems that similar behaviour exists on the old terminal as well though.

EDIT: Similar issue for the old REPL
#113978

[koxudaxi]

@danielhollas
Thank you for sharing the issue.
I think it's the same root cause.

We should think about the direction of the solution.

[eugenetriguba]

@koxudaxi Would this also resolve this issue? https://github.com/python/cpython/pull/113979/files

[koxudaxi]

@eugenetriguba
I think so. We confirmed that the problem will only happen with MacOS.
But, the solution works for all environments.
Should I close my PR as a duplicate?

[ambv]

@koxudaxi, sign the CLA and I think we will still take your PR as it includes a test.

[lysnikolaou]

Thanks all for the input and @koxudaxi for the PR.