v1.0.0 hangs autocompleting haxor-news view command on running Windows

[donnemartin]

Hi Jonathan,

I ran into this issue updating prompt-toolkit to v1.0.0 on haxor-news v0.4.0:

Build

haxor-news v0.4.0, running prompt-toolkit v1.0.0:

https://github.com/donnemartin/haxor-news/releases/tag/0.4.0

Environment

Windows 10 (on VirtualBox 5.0.20)
Python 3.5.1

Steps to reproduce

 $ haxor-news  # start the shell
 $ hn view 1  # works the first time
 $ hn view 1  # hangs on the second time*

*Hangs for awhile but eventually comes back, I'm attempting to type hn view 1 -c.

Previous version of prompt-toolkit v0.52 doesn't have the same issue. I've reverted back to v0.52 for now.

-Donne

[jonathanslenders]

Thanks for reporting. I'll look into this asap. (A few hours ago, I tried to reproduce it, but actually, I was unable.)

[donnemartin]

Thanks Jonathan. One other thing I forgot to mention, not sure if it makes a difference: I'm running Windows 10 on VirtualBox (edited issue description above). I'll try again tomorrow and try to create a gif of a screen capture of what I'm seeing.

[donnemartin]

Here's a video of what I'm seeing on v1.0.0. No issues with v0.52.

https://www.dropbox.com/s/du8v67rzaw4l99e/haxor-news-bug.mov?dl=0

Video is a little choppy because I'm running QuickTime screen record + Windows 10 on a VM with limited resources allocated to it.

[jonathanslenders]

Trying to reproduce: I can reproduce it, but at some point, I got the exception below:

 File "c:\users\jonat\documents\github\haxor-news\haxor_news\hacker_news_cli.py", line 367, in view
    browser)
  File "c:\users\jonat\documents\github\haxor-news\haxor_news\hacker_news.py", line 638, in view_setup
    browser)
  File "c:\users\jonat\documents\github\haxor-news\haxor_news\hacker_news.py", line 553, in view
    comments_hide_non_matching=comments_hide_non_matching)
  File "c:\users\jonat\documents\github\haxor-news\haxor_news\hacker_news.py", line 255, in print_comments
    depth=depth)
  File "c:\users\jonat\documents\github\haxor-news\haxor_news\hacker_news.py", line 243, in print_comments
    self.print_comment(item, regex_query, comments_hide_non_matching, depth)
  File "c:\users\jonat\documents\github\haxor-news\haxor_news\hacker_news.py", line 216, in print_comment
    click.echo(formatted_comment, color=True)
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python35\lib\site-packages\click-6.6-py3.5.egg\click\utils.py", line 259, in echo
    file.write(message)
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python35\lib\site-packages\click-6.6-py3.5.egg\click\_compat.py", line 571, in _safe_write
    return _write(s)
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python35\lib\site-packages\colorama-0.3.7-py3.5.egg\colorama\ansitowin32.py", line 40, in write
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python35\lib\site-packages\colorama-0.3.7-py3.5.egg\colorama\ansitowin32.py", line 141, in write
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python35\lib\site-packages\colorama-0.3.7-py3.5.egg\colorama\ansitowin32.py", line 166, in write_and_convert
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python35\lib\site-packages\colorama-0.3.7-py3.5.egg\colorama\ansitowin32.py", line 174, in write_plain_text
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python35\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u25ce' in position 2: character maps to <undefined>

[jonathanslenders]

Hi @donnemartin,

The bug is actually at this line:
https://github.com/donnemartin/haxor-news/blob/master/haxor_news/haxor.py#L205

prompt_toolkit.document.Document is supposed to be immutable. One reason for this is that it makes the caching significantly easier and a lot more performant. (If the same Document object is created, then we have the same lines, and everything in there can be cached. That's very important for bigger inputs.) At this line, for instance is a Document instantiated, but if a Document with a similar input was already created, that is reused. https://github.com/jonathanslenders/python-prompt-toolkit/blob/master/prompt_toolkit/buffer.py#L394

By changing the text field of a document, we got some weird inconsistencies, like document.buffer.text != document.text which made the completer algorithm loop forever. After generating the completions, it thought that the input was changed, but it was not.

Anyway, just don't assign to Document.text and that should fix it. I will push a new change that will disallow assignments in order to avoid making this mistake in the future.

Edit: there are a few small things in the completer code that I'm going to improve as well. Not sure how this all plays together, but I guess the above change should be sufficient.

Jonathan

[jonathanslenders]

Here I made Document completely read-only: this should have given a more meaningful error instead of a enternal completion loop: fe8f1c7

[donnemartin]

Thanks for the quick investigation @jonathanslenders! I'll try to check this out tomorrow and report back.

[donnemartin]

Anyway, just don't assign to Document.text and that should fix it.

@jonathanslenders that did the trick, thank you!

I got the exception below

Hmm, curious if you see this error for all posts or just a few? And if it's not all posts, please let me know which one and I can try to get a fix. I'm not able to reproduce it on my end, I think it should have been fixed by donnemartin/haxor-news#36.

[jonathanslenders]

If you don't mind, I'm closing this issue. The bug, related to modifying a Document has been covered. Feel free to open again.

About the bug that I generated, I was not able to reproduce it.

[donnemartin]

No problem thanks for the support Jonathan!