Crash on startup when reading malformed history file

[janmayer]

If the hdtv_history file is damaged or malformed, hdtv will crash with a unhelpful error.

Traceback (most recent call last):
  File "./bin/hdtv", line 235, in <module>
    App()
  File "./bin/hdtv", line 178, in __init__
    hdtv.cmdline.MainLoop()
  File "/srv/storage/gpfs/user/jmayer/Development/hdtv/hdtv/cmdline.py", line 663, in MainLoop
    self.session = PromptSession(
  File "/home/jmayer/.pyenv/versions/3.8.5/lib/python3.8/site-packages/prompt_toolkit/shortcuts/prompt.py", line 463, in __init__
    self.default_buffer = self._create_default_buffer()
  File "/home/jmayer/.pyenv/versions/3.8.5/lib/python3.8/site-packages/prompt_toolkit/shortcuts/prompt.py", line 498, in _create_default_buffer
    return Buffer(
  File "/home/jmayer/.pyenv/versions/3.8.5/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 316, in __init__
    self.history.load(new_history_item)
  File "/home/jmayer/.pyenv/versions/3.8.5/lib/python3.8/site-packages/prompt_toolkit/history.py", line 70, in load
    for item in self.load_history_strings():
  File "/home/jmayer/.pyenv/versions/3.8.5/lib/python3.8/site-packages/prompt_toolkit/history.py", line 210, in load_history_strings
    line = line_bytes.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 0: unexpected end of data

To reproduce: echo -n -e '\xe9' >> ~/.local/share/hdtv/hdtv_history

We had one case of this, and I don't know what caused the damage to the file in the first place.
I'm not sure how to best mitigate that error with prompt_toolkit or tell the user his history file is damaged.

[op3]

I guess this might happen if the user still has an old readline history that was not encoded using UTF-8?

This should probably be fixed in prompt_toolkit: prompt-toolkit/python-prompt-toolkit#1303

In the meantime, we can just catch the exception and replace malformed characters if necessary.