Apostrophe makes cell execute forever

[Dene33]

It seems that ' is not processed right. Cells go to infinite loop or something with inputs like What's your name?

[Dene33]

Checked one more time - everything works fine if I use phrases with apostrophes in the beginning. But previously I had this problem with apostrophes two times in a row. Maybe it's just a coincidence. I use colab version, by the way.

[MrKrzYch00]

For sure something hanged for me too in infinite loop (cpu once at 8 cores, once at 2 - this is what GPT-2 does when processing the input once, but it was repeating like that for half an hour or so until I ^C'ed). One of my prompts had apostrophe at the end but it wasn't the last one that hanged the AI in a loop.

^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^CTraceback (most recent call last):
  File "play.py", line 211, in <module>

  File "play.py", line 180, in play_aidungeon_2

  File "/data/AIDungeon/story/story_manager.py", line 181, in act
    result = self.generate_result(action_choice)
  File "/data/AIDungeon/story/story_manager.py", line 186, in generate_result
    block = self.generator.generate(self.story_context() + action)
  File "/data/AIDungeon/generator/gpt2/gpt2_generator.py", line 118, in generate
    return self.generate(prompt)
  File "/data/AIDungeon/generator/gpt2/gpt2_generator.py", line 118, in generate
    return self.generate(prompt)
  [Previous line repeated 37 more times]
  File "/data/AIDungeon/generator/gpt2/gpt2_generator.py", line 109, in generate
    text = self.generate_raw(prompt)
  File "/data/AIDungeon/generator/gpt2/gpt2_generator.py", line 92, in generate_raw
    self.context: [context_tokens for _ in range(self.batch_size)]
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
KeyboardInterrupt

[ghost]

tried this input and it appears to be hanging:
say "I'm sorry, I'm sorry, I don't know what I'm doing"

[ben-bay]

Weird! Can we verify that this error is consistently reproducible?

[Sporking]

Are we sure that absolutely all text that is submitted via JSON is autoescaped properly (on write) and autounescaped properly (on read) so that it is impossible for any submitted text in a JSON string to be interpreted as JSON special characters (quotes, apostrophes, brackets, etc.)? I have seen clear problems with this in earlier versions of AI Dungeon, resulting in JSON error messages that suggested that the code was probably vulnerable to JSON injection attacks. I haven't seen these problems in recent versions of AI Dungeon, but questions like this one make me wonder. Could there still be lurking problems of this sort? (See also https://xkcd.com/327/ and https://kevinsmith.io/sanitize-your-inputs , which despite its title actually recommends escaping outputs, not sanitizing inputs.)