an attacker can crash the watcher thread with a non-utf8 filename

[lovasoa]

Hello,

We recently encountered a serious security issue while using this library.

When an attacker (or a normal user, as a mistake) is able to create a file in a folder watched by watchdog v0.x for python 2, they can crash the event processing thread. Only that thread crashes, not the main thread, so a service using this library would stay alive, but unable to process any subsequent file. This allows for a very powerful and surreptitious denial of service attack against any file processing service using this library. The attack would not be immediately noticeable because the service would still be running, just not processing any more file.

stack trace

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/myservice/venv/local/lib/python2.7/site-packages/watchdog/observers/api.py", line 199, in run
    self.dispatch_events(self.event_queue, self.timeout)
  File "/myservice/venv/local/lib/python2.7/site-packages/watchdog/observers/api.py", line 368, in dispatch_events
    handler.dispatch(event)
  File "/myservice/venv/local/lib/python2.7/site-packages/watchdog/events.py", line 440, in dispatch
    paths.append(unicode_paths.decode(event.src_path))
  File "/myservice/venv/local/lib/python2.7/site-packages/watchdog/utils/unicode_paths.py", line 63, in decode
    path = path.decode(fs_fallback_encoding, 'strict')
  File "/myservice/venv/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe8 in position 47: invalid continuation byte

[BoboTiG]

Python 2.7 is no more supported. Do you reproduce with the latest version, if you can make the upgrade?

[lovasoa]

Unfortunately, Python 2.7 is still far from being dead. It is supported, and is the default python version in several stable long term support distributions. It is the most recent python version on the environment we deploy to.

In any case, a file name can be any sequence of bytes (depending on the filesystem), it does not have to be a valid string in any particular encoding. Maybe it would be possible to backport a small security fix that simply avoids crashing the event processing thread in case of error ?

[lovasoa]

Would you accept a pull request to the python2.7 branch that fixes the issue ?

[BoboTiG]

Give it a try and I'll see what I can do :)

[lovasoa]

@BoboTiG : I made the PR. This would be of a tremendous help to us if this could be released 🙂

[BoboTiG]

It is merged. I just checked and the Python 2.7 branch contains a lot more work than just your patch. Would you be able to try it out and let me know if all works fine for you?

[BoboTiG]

@CCP-Aporia Do you need other backports to the Python 2.7 branch?

[CCP-Aporia]

Not as far as I can tell at the moment, but it's been a few weeks since I last had the chance to spend time on watchdog. If anything, I'd be creating a PR to backport more of the recent changes if necessary. 🙂

[lovasoa]

The last version of the python2 branch is ok for us !

[BoboTiG]

Great! Let's do a new release then ;)

[lovasoa]

@BoboTiG : So, you are going to release that as 0.10.7 ?
When do you think this can be done ?

https://pypi.org/project/watchdog/#history

[BoboTiG]

0.10.7 is online ;)