Add ‘ignore’ for UTF8 decoding to avoid error on non-UTF8 characters

[chrisapplegate]

• If this is a bugfix, I have provided code that clearly demonstrates the problem and that works when used with this PR
• I have added a test to the test suite, if necessary
• I have properly documented new or changed features in the documention, or the docstrings
• I have properly documented unusual changes to the code in the comments around it
• I have made note of any breaking/backwards incompatible changes

I encountered an issue using moviepy with a video that had been produced by a Spanish-speaking client. In the metadata of the video, the description of the video had a non-UTF8 character in it - the letter "í" of "vídeo" was encoded in some non-UTF8 encoding.

    Metadata:
      creation_time   : 2018-10-11T14:40:32.000000Z
      handler_name    : Gestor de contenido de vídeo Apple

This was causing line 262 of moviepy/video/io/ffmpeg_reader.py, the function ffmpeg_parse_infos, to bug out with a UnicodeDecodeError:

'utf-8' codec can't decode byte 0x92 in position 1712: invalid start byte

I am not overtly familiar with this code base but I don't think adding ignore causes the code to break in any meaningful way. The code is not parsing or relying on that Metadata field, and as the exception wasn't being caught or handled, I don't think there is an expected input for this function that was working, and now no longer works thanks to this change.

[vildric]

this fix raise an OSError for me :

Traceback (most recent call last):
File "/vagrant/moviepy/video/io/ffmpeg_writer.py", line 134, in write_frame
self.proc.stdin.write(img_array.tobytes())
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/vagrant/apps/videos/models.py", line 98, in update_taster
clip.write_videofile(filename=self.video3.path, threads=4, progress_bar=False)
File "", line 2, in write_videofile
File "/vagrant/moviepy/decorators.py", line 54, in requires_duration
return f(clip, *a, **k)
File "", line 2, in write_videofile
File "/vagrant/moviepy/decorators.py", line 137, in use_clip_fps_by_default
return f(clip, *new_a, **new_kw)
File "", line 2, in write_videofile
File "/vagrant/moviepy/decorators.py", line 22, in convert_masks_to_RGB
return f(clip, *a, **k)
File "/vagrant/moviepy/video/VideoClip.py", line 327, in write_videofile
progress_bar=progress_bar)
File "/vagrant/moviepy/video/io/ffmpeg_writer.py", line 225, in ffmpeg_write_video
writer.write_frame(frame)
File "/vagrant/moviepy/video/io/ffmpeg_writer.py", line 178, in write_frame
raise IOError(error)
OSError: [Errno 32] Broken pipe

MoviePy error: FFMPEG encountered the following error while writing file /vagrant/media/videos/video/tasters/81_5I1Zgx6.mp4:

b''

[tburrows13]

@chrisapplegate would you be able to provide a video that causes this behaviour?

[mylyu]

I support this fix

[mylyu]

No problem for me using this fix.

[chrisapplegate]

@chrisapplegate would you be able to provide a video that causes this behaviour?

I am afraid not - when I filed this PR I was contracting for a client and I am no longer working for them. In any case, the video was confidential. It would not be too hard for me to annotate a public-domain video with the same, if you like?

[coveralls]

Coverage remained the same at 67.575% when pulling bee616d on chrisapplegate:feature/fix-utf8-issue into b65d293 on Zulko:master.

[believeinnovation]

Here is one. We have the same problem from our side with this video : https://www.dropbox.com/s/k41malnj0cod4zt/test-videoblocks-footage-of-a-crowd-partying-at-a-rock-concert-or-dj-party_srgs6suqzb__a3c1a2474db660108239cf12e44a8ca7__P360.mp4?dl=0
It results in this error :
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/moviepy/video/io/VideoFileClip.py", line 91, in init
fps_source=fps_source)
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/moviepy/video/io/ffmpeg_reader.py", line 33, in init
fps_source)
File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/moviepy/video/io/ffmpeg_reader.py", line 262, in ffmpeg_parse_infos
infos = proc.stderr.read().decode('utf8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xce in position 2013: invalid continuation byte

[tburrows13]

@believeinnovation, does it work if you update to v2.0.0.dev2? (Use --pre with pip to access pre-release versions).

[believeinnovation]

Hello Tom, Yes with 2.0.0.dev2 this video file works 👍🏼 Is there any change file for this new version ?

[tburrows13]

Good to hear! Changelog is at https://github.com/Zulko/moviepy/blob/master/CHANGELOG.md#v200dev2-2020-10-05

[Loskir]

is this still not released? :(

[tburrows13]

You can install v2.0.0.dev2 with pip via using the --pre flag :)

[liulijiexian]

You can install v2.0.0.dev2 with pip via using the --pre flag :)您可以通过使用 --pre 标志 :) 来安装 v2.0.0.dev2 pip
Even if updated to v2.0.0.dev2

The following error still occurs：

ffmpeg_reader.py

Attempt to delete the .decode("utf8", errors="ignore")

new error occurs UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 241: invalid start byte

python: Python 3.8.8。