Tips to improve translation.

[JeanDown123]

Tips to improve translation.

For Video YouTube: ¡Bienvenidos a Extraordinerd!

Audio

Result Run Script Autosub

Results: Very BAD

Methods Improve Translation..

Method 1 : Trim/Crop

Crop video, 10 seconds Audio.

Results:

Speech: Medium
Timing: Good

Method 2 Silence noise

Silence Audio (Noise,music, etc.)

Observations: The best method, but it requires a long time to identify the sounds, voices, ambient noise.

Results: Very good

Method 3 Use Filters

Use Filters of FFmpeg, loudnorm or dynaudnorm
ffmpeg -i myvideo.mp4 -af loudnorm=i=-5 myvideoFilter.wav

Results: Very good

Observations: It requires a special compilation of ffmpeg, but can be downloaded from here.

Method 4 Change threshold

Modify the script, and Zero the value of threshold=0

Observations:this case, it worked like that with the filter, but it is not always so. LOST TOTAL TIMING.

Notes

Note 1 : For speed, the audio must be converted to mono and frequency to 16000
Note 2 : For methods 2 to 4, the script could not identify the silences, so cutting it done automatically at 6.1 seconds (min_region_size = 0.5, max_region_size = 6)

[agermanidis]

Thanks for doing all this work, @JeanDown123. I'll try to incorporate the filtering to autosub shortly.

[stevenj]

I too am playing with filters and your program. I notice that in the extract_audio function the audio is down mixed to mono. I found that by applying the filter to help eliminate phase errors before the down mix, the resulting audio was higher quality and resulted in more text being converted from the file.
See: https://trac.ffmpeg.org/wiki/AudioChannelManipulation

A small problem with this filter, is sometimes (for some reason i can't work out) that filter causes ffmpeg to crash, in which case I just fall back to a straight conversion to mono with no phase correction.

Also, filtering ambient noise using the tips from: https://manerosss.wordpress.com/2017/07/24/ffmpeg-%C2%B7-apply-a-filter-to-enhance-voice-by-removing-low-and-high-frequency-noises/

Improved word recognition. More words in my samples were recognised, and the translations made more sense, generally.

Finally I then used the tool: ffmpeg-normalize to do an automatic normalisation, as in Method 3 above.

Using all three methods in this order resulted in a significant increase in detected words, and translation accuracy.

[GunGunGun]

@stevenj Hi Steven, can you please share some of your configs ? Thanks!

[stevenj]

@GunGunGun Here is the script I am currently using to automatically filter the audio and then generate the subtitles. If you come up with any improvements, let me know.
https://gist.github.com/stevenj/4a4af2723c1c4aa6898bbaf8d8a6ec69

You will also need this tool:
https://github.com/slhck/ffmpeg-normalize

[Wolfenk]

Thank you for the tips @JeanDown123

Thank you too @stevenj ! Is it possible to share your script again? The link don't work 😞

[BingLingGroup]

Thank you for the tips @JeanDown123

Thank you too @stevenj ! Is it possible to share your script again? The link don't work 😞

@Wineliva I found the script here. Perhaps it's what you need.

[stevenj]

@Wineliva Yes the link shared by @BingLingGroup is the one i use.

[BingLingGroup]

@Wineliva @stevenj

I just write the pre-process script into the autosub codes. Now you can pre-process the audio directly from my version of autosub.

Default pre-process commands need ffmpeg-normalize. Of course you can write it youself by using the -ap input options. But remember to set pre-processing output format to 44.kHz/24bit/mono flac. Currently I don't write the logic to judge the format. It will be used directly by speech-to-text method. And when that method cut the clips, it use copy arg so it is very risky when your format isn't proper.

You can install it from my repo by using pip. Or wait for me to release. I write pretty some features now. I think I will release it in a few more days.

[BingLingGroup]

Thank you for the tips @JeanDown123

Thank you too @stevenj ! Is it possible to share your script again? The link don't work 😞

Finally, I release a standalone version. You can check it here.