Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MP3: Mismatch with max delta from 0.002 to 2.0 #78

Closed
Shnatsel opened this issue Dec 4, 2021 · 7 comments
Closed

MP3: Mismatch with max delta from 0.002 to 2.0 #78

Shnatsel opened this issue Dec 4, 2021 · 7 comments

Comments

@Shnatsel
Copy link
Contributor

Shnatsel commented Dec 4, 2021

These files show max delta ranging from 0.002 to 2.0 when compared against ffmpeg with symphonia-check.

Tested on commit 33f0701 (latest as of this writing).

As we have learned in #72, ffmpeg has divergences with other decoders, so running the check on these files against other decoders seems prudent.

The files are from the Pony Music Archive corpus and are free to redistribute. I have run the check on half of it because I don't have enough disk space for the entire thing. This amounts to roughly 90Gb of MP3 (12831 individual files).

@Shnatsel
Copy link
Contributor Author

Shnatsel commented Dec 4, 2021

The chunk I've tested covers bitrates from 32kbit/s to 320kbit/s and a wide range of encoders.

List of distinct encoders from ID3 metadata
    encoded by: Adobe Systems Inc
    encoded by: Archie
    encoded by: dBpoweramp Release 15
    encoded by: dBpoweramp Release 15.3
    encoded by: Dustin Branscum (itstartsatdusk@gmail.com)
    encoded by: FL Studio 10
    encoded by: Free Mp3 Wma Converter
    encoded by: https://pony.fm/
    encoded by: https://pony.fm/, LAME in FL Studio 10, LAME in FL Studio 10
    encoded by: https://pony.fm/, Lavf52.32.0, Lavf52.32.0
    encoded by: https://soundcloud.com/brenton-mattheus
    encoded by: https://www.facebook.com/PunxxMusic
    encoded by: iTunes 10.1.1.4
    encoded by: iTunes 10.4
    encoded by: iTunes 10.4.0.80
    encoded by: iTunes 10.5
    encoded by: iTunes 10.5.0.142
    encoded by: iTunes 10.5.1
    encoded by: iTunes 10.5.2
    encoded by: iTunes 10.5.3
    encoded by: iTunes 10.5.3.3
    encoded by: iTunes 10.6.1
    encoded by: iTunes 10.6.1.7
    encoded by: iTunes 10.6.3.25
    encoded by: iTunes 10.7.0.21
    encoded by: iTunes 11.0.1
    encoded by: iTunes 11.0.1.12
    encoded by: iTunes 11.0.2.26
    encoded by: iTunes 11.0.4.4
    encoded by: iTunes 11.1.3
    encoded by: iTunes 11.1.3.8
    encoded by: iTunes 11.1.4
    encoded by: iTunes 11.2.2.3
    encoded by: iTunes 11.3.0.54
    encoded by: iTunes 11.3.1.2
    encoded by: iTunes 12.0.1.26
    encoded by: iTunes 12.1.0.50
    encoded by: iTunes 12.2.1.16
    encoded by: iTunes 12.3.0.44
    encoded by: iTunes 12.3.3.17
    encoded by: Johan K & Desso
    encoded by: LAME
    encoded by: LAME in FL Studio 10
    encoded by: LAME in FL Studio 11
    encoded by: LAME in FL Studio 12
    encoded by: LAME in FL Studio 20
    encoded by: LAME in FL Studio 8
    encoded by: LAME in FL Studio 9
    encoded by: Lavf51.12.2
    encoded by: Lavf52.16.0
    encoded by: Lavf52.32.0
    encoded by: Logic Pro
    encoded by: LOOPLABS
    encoded by: Mixcraft 3.1 Build 41
    encoded by: Mixcraft 5.2 Build 151
    encoded by: Mixcraft 5.2 Build 152
    encoded by: Mixcraft 6.0 Build 191
    encoded by: Mixcraft 6.0 Build 196
    encoded by: Mixcraft 6.0 Bygget 196
    encoded by: Mixcraft 6.1 Build 204
    encoded by: Mixcraft 6.1 Build 213
    encoded by: Mixcraft 6.1 Build 216
    encoded by: Mixcraft 6.1 Build 217
    encoded by: Mixcraft 6.1 Version 204
    encoded by: Mixcraft 6.1 Version 217
    encoded by: Mixcraft 7.1 64-Bit Build 264
    encoded by: Mixcraft 7.1 64-Bit Build 267
    encoded by: Mixcraft 7.5 32-Bit Build 292
    encoded by: Mixcraft 7.7 32-Bit Build 316
    encoded by: N/A
    encoded by: No
    encoded by: Oz
    encoded by: Pro Tools
    encoded by: REAPER
    encoded by: Ripped with Streamripper
    encoded by: SUPER(C)   
    encoded by: www.ytbmp3.com

@pdeljanov
Copy link
Owner

Hi @Shnatsel,

This may take me a while to get to. Do you know if these also all fail with mpg123?

@Shnatsel
Copy link
Contributor Author

Shnatsel commented Dec 10, 2021

20 out of the 88 files in the archive have significant divergences from both ffmpeg and mpg123.

@Shnatsel
Copy link
Contributor Author

The file sharing service seems to be having issues with uploads, so here's a list:

Mismatches with both ffmpeg and mpg123 (listed max delta is against mpg123)
 2.00000000 Artists/Dj Shadow Music/The Pony I Wanna Be (Dj Shadow Music Remix)/The Pony I Wanna Be (Dj Shadow Music Remix).mp3
 1.25388896 Artists/Flyghtning/Trixie.mp3
 1.11745858 Artists/AJ4Y/My Little Dashie - Fan Made Song.mp3
 1.08737457 Artists/AnalogShift/Changling (The Quick AnalogShift Remix).mp3
 1.08079827 Artists/FiMFlamFilosophy/Bones And Skin.mp3
 1.00200605 Artists/Doctor Whooves, PhD/Ponies/Ballad of the Crystal Fair (DoctorWhoovesPhD Cover).mp3
 0.92904425 Artists/Cpl. Bradley/d.licious (Cpl. Bradley Remix).mp3
 0.87464571 Artists/Buffalo Brony/Nightfall prod. Cherax Destructor.mp3
 0.79491532 Artists/Cuaii/Crystal Hearts.mp3
 0.67264771 Artists/Buffalo Brony/Silva Hound - Hooves Up High Stampede Remix (Buffalo Brony).mp3
 0.59592271 Artists/AspectOfTheStorm/Octavia's Sorrow (Remastered).mp3
 0.42231899 Artists/Dark Symphony/Light in the Rainy Darkness.mp3
 0.26129109 Artists/DannyBrony/Me and You (Let Friendship Lead the Way).mp3
 0.25381893 Artists/Fortissimo/Boggy Brew.mp3
 0.09575542 Artists/Laservega/Giggle at the Ghostly (MLP space mix).mp3
 0.02876295 Artists/Assertive Fluttershy/Luna (Assertive Fluttershy Remix).mp3
 0.00333370 Artists/Arkane Metal/Philippine Bronies OC Themesongs Volume 1/Silverite's Theme (Digital Evolution).mp3
 0.00242868 Artists/Chang31ing/Hearts (Chang31ing's Retwerk).mp3
 0.00034976 Artists/Elision/Boop (Theme of Marker Pony).mp3
 0.00003223 Artists/FlyingNinjaBannanas/Open You Heart ~ Wonders Mix V.mp3

@Shnatsel
Copy link
Contributor Author

File sharing is back up, so here's the archive with only the files that fail in both mpg123 and ffmpeg:
https://mega.nz/file/ZglWCTqT#KDFuyeWLwzGqLyjIa_Lp3wStBsVEl-6tMv8NeiKPZB4

@Shnatsel
Copy link
Contributor Author

Shnatsel commented Jan 22, 2022

Would it make sense to re-run the test on the MP3 corpus with the new FFT implementation? Edit: nevermind, I'll get a bigger corpus and re-run.

I am particularly eager to get the MP3 decoder fully working because Symphonia provides the only pure-Rust, 100% safe MP3 decoder, and I'd like to make it the default in Rodio which currently relies on C code.

pdeljanov added a commit that referenced this issue Jan 23, 2022
pdeljanov added a commit that referenced this issue Jan 23, 2022
Only the first packet should contain these tags. If they appear
elsewhere in the stream they will decode to silence.

Addresses #78.
@pdeljanov
Copy link
Owner

Hi @Shnatsel,

No, you don't need to re-run the corpus. The FFT is not used by the MP3 decoder since it has no benefit in terms of accuracy or performance.

I went through all the files and they are all either now fixed, or not a bug. Details below.

I'm hoping that 0.5 can be released in 2 weeks or so. The only outstanding issue I really want to solve before then is #96. I also want to get #66 merged for the release as well.


This file had multiple LAME Xing/Info packets which were rightly discarded by ffmpeg but not Symphonia. We now discard them properly.

/AJ4Y/My Little Dashie - Fan Made Song.mp3

These files have a VBRI packet which was correctly being decoded as silence, but should've been consumed by the demuxer. This was actually on my todo list, I just didn't have a sample file. It is now implemented and these files are fixed.

/Elision/Boop (Theme of Marker Pony).mp3
/Arkane Metal/Philippine Bronies OC Themesongs Volume 1/Silverite's Theme (Digital Evolution).mp3
/FlyingNinjaBannanas/Open You Heart ~ Wonders Mix V.mp3

The following files are corrupted and produce audible artifacts in both ffmpeg and Symphonia. I believe that fixing #96 may improve the situation a bit, but overall these ones aren't bugs.

/Chang31ing/Hearts (Chang31ing's Retwerk).mp3
/FiMFlamFilosophy/Bones And Skin.mp3
/Cuaii/Crystal Hearts.mp3
/Dj Shadow Music/The Pony I Wanna Be (Dj Shadow Music Remix)/The Pony I Wanna Be (Dj Shadow Music Remix).mp3
/Dark Symphony/Light in the Rainy Darkness.mp3
/DannyBrony/Me and You (Let Friendship Lead the Way).mp3

The following files have junk data after some packets that cause ffmpeg to desynchronize and skip a packet. Symphonia remains synchronized and plays the packet ffmpeg dropped.

/AspectOfTheStorm/Octavia's Sorrow (Remastered).mp3
/Cpl. Bradley/d.licious (Cpl. Bradley Remix).mp3
/Laservega/Giggle at the Ghostly (MLP space mix).mp3

This file is perfectly okay, but ffmpeg is using an incorrect value in one of their lookup tables. I think at some point the value was ambiguous, but we use the value that's stated in the official standard and so do most other decoders. The table ffmpeg uses hasn't changed in 21 years according to the Git history so I'm guessing its just been missed since it's for an odd sample rate (24kHz).

/Fortissimo/Boggy Brew.mp3

The rest of the files were fixed when I added proper support for gapless playback a few weeks ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants