-
Notifications
You must be signed in to change notification settings - Fork 14
Results Validation
Validation has been done against Matlab script results using the IVC and LIVE databases compared with the javascript implementation.
They match the original results exactly when run in Octave.
Note that the original Matlab scripts published by Wang et al., 2004 generate slightly different results than the ones they published (~1%). There are several potential sources for these discrepancies:
- Octave / Matlab implementation differences
- Usage of unreported non default values in the original study, like using a different downsizing factor by taking into account the specific distance between subjects and images.
- LIVE results are validated against the "Release 2" dataset which is more complete but computes MOS scores slightly differently. The original study used "Release 1" since it was the only one available at the time.
Thus, this project focuses on the exact reproduction of the Matlab scripts results in the JS implementation. That being said, differences between reported and computed results don't seem significant. The following plots illustrate both cases:
Reported Results | Computed Results |
---|---|
correlation: 0.828, root-mean-square-error: 68.752 | correlation: 0.821, root-mean-square-error: 68.752 |
You can find more information on how these graphs where generated here.
Bezkrovny's SSIM implementation is also available. The following graph shows how these different algorithms correlate with MOS for different kinds of distorsions from the LIVE dataset Release 2:
JPEG Results | Gaussian Blur |
---|---|
White Noise | Fast Fading |
Additional information on this graph and how to replicate it is available here.