Chroma 422 mode causes enc/dec mismatch and its test cannot detect this

[ycho]

It passes "cargo test --release --features=decode_test chroma_sampling_422_aom",
but encode and decoder mismatches if tried for larger size input like:
./target/release/rav1e https://media.xiph.org/video/derf/y4m/flower_garden_422_ntsc.y4m -o test.ivf -r test_rec.y4m --quantizer 100 --speed=5 --limit=1

FYI, that current chroma 422 and 444 are all encoded as INTRA frames, i.e. inter modes are NOT used.

[lu-zero]

aom:

Diff < left / right > :
 [
     43,
     173,
     102,
     133,
     233,
     188,
     76,
     216,
     62,
     170,
     109,
     60,
     86,
     124,
     202,
     36,
     42,
     134,
     240,
     183,
     105,
     215,
     144,
     21,
     16,
     237,
     109,
     130,
     143,
     162,
     6,
     14,
     39,
     136,
     127,
     255,
     118,
     37,
     22,
     41,
     106,
     139,
     167,
     226,
     251,
     185,
     58,
     1,
     94,
     192,
     117,
     14,
     148,
     62,
     7,
     181,
     164,
     15,
     16,
     212,
     202,
     4,
     129,
     1,
     12,
     73,
     201,
     35,
     164,
     97,
     109,
     87,
     248,
     36,
     209,
     126,
     54,
     155,
     35,
     141,
     248,
     58,
     13,
     73,
     39,
     75,
     249,
     135,
     30,
     78,
     53,
     246,
     122,
     55,
     39,
     230,
     174,
     12,
     79,
     178,
     81,
     93,
     41,
     20,
     125,
     197,
     189,
     134,
     248,
     100,
     22,
     138,
     125,
     154,
     126,
     232,
     69,
     179,
     210,
     118,
     64,
     163,
     16,
     120,
     163,
     97,
     230,
     102,
     103,
     43,
     198,
     139,
     219,
     185,
     80,
     137,
     42,
     160,
     67,
     22,
     135,
     128,
     27,
     199,
     159,
     71,
     199,
     210,
     89,
     173,
     231,
     54,
     215,
     104,
     229,
     123,
     138,
     191,
     180,
     228,
<    40,
>    39,
     254,
     48,
     17,
     67,
     229,
<    69,
>    70,
     109,
     105,
     218,
     218,
     80,
     136,
     123,
     61,
     223,
     151,
     126,
     98,
     188,
     236,
     232,
     87,
     199,
     170,
     6,
     20,
     244,
     181,
     94,
     91,
     160,
     66,
     106,
     24,
     158,
     214,
     114,
     153,
     150,
     224,
     160,
     91,
     217,
     230,
     206,
     41,
     168,
     195,
     149,
     185,
     200,
     18,
     235,
     3,
     163,
     117,
     162,
     195,
     43,
     149,
     164,
     105,
     239,
     64,
     151,
     122,
     226,
     160,
     254,
     157,
     68,
     216,
     252,
     137,
     59,
     167,
     132,
     57,
     90,
     66,
     208,
     253
 ]

dav1d

Diff < left / right > :
 [
     43,
     173,
     102,
     133,
     233,
     188,
     76,
     216,
     62,
     170,
     109,
     60,
     86,
     124,
     202,
     36,
     42,
     134,
     240,
     183,
     105,
     215,
     144,
     21,
     16,
     237,
     109,
     130,
     143,
     162,
     6,
     14,
     39,
     136,
     127,
     255,
     118,
     37,
     22,
     41,
     106,
     139,
     167,
     226,
     251,
     185,
     58,
     1,
     94,
     192,
     117,
     14,
     148,
     62,
     7,
     181,
     164,
     15,
     16,
     212,
     202,
     4,
     129,
     1,
     12,
     73,
     201,
     35,
     164,
     97,
     109,
     87,
     248,
     36,
     209,
     126,
     54,
     155,
     35,
     141,
     248,
     58,
     13,
     73,
     39,
     75,
     249,
     135,
     30,
     78,
     53,
     246,
     122,
     55,
     39,
     230,
     174,
     12,
     79,
     178,
     81,
     93,
     41,
     20,
     125,
     197,
     189,
     134,
     248,
     100,
     22,
     138,
     125,
     154,
     126,
     232,
     69,
     179,
     210,
     118,
     64,
     163,
     16,
     120,
     163,
     97,
     230,
     102,
     103,
     43,
     198,
     139,
     219,
     185,
     80,
     137,
     42,
     160,
     67,
     22,
     135,
     128,
     27,
     199,
     159,
     71,
     199,
     210,
     89,
     173,
     231,
     54,
     215,
     104,
     229,
     123,
     138,
     191,
     180,
     228,
<    40,
>    39,
     254,
     48,
     17,
     67,
     229,
<    69,
>    70,
     109,
     105,
     218,
     218,
     80,
     136,
     123,
     61,
     223,
     151,
     126,
     98,
     188,
     236,
     232,
     87,
     199,
     170,
     6,
     20,
     244,
     181,
     94,
     91,
     160,
     66,
     106,
     24,
     158,
     214,
     114,
     153,
     150,
     224,
     160,
     91,
     217,
     230,
     206,
     41,
     168,
     195,
     149,
     185,
     200,
     18,
     235,
     3,
     163,
     117,
     162,
     195,
     43,
     149,
     164,
     105,
     239,
     64,
     151,
     122,
     226,
     160,
     254,
     157,
     68,
     216,
     252,
     137,
     59,
     167,
     132,
     57,
     90,
     66,
     208,
     253
 ]

I updated the test to use a larger resolution and looks like some rounding somewhere is causing the problem.

[lu-zero]

On power9:

---- test_encode_decode::chroma_sampling_422_dav1d stdout ----
Encoding 98x80 speed 0 quantizer 100 bit-depth 8
Encoded packet 0
Decoding frame 0
Decoded. -> 0
Retrieving frame
Retrieved.
Retrieving frame
Retrieved.
Encoded packet 1
Decoding frame 1
Decoded. -> 0
Retrieving frame
Retrieved.
thread 'test_encode_decode::chroma_sampling_422_dav1d' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 [
     48,
     48,
     74,
     24,
     209,
     161,
     229,
     47,
     52,
     152,
     66,
     99,
     47,
     61,
     18,
     36,
     64,
     145,
     109,
     239,
     37,
     66,
     207,
     7,
     246,
     31,
     211,
     211,
<    178,
>    177,
     141,
     78,
     165,
     71,
     134,
     27,
     248,
     226,
     127,
     23,
     228,
     56,
     126,
     78,
     245,
     232,
     227,
     43,
     15,
     194
 ]

', src/test_encode_decode.rs:110:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.


failures:
    test_encode_decode::chroma_sampling_422_dav1d

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 63 filtered out

[rzumer]

I confirm this also happens on nyan.y4m when converted to 4:2:2. It seems it was still occurring when 4:2:2 input was enabled, but since automated tests were passing I had not noticed it. 4:4:4 input appears to work. The culprit is CDEF.

[shssoichiro]

Is this possibly related to chroma sample positioning? In rav1e we say ChromaSamplePosition is Colocated for 4:2:2, but in daala it is treated as Vertical. They also have a conversion function that they run on 4:2:2 or yuv420mpeg2 input to convert from Vertical to Colocated, which we do not have. This was responsible for small mismatches when I was developing the av-metrics crate. The code I wrote in that crate can probably be reused for this as well (better, it would be nice to have that all in the y4m crate or some crate that wraps y4m with higher level abstractions and generic pixel types).

[ycho]

Is this possibly related to chroma sample positioning? In rav1e we say ChromaSamplePosition is Colocated for 4:2:2, but in daala it is treated as Vertical. They also have a conversion function that they run on 4:2:2 or yuv420mpeg2 input to convert from Vertical to Colocated, which we do not have. This was responsible for small mismatches when I was developing the av-metrics crate. The code I wrote in that crate can probably be reused for this as well (better, it would be nice to have that all in the y4m crate or some crate that wraps y4m with higher level abstractions and generic pixel types).

I think not. When I check for desync, I compare yuv files. For it, I first convert any reconstructed (encoder side) y4m and decoded (decoder side) y4m to yuv using non-daala tools, like ffmpeg. Also, I run aomdec to directly output yuv.

[barrbrain]

The PR #2224 will resolve the desync but there exists a more general issue of the bitstream feature coverage of encode-decode tests.