#68: account for output transform

[russelltg]

Fixes #68

This was all surprisingly complicated, ha! @Andeskjerf can you test and confirm? Tested locally with all transforms

[Andeskjerf]

Recording my main monitor now works without problem when using slurp :) Thank you!

However, trying to record my rotated monitor fails. I tried both with and without hardware encoding, both failing with different errors.

Hardware

Opening libva device from DRM device /dev/dri/renderD128
[Parsed_transpose_vaapi_2 @ 0x621f7b13e0c0] VAAPI driver doesn't support transpose
[Parsed_transpose_vaapi_2 @ 0x621f7b13e0c0] Failed to configure output pad on Parsed_transpose_vaapi_2
thread 'main' panicked at src/main.rs:1762:18:
called `Result::unwrap()` on an `Err` value: ffmpeg::Error(22: Invalid argument)
stack backtrace:
   0: rust_begin_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1649:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1073:23
   4: wl_screenrec::video_filter
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:1762:5
   5: wl_screenrec::EncState::new
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:1361:47
   6: <wl_screenrec::State as wayland_client::event_queue::Dispatch<wayland_protocols_wlr::screencopy::v1::generated::client::zwlr_screencopy_frame_v1::ZwlrScreencopyFrameV1,()>>::event
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:538:35
   7: wayland_client::event_queue::queue_callback
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:656:5
   8: wayland_client::event_queue::EventQueue<State>::dispatching_impl
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:478:13
   9: wayland_client::event_queue::EventQueue<State>::dispatch_pending
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:384:9
  10: wayland_client::event_queue::EventQueue<State>::blocking_dispatch
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:406:9
  11: wl_screenrec::main
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:1839:9
  12: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Software

Opening libva device from DRM device /dev/dri/renderD128
[Parsed_scale_vaapi_1 @ 0x5dad99ccc4c0] Hardware does not support output format yuv420p.
[Parsed_scale_vaapi_1 @ 0x5dad99ccc4c0] Failed to configure output pad on Parsed_scale_vaapi_1
thread 'main' panicked at src/main.rs:1762:18:
called `Result::unwrap()` on an `Err` value: ffmpeg::Error(22: Invalid argument)
stack backtrace:
   0: rust_begin_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1649:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1073:23
   4: wl_screenrec::video_filter
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:1762:5
   5: wl_screenrec::EncState::new
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:1361:47
   6: <wl_screenrec::State as wayland_client::event_queue::Dispatch<wayland_protocols_wlr::screencopy::v1::generated::client::zwlr_screencopy_frame_v1::ZwlrScreencopyFrameV1,()>>::event
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:538:35
   7: wayland_client::event_queue::queue_callback
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:656:5
   8: wayland_client::event_queue::EventQueue<State>::dispatching_impl
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:478:13
   9: wayland_client::event_queue::EventQueue<State>::dispatch_pending
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:384:9
  10: wayland_client::event_queue::EventQueue<State>::blocking_dispatch
             at /home/vlad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wayland-client-0.31.2/src/event_queue.rs:406:9
  11: wl_screenrec::main
             at /home/vlad/Documents/Git/wl-screenrec/src/main.rs:1839:9
  12: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I also tried using -o to record my monitors, and this fails for the transformed monitor. The errors appear identical to the above.

For reference, I use a RX 6800 XT with mesa drivers.

Edit

Updated the logs with the debug backtrace rather than release

[russelltg]

Ah, darn...

VAAPI driver doesn't support transpose

I'll need to look at other ways to do this. Maybe vulkan or something...

[rj1]

can confirm I am experiencing this exact issue as well

[svenstaro]

I'm still getting VAAPI driver doesn't support transpose on my Radeon 780M. Shouldn't that particular issue still be tracked?

Funnily enough, it also crashes with --no-hw:

wl-screenrec --no-hw -o DP-6 -f lol.mkv
Opening libva device from DRM device /dev/dri/renderD128
[Parsed_scale_vaapi_1 @ 0x5eb38a66e740] Hardware does not support output format yuv420p.
[Parsed_scale_vaapi_1 @ 0x5eb38a66e740] Failed to configure output pad on Parsed_scale_vaapi_1

[russelltg]

Yeah, --no-hw only disables HW encoding, not HW scale/flip/pixel format conversion.

For output transform on radv, I think the best best is wait for the vulkan support. Would be curious to have someone test that (See #90). I still need to add transform support to that PR, but would be interesting to see if it works on radv with no transform

[svenstaro]

Sure, will get right on that. Can we chat somewhere real time (Discord, Matrix) for debugging, multiple stabs at this, etc?