Reducing mouse+display lag investigations (mainly for Dear ImGui apps)

[floooh]

Sort of a reminder ticket. See here:

floooh/cimgui-sokol-starterkit#2

And this also overlaps with the "only render on input" mode for sokol_app.h (because at least on Windows switching off vsync and rendering from within WM_MOUSEMOVE already makes a dramatic difference).

Also see:

#341

#301

#292

#248

...these are all more or less related. An idea would be to add a couple different "update modes" to sokol_app.h which would tune the swapchain setup and the place where the frame callback is called for different scenarios. Alternatively instead of modes, add a number of sapp_desc options to tweak unique things instead of grouping them into "modes".

[floooh]

...probably not an option but:

https://github.com/nlguillemot/LateLatching

[floooh]

...another information tidbit:

https://docs.microsoft.com/en-us/windows/uwp/gaming/reduce-latency-with-dxgi-1-3-swap-chains

[slmjkdbtl]

not sure if really related but I fixed #341 by changing the macOS timer based loop to manual input polling loop like this (borrowed code from glfw, all changes are marked by // CHANGE), haven't tested if it breaks metal so haven't submitted a PR

[tanoxyz]

I'm struggling with the same issue with linux/opengl, i did this quick experiment to reduce the lag.

This adds some padding time before the frame based on the time wasted on the previous frame. The idea is to keep the en of the logic close to the actual swap of the buffer.

Pad -> Logic -> vsync -> Swap buffer -\
 ^                                    |
 |____________________________________|

APP_PUBLIC int
APP_Run_application_loop(int(*application_frame)(void)) {
    assert(application_frame);
    int loop_result = 0;
    static i64 pad_time = 0;
    for (;;) {
        i64 frame_begin = APP_Time();
        // Sleep based on the padding time
        struct timespec remaining, request = {0, pad_time};
        nanosleep(&request, &remaining);

        APP__Process_events();
        loop_result = application_frame();
        if (loop_result != 0) {break;}
        APP__Clear_events();

        i64 frame_end  = APP_Time();
        __builtin_printf("Frame: %fms\n", (f32)(frame_end-frame_begin)*1e-6f);

        i64 swap_begin = APP_Time();
        APP__Swap_buffers();
        APP_Set_swap_interval(0); //_The only way I found to really syncronize :/
        APP_Set_swap_interval(1); //
        i64 swap_end = APP_Time();

        i64 swap_time = swap_end-swap_begin; // This will be the wasted time
        __builtin_printf("Swap: %fms\n", (f32)(swap_time)*1e-6f);

        // If we dont have pad time we add it as 3/4 of the wasted time
        // If we have pad time but the wasted time is greater, we reset it.
        if      (pad_time == 0)        pad_time = 3*swap_time/4;
        else if (swap_time > pad_time) pad_time = 0;

    }
#if defined(APP_WINDOWS)
    PostQuitMessage(0);
#endif
    if (loop_result == 1) {
        return 0;
    }
    return loop_result;
}

[floooh]

Btw one (embarassingly simple) latency improvement on Windows with D3D/DXGI was this recent fix:

sokol/sokol_app.h

Line 5831 in a180313

_sapp_dxgi_SetMaximumFrameLatency(_sapp.d3d11.dxgi_device, 1);

For some reason DXGI has 3 frames of latency by default, which is good for video playback, but bad for interactive applications. Dragged items still don't stick to the mouse cursor, but it's at least much better than before.