Precompilation for faster Pluto startup

[lungben]

In SciML a trick is used for reducing startup times - essentially it is just including a minimal example inside the source code module itself. This example is then executed in precompilation stage and the functions it requires are then already compiled when the user need them.

See
https://discourse.julialang.org/t/22-seconds-to-3-and-now-more-lets-fix-all-of-the-differentialequations-jl-universe-compile-times/66313

and

https://github.com/SciML/OrdinaryDiffEq.jl/blob/v5.61.1/src/OrdinaryDiffEq.jl#L175-L193

For measuring startup performance, I used the following script:

using Pluto

test_notebook = "../sample/Getting started.jl"

@time let
    session = Pluto.ServerSession()

    fakeclient = Pluto.ClientSession(:fake, nothing)
    session.connected_clients[fakeclient.id] = fakeclient

    Pluto.SessionActions.open(session, test_notebook)

    notebook = first(values(session.notebooks))
    Pluto.update_run!(session, notebook, notebook.cells)
    Pluto.SessionActions.shutdown(session, notebook)
end

@time let # cross check - everything already compiled here
    session = Pluto.ServerSession()

    fakeclient = Pluto.ClientSession(:fake, nothing)
    session.connected_clients[fakeclient.id] = fakeclient

    Pluto.SessionActions.open(session, test_notebook)

    notebook = first(values(session.notebooks))
    Pluto.update_run!(session, notebook, notebook.cells)
    Pluto.SessionActions.shutdown(session, notebook)
end

Before:

julia .\notebook_startup.jl
 34.312971 seconds (35.99 M allocations: 2.060 GiB, 2.06% gc time, 21.23% compilation time)
 11.442876 seconds (1.25 M allocations: 79.949 MiB, 0.18% gc time, 0.33% compilation time)

After:

julia .\notebook_startup.jl
 30.483048 seconds (39.93 M allocations: 2.286 GiB, 2.32% gc time, 8.33% compilation time)
 11.485506 seconds (1.64 M allocations: 100.089 MiB, 0.29% gc time, 0.45% compilation time)

Edit: updated timings not running on battery power anymore. This gives more consistent timings, but unfortunately a less dramatic improvement with this PR. But still the startup time is improved significantly.

Also the "standard" Pluto startup feels significantly faster.

Pretty sure more could be done here by including more Pluto functionality for precompilation.

[VarLad]

This is exciting!
So it'll only take 11secs to start Pluto, in exchange of larger precompile times?

[lungben]

It looks like (from the timings above) that about 20% of the startup time is due to compilations. Ideally we could be able to move this part mostly to precompilation. In the current state, this PR reduces precompilation times by about 2/3.
The main difficulty with the remaining 1/3 is that we cannot just call "standard" Pluto functions in the precompilation phase because they contain objects that cannot be serialized (especially Tasks). Thus, we need to break up Pluto functions to call only the supported ones in precompilation phase.

The remaining 80% of the Pluto startup time seems not to be related to compilation times.

[fonsp]

Cool! I tried this but didn't manage, but it might still be useful to look at: https://github.com/fonsp/Pluto.jl/tree/precompile-during-precompile

I'll have more time to look at the PR after my break! Be sure to also time using Pluto

[lungben]

Maybe workspace_use_distributed=true prevented some precompilation?
On the flip side, run_reactive! crashes for me if called during precompilation, most likely due to task / async functionality (which is not allowed in precompilation) in WorkspaceManager.

[fonsp]

Can you time using Pluto?

[lungben]

Current main:

PS C:\Users\benlu.julia\dev\Pluto\test> julia -e '@time using Pluto'

0.773140 seconds (1.08 M allocations: 69.001 MiB, 3.10% gc time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia -e '@time using Pluto'

0.792773 seconds (1.08 M allocations: 69.002 MiB, 3.01% gc time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia -e '@time using Pluto'

0.830735 seconds (1.08 M allocations: 69.001 MiB, 3.61% gc time)

PS C:\Users\benlu.julia\dev\Pluto\test> julia notebook_startup.jl

34.839933 seconds (36.48 M allocations: 2.089 GiB, 2.14% gc time, 18.12% compilation time)
10.727400 seconds (598.29 k allocations: 40.657 MiB, 0.11% gc time, 0.36% compilation time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia notebook_startup.jl

30.045399 seconds (36.48 M allocations: 2.090 GiB, 2.56% gc time, 21.94% compilation time)
10.591936 seconds (594.17 k allocations: 40.401 MiB, 0.12% gc time, 0.38% compilation time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia notebook_startup.jl

31.160570 seconds (36.48 M allocations: 2.090 GiB, 2.53% gc time, 23.23% compilation time)
10.726944 seconds (595.25 k allocations: 40.477 MiB, 0.12% gc time, 0.37% compilation time)

With this PR:

PS C:\Users\benlu.julia\dev\Pluto\test> julia -e '@time using Pluto'

0.965575 seconds (1.38 M allocations: 88.982 MiB, 2.54% gc time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia -e '@time using Pluto'

0.944487 seconds (1.38 M allocations: 88.982 MiB, 2.36% gc time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia -e '@time using Pluto'

0.998248 seconds (1.38 M allocations: 88.980 MiB, 2.48% gc time)

PS C:\Users\benlu.julia\dev\Pluto\test> julia notebook_startup.jl

29.648578 seconds (40.87 M allocations: 2.339 GiB, 2.48% gc time, 7.97% compilation time)
10.604080 seconds (699.69 k allocations: 43.801 MiB, 0.08% gc time, 0.39% compilation time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia notebook_startup.jl

33.155839 seconds (40.87 M allocations: 2.339 GiB, 2.42% gc time, 8.51% compilation time)
11.402447 seconds (703.24 k allocations: 44.066 MiB, 0.08% gc time, 0.37% compilation time)
PS C:\Users\benlu.julia\dev\Pluto\test> julia notebook_startup.jl

31.700384 seconds (40.86 M allocations: 2.339 GiB, 2.34% gc time, 7.49% compilation time)
10.511332 seconds (703.15 k allocations: 44.064 MiB, 0.09% gc time, 0.39% compilation time)

Looks like there are no consistent speed-ups with this PR (anymore?). Pure using timing seems even slightly worse. However, the compilation time fraction is still decreased by a factor of 3 (I wonder why this does not result in a faster startup anymore).