Loading DiffEqBase invalidates package loading

[KristofferC]

julia> using DiffEqBase

julia> @time using Random
  0.389889 seconds (593.85 k allocations: 41.506 MiB, 2.73% gc time, 98.76% compilation time: 100% of which was recompilation)

I tried to diagnose but loading DiffEqBase also breaks everything

julia> @snoopr using DiffEqBase
7817-element Vector{Any}:
Error showing value of type Vector{Any}:
ERROR: UndefVarError: `isinplace` not defined

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate `TruncatedStacktraces.VERBOSE[] = true` and re-run the code.

Stacktrace:
  [1] show(io::IOContext{IOBuffer}, t::
SYSTEM (REPL): showing an error caused an error
ERROR: UndefVarError: `isinplace` not defined

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate `TruncatedStacktraces.VERBOSE[] = true` and re-run the code.

Stacktrace:
  [1] show(io::IOContext{IOBuffer}, t::
SYSTEM (REPL): caught exception of type UndefVarError while trying to handle a nested exception; giving up

julia> DiffEqBase.TruncatedStacktraces.VERBOSE[] = true
true

julia> trees = invalidation_trees(invalidations)
16-element Vector{SnoopCompile.MethodInvalidations}:
Error showing value of type Vector{SnoopCompile.MethodInvalidations}:
ERROR: UndefVarError: `uType` not defined
Stacktrace:
  [1] show(io::IOContext{IOBuffer}, t::
SYSTEM (REPL): showing an error caused an error
ERROR: UndefVarError: `uType` not defined
Stacktrace:
  [1] show(io::IOContext{IOBuffer}, t::
SYSTEM (REPL): caught exception of type UndefVarError while trying to handle a nested exception; giving up

[oscardssmith]

the problem here is that type printing was overloaded incorrectly (and there's no correct way to overload it).

[ChrisRackauckas]

Then what's the right way to do it? I'm happy to implement it in any way that is suggested, but I was told JuliaLang/julia#48444 isn't going to be making it into v1.9 even as experimental, and this is the best way we have right now.

[daviehh]

This recompilation looks to be triggered when one overloads show when the type is specified as Type{<:foo} as implemented in https://github.com/SciML/TruncatedStacktraces.jl/blob/c457db3d350b9d8f52c42715c373175aeadad3c1/src/TruncatedStacktraces.jl#L67

e.g. on a fresh julia session, one may trigger it w/

function Base.show(io::IO, t::Type{<:Val{a}}) where {a} end
@time using Random

0.372676 seconds (586.34 k allocations: 40.771 MiB, 98.86% compilation time: 100% of which was recompilation)

snoopr shows

using SnoopCompileCore
i = @snoopr function Base.show(io::IO, t::Type{<:Val{a}}) where {a} end
using SnoopCompile
trees = SnoopCompile.invalidation_trees(i)

1-element Vector{SnoopCompile.MethodInvalidations}:
inserting show(io::IO, t::Type{<:}) where a @ Main REPL[2]:1 invalidated:
backedges: 1: superseding show(io::IO, x::Type) @ Base show.jl:950 with MethodInstance for show(::IOContext{IOBuffer}, ::Type) (107 children)
2: superseding show(io::IO, x::Type) @ Base show.jl:950 with MethodInstance for show(::IOBuffer, ::Type) (2993 children)
3 mt_cache

[oscardssmith]

If you wanted to pirate something it would be a lot better to pirate exception printing rather than type showing. The problem with pirating type showing is that a ton of functionality uses it (e.g. help mode), and showing non-concrete types correctly is very tricky (since e.g. type parameters might not be assigned).

[KristofferC]

One solution is to run package loading in a fixed world. That would make it immune to invalidations, but you couldn't Revise it without removing that.

[ChrisRackauckas]

Fixed by making stacktraces huge.

[KristofferC]

I might have an idea that could possibly reduce invalidations but provide the same functionality (albeit only in stacktraces). The idea would be to take over a part of the stacktrace printing and redirect that to a custom function that can be overloaded (so not show on type). I will experiment and see how it goes.

[ChrisRackauckas]

Thanks. I think we've proven this experiment has failed though it's very close to having something that can work with just a tweak to Base like that, and we have everything setup so the TruncatedStacktraces macro deprecation path can be to just use that and lots of the printing choices are already in here. So we're ready to go once something shows up that won't invalidate.