[WIP] Alternative attempt at compile-on-demand layer integration

[pchintalapudi]

This PR is an alternative to #44575. As opposed to that PR, this version works^TM on LLVM 13 and with our current memory manager, by not splitting up modules (thus avoiding a recursive memory manager allocation). Where JITLink is available, we will split up modules more aggressively, although the performance impact from this is unknown as we emit modules with fairly fine granularity anyways.

Known failures include something in the ccall testset and in InteractiveUtils.

Depends on #44605 for increased parallelism gains from multiple contexts. Depends on #44926 for various thread safety improvements.

[gbaraldi]

Deadlock on non-debug builds aside, this is about 5% faster on TTFP than master on my tests. Same thing for time to first solve.

[pchintalapudi]

Conclusions so far: the ccall test failure should be fixed by #44967 when it is merged into master, InteractiveUtils failure is a failure of code_native which probably relates to compile-on-demand, and I need to investigate the deadlocks and windows misc test failures.

[gbaraldi]

From what I've seen the deadlock happens because allocateCodeSection runs twice on the same thread without running finalizeMemory so allocating_thread == std::this_thread::get_id() and no other thread will run finalize so it just gets stuck.

[pchintalapudi]

That is a problem, that was unfortunately not resolved by the partition function fix. I'll see if we can simply ignore the context of that assert and if libunwind has improved its support in the last 6 years.

[gbaraldi]

It works for me, time to first solve is about 12% better and TTFP is about 6% better so 🚀

[giordano]

What's "time to first solve" now? 😅

[gbaraldi]

How long it takes for a solve to happen in OrdinaryDiffEq.

using OrdinaryDiffEq
f(u,p,t) = 1.01*u
u0=1/2
tspan = (0.0,1.0)
prob = ODEProblem(f,u0,tspan)
sol = solve(prob,Tsit5(),reltol=1e-8,abstol=1e-8)

I can take other ideas for Time to First *

[gbaraldi]

Interactive utils failure happens because this check fails

julia/src/debuginfo.cpp

Line 1167 in 6f8662b

if (fit != objmap.end() && fptr < fit->first + fit->second.SectionSize) {

From a random run while printing

std::cout << "fitobj: " << (fit != objmap.end()) << " fptr: " << fptr << " fitfir :" << fit->first << " sec: " << fit->second.SectionSize<< std::endl;

this pr:

julia> function linear_foo()
                      return 5
                  end
linear_foo (generic function with 1 method)

julia> @code_native dump_module=false linear_foo()
fitobj: 1 fptr: 140663119466160 fitfir :140660438813392 sec: 3593
WARNING: Unable to find function pointer

master:

@code_native dump_module=false linear_foo()
fitobj: 1 fptr: 139721520063056 fitfir :139721520063056 sec: 49
fitobj: 0 fptr: 5 fitfir :23 sec: 0
fitobj: 0 fptr: 0 fitfir :23 sec: 0
fitobj: 0 fptr: 1 fitfir :23 sec: 0
        .text
; ┌ @ REPL[1]:1 within `linear_foo`
        movl    $5, %eax
        retq
        nopw    %cs:(%rax,%rax)
; └

The section size seems suspiciously large

[pchintalapudi]

So I think the root cause of the code_native failures is somewhat different.

1. The problem lies earlier in jl_dump_method_asm_impl, where we first attempt to fully compile the method, and if we succeed then we try to read the assembly directly from the compiled method and annotate it as we see fit.
2. Unfortunately, with compile-on-demand, this does not work in the general case because our function pointer may point to a compilation thunk, which is for all intents and purposes complete garbage for the assembly reader.
3. Instead we need to rely on the fallback implementation of dump_method_asm that recompiles the function and dumps the resultant IR's assembly.
4. But this fallback method does not appear to respect dump_module=false, so the InteractiveUtils test will still fail (although any output is still better than returning an empty string).

I will patch up the PR to ignore our disassembly implementation, but there will likely be no discernible change in number of tests passing.

[pchintalapudi]

Compile-on-demand summary statistics during sysimg build

Core.Compiler

4035 jitlayers              - Number of modules added to the JIT
3956 jitlayers              - Number of modules compiled by the JIT

sys.ji

5274 jitlayers              - Number of modules added to the JIT
4562 jitlayers              - Number of modules compiled by the JIT

sys-o.a (various points)

1440 jitlayers             - Number of modules added to the JIT
860 jitlayers              - Number of modules compiled by the JIT

---

1729 jitlayers              - Number of modules added to the JIT
1205 jitlayers              - Number of modules compiled by the JIT

---

2377 jitlayers                    - Number of modules added to the JIT
1851 jitlayers                    - Number of modules compiled by the JIT
These stats are during the actual system image build stage

---

3 jitlayers                    - Number of modules added to the JIT
3 jitlayers                    - Number of modules compiled by the JIT

Total

14858 modules added to the JIT
12437 modules compiled by the JIT (passed the compile-on-demand layer)

83.7% of modules were actually compiled by the JIT during overall julia compilation
77.9% of modules were actually compiled by the JIT during system image build (with optimization turned on)