Julia GC sometimes crashes in GAP

[fingolfin]

There is a crash in the Julia-GC-integration in GAP, that sometimes occurs when running the GAP testinstall test suite several times in the row. @rbehrends and me reproduced it in the past, and could not track it down so far.

In addition, recently we have been seeing crashes in the JuliaInterface test suite (e.g. here) that look like this:

...
     252 ms (192 ms GC) and 124KB allocated for import.tst
testing: /home/travis/build/oscar-system/GAPJulia/gap/pkg/JuliaInterface/tst/u\
tils.tst
# line 2 of 27 (7%)fatal: error thrown and no exception handler available.
ReadOnlyMemoryError()
TryMarkRange at /home/travis/build/oscar-system/GAPJulia/gap/src/julia_gc.c:484
GapTaskScanner at /home/travis/build/oscar-system/GAPJulia/gap/src/julia_gc.c:534
gc_mark_loop at /buildworker/worker/package_linux64/build/src/gc.c:2310
_jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:2666
jl_gc_collect at /buildworker/worker/package_linux64/build/src/gc.c:2837
CollectBags at /home/travis/build/oscar-system/GAPJulia/gap/src/julia_gc.c:643
FuncGASMAN at /home/travis/build/oscar-system/GAPJulia/gap/src/gap.c:904
...

Note that ReadOnlyMemoryError is actually a misnomer; this exception is also thrown when trying to read from memory we don't have access to. So my guess is that GapTaskScanner is scanning too much. Perhaps jl_task_stack_buffer needs to be adjusted again on the Julia side.

[sebasguts]

Indeed, I am getting this constantly when trying to load GAP from Julia.

You can reproduce it by getting my minimal version of LibGAP.jl here: https://github.com/sebasguts/GAPJulia/tree/startup_segfault

and then executing

using Libdl
push!( DL_LOAD_PATH, "<path_to_gap>/.libs/" )
include("<path_to_gapjulia>/LibGAP.jl/src/initialization.jl")
libgap.initialize( [ ""
                                   , "-l", "<path_to_gap>"
                                   , "-T", "-r", "-A", "--nointeract"
                                   , "-m", "512m" ], ["\0"] )

[fingolfin]

The following code snippet always crashes Julia with LibGAP.jl loaded, at least for me:

n = GAP.GAPFuncs.PROD(2^59,2^59)

function sum_bench()
    for i in 1:1000000
        s = GAP.GAPFuncs.SUM(n, n)
    end
    return s
end

@time sum_bench()

Here is the stack trace:

signal (11): Segmentation fault: 11
in expression starting at no file:0
TryMarkRange at /Users/mhorn/Projekte/OSCAR/GAPJulia/gap/.libs/libgap.dylib (unknown line)
GapRootScanner at /Users/mhorn/Projekte/OSCAR/GAPJulia/gap/.libs/libgap.dylib (unknown line)
_jl_gc_collect at /Users/mhorn/Projekte/foreign/julia/src/gc.c:2699
jl_gc_collect at /Users/mhorn/Projekte/foreign/julia/src/gc.c:2874
jl_gc_pool_alloc at /Users/mhorn/Projekte/foreign/julia/src/gc.c:1102
jl_gc_alloc_ at /Users/mhorn/Projekte/foreign/julia/src/./julia_internal.h:263 [inlined]
jl_gc_alloc at /Users/mhorn/Projekte/foreign/julia/src/gc.c:2911
_new_array_ at /Users/mhorn/Projekte/foreign/julia/src/array.c:100
_new_array at /Users/mhorn/Projekte/foreign/julia/src/array.c:158 [inlined]
jl_alloc_array_1d at /Users/mhorn/Projekte/foreign/julia/src/array.c:418
Type at ./boot.jl:393 [inlined]
Type at ./boot.jl:402 [inlined]
similar at ./abstractarray.jl:583 [inlined]
similar at ./abstractarray.jl:575 [inlined]
_similar_for at ./array.jl:532 [inlined]
_collect at ./array.jl:563 [inlined]
collect at ./array.jl:557
GapFunc at /Users/mhorn/Projekte/GAP/gap.reimer/pkg/GAPJulia/JuliaInterface/julia/gaptypes.jl:102
sum_bench at ./REPL[2]:3
jl_fptr_trampoline at /Users/mhorn/Projekte/foreign/julia/src/gf.c:1854
do_call at /Users/mhorn/Projekte/foreign/julia/src/interpreter.c:323
eval_body at /Users/mhorn/Projekte/foreign/julia/src/interpreter.c:623
jl_interpret_toplevel_thunk_callback at /Users/mhorn/Projekte/foreign/julia/src/interpreter.c:883
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x1319c113f)
unknown function (ip: 0x1)
jl_interpret_toplevel_thunk at /Users/mhorn/Projekte/foreign/julia/src/interpreter.c:892
jl_toplevel_eval_flex at /Users/mhorn/Projekte/foreign/julia/src/toplevel.c:764
jl_toplevel_eval at /Users/mhorn/Projekte/foreign/julia/src/toplevel.c:773 [inlined]
jl_toplevel_eval_in at /Users/mhorn/Projekte/foreign/julia/src/toplevel.c:793
eval at ./boot.jl:319
eval_user_input at /Users/mhorn/Projekte/foreign/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:85
macro expansion at /Users/mhorn/Projekte/foreign/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:117 [inlined]
#26 at ./task.jl:259
jl_apply at /Users/mhorn/Projekte/foreign/julia/src/./julia.h:1571 [inlined]
start_task at /Users/mhorn/Projekte/foreign/julia/src/task.c:572
Allocations: 7161430 (Pool: 7142511; Big: 18919); GC: 5

@rbehrends it would be great if you could have a look at this, too.

[fingolfin]

This was improved a lot by
gap-system/gap#2969