Repeated creation of triply-nested array in local variable crashes without explicit gc

[grahamas]

This seems like an un-aggressive gc problem, but it doesn't seem nearly complicated enough to require an explicit call to gc.

In both the nightly build and 1.3, the following code called repeatedly will eventually lead julia to run out of memory.

julia> N=2; N_RAND=2_000_000; for i in 1:N
           mdb = [[[randn(N_RAND) for _ in 1:10] for _ in 1:2] for n in 1:5] 
       end

For my machine with 8GB RAM, I run this about 6 times before julia is killed. You can watch the memory usage growing on htop.

Curiously, wrapping this loop inside another loop fixes the problem. Julia only crashes when you run this command sequentially by hand, not in a loop.

It doesn't matter what N equals.

It's somehow important that the arrays are triply nested. If you remove a layer of nesting, but change N_RAND to keep the data size the same, julia won't crash.

If you explicitly call GC.gc() after every invocation of the loop, the problem goes away.

julia> versioninfo()
Julia Version 1.4.0-DEV.657
Commit 6f76d16a9c (2019-12-23 01:48 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, haswell)

Seems similar to #26101, except not parallel.

[JeffBezanson]

Interestingly, also does not happen if the code is in a function. Will also be interesting to try this on different platforms. This seems to be some kind of adverse allocation pattern, perhaps due to alternating compilation with the larger array allocations. Might be similar to #30653.