Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Precompilation failure on Windows with julia 1.6 beta1 #2323

Closed
jebej opened this issue Jan 8, 2021 · 12 comments · Fixed by #2366
Closed

Precompilation failure on Windows with julia 1.6 beta1 #2323

jebej opened this issue Jan 8, 2021 · 12 comments · Fixed by #2366
Milestone
1.6

Comments

@jebej
Copy link

jebej commented Jan 8, 2021

I am getting errors where packages fail to precompile in parallel, either with a crash or just never terminate, even though things work when precompiling normally.

Here is the environment, though I have seen this happen consistently with various sets of packages:

[deps]
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
OhMyREPL = "5fb14364-9ced-5910-84b2-373655c76a03"
Pluto = "c3e4b0f8-55cb-11ea-2926-15256bba5781"
ProgressMeter = "92933f4c-e287-5a05-a399-4b506db050ca"

(@v1.6) pkg> up
    Updating registry at `C:\Users\Jeremy\.julia\registries\General`
   Installed JLLWrappers ─ v1.2.0
No Changes to `C:\Users\Jeremy\.julia\environments\v1.6\Project.toml`
Updating `C:\Users\Jeremy\.julia\environments\v1.6\Manifest.toml`
  [692b3bcd] ↑ JLLWrappers v1.1.4 ⇒ v1.2.0
Precompiling project...
  Progress [======>                                  ]  3/23
  ◐ URIs
  ◒ Parsers
  ✓ JLLWrappers
  ✗ DataAPI
  ◐ IniFile
  ◐ MsgPack
  ◐ IteratorInterfaceExtensions
  ◒ ProgressMeter
  ✗ MbedTLS_jll
  ◓ FuzzyCompletions
  ◒ DataValueInterfaces
  ◓ Zlib_jll
  ◐ MbedTLS
  ◒ LibSSH2_jll
Internal error: encountered unexpected error in runtime:
ReadOnlyMemoryError()
reset_page at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1127 [inlined]
add_page at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1159
jl_gc_pool_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1212
UseRef at .\compiler/ssair\ir.jl:307 [inlined]
UseRefIterator at .\compiler/ssair\ir.jl:312 [inlined]
userefs at .\compiler/ssair\ir.jl:427 [inlined]
count_added_node! at .\compiler/ssair\ir.jl:646
setindex! at .\compiler/ssair\ir.jl:758
setindex! at .\compiler/ssair\ir.jl:766 [inlined]
ir_inline_item! at .\compiler/ssair\inlining.jl:337
batch_inline! at .\compiler/ssair\inlining.jl:549
ssa_inlining_pass! at .\compiler/ssair\inlining.jl:71
jfptr_ssa_inlining_passNOT._9828.clone_1 at C:\Users\Jeremy\AppData\Local\Julia-1.6.0-beta1\lib\julia\sys.dll (unknown line)
run_passes at .\compiler/ssair\driver.jl:128
optimize at .\compiler\optimize.jl:277 [inlined]
_typeinf at .\compiler\typeinfer.jl:239
typeinf at .\compiler\typeinfer.jl:211
abstract_call_method_with_const_args at .\compiler\abstractinterpretation.jl:319
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:166
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
abstract_call_method_with_const_args at .\compiler\abstractinterpretation.jl:319
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:166
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_apply at .\compiler\abstractinterpretation.jl:727
abstract_call_known at .\compiler\abstractinterpretation.jl:934
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call at .\compiler\abstractinterpretation.jl:1036
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1383
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_apply at .\compiler\abstractinterpretation.jl:727
abstract_call_known at .\compiler\abstractinterpretation.jl:934
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_apply at .\compiler\abstractinterpretation.jl:727
abstract_call_known at .\compiler\abstractinterpretation.jl:934
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_edge at .\compiler\typeinfer.jl:745
abstract_call_method at .\compiler\abstractinterpretation.jl:478
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:141
abstract_call_known at .\compiler\abstractinterpretation.jl:1015
abstract_call at .\compiler\abstractinterpretation.jl:1038
abstract_call at .\compiler\abstractinterpretation.jl:1022
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1144
typeinf_local at .\compiler\abstractinterpretation.jl:1398
typeinf_nocycle at .\compiler\abstractinterpretation.jl:1454
_typeinf at .\compiler\typeinfer.jl:216
typeinf at .\compiler\typeinfer.jl:211
typeinf_ext at .\compiler\typeinfer.jl:835
typeinf_ext_toplevel at .\compiler\typeinfer.jl:868
typeinf_ext_toplevel at .\compiler\typeinfer.jl:864
jfptr_typeinf_ext_toplevel_13768.clone_1 at C:\Users\Jeremy\AppData\Local\Julia-1.6.0-beta1\lib\julia\sys.dll (unknown line)
_jl_invoke at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2224 [inlined]
jl_apply_generic at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2406 [inlined]
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
jl_type_infer at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:296
jl_generate_fptr at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:340
jl_compile_method_internal at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:1957
jl_compile_method_internal at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:1911 [inlined]
_jl_invoke at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2216 [inlined]
jl_apply_generic at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2406
_spawn_primitive at .\process.jl:99
#636 at .\process.jl:112 [inlined]
setup_stdios at .\process.jl:196
_spawn at .\process.jl:111 [inlined]
_spawn at .\process.jl:139
_spawn at .\process.jl:139
#open#645 at .\process.jl:374
open##kw at .\process.jl:361 [inlined]
open at .\process.jl:343
unknown function (ip: 0000000054a839c8)
create_expr_cache at .\loading.jl:1244
compilecache at .\loading.jl:1322
#212 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1125
unknown function (ip: 0000000054aa7283)
with_logstate at .\logging.jl:491
with_logger at .\logging.jl:603 [inlined]
macro expansion at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1124 [inlined]
#209 at .\task.jl:406
unknown function (ip: 0000000054aa46d3)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:839
┌ Error: Pkg.precompile error
│   exception =
│    IOError: could not spawn `'C:\Users\Jeremy\AppData\Local\Julia-1.6.0-beta1\bin\julia.exe' -Cnative '-JC:\Users\Jeremy\AppData\Local\Julia-1.6.0-beta1\lib\julia\sys.dll' -g1 -O0 --output-ji 'C:\Users\Jeremy\.julia\compiled\v1.6\fzf_jll\jl_5179.tmp' --output-incremental=yes --startup-file=no --history-file=no --warn-overwrite=yes --color=yes --eval 'eval(Meta.parse(read(stdin,String)))'`: unknown error (UNKNOWN)
│    Stacktrace:
│      [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Vector{Any})
│        @ Base .\process.jl:99
│      [2] #636
│      [3] setup_stdios(f::Base.var"#636#637"{Cmd}, stdios::Vector{Any})
│        @ Base .\process.jl:196
│      [4] _spawn
│        @ .\process.jl:111 [inlined]
│      [5] _spawn(::Base.CmdRedirect, ::Vector{Any})
│        @ Base .\process.jl:139
│      [6] open(cmds::Base.CmdRedirect, stdio::Base.TTY; write::Bool, read::Bool)
│        @ Base .\process.jl:374
│      [7] open(cmds::Base.CmdRedirect, mode::String, stdio::Base.TTY)
│        @ Base .\process.jl:343
│      [8] create_expr_cache(pkg::Base.PkgId, input::String, output::String, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, internal_stderr::IOBuffer, internal_stdout::Base.DevNull)
│        @ Base .\loading.jl:1244
│      [9] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IOBuffer, internal_stdout::Base.DevNull)
  Progress [=======>                                 ]  4/23
  ◓ URIs
  ◐ Parsers
  ✗ DataAPI
  ◓ IniFile
  ◓ MsgPack
  ◓ IteratorInterfaceExtensions
  ◐ ProgressMeter
  ✗ MbedTLS_jll
  ◑ FuzzyCompletions
  ◐ DataValueInterfaces
  ✗ Zlib_jll
  ◓ MbedTLS
  ◐ LibSSH2_jll
  ◑ fzf_jll
Jeremy@NEWTON  ~                                                                                                             [17:03]
❯ julia16
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-beta1.0 (2021-01-08)
 _/ |\__'_|_|_|\__'_|  |  release-1.6/b84990e1ac (fork: 107 commits, 30 days)
|__/                   |

(@v1.6) pkg> up
    Updating registry at `C:\Users\Jeremy\.julia\registries\General`
No Changes to `C:\Users\Jeremy\.julia\environments\v1.6\Project.toml`
No Changes to `C:\Users\Jeremy\.julia\environments\v1.6\Manifest.toml`
Precompiling project...
  Progress [======================================>  ]  18/19
  ✗ Parsers
  ✗ DataAPI
  ✗ IniFile
  ✗ MsgPack
  ✗ IteratorInterfaceExtensions
  ✗ ProgressMeter
  ✗ MbedTLS_jll
  ✗ FuzzyCompletions
  ✗ DataValueInterfaces
  ✗ Zlib_jll
  ✗ MbedTLS
  ✗ LibSSH2_jll
  ✗ TableTraits
  ✗ JSON
  ◐ BenchmarkTools
 Interrupted: Exiting precompilation...

julia> using BenchmarkTools
[ Info: Precompiling BenchmarkTools [6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf]

julia> using ProgressMeter
[ Info: Precompiling ProgressMeter [92933f4c-e287-5a05-a399-4b506db050ca]
@IanButterworth
Copy link
Member

IanButterworth commented Jan 9, 2021 •
edited
Loading

Thanks for the report. Could you delete your .julia/compiled/v1.6 folder and try again, while monitoring your system monitor for memory usage?

I'm not familiar with what could cause a ReadOnlyMemoryError on windows, but perhaps there's too much memory strain..

You could also try setting ENV["JULIA_NUM_PRECOMPILE_TASKS"] to something small, starting at 2 and working up to the number of CPU cores you have,for instance and see where it breaks.

@jebej
Copy link
Author

jebej commented Jan 9, 2021

I tried setting the variable from 2 to 6, and precompilation works properly. However, it still failed consistently without setting the variable, so I tried setting the variable to 12, and there it failed.

My CPU is an intel 8086k, with 6 cores & 12 threads, and I have 16GB of RAM (usage was stable around 50%, even with the 12 threads).

Log: 

Jeremy@NEWTON  ~                                                                                                             [11:23]
❯ $env:JULIA_NUM_PRECOMPILE_TASKS=2; julia16 --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-beta1.0 (2021-01-08)
 _/ |\__'_|_|_|\__'_|  |  release-1.6/b84990e1ac (fork: 107 commits, 31 days)
|__/                   |

julia> ENV["JULIA_NUM_PRECOMPILE_TASKS"]
"2"

(@v1.6) pkg> precompile
  Progress [========================================>]  81/81
81 dependencies successfully precompiled in 63 seconds

julia>
Jeremy@NEWTON  ~                                                                                                             [11:26]
❯ $env:JULIA_NUM_PRECOMPILE_TASKS=3; julia16 --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-beta1.0 (2021-01-08)
 _/ |\__'_|_|_|\__'_|  |  release-1.6/b84990e1ac (fork: 107 commits, 31 days)
|__/                   |

(@v1.6) pkg> precompile
Precompiling project...
  Progress [========================================>]  81/81
81 dependencies successfully precompiled in 50 seconds

(@v1.6) pkg>
Jeremy@NEWTON  ~                                                                                                             [11:27]
❯ $env:JULIA_NUM_PRECOMPILE_TASKS=4; julia16 --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-beta1.0 (2021-01-08)
 _/ |\__'_|_|_|\__'_|  |  release-1.6/b84990e1ac (fork: 107 commits, 31 days)
|__/                   |

(@v1.6) pkg> precompile
  Progress [========================================>]  81/81
81 dependencies successfully precompiled in 47 seconds

(@v1.6) pkg>
Jeremy@NEWTON  ~                                                                                                             [11:28]
❯ $env:JULIA_NUM_PRECOMPILE_TASKS=5; julia16 --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-beta1.0 (2021-01-08)
 _/ |\__'_|_|_|\__'_|  |  release-1.6/b84990e1ac (fork: 107 commits, 31 days)
|__/                   |

(@v1.6) pkg> precompile
  Progress [========================================>]  81/81
81 dependencies successfully precompiled in 46 seconds

(@v1.6) pkg>
Jeremy@NEWTON  ~                                                                                                             [11:29]
❯ $env:JULIA_NUM_PRECOMPILE_TASKS=6; julia16 --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-beta1.0 (2021-01-08)
 _/ |\__'_|_|_|\__'_|  |  release-1.6/b84990e1ac (fork: 107 commits, 31 days)
|__/                   |

(@v1.6) pkg> precompile
  Progress [========================================>]  81/81
81 dependencies successfully precompiled in 43 seconds

(@v1.6) pkg>
Jeremy@NEWTON  ~                                                                                                             [11:31]
❯ $env:JULIA_NUM_PRECOMPILE_TASKS=12; julia16 --startup-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.0-beta1.0 (2021-01-08)
 _/ |\__'_|_|_|\__'_|  |  release-1.6/b84990e1ac (fork: 107 commits, 31 days)
|__/                   |

(@v1.6) pkg> precompile
Precompiling project...
  Progress [===================================>     ]  69/81
  ✗ URIs
  ✗ Parsers
  ✗ ArrayInterface
  ✗ TableTraits
  ✗ ArnoldiMethod
  ◐ DiffResults
  ✗ ExponentialUtilities
  ✗ MuladdMacro
  ✗ JSON
  ✗ DataStructures
  ✗ fzf_jll
  ✗ BenchmarkTools
  ✗ MbedTLS
  ✗ CommonSubexpressions
  ✗ SimpleTraits
  ✗ ZygoteRules
  ✗ Setfield
  ✗ HTTP
  ✗ LibSSH2_jll
  ✗ JLFzf
  ✗ OpenSpecFun_jll
  ✗ ChainRulesCore
  ✗ OhMyREPL
  ◒ SpecialFunctions
  ✗ Tables
  ✗ FiniteDiff
  ✗ LabelledArrays
  ✗ RecursiveArrayTools
  ✗ Pluto
 Interrupted: Exiting precompilation...

@jebej
Copy link
Author

jebej commented Jan 9, 2021

It seems to work with 7, but 8 tasks failed.

@IanButterworth
Copy link
Member

What does Sys.CPU_THREADS report for you?

Precompilation defaults to Sys.CPU_THREADS + 1 tasks, which should result in a balanced load, but perhaps that strategy is less stable on windows

@jebej
Copy link
Author

jebej commented Jan 9, 2021

Sys.CPU_THREADS returns 12.

Is there a way to see what the error is? Why would it work for a smaller number of threads but not at a larger number?

@IanButterworth
Copy link
Member

IanButterworth commented Jan 9, 2021 •
edited
Loading

I'm guessing that there's just some underlying bug that's more likely to occur the harder you load the system.

Perhaps we can see if there are more reports of this, and decide whether to special-case fewer threads on windows.

In the mean time, manually setting the number of precomp threads to 6/7 seems stable for you.

@IanButterworth
Copy link
Member

@Keno would this be a good bug to get an rr trace of?

@Keno
Copy link
Member

Keno commented Jan 9, 2021

If it's windows only then rr is not available. Otherwise sure.

@IanButterworth
Copy link
Member

Unfortuantely I don't think we've seen this on platforms other than windows

@fredrikekre fredrikekre added this to the 1.6 milestone Jan 11, 2021
@JeffreySarnoff
Copy link
Contributor

I encountered this. Deleting .julia/registries/General made it go away

@fredrikekre
Copy link
Member

Could this happen if there are old precompile files with bad permissions? Julia 1.5.3 on Windows did that, see e.g.JuliaLang/julia#38411

@JeffreySarnoff
Copy link
Contributor

This problem or something loosely similar after installing and running JuliaPro and then uninstalling JuliaPro and then installing 1.6-beta1.

@IanButterworth IanButterworth mentioned this issue Jan 27, 2021
Closed
IanButterworth added a commit to IanButterworth/Pkg.jl that referenced this issue Jan 28, 2021
@IanButterworth
halve the default number of precompilation tasks in windows to avoid J…
5514b1f 
…uliaLang#2323
@IanButterworth IanButterworth mentioned this issue Jan 28, 2021
Merged
KristofferC pushed a commit that referenced this issue Jan 28, 2021
@IanButterworth @fredrikekre
Precomp: Halve the default number of precompilation tasks in windows …
320ec76 
…to avoid ReadOnlyMemoryError (#2366)

* halve the default number of precompilation tasks in windows to avoid #2323


Co-authored-by: Fredrik Ekre <ekrefredrik@gmail.com>
KristofferC pushed a commit to IanButterworth/Pkg.jl that referenced this issue Jan 28, 2021
@IanButterworth @KristofferC
Precomp: Halve the default number of precompilation tasks in windows …
c6f495c 
…to avoid ReadOnlyMemoryError (JuliaLang#2366)

* halve the default number of precompilation tasks in windows to avoid JuliaLang#2323

Co-authored-by: Fredrik Ekre <ekrefredrik@gmail.com>
(cherry picked from commit 320ec76)
This was referenced May 5, 2021
Closed
Closed
@devmotion devmotion mentioned this issue Aug 26, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
1.6
5 participants
@Keno @JeffreySarnoff @IanButterworth @jebej @fredrikekre