Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flux Enzyme tests are failing #1738

Closed
CarloLucibello opened this issue Aug 18, 2024 · 3 comments
Closed

Flux Enzyme tests are failing #1738

CarloLucibello opened this issue Aug 18, 2024 · 3 comments

Comments

@CarloLucibello
Copy link
Collaborator

Not sure why but it can be seen in FluxML/Flux.jl#2464
e.g.
https://github.com/FluxML/Flux.jl/actions/runs/10435007814/job/28898510960?pr=2464#step:6:530

@wsmoses
Copy link
Member

wsmoses commented Aug 19, 2024

@vchuravy @maleadt any idea what this is? Looks like GPUCompiler.jl fails to emit code?

You may need to write `f(x::Vararg{T})` rather than `f(x::Vararg{<:T})` or `f(x::Vararg{T}) where T` instead of `f(x::Vararg{T} where T)`.
: Error During Test at D:\a\Flux.jl\Flux.jl\test\ext_enzyme\enzyme.jl:121
  Got exception outside of a @test
  AssertionError: Static compilation failed
  Stacktrace:
    [1] compile_method_instance(job::GPUCompiler.CompilerJob)
      @ GPUCompiler C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\jlgen.jl:739
    [2] macro expansion
      @ C:\Users\runneradmin\.julia\packages\TimerOutputs\Lw5SP\src\TimerOutput.jl:253 [inlined]
    [3] irgen(job::GPUCompiler.CompilerJob)
      @ GPUCompiler C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\irgen.jl:4
    [4] macro expansion
      @ C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\driver.jl:208 [inlined]
    [5] macro expansion
      @ C:\Users\runneradmin\.julia\packages\TimerOutputs\Lw5SP\src\TimerOutput.jl:253 [inlined]
    [6] macro expansion
      @ C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\driver.jl:207 [inlined]
    [7] emit_llvm(job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, only_entry::Bool, validate::Bool)
      @ GPUCompiler C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\utils.jl:103
    [8] emit_llvm
      @ C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\utils.jl:97 [inlined]
    [9] codegen(output::Symbol, job::GPUCompiler.CompilerJob; libraries::Bool, toplevel::Bool, optimize::Bool, cleanup::Bool, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
      @ GPUCompiler C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\driver.jl:136
   [10] codegen
      @ C:\Users\runneradmin\.julia\packages\GPUCompiler\Y4hSX\src\driver.jl:115 [inlined]
   [11] codegen(output::Symbol, job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, toplevel::Bool, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
      @ Enzyme.Compiler C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\compiler.jl:5525
   [12] codegen
      @ C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\compiler.jl:[549](https://github.com/FluxML/Flux.jl/actions/runs/10435007814/job/28898510960?pr=2464#step:6:552)3 [inlined]
   [13] _thunk(job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams}, postopt::Bool)
      @ Enzyme.Compiler C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\compiler.jl:7103
   [14] _thunk
      @ C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\compiler.jl:7103 [inlined]
   [15] cached_compilation
      @ C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\compiler.jl:7144 [inlined]
   [16] thunkbase(ctx::LLVM.Context, mi::Core.MethodInstance, ::Val{0x0000000000007d27}, ::Type{EnzymeCore.Const{var"#loss#455"}}, ::Type{Active}, tt::Type{Tuple{Duplicated{Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}}, Duplicated{Vector{Float32}}}}, ::Val{Enzyme.API.DEM_ReverseModeCombined}, ::Val{1}, ::Val{(false, false, false)}, ::Val{true}, ::Val{false}, ::Type{EnzymeCore.FFIABI}, ::Val{true})
      @ Enzyme.Compiler C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\compiler.jl:7217
   [17] #s2048#18999
      @ C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\compiler.jl:7269 [inlined]
   [18] var"#s2048#18999"(FA::Any, A::Any, TT::Any, Mode::Any, ModifiedBetween::Any, width::Any, ReturnPrimal::Any, ShadowInit::Any, World::Any, ABI::Any, ErrIfFuncWritten::Any, ::Any, ::Type, ::Type, ::Type, tt::Any, ::Type, ::Type, ::Type, ::Type, ::Type, ::Type, ::Any)
      @ Enzyme.Compiler .\none:0
   [19] (::Core.GeneratedFunctionStub)(::UInt64, ::LineNumberNode, ::Any, ::Vararg{Any})
      @ Core .\boot.jl:602
   [20] autodiff
      @ C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\Enzyme.jl:315 [inlined]
   [21] autodiff(::EnzymeCore.ReverseMode{true, EnzymeCore.FFIABI, false, false}, ::var"#loss#455", ::Type{Active}, ::Duplicated{Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}}, ::Duplicated{Vector{Float32}})
      @ Enzyme C:\Users\runneradmin\.julia\packages\Enzyme\OOd6p\src\Enzyme.jl:332
   [22] gradient_ez(::Function, ::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}, ::Vararg{Any})
      @ Main D:\a\Flux.jl\Flux.jl\test\ext_enzyme\enzyme.jl:30
   [23] test_enzyme_grad(loss::var"#loss#455", model::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}}}, x::Vector{Float32})
      @ Main D:\a\Flux.jl\Flux.jl\test\ext_enzyme\enzyme.jl:53
   [24] macro expansion
      @ D:\a\Flux.jl\Flux.jl\test\ext_enzyme\enzyme.jl:123 [inlined]
   [25] macro expansion
      @ C:\hostedtoolcache\windows\julia\1.10.4\x64\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
   [26] macro expansion
      @ D:\a\Flux.jl\Flux.jl\test\ext_enzyme\enzyme.jl:122 [inlined]
   [27] macro expansion
      @ C:\hostedtoolcache\windows\julia\1.10.4\x64\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
   [28] top-level scope
      @ D:\a\Flux.jl\Flux.jl\test\ext_enzyme\enzyme.jl:96
   [29] include(fname::String)
      @ Base.MainInclude .\client.jl:489
   [30] macro expansion
      @ D:\a\Flux.jl\Flux.jl\test\runtests.jl:146 [inlined]
   [31] macro expansion
      @ C:\hostedtoolcache\windows\julia\1.10.4\x64\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
   [32] macro expansion
      @ D:\a\Flux.jl\Flux.jl\test\runtests.jl:144 [inlined]
   [33] macro expansion
      @ C:\hostedtoolcache\windows\julia\1.10.4\x64\share\julia\stdlib\v1.10\Test\src\Test.jl:1577 [inlined]
   [34] top-level scope
      @ D:\a\Flux.jl\Flux.jl\test\runtests.jl:22
   [35] include(fname::String)
      @ Base.MainInclude .\client.jl:489
   [36] top-level scope
      @ none:6
   [37] eval
      @ .\boot.jl:385 [inlined]
   [38] exec_options(opts::Base.JLOptions)
      @ Base .\client.jl:291
   [39] _start()
      @ Base .\client.jl:[552](https://github.com/FluxML/Flux.jl/actions/runs/10435007814/job/28898510960?pr=2464#step:6:555)
testing f64(Chain(Dense, Dense))

@wsmoses
Copy link
Member

wsmoses commented Aug 26, 2024

cc @gbaraldi if you have any idea what's up

@wsmoses
Copy link
Member

wsmoses commented Sep 28, 2024

@CarloLucibello is this still the case, I don't see failures on Flux CI?

Reopen if so

@wsmoses wsmoses closed this as completed Sep 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants