Function erroring inside @testset but not outside

[giordano]

On Julia master, this works:

julia> begin
           ptr = cfunction(x -> x*x, Cdouble, Tuple{Cdouble})
           g(x) = ccall(ptr, Cdouble, (Cdouble,), x)
           g(3.0)
       end
9.0

but if this is wrapped in a @testset, it errors badly:

julia> Base.Test.@testset begin
           ptr = cfunction(x -> x*x, Cdouble, Tuple{Cdouble})
           g(x) = ccall(ptr, Cdouble, (Cdouble,), x)
           g(3.0)
       end
Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{Any, 1}[Expr(:return, Expr(:foreigncall, Expr(:block, SSAValue(1) = Expr(:call, Core.getfield, SlotNumber(id=1), :(:ptr))::Any, Expr(:if, Expr(:call, Core.isdefined, SSAValue(1), :(:contents))::Any, nothing, Expr(:block, <newvar #<3>>, SlotNumber(id=3))::Any)::Any, Expr(:call, Core.getfield, SSAValue(1), :(:contents))::Any)::Any, Float64, svec(Float64), :(:ccall), 1, SlotNumber(id=2))::Float64)::Any], i=(0,))
rec_backtrace at /home/mose/repo/julia/src/stackwalk.c:87
record_backtrace at /home/mose/repo/julia/src/task.c:246
jl_throw at /home/mose/repo/julia/src/task.c:568
jl_bounds_error_ints at /home/mose/repo/julia/src/rtutils.c:182
getindex at ./array.jl:768
_getfield_elim_pass! at ./inference.jl:5599
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
_getfield_elim_pass! at ./inference.jl:5583
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
_getfield_elim_pass! at ./inference.jl:5583
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
_getfield_elim_pass! at ./inference.jl:5583
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
getfield_elim_pass! at ./inference.jl:5576
optimize at ./inference.jl:3402
typeinf at ./inference.jl:3303
typeinf_frame at ./inference.jl:2984 [inlined]
typeinf_code at ./inference.jl:3066
jlcall_typeinf_code_794 at /home/mose/repo/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
typeinf_ext at ./inference.jl:3105
jlcall_typeinf_ext_0 at /home/mose/repo/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
jl_apply at /home/mose/repo/julia/src/julia.h:1451 [inlined]
jl_apply_with_saved_exception_state at /home/mose/repo/julia/src/rtutils.c:249
jl_type_infer at /home/mose/repo/julia/src/gf.c:263
jl_compile_for_dispatch at /home/mose/repo/julia/src/gf.c:1691
jl_compile_method_internal at /home/mose/repo/julia/src/julia_internal.h:334 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:381 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
macro expansion at ./REPL[1]:4 [inlined]
macro expansion at ./test.jl:942 [inlined]
anonymous at ./<missing> (unknown line)
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_toplevel_eval_flex at /home/mose/repo/julia/src/toplevel.c:628
jl_toplevel_eval_in at /home/mose/repo/julia/src/builtins.c:625
eval at ./repl/REPL.jl:3
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
eval_user_input at ./repl/REPL.jl:69
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
macro expansion at ./repl/REPL.jl:100 [inlined]
#1 at ./event.jl:96
jl_call_fptr_internal at /home/mose/repo/julia/src/julia_internal.h:366 [inlined]
jl_call_method_internal at /home/mose/repo/julia/src/julia_internal.h:385 [inlined]
jl_apply_generic at /home/mose/repo/julia/src/gf.c:1980
jl_apply at /home/mose/repo/julia/src/julia.h:1451 [inlined]
start_task at /home/mose/repo/julia/src/task.c:268
unknown function (ip: 0xffffffffffffffff)
test set: Error During Test
  Got an exception of type ErrorException outside of a @test
  error compiling g: unsupported or misplaced expression "block" in function g
  Stacktrace:
   [1] macro expansion at ./REPL[1]:4 [inlined]
   [2] macro expansion at ./test.jl:942 [inlined]
   [3] anonymous at ./<missing>:?
   [4] eval(::Module, ::Expr) at ./repl/REPL.jl:3
   [5] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./repl/REPL.jl:69
   [6] macro expansion at ./repl/REPL.jl:100 [inlined]
   [7] (::getfield(Base.REPL, Symbol("##1#2")){Base.REPL.REPLBackend})() at ./event.jl:96
Test Summary: | Error  Total
test set      |     1      1
ERROR: Some tests did not pass: 0 passed, 0 failed, 1 errored, 0 broken.

With git-bisect I found this started to happen with PR #23698.

First reported on Discourse: https://discourse.julialang.org/t/function-erroring-inside-testset-but-not-outside/6064

[vtjnash]

It looks like the linearizer in lowering fails to handle the first argument to ccall / foreigncall correctly.

[giordano]

Bump :-)

[giordano]

Bump 2 :-)

Note also this is a regression, it used to work in Julia 0.6 and I bisected to #23698, as indicated in the first message. Perhaps should be labeled as such?

[giordano]

Bump 3 :-) Is there any chance to have this fixed for Julia 0.7/1.0?

[KristofferC]

Seems fixed now?

[giordano]

It wasn't when I wrote the last message. Will check again next week.

[giordano]

Ok, I've checked now, it works, thanks.

Side question: why this?

julia> begin
           f(x) = x*x
           ptr = @cfunction(f, Cdouble, (Cdouble,))
       end
ERROR: UndefVarError: f not defined

I guess there is a scope issue, but I can't see how to fix it. It used to work with Julia 0.6. Looking for "begin" and "scope" in the release notes for Julia 0.7 I don't see anything related.

[KristofferC]

Use $f.

[giordano]

Already tried that, but it wasn't clear to me how to use a CFunction in a ccall:

begin
    f(x) = x*x
    ptr = @cfunction($f, Cdouble, (Cdouble,))
    g(x) = ccall(ptr, Cdouble, (Cdouble,), x)
    g(1.0)
end
ERROR: TypeError: in g, in ccall: first argument not a pointer or valid constant expression, expected Ptr, got Base.CFunction

Base.CFunction's docstring says

Like all cfunction handles, it should be passed to ccall as a Ptr{Cvoid}, and will be converted automatically at the call site to the appropriate type.

So should I convert the CFunction to Ptr{Cvoid}? How? I see that CFunction has a field ptr which is a Ptr{Cvoid}, but I'm not sure I'm supposed to directly use that.

[giordano]

I guess Base.unsafe_convert should do the trick (which simply returns the ptr field).

Edit: continued on discourse: https://discourse.julialang.org/t/using-a-base-cfunction-in-ccall/13621

[KristofferC]

Let's continue this on discourse.