From 02660d20bcd9edb345d38f9516616fd3e81bf255 Mon Sep 17 00:00:00 2001 From: matthieugomez Date: Sun, 2 Aug 2015 14:35:04 -0400 Subject: [PATCH 1/6] Move discussion about abstract types in fields from FAQ to performances [av skip] --- doc/manual/faq.rst | 300 ------------------------------- doc/manual/performance-tips.rst | 310 ++++++++++++++++++++++++++++++-- 2 files changed, 300 insertions(+), 310 deletions(-) diff --git a/doc/manual/faq.rst b/doc/manual/faq.rst index f98a04ae76d29..7db3f6e1982e0 100644 --- a/doc/manual/faq.rst +++ b/doc/manual/faq.rst @@ -447,306 +447,6 @@ in the future, we could consider defaulting to checked integer arithmetic in Julia, but for now, we have to live with the possibility of overflow. -.. _man-abstract-fields: - -How do "abstract" or ambiguous fields in types interact with the compiler? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Types can be declared without specifying the types of their fields: - -.. doctest:: - - julia> type MyAmbiguousType - a - end - -This allows ``a`` to be of any type. This can often be useful, but it -does have a downside: for objects of type ``MyAmbiguousType``, the -compiler will not be able to generate high-performance code. The -reason is that the compiler uses the types of objects, not their -values, to determine how to build code. Unfortunately, very little can -be inferred about an object of type ``MyAmbiguousType``: - -.. doctest:: - - julia> b = MyAmbiguousType("Hello") - MyAmbiguousType("Hello") - - julia> c = MyAmbiguousType(17) - MyAmbiguousType(17) - - julia> typeof(b) - MyAmbiguousType - - julia> typeof(c) - MyAmbiguousType - -``b`` and ``c`` have the same type, yet their underlying -representation of data in memory is very different. Even if you stored -just numeric values in field ``a``, the fact that the memory -representation of a ``UInt8`` differs from a ``Float64`` also means -that the CPU needs to handle them using two different kinds of -instructions. Since the required information is not available in the -type, such decisions have to be made at run-time. This slows -performance. - -You can do better by declaring the type of ``a``. Here, we are focused -on the case where ``a`` might be any one of several types, in which -case the natural solution is to use parameters. For example: - -.. doctest:: - - julia> type MyType{T<:AbstractFloat} - a::T - end - -This is a better choice than - -.. doctest:: - - julia> type MyStillAmbiguousType - a::AbstractFloat - end - -because the first version specifies the type of ``a`` from the type of -the wrapper object. For example: - -.. doctest:: - - julia> m = MyType(3.2) - MyType{Float64}(3.2) - - julia> t = MyStillAmbiguousType(3.2) - MyStillAmbiguousType(3.2) - - julia> typeof(m) - MyType{Float64} - - julia> typeof(t) - MyStillAmbiguousType - -The type of field ``a`` can be readily determined from the type of -``m``, but not from the type of ``t``. Indeed, in ``t`` it's possible -to change the type of field ``a``: - -.. doctest:: - - julia> typeof(t.a) - Float64 - - julia> t.a = 4.5f0 - 4.5f0 - - julia> typeof(t.a) - Float32 - -In contrast, once ``m`` is constructed, the type of ``m.a`` cannot -change: - -.. doctest:: - - julia> m.a = 4.5f0 - 4.5 - - julia> typeof(m.a) - Float64 - -The fact that the type of ``m.a`` is known from ``m``'s type---coupled -with the fact that its type cannot change mid-function---allows the -compiler to generate highly-optimized code for objects like ``m`` but -not for objects like ``t``. - -Of course, all of this is true only if we construct ``m`` with a -concrete type. We can break this by explicitly constructing it with -an abstract type: - -.. doctest:: - - julia> m = MyType{AbstractFloat}(3.2) - MyType{AbstractFloat}(3.2) - - julia> typeof(m.a) - Float64 - - julia> m.a = 4.5f0 - 4.5f0 - - julia> typeof(m.a) - Float32 - -For all practical purposes, such objects behave identically to those -of ``MyStillAmbiguousType``. - -It's quite instructive to compare the sheer amount code generated for -a simple function -:: - - func(m::MyType) = m.a+1 - -using -:: - - code_llvm(func,(MyType{Float64},)) - code_llvm(func,(MyType{AbstractFloat},)) - code_llvm(func,(MyType,)) - -For reasons of length the results are not shown here, but you may wish -to try this yourself. Because the type is fully-specified in the first -case, the compiler doesn't need to generate any code to resolve the -type at run-time. This results in shorter and faster code. - - -.. _man-abstract-container-type: - -How should I declare "abstract container type" fields? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The same best practices that apply in the `previous section -<#man-abstract-fields>`_ also work for container types: - -.. doctest:: - - julia> type MySimpleContainer{A<:AbstractVector} - a::A - end - - julia> type MyAmbiguousContainer{T} - a::AbstractVector{T} - end - -For example: - -.. doctest:: - - julia> c = MySimpleContainer(1:3); - - julia> typeof(c) - MySimpleContainer{UnitRange{Int64}} - - julia> c = MySimpleContainer([1:3;]); - - julia> typeof(c) - MySimpleContainer{Array{Int64,1}} - - julia> b = MyAmbiguousContainer(1:3); - - julia> typeof(b) - MyAmbiguousContainer{Int64} - - julia> b = MyAmbiguousContainer([1:3;]); - - julia> typeof(b) - MyAmbiguousContainer{Int64} - -For ``MySimpleContainer``, the object is fully-specified by its type -and parameters, so the compiler can generate optimized functions. In -most instances, this will probably suffice. - -While the compiler can now do its job perfectly well, there are cases -where *you* might wish that your code could do different things -depending on the *element type* of ``a``. Usually the best way to -achieve this is to wrap your specific operation (here, ``foo``) in a -separate function:: - - function sumfoo(c::MySimpleContainer) - s = 0 - for x in c.a - s += foo(x) - end - s - end - - foo(x::Integer) = x - foo(x::AbstractFloat) = round(x) - -This keeps things simple, while allowing the compiler to generate -optimized code in all cases. - -However, there are cases where you may need to declare different -versions of the outer function for different element types of -``a``. You could do it like this:: - - function myfun{T<:AbstractFloat}(c::MySimpleContainer{Vector{T}}) - ... - end - function myfun{T<:Integer}(c::MySimpleContainer{Vector{T}}) - ... - end - -This works fine for ``Vector{T}``, but we'd also have to write -explicit versions for ``UnitRange{T}`` or other abstract types. To -prevent such tedium, you can use two parameters in the declaration of -``MyContainer``:: - - type MyContainer{T, A<:AbstractVector} - a::A - end - MyContainer(v::AbstractVector) = MyContainer{eltype(v), typeof(v)}(v) - - julia> b = MyContainer(1.3:5); - - julia> typeof(b) - MyContainer{Float64,UnitRange{Float64}} - -Note the somewhat surprising fact that ``T`` doesn't appear in the -declaration of field ``a``, a point that we'll return to in a moment. -With this approach, one can write functions such as:: - - function myfunc{T<:Integer, A<:AbstractArray}(c::MyContainer{T,A}) - return c.a[1]+1 - end - # Note: because we can only define MyContainer for - # A<:AbstractArray, and any unspecified parameters are arbitrary, - # the previous could have been written more succinctly as - # function myfunc{T<:Integer}(c::MyContainer{T}) - - function myfunc{T<:AbstractFloat}(c::MyContainer{T}) - return c.a[1]+2 - end - - function myfunc{T<:Integer}(c::MyContainer{T,Vector{T}}) - return c.a[1]+3 - end - - julia> myfunc(MyContainer(1:3)) - 2 - - julia> myfunc(MyContainer(1.0:3)) - 3.0 - - julia> myfunc(MyContainer([1:3])) - 4 - -As you can see, with this approach it's possible to specialize on both -the element type ``T`` and the array type ``A``. - -However, there's one remaining hole: we haven't enforced that ``A`` -has element type ``T``, so it's perfectly possible to construct an -object like this:: - - julia> b = MyContainer{Int64, UnitRange{Float64}}(1.3:5); - - julia> typeof(b) - MyContainer{Int64,UnitRange{Float64}} - -To prevent this, we can add an inner constructor:: - - type MyBetterContainer{T<:Real, A<:AbstractVector} - a::A - - MyBetterContainer(v::AbstractVector{T}) = new(v) - end - MyBetterContainer(v::AbstractVector) = MyBetterContainer{eltype(v),typeof(v)}(v) - - - julia> b = MyBetterContainer(1.3:5); - - julia> typeof(b) - MyBetterContainer{Float64,UnitRange{Float64}} - - julia> b = MyBetterContainer{Int64, UnitRange{Float64}}(1.3:5); - ERROR: no method MyBetterContainer(UnitRange{Float64},) - -The inner constructor requires that the element type of ``A`` be ``T``. -- _man-packages: diff --git a/doc/manual/performance-tips.rst b/doc/manual/performance-tips.rst index df033f3182fc8..8ca9d704c7add 100644 --- a/doc/manual/performance-tips.rst +++ b/doc/manual/performance-tips.rst @@ -166,6 +166,10 @@ that can be manipulated efficiently. See also the discussion under :ref:`man-parametric-types`. + + + + Type declarations ----------------- @@ -176,25 +180,311 @@ arguments, local variables, and expressions. However, there are a few specific instances where declarations are helpful. -Declare specific types for fields of composite types -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. _man-abstract-fields: +Avoid fields with abstract type +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Types can be declared without specifying the types of their fields: + +.. doctest:: + + julia> type MyAmbiguousType + a + end + +This allows ``a`` to be of any type. This can often be useful, but it +does have a downside: for objects of type ``MyAmbiguousType``, the +compiler will not be able to generate high-performance code. The +reason is that the compiler uses the types of objects, not their +values, to determine how to build code. Unfortunately, very little can +be inferred about an object of type ``MyAmbiguousType``: + +.. doctest:: + + julia> b = MyAmbiguousType("Hello") + MyAmbiguousType("Hello") + + julia> c = MyAmbiguousType(17) + MyAmbiguousType(17) + + julia> typeof(b) + MyAmbiguousType + + julia> typeof(c) + MyAmbiguousType + +``b`` and ``c`` have the same type, yet their underlying +representation of data in memory is very different. Even if you stored +just numeric values in field ``a``, the fact that the memory +representation of a ``UInt8`` differs from a ``Float64`` also means +that the CPU needs to handle them using two different kinds of +instructions. Since the required information is not available in the +type, such decisions have to be made at run-time. This slows +performance. + +You can do better by declaring the type of ``a``. Here, we are focused +on the case where ``a`` might be any one of several types, in which +case the natural solution is to use parameters. For example: + +.. doctest:: + + julia> type MyType{T<:AbstractFloat} + a::T + end + +This is a better choice than + +.. doctest:: + + julia> type MyStillAmbiguousType + a::AbstractFloat + end + +because the first version specifies the type of ``a`` from the type of +the wrapper object. For example: + +.. doctest:: + + julia> m = MyType(3.2) + MyType{Float64}(3.2) + + julia> t = MyStillAmbiguousType(3.2) + MyStillAmbiguousType(3.2) + + julia> typeof(m) + MyType{Float64} + + julia> typeof(t) + MyStillAmbiguousType + +The type of field ``a`` can be readily determined from the type of +``m``, but not from the type of ``t``. Indeed, in ``t`` it's possible +to change the type of field ``a``: + +.. doctest:: + + julia> typeof(t.a) + Float64 + + julia> t.a = 4.5f0 + 4.5f0 + + julia> typeof(t.a) + Float32 + +In contrast, once ``m`` is constructed, the type of ``m.a`` cannot +change: + +.. doctest:: + + julia> m.a = 4.5f0 + 4.5 + + julia> typeof(m.a) + Float64 + +The fact that the type of ``m.a`` is known from ``m``'s type---coupled +with the fact that its type cannot change mid-function---allows the +compiler to generate highly-optimized code for objects like ``m`` but +not for objects like ``t``. + +Of course, all of this is true only if we construct ``m`` with a +concrete type. We can break this by explicitly constructing it with +an abstract type: + +.. doctest:: + + julia> m = MyType{AbstractFloat}(3.2) + MyType{AbstractFloat}(3.2) + + julia> typeof(m.a) + Float64 + + julia> m.a = 4.5f0 + 4.5f0 + + julia> typeof(m.a) + Float32 + +For all practical purposes, such objects behave identically to those +of ``MyStillAmbiguousType``. + +It's quite instructive to compare the sheer amount code generated for +a simple function +:: + + func(m::MyType) = m.a+1 + +using +:: + + code_llvm(func,(MyType{Float64},)) + code_llvm(func,(MyType{AbstractFloat},)) + code_llvm(func,(MyType,)) -Given a user-defined type like the following:: +For reasons of length the results are not shown here, but you may wish +to try this yourself. Because the type is fully-specified in the first +case, the compiler doesn't need to generate any code to resolve the +type at run-time. This results in shorter and faster code. - type Foo - field + +.. _man-abstract-container-type: + +Avoid fields with abstract containers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The same best practices also work for container types: + +.. doctest:: + + julia> type MySimpleContainer{A<:AbstractVector} + a::A + end + + julia> type MyAmbiguousContainer{T} + a::AbstractVector{T} + end + +For example: + +.. doctest:: + + julia> c = MySimpleContainer(1:3); + + julia> typeof(c) + MySimpleContainer{UnitRange{Int64}} + + julia> c = MySimpleContainer([1:3;]); + + julia> typeof(c) + MySimpleContainer{Array{Int64,1}} + + julia> b = MyAmbiguousContainer(1:3); + + julia> typeof(b) + MyAmbiguousContainer{Int64} + + julia> b = MyAmbiguousContainer([1:3;]); + + julia> typeof(b) + MyAmbiguousContainer{Int64} + +For ``MySimpleContainer``, the object is fully-specified by its type +and parameters, so the compiler can generate optimized functions. In +most instances, this will probably suffice. + +While the compiler can now do its job perfectly well, there are cases +where *you* might wish that your code could do different things +depending on the *element type* of ``a``. Usually the best way to +achieve this is to wrap your specific operation (here, ``foo``) in a +separate function:: + + function sumfoo(c::MySimpleContainer) + s = 0 + for x in c.a + s += foo(x) + end + s + end + + foo(x::Integer) = x + foo(x::AbstractFloat) = round(x) + +This keeps things simple, while allowing the compiler to generate +optimized code in all cases. + +However, there are cases where you may need to declare different +versions of the outer function for different element types of +``a``. You could do it like this:: + + function myfun{T<:AbstractFloat}(c::MySimpleContainer{Vector{T}}) + ... + end + function myfun{T<:Integer}(c::MySimpleContainer{Vector{T}}) + ... + end + +This works fine for ``Vector{T}``, but we'd also have to write +explicit versions for ``UnitRange{T}`` or other abstract types. To +prevent such tedium, you can use two parameters in the declaration of +``MyContainer``:: + + type MyContainer{T, A<:AbstractVector} + a::A + end + MyContainer(v::AbstractVector) = MyContainer{eltype(v), typeof(v)}(v) + + julia> b = MyContainer(1.3:5); + + julia> typeof(b) + MyContainer{Float64,UnitRange{Float64}} + +Note the somewhat surprising fact that ``T`` doesn't appear in the +declaration of field ``a``, a point that we'll return to in a moment. +With this approach, one can write functions such as:: + + function myfunc{T<:Integer, A<:AbstractArray}(c::MyContainer{T,A}) + return c.a[1]+1 + end + # Note: because we can only define MyContainer for + # A<:AbstractArray, and any unspecified parameters are arbitrary, + # the previous could have been written more succinctly as + # function myfunc{T<:Integer}(c::MyContainer{T}) + + function myfunc{T<:AbstractFloat}(c::MyContainer{T}) + return c.a[1]+2 + end + + function myfunc{T<:Integer}(c::MyContainer{T,Vector{T}}) + return c.a[1]+3 end -the compiler will not generally know the type of ``foo.field``, since it -might be modified at any time to refer to a value of a different type. -It will help to declare the most specific type possible, such as -``field::Float64`` or ``field::Array{Int64,1}``. + julia> myfunc(MyContainer(1:3)) + 2 + + julia> myfunc(MyContainer(1.0:3)) + 3.0 + + julia> myfunc(MyContainer([1:3])) + 4 + +As you can see, with this approach it's possible to specialize on both +the element type ``T`` and the array type ``A``. + +However, there's one remaining hole: we haven't enforced that ``A`` +has element type ``T``, so it's perfectly possible to construct an +object like this:: + + julia> b = MyContainer{Int64, UnitRange{Float64}}(1.3:5); + + julia> typeof(b) + MyContainer{Int64,UnitRange{Float64}} + +To prevent this, we can add an inner constructor:: + + type MyBetterContainer{T<:Real, A<:AbstractVector} + a::A + + MyBetterContainer(v::AbstractVector{T}) = new(v) + end + MyBetterContainer(v::AbstractVector) = MyBetterContainer{eltype(v),typeof(v)}(v) + + + julia> b = MyBetterContainer(1.3:5); + + julia> typeof(b) + MyBetterContainer{Float64,UnitRange{Float64}} + + julia> b = MyBetterContainer{Int64, UnitRange{Float64}}(1.3:5); + ERROR: no method MyBetterContainer(UnitRange{Float64},) + +The inner constructor requires that the element type of ``A`` be ``T``. Annotate values taken from untyped locations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is often convenient to work with data structures that may contain -values of any type, such as the original ``Foo`` type above, or cell +values of any type, such as cell arrays (arrays of type ``Array{Any}``). But, if you're using one of these structures and happen to know the type of an element, it helps to share this knowledge with the compiler:: From 2bbdc30448ac3ded06f41a5c9d2ce54bdf2b5c67 Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Fri, 18 Sep 2015 18:54:14 -0400 Subject: [PATCH 2/6] eliminate codegen dependence on jl_pvalue_llvmt as a unique type --- src/ccall.cpp | 93 ++++---- src/cgutils.cpp | 97 +++++---- src/codegen.cpp | 513 +++++++++++++++++++++++++-------------------- src/intrinsics.cpp | 73 ++++--- 4 files changed, 429 insertions(+), 347 deletions(-) diff --git a/src/ccall.cpp b/src/ccall.cpp index 4866639609b44..b66b236ac8dbd 100644 --- a/src/ccall.cpp +++ b/src/ccall.cpp @@ -314,13 +314,13 @@ Value *llvm_type_rewrite(Value *v, Type *from_type, Type *target_type, // to = desired LLVM type // jlto = Julia type of formal argument // jvinfo = value of actual argument -static Value *julia_to_native(Type *to, jl_value_t *jlto, const jl_cgval_t &jvinfo, +static Value *julia_to_native(Type *to, bool toboxed, jl_value_t *jlto, const jl_cgval_t &jvinfo, bool addressOf, bool byRef, bool inReg, bool needCopy, bool tojulia, int argn, jl_codectx_t *ctx, bool *needStackRestore) { // We're passing Any - if (to == jl_pvalue_llvmt) { + if (toboxed) { assert(!addressOf && !byRef); // don't expect any ABI to pass pointers by pointer return boxed(jvinfo, ctx); } @@ -586,7 +586,7 @@ static jl_cgval_t emit_cglobal(jl_value_t **args, size_t nargs, jl_codectx_t *ct } JL_GC_POP(); - return mark_julia_type(res, rt); + return mark_julia_type(res, false, rt); } // llvmcall(ir, (rettypes...), (argtypes...), args...) @@ -663,7 +663,8 @@ static jl_cgval_t emit_llvmcall(jl_value_t **args, size_t nargs, jl_codectx_t *c */ for (size_t i = 0; i < nargt; ++i) { jl_value_t *tti = jl_svecref(tt,i); - Type *t = julia_type_to_llvm(tti); + bool toboxed; + Type *t = julia_type_to_llvm(tti, &toboxed); argtypes.push_back(t); if (4+i > nargs) { jl_error("Missing arguments to llvmcall!"); @@ -671,19 +672,20 @@ static jl_cgval_t emit_llvmcall(jl_value_t **args, size_t nargs, jl_codectx_t *c jl_value_t *argi = args[4+i]; jl_cgval_t arg; bool needroot = false; - if (t == jl_pvalue_llvmt || !jl_isbits(tti)) { + if (toboxed || !jl_isbits(tti)) { arg = emit_expr(argi, ctx, true); - if (t == jl_pvalue_llvmt && !arg.isboxed) { + if (toboxed && !arg.isboxed) { needroot = true; } } else { arg = emit_unboxed(argi, ctx); + toboxed = false; } - Value *v = julia_to_native(t, tti, arg, false, false, false, false, false, i, ctx, NULL); + Value *v = julia_to_native(t, toboxed, tti, arg, false, false, false, false, false, i, ctx, NULL); // make sure args are rooted - if (t == jl_pvalue_llvmt && (needroot || might_need_root(argi))) { + if (toboxed && (needroot || might_need_root(argi))) { make_gcroot(v, ctx); } bool issigned = jl_signed_type && jl_subtype(tti, (jl_value_t*)jl_signed_type, 0); @@ -691,7 +693,8 @@ static jl_cgval_t emit_llvmcall(jl_value_t **args, size_t nargs, jl_codectx_t *c } Function *f; - Type *rettype = julia_type_to_llvm(rtt); + bool retboxed; + Type *rettype = julia_type_to_llvm(rtt, &retboxed); if (isString) { // Make sure to find a unique name std::string ir_name; @@ -779,7 +782,7 @@ static jl_cgval_t emit_llvmcall(jl_value_t **args, size_t nargs, jl_codectx_t *c // the actual call assert(f->getParent() == jl_Module); // no prepare_call(f) is needed below, since this was just emitted into the same module - CallInst *inst = builder.CreateCall(f,ArrayRef(&argvals[0],nargt)); + CallInst *inst = builder.CreateCall(f, ArrayRef(&argvals[0], nargt)); ctx->to_inline.push_back(inst); JL_GC_POP(); @@ -788,12 +791,12 @@ static jl_cgval_t emit_llvmcall(jl_value_t **args, size_t nargs, jl_codectx_t *c jl_error("Return type of llvmcall'ed function does not match declared return type"); } - return mark_julia_type(inst, rtt); + return mark_julia_type(inst, retboxed, rtt); } // --- code generator for ccall itself --- -static jl_cgval_t mark_or_box_ccall_result(Value *result, jl_value_t *rt_expr, jl_value_t *rt, bool static_rt, jl_codectx_t *ctx) +static jl_cgval_t mark_or_box_ccall_result(Value *result, bool isboxed, jl_value_t *rt_expr, jl_value_t *rt, bool static_rt, jl_codectx_t *ctx) { if (!static_rt) { // box if concrete type was not statically known @@ -802,9 +805,10 @@ static jl_cgval_t mark_or_box_ccall_result(Value *result, jl_value_t *rt_expr, j int nb = sizeof(void*); return mark_julia_type( init_bits_value(emit_allocobj(nb), runtime_bt, result), + true, (jl_value_t*)jl_pointer_type); } - return mark_julia_type(result, rt); + return mark_julia_type(result, isboxed, rt); } typedef AttributeSet attr_type; @@ -814,6 +818,7 @@ static std::string generate_func_sig( Type **prt, // out parameter of the llvm return type for the function signature int &sret, // out parameter for indicating whether return value has been moved to the first argument position std::vector &fargt, // vector of llvm output types (julia_struct_to_llvm) for arguments (vararg is the last item, if applicable) + std::vector &fargt_isboxed, // vector of whether the llvm output types is boxed for each argument (vararg is the last item, if applicable) std::vector &fargt_sig, // vector of ABI coercion types for call signature Type *&fargt_vasig, // ABI coercion type for vararg list std::vector &inRegList, // vector of "inreg" parameters (vararg is the last item, if applicable) @@ -859,12 +864,14 @@ static std::string generate_func_sig( tti = jl_tparam0(tti); } Type *t = NULL; + bool isboxed; Attribute::AttrKind av = Attribute::None; if (jl_is_abstract_ref_type(tti)) { if (jl_is_typevar(jl_tparam0(tti))) jl_error("ccall: argument type Ref should have an element type, not Ref{T}"); tti = (jl_value_t*)jl_voidpointer_type; t = T_pint8; + isboxed = false; } else { if (jl_is_cpointer_type(tti) && jl_is_typevar(jl_tparam0(tti))) @@ -881,7 +888,7 @@ static std::string generate_func_sig( } } - t = julia_struct_to_llvm(tti); + t = julia_struct_to_llvm(tti, &isboxed); if (t == NULL || t == T_void) { std::stringstream msg; msg << "ccall: the type of argument "; @@ -915,6 +922,7 @@ static std::string generate_func_sig( byRefList.push_back(byRef); inRegList.push_back(inReg); fargt.push_back(t); + fargt_isboxed.push_back(isboxed); if (!current_isVa) fargt_sig.push_back(pat); else @@ -1034,7 +1042,8 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) } JL_TYPECHK(ccall, type, rt); - Type *lrt = julia_struct_to_llvm(rt); + bool retboxed; + Type *lrt = julia_struct_to_llvm(rt, &retboxed); if (lrt == NULL) { emit_error("ccall: return type doesn't correspond to a C type", ctx); JL_GC_POP(); @@ -1111,8 +1120,8 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) assert(!(jl_is_expr(argi) && ((jl_expr_t*)argi)->head == amp_sym)); jl_cgval_t ary = emit_expr(argi, ctx); JL_GC_POP(); - return mark_or_box_ccall_result(builder.CreateBitCast(emit_arrayptr(boxed(ary, ctx)),lrt), - args[2], rt, static_rt, ctx); + return mark_or_box_ccall_result(builder.CreateBitCast(emit_arrayptr(boxed(ary, ctx)), lrt), + retboxed, args[2], rt, static_rt, ctx); } if (fptr == (void *) &jl_value_ptr || ((f_lib==NULL || (intptr_t)f_lib==2) @@ -1132,11 +1141,15 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) } Value *ary; Type *largty; - if (addressOf) - largty = jl_pvalue_llvmt; - else - largty = julia_struct_to_llvm(tti); - if (largty == jl_pvalue_llvmt) { + bool isboxed; + if (addressOf) { + largty = T_pjlvalue; + isboxed = true; + } + else { + largty = julia_struct_to_llvm(tti, &isboxed); + } + if (isboxed) { ary = boxed(emit_expr(argi, ctx),ctx); } else { @@ -1145,7 +1158,7 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) } JL_GC_POP(); return mark_or_box_ccall_result(builder.CreateBitCast(ary, lrt), - args[2], rt, static_rt, ctx); + retboxed, args[2], rt, static_rt, ctx); } if (fptr == (void *) &jl_is_leaf_type || ((f_lib==NULL || (intptr_t)f_lib==2) @@ -1157,7 +1170,7 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) int isleaf = jl_is_leaf_type(jl_tparam0(ty)); JL_GC_POP(); return mark_or_box_ccall_result(ConstantInt::get(T_int32, isleaf), - args[2], rt, static_rt, ctx); + false, args[2], rt, static_rt, ctx); } } if (fptr == (void*)&jl_function_ptr || @@ -1191,7 +1204,7 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) emit_expr(args[8], ctx); JL_GC_POP(); return mark_or_box_ccall_result(builder.CreateBitCast(llvmf, lrt), - args[2], rt, static_rt, ctx); + retboxed, args[2], rt, static_rt, ctx); } JL_CATCH { } @@ -1215,13 +1228,14 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) std::vector fargt(0); std::vector fargt_sig(0); + std::vector fargt_isboxed(0); Type *fargt_vasig = NULL; std::vector inRegList(0); std::vector byRefList(0); attr_type attrs; Type *prt = NULL; int sret = 0; - std::string err_msg = generate_func_sig(&lrt, &prt, sret, fargt, fargt_sig, fargt_vasig, + std::string err_msg = generate_func_sig(&lrt, &prt, sret, fargt, fargt_isboxed, fargt_sig, fargt_vasig, inRegList, byRefList, attrs, rt, tt, (nargs - 3)/2); if (!err_msg.empty()) { JL_GC_POP(); @@ -1236,6 +1250,7 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) // First, if the ABI requires us to provide the space for the return // argument, allocate the box and store that as the first argument type + bool sretboxed; if (sret) { jl_cgval_t sret_val = emit_new_struct(rt,1,NULL,ctx); // TODO: is it valid to be creating an incomplete type this way? assert(sret_val.typ != NULL && "Type was not concrete"); @@ -1246,10 +1261,11 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) argvals[0] = result; } else { - // XXX: result needs a GC root here if result->getType() == jl_pvalue_llvmt + // XXX: result needs a GC root here if result->getType() == T_pjlvalue result = sret_val.V; argvals[0] = builder.CreateBitCast(result, fargt_sig.at(0)); } + sretboxed = sret_val.isboxed; } // save argument depth until after we're done emitting arguments @@ -1271,16 +1287,19 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) } Type *largty; // LLVM type of the current parameter + bool toboxed; jl_value_t *jargty; // Julia type of the current parameter bool byRef, inReg; // Argument attributes if (isVa && ai >= nargt - 1) { largty = fargt.at(nargt - 1); + toboxed = fargt_isboxed.at(nargt - 1); jargty = jl_tparam0(jl_svecref(tt, nargt - 1)); byRef = byRefList.at(nargt - 1); inReg = inRegList.at(nargt - 1); } else { largty = fargt.at(ai); + toboxed = fargt_isboxed.at(ai); jargty = jl_svecref(tt, ai); byRef = byRefList.at(ai); inReg = inRegList.at(ai); @@ -1302,9 +1321,9 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) } jargty = (jl_value_t*)jl_voidpointer_type; } - else if (largty == jl_pvalue_llvmt || largty->isStructTy()) { + else if (toboxed || largty->isStructTy()) { arg = emit_expr(argi, ctx, true); - if (largty == jl_pvalue_llvmt && !arg.isboxed) { + if (toboxed && !arg.isboxed) { needroot = true; } } @@ -1312,10 +1331,10 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) arg = emit_unboxed(argi, ctx); } - Value *v = julia_to_native(largty, jargty, arg, addressOf, byRef, inReg, + Value *v = julia_to_native(largty, toboxed, jargty, arg, addressOf, byRef, inReg, need_private_copy(jargty, byRef), false, ai + 1, ctx, &needStackRestore); // make sure args are rooted - if (largty == jl_pvalue_llvmt && (needroot || might_need_root(argi))) { + if (toboxed && (needroot || might_need_root(argi))) { make_gcroot(v, ctx); } bool issigned = jl_signed_type && jl_subtype(jargty, (jl_value_t*)jl_signed_type, 0); @@ -1411,18 +1430,17 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) // type because the ABI required us to pass a pointer (sret), // then we do not need to do this. if (!sret) { - Type *jlrt = julia_type_to_llvm(rt); + Type *jlrt = julia_type_to_llvm(rt, &retboxed); // compute the real "julian" return type and update retboxed if (type_is_ghost(jlrt)) { return ghostValue(rt); } - else if (lrt->isStructTy() && jlrt == jl_pvalue_llvmt) { + else if (lrt->isStructTy() && retboxed) { assert(jl_is_structtype(rt)); jl_cgval_t newst = emit_new_struct(rt, 1, NULL, ctx); // emit a new, empty struct assert(newst.typ != NULL && "Type was not concrete"); assert(newst.isboxed); // copy the data from the return value to the new struct - // julia gc is aligned 16, otherwise use default alignment for alloca pointers - builder.CreateAlignedStore(result, builder.CreateBitCast(newst.V, prt->getPointerTo()), newst.V->getType() == jl_pvalue_llvmt ? 16 : 0); + builder.CreateAlignedStore(result, builder.CreateBitCast(newst.V, prt->getPointerTo()), 16); // julia gc is aligned 16 return newst; } else if (jlrt != prt) { @@ -1431,9 +1449,10 @@ static jl_cgval_t emit_ccall(jl_value_t **args, size_t nargs, jl_codectx_t *ctx) } } else { - if (result->getType() != jl_pvalue_llvmt) + retboxed = sretboxed; + if (!retboxed) result = builder.CreateLoad(result); // something alloca'd above } - return mark_or_box_ccall_result(result, args[2], rt, static_rt, ctx); + return mark_or_box_ccall_result(result, retboxed, args[2], rt, static_rt, ctx); } diff --git a/src/cgutils.cpp b/src/cgutils.cpp index 2ff4fad667ae3..8deefc2cb9fef 100644 --- a/src/cgutils.cpp +++ b/src/cgutils.cpp @@ -541,9 +541,9 @@ static Value *julia_gv(const char *cname, void *addr) std::stringstream gvname; gvname << cname << globalUnique++; // no existing GlobalVariable, create one and store it - GlobalVariable *gv = new GlobalVariable(*jl_Module, jl_pvalue_llvmt, + GlobalVariable *gv = new GlobalVariable(*jl_Module, T_pjlvalue, false, imaging_mode ? GlobalVariable::InternalLinkage : GlobalVariable::ExternalLinkage, - ConstantPointerNull::get((PointerType*)jl_pvalue_llvmt), gvname.str()); + ConstantPointerNull::get((PointerType*)T_pjlvalue), gvname.str()); addComdat(gv); // make the pointer valid for this session @@ -595,7 +595,7 @@ static Value *literal_pointer_val(jl_value_t *p) // emit a pointer to any jl_value_t which will be valid across reloading code // also, try to give it a nice name for gdb, for easy identification if (p == NULL) - return ConstantPointerNull::get((PointerType*)jl_pvalue_llvmt); + return ConstantPointerNull::get((PointerType*)T_pjlvalue); // some common constant values if (p == jl_false) return tbaa_decorate(tbaa_const, builder.CreateLoad(prepare_global(jlfalse_var))); @@ -604,7 +604,7 @@ static Value *literal_pointer_val(jl_value_t *p) if (p == (jl_value_t*)jl_emptysvec) return tbaa_decorate(tbaa_const, builder.CreateLoad(prepare_global(jlemptysvec_var))); if (!imaging_mode) - return literal_static_pointer_val(p, jl_pvalue_llvmt); + return literal_static_pointer_val(p, T_pjlvalue); if (jl_is_datatype(p)) { jl_datatype_t *addr = (jl_datatype_t*)p; // DataTypes are prefixed with a + @@ -640,9 +640,9 @@ static Value *literal_pointer_val(jl_binding_t *p) { // emit a pointer to any jl_value_t which will be valid across reloading code if (p == NULL) - return ConstantPointerNull::get((PointerType*)jl_pvalue_llvmt); + return ConstantPointerNull::get((PointerType*)T_pjlvalue); if (!imaging_mode) - return literal_static_pointer_val(p, jl_pvalue_llvmt); + return literal_static_pointer_val(p, T_pjlvalue); // bindings are prefixed with jl_bnd# return julia_gv("jl_bnd#", p->name, p->owner, p); } @@ -659,23 +659,26 @@ static Value *julia_binding_gv(jl_binding_t *b) // emit a literal_pointer_val to the value field of a jl_binding_t // binding->value are prefixed with * Value *bv = imaging_mode ? - builder.CreateBitCast(julia_gv("*", b->name, b->owner, b), jl_ppvalue_llvmt) : - literal_static_pointer_val(b,jl_ppvalue_llvmt); + builder.CreateBitCast(julia_gv("*", b->name, b->owner, b), T_ppjlvalue) : + literal_static_pointer_val(b,T_ppjlvalue); return julia_binding_gv(bv); } // --- mapping between julia and llvm types --- -static Type *julia_struct_to_llvm(jl_value_t *jt); +static Type *julia_struct_to_llvm(jl_value_t *jt, bool *isboxed); extern "C" { -DLLEXPORT Type *julia_type_to_llvm(jl_value_t *jt) +DLLEXPORT Type *julia_type_to_llvm(jl_value_t *jt, bool *isboxed) { // this function converts a Julia Type into the equivalent LLVM type + if (isboxed) *isboxed = false; if (jt == (jl_value_t*)jl_bool_type) return T_int1; if (jt == (jl_value_t*)jl_bottom_type) return T_void; - if (!jl_is_leaf_type(jt)) - return jl_pvalue_llvmt; + if (!jl_is_leaf_type(jt)) { + if (isboxed) *isboxed = true; + return T_pjlvalue; + } if (jl_is_cpointer_type(jt)) { Type *lt = julia_type_to_llvm(jl_tparam0(jt)); if (lt == NULL) @@ -705,18 +708,20 @@ DLLEXPORT Type *julia_type_to_llvm(jl_value_t *jt) if (((jl_datatype_t*)jt)->size == 0) { return T_void; } - return julia_struct_to_llvm(jt); + return julia_struct_to_llvm(jt, isboxed); } - return jl_pvalue_llvmt; + if (isboxed) *isboxed = true; + return T_pjlvalue; } } -static Type *julia_struct_to_llvm(jl_value_t *jt) +static Type *julia_struct_to_llvm(jl_value_t *jt, bool *isboxed) { // this function converts a Julia Type into the equivalent LLVM struct // use this where C-compatible (unboxed) structs are desired // use julia_type_to_llvm directly when you want to preserve Julia's type semantics bool isTuple = jl_is_tuple_type(jt); + if (isboxed) *isboxed = false; if ((isTuple || jl_is_structtype(jt)) && !jl_is_array_type(jt)) { if (!jl_is_leaf_type(jt)) return NULL; @@ -738,7 +743,7 @@ static Type *julia_struct_to_llvm(jl_value_t *jt) jl_value_t *ty = jl_svecref(jst->types, i); Type *lty; if (jl_field_isptr(jst, i)) - lty = jl_pvalue_llvmt; + lty = T_pjlvalue; else lty = ty==(jl_value_t*)jl_bool_type ? T_int8 : julia_type_to_llvm(ty); if (lasttype != NULL && lasttype != lty) @@ -767,7 +772,7 @@ static Type *julia_struct_to_llvm(jl_value_t *jt) } return (Type*)jst->struct_decl; } - return julia_type_to_llvm(jt); + return julia_type_to_llvm(jt, isboxed); } static bool is_datatype_all_pointers(jl_datatype_t *dt) @@ -806,13 +811,13 @@ static bool deserves_sret(jl_value_t *dt, Type *T) static Value *emit_nthptr_addr(Value *v, ssize_t n) { - return builder.CreateGEP(builder.CreateBitCast(v, jl_ppvalue_llvmt), + return builder.CreateGEP(builder.CreateBitCast(v, T_ppjlvalue), ConstantInt::get(T_size, (ssize_t)n)); } static Value *emit_nthptr_addr(Value *v, Value *idx) { - return builder.CreateGEP(builder.CreateBitCast(v, jl_ppvalue_llvmt), idx); + return builder.CreateGEP(builder.CreateBitCast(v, T_ppjlvalue), idx); } static Value *emit_nthptr(Value *v, ssize_t n, MDNode *tbaa) @@ -845,12 +850,12 @@ static Value *emit_typeptr_addr(Value *p) static Value *emit_typeof(Value *tt) { // given p, a jl_value_t*, compute its type tag - assert(tt->getType() == jl_pvalue_llvmt); + assert(tt->getType() == T_pjlvalue); tt = builder.CreateLoad(emit_typeptr_addr(tt), false); tt = builder.CreateIntToPtr(builder.CreateAnd( builder.CreatePtrToInt(tt, T_size), ConstantInt::get(T_size,~(uptrint_t)15)), - jl_pvalue_llvmt); + T_pjlvalue); return tt; } static Value *emit_typeof(const jl_cgval_t &p) @@ -874,7 +879,7 @@ static Value *emit_datatype_types(const jl_cgval_t &dt) CreateBitCast(builder. CreateGEP(builder.CreateBitCast(dt.V, T_pint8), ConstantInt::get(T_size, offsetof(jl_datatype_t, types))), - jl_ppvalue_llvmt)); + T_ppjlvalue)); } static Value *emit_datatype_nfields(const jl_cgval_t &dt) @@ -1085,7 +1090,8 @@ static Value *emit_unbox(Type *to, const jl_cgval_t &x, jl_value_t *jt); static jl_cgval_t typed_load(Value *ptr, Value *idx_0based, jl_value_t *jltype, jl_codectx_t *ctx, MDNode* tbaa, size_t alignment = 0) { - Type *elty = julia_type_to_llvm(jltype); + bool isboxed; + Type *elty = julia_type_to_llvm(jltype, &isboxed); assert(elty != NULL); if (type_is_ghost(elty)) return ghostValue(jltype); @@ -1116,13 +1122,13 @@ static jl_cgval_t typed_load(Value *ptr, Value *idx_0based, jl_value_t *jltype, else { elt = load; } - if (elty == jl_pvalue_llvmt) { + if (isboxed) { null_pointer_check(elt, ctx); } //} if (isbool) - return mark_julia_type(builder.CreateTrunc(elt, T_int1), jltype); - return mark_julia_type(elt, jltype); + return mark_julia_type(builder.CreateTrunc(elt, T_int1), false, jltype); + return mark_julia_type(elt, isboxed, jltype); } static void typed_store(Value *ptr, Value *idx_0based, const jl_cgval_t &rhs, @@ -1260,7 +1266,7 @@ static void jl_add_linfo_root(jl_lambda_info_t *li, jl_value_t *val); static Value *data_pointer(Value *x) { - return builder.CreateBitCast(x, jl_ppvalue_llvmt); + return builder.CreateBitCast(x, T_ppjlvalue); } static bool emit_getfield_unknownidx(jl_cgval_t *ret, const jl_cgval_t &strct, Value *idx, jl_datatype_t *stt, jl_codectx_t *ctx) @@ -1271,11 +1277,11 @@ static bool emit_getfield_unknownidx(jl_cgval_t *ret, const jl_cgval_t &strct, V idx = emit_bounds_check(strct, (jl_value_t*)stt, idx, ConstantInt::get(T_size, nfields), ctx); Value *fld = tbaa_decorate(tbaa_user, builder.CreateLoad( builder.CreateGEP( - builder.CreateBitCast(strct.V, jl_ppvalue_llvmt), + builder.CreateBitCast(strct.V, T_ppjlvalue), idx))); if ((unsigned)stt->ninitialized != nfields) null_pointer_check(fld, ctx); - *ret = mark_julia_type(fld, jl_any_type); + *ret = mark_julia_type(fld, true, jl_any_type); return true; } else if (is_tupletype_homogeneous(stt->types)) { @@ -1293,7 +1299,7 @@ static bool emit_getfield_unknownidx(jl_cgval_t *ret, const jl_cgval_t &strct, V #else Value *fld = builder.CreateCall2(prepare_call(jlgetnthfieldchecked_func), strct.V, idx); #endif - *ret = mark_julia_type(fld, jl_any_type); + *ret = mark_julia_type(fld, true, jl_any_type); return true; } } @@ -1323,7 +1329,7 @@ static bool emit_getfield_unknownidx(jl_cgval_t *ret, const jl_cgval_t &strct, V if (jt == (jl_value_t*)jl_bool_type) { fld = builder.CreateTrunc(fld, T_int1); } - *ret = mark_julia_type(fld, jt); + *ret = mark_julia_type(fld, false, jt); return true; } return false; @@ -1347,10 +1353,10 @@ static jl_cgval_t emit_getfield_knownidx(const jl_cgval_t &strct, unsigned idx, ConstantInt::get(T_size, jl_field_offset(jt,idx))); MDNode *tbaa = strct.isimmutable ? tbaa_immut : tbaa_user; if (jl_field_isptr(jt,idx)) { - Value *fldv = tbaa_decorate(tbaa, builder.CreateLoad(builder.CreateBitCast(addr,jl_ppvalue_llvmt))); + Value *fldv = tbaa_decorate(tbaa, builder.CreateLoad(builder.CreateBitCast(addr, T_ppjlvalue))); if (idx >= (unsigned)jt->ninitialized) null_pointer_check(fldv, ctx); - return mark_julia_type(fldv, jfty); + return mark_julia_type(fldv, true, jfty); } else { int align = jl_field_offset(jt,idx); @@ -1376,7 +1382,7 @@ static jl_cgval_t emit_getfield_knownidx(const jl_cgval_t &strct, unsigned idx, fldv = builder.CreateTrunc(fldv, T_int1); } assert(!jl_field_isptr(jt, idx)); - return mark_julia_type(fldv, jfty); + return mark_julia_type(fldv, false, jfty); } } @@ -1447,7 +1453,7 @@ static Value *emit_arraylen_prim(Value *t, jl_value_t *ty) } else { std::vector fargt(0); - fargt.push_back(jl_pvalue_llvmt); + fargt.push_back(T_pjlvalue); FunctionType *ft = FunctionType::get(T_size, fargt, false); Value *alen = jl_Module->getOrInsertFunction("jl_array_len_", ft); return builder.CreateCall(prepare_call(alen), t); @@ -1703,7 +1709,7 @@ static Value *boxed(const jl_cgval_t &vinfo, jl_codectx_t *ctx, jl_value_t *jt) } if (jt == jl_bottom_type || jt == NULL) { // We have an undef value on a (hopefully) dead branch - return UndefValue::get(jl_pvalue_llvmt); + return UndefValue::get(T_pjlvalue); } if (vinfo.isghost) { jl_value_t *s = static_void_instance(jt); @@ -1770,7 +1776,7 @@ static void emit_cpointercheck(const jl_cgval_t &x, const std::string &msg, jl_codectx_t *ctx) { Value *t = emit_typeof(x); - emit_typecheck(mark_julia_type(t, jl_any_type), (jl_value_t*)jl_datatype_type, msg, ctx); + emit_typecheck(mark_julia_type(t, true, jl_any_type), (jl_value_t*)jl_datatype_type, msg, ctx); Value *istype = builder.CreateICmpEQ(emit_nthptr(t, (ssize_t)(offsetof(jl_datatype_t,name)/sizeof(char*)), tbaa_datatype), @@ -1835,7 +1841,7 @@ static void emit_write_barrier(jl_codectx_t* ctx, Value *parent, Value *ptr) Value* ptr_not_marked = builder.CreateICmpEQ(ptr_mark_bit, ConstantInt::get(T_size, 0)); builder.CreateCondBr(ptr_not_marked, barrier_trigger, cont); builder.SetInsertPoint(barrier_trigger); - builder.CreateCall(prepare_call(queuerootfun), builder.CreateBitCast(parent, jl_pvalue_llvmt)); + builder.CreateCall(prepare_call(queuerootfun), builder.CreateBitCast(parent, T_pjlvalue)); builder.CreateBr(cont); ctx->f->getBasicBlockList().push_back(cont); builder.SetInsertPoint(cont); @@ -1867,7 +1873,7 @@ static void emit_setfield(jl_datatype_t *sty, const jl_cgval_t &strct, size_t id if (jl_field_isptr(sty, idx0)) { Value *r = boxed(rhs, ctx); builder.CreateStore(r, - builder.CreateBitCast(addr, jl_ppvalue_llvmt)); + builder.CreateBitCast(addr, T_ppjlvalue)); if (wb) emit_checked_write_barrier(ctx, strct.V, r); } else { @@ -1916,7 +1922,7 @@ static jl_cgval_t emit_new_struct(jl_value_t *ty, size_t nargs, jl_value_t **arg } idx++; } - return mark_julia_type(strct, ty); + return mark_julia_type(strct, false, ty); } Value *f1 = NULL; int fieldStart = ctx->gc.argDepth; @@ -1940,11 +1946,11 @@ static jl_cgval_t emit_new_struct(jl_value_t *ty, size_t nargs, jl_value_t **arg make_gcroot(f1, ctx); } Value *strct = emit_allocobj(sty->size); - jl_cgval_t strctinfo = mark_julia_type(strct, ty); + jl_cgval_t strctinfo = mark_julia_type(strct, true, ty); builder.CreateStore(literal_pointer_val((jl_value_t*)ty), emit_typeptr_addr(strct)); if (f1) { - jl_cgval_t f1info = mark_julia_type(f1, jl_any_type); + jl_cgval_t f1info = mark_julia_type(f1, true, jl_any_type); if (!jl_subtype(expr_type(args[1],ctx), jl_field_type(sty,0), 0)) emit_typecheck(f1info, jl_field_type(sty,0), "new", ctx); emit_setfield(sty, strctinfo, 0, f1info, ctx, false, false); @@ -1962,7 +1968,7 @@ static jl_cgval_t emit_new_struct(jl_value_t *ty, size_t nargs, jl_value_t **arg builder.CreatePointerCast( builder.CreateGEP(builder.CreateBitCast(strct, T_pint8), ConstantInt::get(T_size, jl_field_offset(sty,i))), - jl_ppvalue_llvmt)); + T_ppjlvalue)); } } bool need_wb = false; @@ -1995,11 +2001,12 @@ static jl_cgval_t emit_new_struct(jl_value_t *ty, size_t nargs, jl_value_t **arg if (nargs >= 2) return emit_unboxed(args[1], ctx); // do side effects Type *lt = julia_type_to_llvm(ty); - return mark_julia_type(UndefValue::get(lt), ty); + assert(lt != T_pjlvalue); + return mark_julia_type(UndefValue::get(lt), false, ty); } else { // 0 fields, singleton assert(sty->instance != NULL); - return mark_julia_type(literal_pointer_val(sty->instance), sty); + return mark_julia_const(sty->instance); } } diff --git a/src/codegen.cpp b/src/codegen.cpp index 1d6e59b9140a6..12aa0358db5f5 100644 --- a/src/codegen.cpp +++ b/src/codegen.cpp @@ -115,6 +115,10 @@ using namespace llvm::legacy; #define LLVM37_param(x) #endif +#ifndef LLVM35 +#define AddrSpaceCastInst BitCastInst +#endif + extern "C" { #include "builtin_proto.h" @@ -193,9 +197,9 @@ static DataLayout *jl_data_layout; static bool imaging_mode = false; // types -static Type *jl_value_llvmt; -static Type *jl_pvalue_llvmt; -static Type *jl_ppvalue_llvmt; +static Type *T_jlvalue; +static Type *T_pjlvalue; +static Type *T_ppjlvalue; static Type* jl_parray_llvmt; static FunctionType *jl_func_sig; static Type *jl_pfptr_llvmt; @@ -225,6 +229,9 @@ static Type *T_psize; static Type *T_pfloat32; static Type *T_pfloat64; +static Type *T_ppint8; +static Type *T_pppint8; + static Type *T_void; // type-based alias analysis nodes. Indentation of comments indicates hierarchy. @@ -267,7 +274,7 @@ static Value *V_null; static Type *NoopType; static Value *literal_pointer_val(jl_value_t *p); extern "C" { -DLLEXPORT Type *julia_type_to_llvm(jl_value_t *jt); +DLLEXPORT Type *julia_type_to_llvm(jl_value_t *jt, bool *isboxed=NULL); } static bool type_is_ghost(Type *ty) { @@ -390,15 +397,14 @@ struct jl_cgval_t { bool isghost; // whether this value is "ghost" bool ispointer; // whether this value is actually pointer to the value bool isimmutable; // V points to something that is definitely immutable (e.g. not stack allocated) - //bool isbox; // points to memory inside a jl_box_t* //bool isstack; // points to stack-allocated memory //bool isarg; // derived from an argument bool needsgcroot; // this value needs a gcroot - jl_cgval_t(Value *V, jl_value_t *typ) : // general constructor (with pointer type auto-detect) + jl_cgval_t(Value *V, bool isboxed, jl_value_t *typ) : // general constructor (with pointer type auto-detect) V(V), // V is allowed to be NULL in a jl_varinfo_t context, but not during codegen contexts typ(typ), //T(julia_type_to_llvm(typ)), - isboxed(V && V->getType() == jl_pvalue_llvmt), + isboxed(isboxed), isghost(false), ispointer(this->isboxed), isimmutable(this->isboxed && jl_is_immutable_datatype(typ)), @@ -584,7 +590,7 @@ static AllocaInst *emit_static_alloca(Type *lty) static inline jl_cgval_t ghostValue(jl_value_t *typ) { - assert(typ == jl_bottom_type || jl_is_datatype(typ)); + assert(typ == jl_bottom_type || (jl_is_datatype(typ) && jl_datatype_size(typ) == 0)); return jl_cgval_t(typ); } static inline jl_cgval_t ghostValue(jl_datatype_t *typ) @@ -595,29 +601,31 @@ static inline jl_cgval_t ghostValue(jl_datatype_t *typ) static inline jl_cgval_t mark_julia_slot(Value *v, jl_value_t *typ) { // eagerly put this back onto the stack - jl_cgval_t tagval(v, typ); + assert(v->getType() != T_pjlvalue); + jl_cgval_t tagval(v, false, typ); tagval.ispointer = true; return tagval; } -static inline jl_cgval_t mark_julia_type(Value *v, jl_value_t *typ) +static inline jl_cgval_t mark_julia_type(Value *v, bool isboxed, jl_value_t *typ) { Type *T = julia_type_to_llvm(typ); if (type_is_ghost(T)) { return ghostValue(typ); } - if (v && T->isAggregateType() && v->getType() != jl_pvalue_llvmt) { + if (v && T->isAggregateType() && !isboxed) { + assert(v->getType() != T_pjlvalue); // eagerly put this back onto the stack // llvm mem2reg pass will remove this if unneeded Value *loc = emit_static_alloca(T); builder.CreateStore(v, loc); return mark_julia_slot(loc, typ); } - return jl_cgval_t(v, typ); + return jl_cgval_t(v, isboxed, typ); } -static inline jl_cgval_t mark_julia_type(Value *v, jl_datatype_t *typ) +static inline jl_cgval_t mark_julia_type(Value *v, bool isboxed, jl_datatype_t *typ) { - return mark_julia_type(v, (jl_value_t*)typ); + return mark_julia_type(v, isboxed, (jl_value_t*)typ); } static inline jl_cgval_t remark_julia_type(const jl_cgval_t &v, jl_value_t *typ) @@ -630,11 +638,15 @@ static inline jl_cgval_t remark_julia_type(const jl_cgval_t &v, jl_value_t *typ) } static inline jl_cgval_t mark_julia_const(jl_value_t *jv) { - jl_value_t *typ = jl_typeof(jv); + jl_value_t *typ; + if (jl_is_datatype(jv) || jl_is_uniontype(jv) || jl_is_typector(jv)) + typ = (jl_value_t*)jl_wrap_Type(jv); + else + typ = jl_typeof(jv); if (type_is_ghost(julia_type_to_llvm(typ))) { return ghostValue(typ); } - return jl_cgval_t(literal_pointer_val(jv), typ); + return jl_cgval_t(literal_pointer_val(jv), true, typ); } @@ -708,8 +720,8 @@ static void alloc_local(jl_sym_t *s, jl_codectx_t *ctx) jl_varinfo_t &vi = ctx->vars[s]; jl_value_t *jt = vi.value.typ; assert(store_unboxed_p(s,ctx)); - Type *vtype = julia_struct_to_llvm(jt); - assert(vtype != jl_pvalue_llvmt); + Type *vtype = julia_type_to_llvm(jt); + assert(vtype != T_pjlvalue); if (type_is_ghost(vtype)) { vi.value = ghostValue(jt); return; @@ -1834,7 +1846,7 @@ static jl_cgval_t emit_boxed_rooted(jl_value_t *e, jl_codectx_t *ctx) // TODO: m if (!v.isboxed) { Value *vbox = boxed(v, ctx); make_gcroot(vbox, ctx); - v = jl_cgval_t(vbox, v.typ); // bypass normal auto-unbox behavior for isghost + v = jl_cgval_t(vbox, true, v.typ); // XXX: bypasses the normal auto-unbox behavior for isghost! } else if (might_need_root(e)) { // TODO: v.needsgcroot make_gcroot(v.V, ctx); @@ -1959,7 +1971,7 @@ static jl_cgval_t emit_getfield(jl_value_t *expr, jl_sym_t *name, jl_codectx_t * if (bnd && bnd->value != NULL) { if (bnd->constp && jl_isbits(jl_typeof(bnd->value))) return emit_unboxed(bnd->value, ctx); - return mark_julia_type(builder.CreateLoad(bp), bnd->constp ? jl_typeof(bnd->value) : (jl_value_t*)jl_any_type); + return mark_julia_type(builder.CreateLoad(bp), true, bnd->constp ? jl_typeof(bnd->value) : (jl_value_t*)jl_any_type); } // todo: use type info to avoid undef check return emit_checked_var(bp, name, ctx); @@ -1998,7 +2010,7 @@ static jl_cgval_t emit_getfield(jl_value_t *expr, jl_sym_t *name, jl_codectx_t * ConstantInt::get(T_int32,2)); #endif ctx->gc.argDepth = argStart; - return mark_julia_type(result, jl_any_type); // (typ will be patched up by caller) + return mark_julia_type(result, true, jl_any_type); // (typ will be patched up by caller) } static Value *emit_bits_compare(const jl_cgval_t &arg1, const jl_cgval_t &arg2, jl_codectx_t *ctx) @@ -2023,7 +2035,7 @@ static Value *emit_bits_compare(const jl_cgval_t &arg1, const jl_cgval_t &arg2, Value *subAns, *fld1, *fld2; fld1 = builder.CreateExtractElement(varg1, ConstantInt::get(T_int32,i)), fld2 = builder.CreateExtractElement(varg2, ConstantInt::get(T_int32,i)), - subAns = emit_bits_compare(mark_julia_type(fld1, fldty), mark_julia_type(fld2, fldty), ctx); + subAns = emit_bits_compare(mark_julia_type(fld1, false, fldty), mark_julia_type(fld2, false, fldty), ctx); answer = builder.CreateAnd(answer, subAns); } return answer; @@ -2187,7 +2199,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, if (rt1) { rt2 = static_eval(args[2], ctx, true); if (rt2) { - *ret = mark_julia_type(ConstantInt::get(T_int1, jl_egal(rt1, rt2)), jl_bool_type); + *ret = mark_julia_type(ConstantInt::get(T_int1, jl_egal(rt1, rt2)), false, jl_bool_type); JL_GC_POP(); return true; } @@ -2206,7 +2218,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, // emit comparison test Value *ans = emit_f_is(v1, v2, ctx); ctx->gc.argDepth = last_depth; - *ret = mark_julia_type(ans, jl_bool_type); + *ret = mark_julia_type(ans, false, jl_bool_type); JL_GC_POP(); return true; } @@ -2214,7 +2226,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, else if (f->fptr == &jl_f_typeof && nargs==1) { jl_cgval_t arg1 = emit_expr(args[1], ctx); Value *lty = emit_typeof(arg1); - *ret = mark_julia_type(lty, jl_datatype_type); + *ret = mark_julia_type(lty, true, jl_datatype_type); JL_GC_POP(); return true; } @@ -2273,13 +2285,13 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, if (jl_is_type_type(ty) && !jl_has_typevars(jl_tparam0(ty))) { jl_value_t *tp0 = jl_tparam0(ty); if (jl_subtype(arg, tp0, 0)) { - *ret = mark_julia_type(ConstantInt::get(T_int1,1), jl_bool_type); + *ret = mark_julia_type(ConstantInt::get(T_int1, 1), false, jl_bool_type); JL_GC_POP(); return true; } if (!jl_subtype(tp0, (jl_value_t*)jl_type_type, 0)) { if (jl_is_leaf_type(arg)) { - *ret = mark_julia_type(ConstantInt::get(T_int1,0), jl_bool_type); + *ret = mark_julia_type(ConstantInt::get(T_int1, 0), false, jl_bool_type); JL_GC_POP(); return true; } @@ -2288,6 +2300,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, *ret = mark_julia_type( builder.CreateICmpEQ(emit_typeof(arg1), literal_pointer_val(tp0)), + false, jl_bool_type); JL_GC_POP(); return true; @@ -2303,7 +2316,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, jl_is_type_type(rt2) && !jl_is_typevar(jl_tparam0(rt2))) { int issub = jl_subtype(jl_tparam0(rt1), jl_tparam0(rt2), 0); // TODO: emit args[1] and args[2] in case of side effects? - *ret = mark_julia_type(ConstantInt::get(T_int1, issub), jl_bool_type); + *ret = mark_julia_type(ConstantInt::get(T_int1, issub), false, jl_bool_type); JL_GC_POP(); return true; } @@ -2334,7 +2347,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, ConstantInt::get(T_size, ctx->nReqArgs)), nva); #endif - *ret = mark_julia_type(r, expr_type(expr, ctx)); + *ret = mark_julia_type(r, true, expr_type(expr, ctx)); JL_GC_POP(); return true; } @@ -2369,7 +2382,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, if (jl_is_array_type(aty)) { // todo: also allow e.g. Union of several array types jl_cgval_t arg1 = emit_expr(args[1], ctx); - *ret = mark_julia_type(emit_arraylen(arg1, args[1], ctx), jl_long_type); + *ret = mark_julia_type(emit_arraylen(arg1, args[1], ctx), false, jl_long_type); JL_GC_POP(); return true; } @@ -2386,12 +2399,12 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, if (jl_is_long(args[2])) { uint32_t idx = (uint32_t)jl_unbox_long(args[2]); if (idx > 0 && idx <= ndims) { - *ret = mark_julia_type(emit_arraysize(ary, args[1], idx, ctx), jl_long_type); + *ret = mark_julia_type(emit_arraysize(ary, args[1], idx, ctx), false, jl_long_type); JL_GC_POP(); return true; } else if (idx > ndims) { - *ret = mark_julia_type(ConstantInt::get(T_size, 1), jl_long_type); + *ret = mark_julia_type(ConstantInt::get(T_size, 1), false, jl_long_type); JL_GC_POP(); return true; } @@ -2420,7 +2433,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, PHINode *result = builder.CreatePHI(T_size, 2); result->addIncoming(v_one, outBB); result->addIncoming(v_sz, inBB); - *ret = mark_julia_type(result, jl_long_type); + *ret = mark_julia_type(result, false, jl_long_type); JL_GC_POP(); return true; } @@ -2505,10 +2518,10 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, Value *own_ptr = builder.CreateLoad( builder.CreateBitCast(builder.CreateConstGEP1_32( builder.CreateBitCast(ary.V,T_pint8), jl_array_data_owner_offset(nd)), - jl_ppvalue_llvmt)); + T_ppjlvalue)); builder.CreateBr(mergeBB); builder.SetInsertPoint(mergeBB); - data_owner = builder.CreatePHI(jl_pvalue_llvmt, 2); + data_owner = builder.CreatePHI(T_pjlvalue, 2); data_owner->addIncoming(ary.V, curBB); data_owner->addIncoming(own_ptr, ownedBB); } @@ -2541,11 +2554,12 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, Value *idx = emit_unbox(T_size, emit_unboxed(args[2], ctx), fldt); idx = emit_bounds_check( - jl_cgval_t(builder.CreateGEP(ctx->argArray, ConstantInt::get(T_size, ctx->nReqArgs)), NULL), + jl_cgval_t(builder.CreateGEP(ctx->argArray, ConstantInt::get(T_size, ctx->nReqArgs)), false, NULL), NULL, idx, valen, ctx); idx = builder.CreateAdd(idx, ConstantInt::get(T_size, ctx->nReqArgs)); *ret = mark_julia_type( - tbaa_decorate(tbaa_user,builder.CreateLoad(builder.CreateGEP(ctx->argArray,idx))), + tbaa_decorate(tbaa_user, builder.CreateLoad(builder.CreateGEP(ctx->argArray, idx))), + true, expr_type(expr, ctx)); JL_GC_POP(); return true; @@ -2606,7 +2620,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, else if (f->fptr == &jl_f_nfields && nargs==1) { if (ctx->vaStack && symbol_eq(args[1], ctx->vaName) && !ctx->vars[ctx->vaName].isAssigned) { - *ret = mark_julia_type(emit_n_varargs(ctx), jl_long_type); + *ret = mark_julia_type(emit_n_varargs(ctx), false, jl_long_type); JL_GC_POP(); return true; } @@ -2616,7 +2630,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, if (jl_is_leaf_type(tp0)) { emit_expr(args[1], ctx); assert(jl_is_datatype(tp0)); - *ret = mark_julia_type(ConstantInt::get(T_size, jl_datatype_nfields(tp0)), jl_long_type); + *ret = mark_julia_type(ConstantInt::get(T_size, jl_datatype_nfields(tp0)), false, jl_long_type); JL_GC_POP(); return true; } @@ -2628,7 +2642,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, sz = emit_datatype_nfields(arg1); else sz = ConstantInt::get(T_size, jl_datatype_nfields(aty)); - *ret = mark_julia_type(sz, jl_long_type); + *ret = mark_julia_type(sz, false, jl_long_type); JL_GC_POP(); return true; } @@ -2645,8 +2659,8 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, Value *types_len = emit_datatype_nfields(ty); Value *idx = emit_unbox(T_size, emit_unboxed(args[2], ctx), (jl_value_t*)jl_long_type); emit_bounds_check(ty, (jl_value_t*)jl_datatype_type, idx, types_len, ctx); - Value *fieldtyp = builder.CreateLoad(builder.CreateGEP(builder.CreateBitCast(types_svec, jl_ppvalue_llvmt), idx)); - *ret = mark_julia_type(fieldtyp, expr_type(expr, ctx)); + Value *fieldtyp = builder.CreateLoad(builder.CreateGEP(builder.CreateBitCast(types_svec, T_ppjlvalue), idx)); + *ret = mark_julia_type(fieldtyp, true, expr_type(expr, ctx)); JL_GC_POP(); return true; } @@ -2666,7 +2680,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, sty != jl_datatype_type) { if (jl_is_leaf_type((jl_value_t*)sty) || (sty->name->names == jl_emptysvec && sty->size > 0)) { - *ret = mark_julia_type(ConstantInt::get(T_size, sty->size), jl_long_type); + *ret = mark_julia_type(ConstantInt::get(T_size, sty->size), false, jl_long_type); JL_GC_POP(); return true; } @@ -2705,7 +2719,7 @@ static Value *emit_jlcall(Value *theFptr, Value *theF, int argStart, if (nargs > 0) myargs = emit_temp_slot(argStart, ctx); else - myargs = Constant::getNullValue(jl_ppvalue_llvmt); + myargs = Constant::getNullValue(T_ppjlvalue); #ifdef LLVM37 Value *result = builder.CreateCall(prepare_call(theFptr), {theF, myargs, ConstantInt::get(T_int32,nargs)}); @@ -2735,9 +2749,11 @@ static jl_cgval_t emit_call_function_object(jl_function_t *f, Value *theF, Value jl_value_t **args, size_t nargs, jl_codectx_t *ctx) { - if (f!=NULL && specialized && f->linfo!=NULL && f->linfo->specFunctionObject!=NULL) { + if (f!=NULL && specialized && f->linfo!=NULL && f->linfo->specFunctionObject != NULL) { // emit specialized call site jl_value_t *jlretty = jl_ast_rettype(f->linfo, f->linfo->ast); + bool retboxed; + (void)julia_type_to_llvm(jlretty, &retboxed); Function *cf = (Function*)f->linfo->specFunctionObject; FunctionType *cft = cf->getFunctionType(); size_t nfargs = cft->getNumParams(); @@ -2753,13 +2769,15 @@ static jl_cgval_t emit_call_function_object(jl_function_t *f, Value *theF, Value for(size_t i=0; i < nargs; i++) { Type *at = cft->getParamType(idx); jl_value_t *jt = jl_nth_slot_type(f->linfo->specTypes,i); - Type *et = julia_type_to_llvm(jt); + bool isboxed; + Type *et = julia_type_to_llvm(jt, &isboxed); if (type_is_ghost(et)) { // Still emit the expression in case it has side effects emit_expr(args[i+1], ctx); continue; } - if (at == jl_pvalue_llvmt) { + if (isboxed) { + assert(at == T_pjlvalue && et == T_pjlvalue); jl_cgval_t origval = emit_expr(args[i+1], ctx); argvals[idx] = boxed(origval, ctx,expr_type(args[i+1],ctx)); assert(dyn_cast(argvals[idx]) == 0); @@ -2796,9 +2814,9 @@ static jl_cgval_t emit_call_function_object(jl_function_t *f, Value *theF, Value assert(idx == nfargs); CallInst *call = builder.CreateCall(prepare_call(cf), ArrayRef(&argvals[0], nfargs)); call->setAttributes(cf->getAttributes()); - return sret ? mark_julia_slot(result, jlretty) : mark_julia_type(call, jlretty); + return sret ? mark_julia_slot(result, jlretty) : mark_julia_type(call, retboxed, jlretty); } - return mark_julia_type(emit_jlcall(theFptr, theF, &args[1], nargs, ctx), jl_any_type); // (typ will be patched up by caller) + return mark_julia_type(emit_jlcall(theFptr, theF, &args[1], nargs, ctx), true, jl_any_type); // (typ will be patched up by caller) } static Value *emit_is_function(Value *x, jl_codectx_t *ctx) @@ -2915,7 +2933,7 @@ static jl_cgval_t emit_call(jl_value_t **args, size_t arglen, jl_codectx_t *ctx, if (nargs > 0) myargs = emit_temp_slot(argStart + 1, ctx); // argStart holds theFunc, argStart + 1 holds the start of the argument list else - myargs = Constant::getNullValue(jl_ppvalue_llvmt); // no arguments + myargs = Constant::getNullValue(T_ppjlvalue); // no arguments theFptr = emit_nthptr_recast(theFunc, (ssize_t)(offsetof(jl_function_t,fptr)/sizeof(void*)), tbaa_func, jl_pfptr_llvmt); #ifdef LLVM37 Value *r1 = builder.CreateCall(prepare_call(theFptr), {theFunc, myargs, @@ -2933,7 +2951,7 @@ static jl_cgval_t emit_call(jl_value_t **args, size_t arglen, jl_codectx_t *ctx, Value *r2; if (!jl_is_gf(call_func)) { just_emit_error("\"call\" is not a generic function", ctx); - r2 = UndefValue::get(jl_pvalue_llvmt); + r2 = UndefValue::get(T_pjlvalue); } else { #ifdef LLVM37 @@ -2951,10 +2969,10 @@ static jl_cgval_t emit_call(jl_value_t **args, size_t arglen, jl_codectx_t *ctx, builder.CreateBr(mergeBB1); ctx->f->getBasicBlockList().push_back(mergeBB1); builder.SetInsertPoint(mergeBB1); - PHINode *ph = builder.CreatePHI(jl_pvalue_llvmt, 2); + PHINode *ph = builder.CreatePHI(T_pjlvalue, 2); ph->addIncoming(r1, funcBB1); ph->addIncoming(r2, elseBB1); - result = mark_julia_type(ph, jl_any_type); + result = mark_julia_type(ph, true, jl_any_type); } if (result.typ == (jl_value_t*)jl_any_type) result = remark_julia_type(result, expr_type(expr, ctx)); // patch up typ if necessary @@ -3000,9 +3018,9 @@ static Value *global_binding_pointer(jl_module_t *m, jl_sym_t *s, b = jl_get_binding(m, s); if (b == NULL) { // var not found. switch to delayed lookup. - Constant *initnul = ConstantPointerNull::get((PointerType*)jl_pvalue_llvmt); + Constant *initnul = ConstantPointerNull::get((PointerType*)T_pjlvalue); GlobalVariable *bindinggv = - new GlobalVariable(*jl_Module, jl_pvalue_llvmt, + new GlobalVariable(*jl_Module, T_pjlvalue, false, GlobalVariable::PrivateLinkage, initnul, "delayedvar"); Value *cachedval = builder.CreateLoad(bindinggv); @@ -3025,10 +3043,10 @@ static Value *global_binding_pointer(jl_module_t *m, jl_sym_t *s, builder.CreateBr(have_val); ctx->f->getBasicBlockList().push_back(have_val); builder.SetInsertPoint(have_val); - PHINode *p = builder.CreatePHI(jl_pvalue_llvmt, 2); + PHINode *p = builder.CreatePHI(T_pjlvalue, 2); p->addIncoming(cachedval, currentbb); p->addIncoming(bval, not_found); - return julia_binding_gv(builder.CreateBitCast(p, jl_ppvalue_llvmt)); + return julia_binding_gv(builder.CreateBitCast(p, T_ppjlvalue)); } if (b->deprecated) cg_bdw(b, ctx); } @@ -3038,14 +3056,10 @@ static Value *global_binding_pointer(jl_module_t *m, jl_sym_t *s, static jl_cgval_t emit_checked_var(Value *bp, jl_sym_t *name, jl_codectx_t *ctx, bool isvol) { - assert(bp->getType() == jl_ppvalue_llvmt); + assert(bp->getType() == T_ppjlvalue); Value *v = builder.CreateLoad(bp, isvol); - // in unreachable code, there might be a poorly-typed instance of a variable - // that has a concrete type everywhere it's actually used. tolerate this - // situation by just skipping the NULL check if it wouldn't be valid. (issue #7836) - if (v->getType() == jl_pvalue_llvmt) - undef_var_error_if_null(v, name, ctx); - return mark_julia_type(v, jl_any_type); + undef_var_error_if_null(v, name, ctx); + return mark_julia_type(v, true, jl_any_type); } static jl_cgval_t emit_var(jl_sym_t *sym, jl_codectx_t *ctx, bool isboxed) @@ -3067,11 +3081,13 @@ static jl_cgval_t emit_var(jl_sym_t *sym, jl_codectx_t *ctx, bool isboxed) if (jbp->constp) { if (!isboxed && jl_isbits(jl_typeof(jbp->value))) return emit_unboxed(jbp->value, ctx); + else + return mark_julia_const(jbp->value); } // double-check that a global variable is actually defined. this // can be a problem in parallel when a definition is missing on // one machine. - return mark_julia_type(builder.CreateLoad(bp), jbp->constp ? jl_typeof(jbp->value) : (jl_value_t*)jl_any_type); + return mark_julia_type(builder.CreateLoad(bp), true, jl_any_type); } return emit_checked_var(bp, sym, ctx); } @@ -3085,7 +3101,7 @@ static jl_cgval_t emit_var(jl_sym_t *sym, jl_codectx_t *ctx, bool isboxed) // if the jl_box_t in the closure env, it will be const in the function load = tbaa_decorate(tbaa_const, load); } - bp = builder.CreatePointerCast(load, jl_ppvalue_llvmt); + bp = builder.CreatePointerCast(load, T_ppjlvalue); } if (vi.isArgument || // arguments are always defined ((vi.closureidx == -1 || !vi.isAssigned) && !vi.usedUndef)) { @@ -3095,7 +3111,7 @@ static jl_cgval_t emit_var(jl_sym_t *sym, jl_codectx_t *ctx, bool isboxed) // if it's in the closure env, but only assigned by the parent function, it will be const while in the child function v = tbaa_decorate(tbaa_const, v); } - return mark_julia_type(v, vi.value.typ); + return mark_julia_type(v, true, vi.value.typ); } else { jl_cgval_t v = emit_checked_var(bp, sym, ctx, vi.isVolatile); @@ -3110,7 +3126,7 @@ static jl_cgval_t emit_var(jl_sym_t *sym, jl_codectx_t *ctx, bool isboxed) Value *v = vi.value.V; if (v->getType() != T) v = builder.CreatePointerCast(v, T); - return mark_julia_type(builder.CreateLoad(v, true), vi.value.typ); + return mark_julia_type(builder.CreateLoad(v, true), false, vi.value.typ); } else { return vi.value; @@ -3127,8 +3143,8 @@ static void emit_assignment(jl_value_t *l, jl_value_t *r, jl_codectx_t *ctx) jl_value_t *declType = (jl_is_array(gensym_types) ? jl_cellref(gensym_types, idx) : (jl_value_t*)jl_any_type); jl_cgval_t slot; // slot is a jl_value_t or jl_value_t* if (store_unboxed_p(declType)) { - Type *vtype = julia_struct_to_llvm(declType); - assert(vtype != jl_pvalue_llvmt); + Type *vtype = julia_type_to_llvm(declType); + assert(vtype != T_pjlvalue); if (type_is_ghost(vtype)) { slot = emit_expr(r, ctx); } @@ -3137,6 +3153,7 @@ static void emit_assignment(jl_value_t *l, jl_value_t *r, jl_codectx_t *ctx) if (slot.ispointer) { // emit a copy of boxed isbits values. TODO: elid this copy if unnecessary slot = mark_julia_type( emit_unbox(julia_type_to_llvm(declType), slot, declType), + false, declType); } } @@ -3148,7 +3165,7 @@ static void emit_assignment(jl_value_t *l, jl_value_t *r, jl_codectx_t *ctx) Value *bp = emit_local_slot(ctx->gc.argSpaceSize++, ctx); builder.CreateStore(rval, bp); } - slot = mark_julia_type(rval, declType); + slot = mark_julia_type(rval, true, declType); } ctx->gensym_SAvalues.at(idx) = slot; // now gensym_SAvalues[idx] contains the SAvalue assert(ctx->gensym_assigned.at(idx) = true); // (assignment, not comparison test) @@ -3219,7 +3236,7 @@ static void emit_assignment(jl_value_t *l, jl_value_t *r, jl_codectx_t *ctx) return; } if (vi.isBox) { - bp = builder.CreatePointerCast(builder.CreateLoad(bp), jl_ppvalue_llvmt); + bp = builder.CreatePointerCast(builder.CreateLoad(bp), T_ppjlvalue); } builder.CreateStore(rval, bp, vi.isVolatile); if (vi.isBox) { @@ -3233,6 +3250,7 @@ static void emit_assignment(jl_value_t *l, jl_value_t *r, jl_codectx_t *ctx) if (store_unboxed_p(vi.value.typ) && rval_info.ispointer) { // emit a copy of boxed isbits values. TODO: elid this copy if unnecessary rval_info = mark_julia_type( emit_unbox(julia_type_to_llvm(vi.value.typ), rval_info, vi.value.typ), + false, vi.value.typ); } vi.value = rval_info; @@ -3343,11 +3361,10 @@ static jl_cgval_t emit_expr(jl_value_t *expr, jl_codectx_t *ctx, bool isboxed, b jl_binding_t *b = jl_get_binding(mod, var); if (b == NULL) b = jl_get_binding_wr(mod, var); - Value *bp = julia_binding_gv(b); - if (b->constp && b->value!=NULL) { - return mark_julia_type(builder.CreateLoad(bp), jl_typeof(b->value)); + if (b->constp && b->value != NULL) { + return mark_julia_const(b->value); } - return emit_checked_var(bp, var, ctx); + return emit_checked_var(julia_binding_gv(b), var, ctx); } if (jl_is_newvarnode(expr)) { assert(!valuepos); @@ -3361,14 +3378,14 @@ static jl_cgval_t emit_expr(jl_value_t *expr, jl_codectx_t *ctx, bool isboxed, b if (vi.isBox) { builder.CreateStore(builder.CreateCall(prepare_call(jlbox_func), V_null), lv); } - else if (lv->getType() == jl_ppvalue_llvmt && vi.usedUndef) { + else if (vi.usedUndef) { builder.CreateStore(V_null, lv); } } return jl_cgval_t(); } if (jl_is_lambda_info(expr)) { - return mark_julia_type(emit_lambda_closure(expr, ctx), jl_function_type); + return mark_julia_type(emit_lambda_closure(expr, ctx), true, jl_function_type); } if (!jl_is_expr(expr)) { int needroot = true; @@ -3476,8 +3493,8 @@ static jl_cgval_t emit_expr(jl_value_t *expr, jl_codectx_t *ctx, bool isboxed, b bp = vi.memloc; if (vi.isBox) { // bp is a jl_box_t* - bp = builder.CreatePointerCast(builder.CreateLoad(bp), jl_ppvalue_llvmt); - bp_owner = builder.CreateBitCast(bp, jl_pvalue_llvmt); + bp = builder.CreatePointerCast(builder.CreateLoad(bp), T_ppjlvalue); + bp_owner = builder.CreateBitCast(bp, T_pjlvalue); } } } @@ -3485,6 +3502,7 @@ static jl_cgval_t emit_expr(jl_value_t *expr, jl_codectx_t *ctx, bool isboxed, b Value *mdargs[4] = { name, bp, bp_owner, literal_pointer_val(bnd) }; return mark_julia_type( builder.CreateCall(prepare_call(jlgenericfunction_func), ArrayRef(&mdargs[0], 4)), + true, jl_function_type); } else { @@ -3499,6 +3517,7 @@ static jl_cgval_t emit_expr(jl_value_t *expr, jl_codectx_t *ctx, bool isboxed, b ctx->gc.argDepth = last_depth; return mark_julia_type( builder.CreateCall(prepare_call(jlmethod_func), ArrayRef(&mdargs[0], 9)), + true, jl_function_type); } } @@ -3539,10 +3558,10 @@ static jl_cgval_t emit_expr(jl_value_t *expr, jl_codectx_t *ctx, bool isboxed, b } Value *typ = boxed(emit_expr(args[0], ctx), ctx); Value *val = emit_jlcall(jlnew_func, typ, &args[1], nargs-1, ctx); - return mark_julia_type(val, ty); + return mark_julia_type(val, true, ty); } else if (head == exc_sym) { // *jl_exception_in_transit - return mark_julia_type(builder.CreateLoad(prepare_global(jlexc_var), /*isvolatile*/true), jl_any_type); + return mark_julia_type(builder.CreateLoad(prepare_global(jlexc_var), /*isvolatile*/true), true, jl_any_type); } else if (head == leave_sym) { assert(jl_is_long(args[0])); @@ -3620,7 +3639,7 @@ static jl_cgval_t emit_expr(jl_value_t *expr, jl_codectx_t *ctx, bool isboxed, b } } jl_cgval_t ast = emit_expr(arg, ctx); - return mark_julia_type(builder.CreateCall(prepare_call(jlcopyast_func), boxed(ast, ctx)), ast.typ); + return mark_julia_type(builder.CreateCall(prepare_call(jlcopyast_func), boxed(ast, ctx)), true, ast.typ); } else if (head == simdloop_sym) { if (!llvm::annotateSimdLoop(builder.GetInsertBlock())) @@ -3673,7 +3692,7 @@ static void allocate_gc_frame(size_t n_roots, BasicBlock *b0, jl_codectx_t *ctx) gc->argDepth = 0; gc->maxDepth = 0; - gc->gcframe = builder.CreateAlloca(jl_pvalue_llvmt, ConstantInt::get(T_int32, 0)); + gc->gcframe = builder.CreateAlloca(T_pjlvalue, ConstantInt::get(T_int32, 0)); #ifdef JL_DEBUG_BUILD gc->gcframe->setName("gcrootframe"); #endif @@ -3719,7 +3738,7 @@ emit_gcpops(jl_codectx_t *ctx) Instruction *gcpop = (Instruction*)builder.CreateConstGEP1_32(ctx->gc.gcframe, 1); builder.CreateStore(builder.CreatePointerCast(builder.CreateLoad(gcpop), - jl_ppvalue_llvmt), + T_ppjlvalue), prepare_global(jlpgcstack_var)); } } @@ -3743,7 +3762,7 @@ static void finalize_gc_frame(jl_codectx_t *ctx) builder.CreateStore(ConstantInt::get(T_size, (gc->argSpaceSize + gc->maxDepth) << 1), builder.CreateBitCast(builder.CreateConstGEP1_32(newgcframe, 0), T_psize)); builder.CreateStore(builder.CreateLoad(prepare_global(jlpgcstack_var)), - builder.CreatePointerCast(builder.CreateConstGEP1_32(newgcframe, 1), PointerType::get(jl_ppvalue_llvmt,0))); + builder.CreatePointerCast(builder.CreateConstGEP1_32(newgcframe, 1), PointerType::get(T_ppjlvalue,0))); builder.CreateStore(newgcframe, prepare_global(jlpgcstack_var)); // Initialize the slots for temporary variables to NULL for (int i = 0; i < gc->argSpaceSize; i++) { @@ -3770,7 +3789,8 @@ static Function *gen_cfun_wrapper(jl_function_t *ff, jl_value_t *jlrettype, jl_t } } // Generate a c-callable wrapper - Type *crt = ((isref&1) ? jl_pvalue_llvmt : julia_struct_to_llvm(jlrettype)); + bool toboxed; + Type *crt = ((isref & 1) ? T_pjlvalue : julia_struct_to_llvm(jlrettype, &toboxed)); if (crt == NULL) jl_error("cfunction: return type doesn't correspond to a C type"); size_t i; @@ -3783,6 +3803,7 @@ static Function *gen_cfun_wrapper(jl_function_t *ff, jl_value_t *jlrettype, jl_t } std::vector fargt(0); + std::vector fargt_isboxed(0); std::vector fargt_sig(0); Type* fargt_vasig; std::vector inRegList(0); @@ -3790,7 +3811,7 @@ static Function *gen_cfun_wrapper(jl_function_t *ff, jl_value_t *jlrettype, jl_t attr_type attrs; Type *prt = NULL; int sret = 0; - std::string err_msg = generate_func_sig(&crt, &prt, sret, fargt, fargt_sig, fargt_vasig, inRegList, byRefList, attrs, + std::string err_msg = generate_func_sig(&crt, &prt, sret, fargt, fargt_isboxed, fargt_sig, fargt_vasig, inRegList, byRefList, attrs, ((isref&1) ? (jl_value_t*)jl_any_type : jlrettype), argt->parameters, nargs); if (!err_msg.empty()) jl_error(err_msg.c_str()); @@ -3889,96 +3910,102 @@ static Function *gen_cfun_wrapper(jl_function_t *ff, jl_value_t *jlrettype, jl_t for (size_t i = 0; i < nargs; i++) { Value *val = AI++; jl_value_t *jargty = jl_nth_slot_type(lam->specTypes, i); + bool isboxed, argboxed; + Type *t = julia_type_to_llvm(jargty, &isboxed); + (void)julia_struct_to_llvm(jargty, &argboxed); + jl_cgval_t inputarg; // figure out how to unpack this type if (isref & (2<getPointerTo()); val = builder.CreateAlignedLoad(val, 1); // make no alignment assumption about pointer from C + inputarg = mark_julia_type(val, false, jargty); } } } + else if (argboxed) { + inputarg = mark_julia_type(val, true, jargty); + } else { // undo whatever we might have done to this poor argument bool issigned = jl_signed_type && jl_subtype(jargty, (jl_value_t*)jl_signed_type, 0); val = llvm_type_rewrite(val, val->getType(), fargt[i], true, byRefList[i], issigned, &ctx); + if (isboxed) { + Value *mem = emit_allocobj(jl_datatype_size(jargty)); + builder.CreateStore(literal_pointer_val((jl_value_t*)jargty), + emit_typeptr_addr(mem)); + builder.CreateAlignedStore(val, + builder.CreateBitCast(mem, val->getType()->getPointerTo()), + 16); // julia's gc gives 16-byte aligned addresses + inputarg = mark_julia_type(mem, true, jargty); + } + else { + inputarg = mark_julia_type(val, false, jargty); + } } // figure out how to repack this type - Type *at = specsig ? theFptr->getFunctionType()->getParamType(FParamIndex++) : jl_pvalue_llvmt; - if (val->getType() != at) { - if (at == jl_pvalue_llvmt) { - assert(jl_is_leaf_type(jargty)); - if (jl_datatype_nfields(jargty) == 0) { - val = literal_pointer_val(jl_new_struct_uninit((jl_datatype_t*)jargty)); - } - else if (jl_isbits(jargty)) { - val = boxed(mark_julia_type(val, jargty), &ctx, jargty); - } - else { - Value *mem = emit_allocobj(jl_datatype_size(jargty)); - builder.CreateStore(literal_pointer_val((jl_value_t*)jargty), - emit_typeptr_addr(mem)); - builder.CreateAlignedStore(val, builder.CreateBitCast(mem, val->getType()->getPointerTo()), 16); // julia's gc gives 16-byte aligned addresses - val = mem; - } - if (specsig) - make_gcroot(val, &ctx); - } - else if (val->getType()->isAggregateType()) { - assert(at->isPointerTy() && at->getContainedType(0) == val->getType()); - // aggregate types are passed by pointer - Value *loc = emit_static_alloca(val->getType(), &ctx); - builder.CreateStore(val, loc); - val = loc; + if (!specsig) { + Value *arg = boxed(inputarg, &ctx); + make_gcroot(arg, &ctx); + } + else { + Value *arg; + Type *at = theFptr->getFunctionType()->getParamType(FParamIndex++); + if (isboxed) { + arg = boxed(inputarg, &ctx); + make_gcroot(arg, &ctx); } else { - val = emit_unbox(at, mark_julia_type(val, jargty), jargty); - assert(dyn_cast(val) == 0); + arg = emit_unbox(t, inputarg, jargty); + assert(!isa(arg)); + if (t->isAggregateType()) { + assert(at->isPointerTy() && at->getContainedType(0) == t); + // aggregate types are passed by pointer + Value *loc = emit_static_alloca(t, &ctx); + builder.CreateStore(arg, loc); + arg = loc; + } } - } - // add to argument list - if (specsig) - args.push_back(val); - else - make_gcroot(val, &ctx); + // add to argument list + args.push_back(arg); + } } // Create the call - Value *r; + jl_cgval_t retval; if (specsig) { + bool retboxed; CallInst *call = builder.CreateCall(prepare_call(theFptr), ArrayRef(args)); call->setAttributes(theFptr->getAttributes()); - if (jlfunc_sret) - r = builder.CreateLoad(result); - else - r = call; + (void)julia_type_to_llvm(jlrettype, &retboxed); + retval = mark_julia_type(jlfunc_sret ? (Value*)builder.CreateLoad(result) : (Value*)call, retboxed, jlrettype); } else { - r = emit_jlcall(theFptr, literal_pointer_val((jl_value_t*)ff), 0, nargs, &ctx); + Value *ret = emit_jlcall(theFptr, literal_pointer_val((jl_value_t*)ff), 0, nargs, &ctx); + retval = mark_julia_type(ret, true, jlrettype); } // Prepare the return value - if (isref&1) { + Value *r; + if (isref & 1) { assert(!sret); // return a jl_value_t* - if (r->getType() != jl_pvalue_llvmt) { - r = boxed(mark_julia_type(r, jlrettype), &ctx, jlrettype); - } + r = boxed(retval, &ctx); } else if (sret && jlfunc_sret) { // nothing to do @@ -3987,7 +4014,7 @@ static Function *gen_cfun_wrapper(jl_function_t *ff, jl_value_t *jlrettype, jl_t if (sret) prt = fargt_sig[0]->getContainedType(0); // sret is a PointerType bool issigned = jl_signed_type && jl_subtype(jlrettype, (jl_value_t*)jl_signed_type, 0); - Value *v = julia_to_native(crt, jlrettype, mark_julia_type(r, jlrettype), + Value *v = julia_to_native(crt, toboxed, jlrettype, retval, false, false, false, false, false, 0, &ctx, NULL); r = llvm_type_rewrite(v, crt, prt, false, false, issigned, &ctx); if (sret) @@ -4088,13 +4115,14 @@ static Function *gen_jlcall_wrapper(jl_lambda_info_t *lam, jl_expr_t *ast, Funct } for(size_t i=0; i < nargs; i++) { jl_value_t *ty = jl_nth_slot_type(lam->specTypes, i); - Type *lty = julia_type_to_llvm(ty); + bool isboxed; + Type *lty = julia_type_to_llvm(ty, &isboxed); if (lty != NULL && type_is_ghost(lty)) continue; Value *argPtr = builder.CreateGEP(argArray, ConstantInt::get(T_size, i)); Value *theArg = builder.CreateLoad(argPtr); - if (lty != NULL && lty != jl_pvalue_llvmt) { + if (lty != NULL && !isboxed) { theArg = builder.CreatePointerCast(theArg, PointerType::get(lty,0)); if (!lty->isAggregateType()) // keep "aggregate" type values in place as pointers theArg = builder.CreateLoad(theArg); @@ -4107,17 +4135,13 @@ static Function *gen_jlcall_wrapper(jl_lambda_info_t *lam, jl_expr_t *ast, Funct // wrappers can be reused for different functions of the same type. CallInst *call = builder.CreateCall(prepare_call(f), ArrayRef(&args[0], nfargs)); call->setAttributes(f->getAttributes()); - Value *r; - if (sret || call->getType() != jl_pvalue_llvmt) { - jl_value_t *ty = jl_ast_rettype(lam, (jl_value_t*)ast); - jl_cgval_t r_typed = sret ? mark_julia_slot(result, ty) : mark_julia_type(call, ty); - r = boxed(r_typed, &ctx); - } - else { - r = call; - } - builder.CreateRet(r); + jl_value_t *jlretty = jl_ast_rettype(lam, (jl_value_t*)ast); + bool retboxed; + (void)julia_type_to_llvm(jlretty, &retboxed); + if (sret) { assert(!retboxed); } + jl_cgval_t retval = sret ? mark_julia_slot(result, jlretty) : mark_julia_type(call, retboxed, jlretty); + builder.CreateRet(boxed(retval, &ctx)); finalize_gc_frame(&ctx); FPM->run(*w); @@ -4181,13 +4205,13 @@ static Function *emit_function(jl_lambda_info_t *lam) jl_sym_t *argname = jl_decl_var(arg); jl_varinfo_t &varinfo = ctx.vars[argname]; varinfo.isArgument = true; - jl_value_t *ty = lam->specTypes ? jl_nth_slot_type(lam->specTypes, i) : (jl_value_t*)jl_any_type; - varinfo.value = mark_julia_type((Value*)NULL, ty); + jl_value_t *ty = jl_nth_slot_type(lam->specTypes, i); + varinfo.value = mark_julia_type((Value*)NULL, false, ty); } if (va) { jl_varinfo_t &varinfo = ctx.vars[ctx.vaName]; varinfo.isArgument = true; - varinfo.value = mark_julia_type((Value*)NULL, jl_tuple_type); + varinfo.value = mark_julia_type((Value*)NULL, false, jl_tuple_type); } for(i=0; i < vinfoslen; i++) { @@ -4208,7 +4232,7 @@ static Function *emit_function(jl_lambda_info_t *lam) jl_value_t *typ = jl_cellref(vi,1); if (!jl_is_type(typ)) typ = (jl_value_t*)jl_any_type; - varinfo.value = mark_julia_type((Value*)NULL, typ); + varinfo.value = mark_julia_type((Value*)NULL, false, typ); } bool hasCapt = (captvinfoslen > 0); for(i=0; i < captvinfoslen; i++) { @@ -4228,7 +4252,7 @@ static Function *emit_function(jl_lambda_info_t *lam) jl_value_t *typ = jl_cellref(vi,1); if (!jl_is_type(typ)) typ = (jl_value_t*)jl_any_type; - varinfo.value = mark_julia_type((Value*)NULL, typ); + varinfo.value = mark_julia_type((Value*)NULL, false, typ); varinfo.hasGCRoot = true; } @@ -4282,8 +4306,16 @@ static Function *emit_function(jl_lambda_info_t *lam) ctx.sret = false; if (specsig) { // assumes !va std::vector fsig(0); - Type *rt = (jlrettype == (jl_value_t*)jl_void_type ? T_void : julia_type_to_llvm(jlrettype)); - if (rt != jl_pvalue_llvmt && rt != T_void && deserves_sret(jlrettype, rt)) { + Type *rt; + bool retboxed; + if (jlrettype == (jl_value_t*)jl_void_type) { + rt = T_void; + retboxed = false; + } + else { + rt = julia_type_to_llvm(jlrettype, &retboxed); + } + if (!retboxed && rt != T_void && deserves_sret(jlrettype, rt)) { ctx.sret = true; fsig.push_back(rt->getPointerTo()); rt = T_void; @@ -4677,7 +4709,7 @@ static Function *emit_function(jl_lambda_info_t *lam) if (!varinfo.value.isghost) { varinfo.memloc = builder.CreatePointerCast( emit_nthptr_addr(envArg, i + offsetof(jl_svec_t,data) / sizeof(void*)), - jl_ppvalue_llvmt); + T_ppjlvalue); } } } @@ -4778,17 +4810,18 @@ static Function *emit_function(jl_lambda_info_t *lam) if (!vi.value.isghost) { if (specsig) { jl_value_t *argType = jl_nth_slot_type(lam->specTypes, i); - Type *llvmArgType = julia_type_to_llvm(argType); + bool isboxed; + Type *llvmArgType = julia_type_to_llvm(argType, &isboxed); if (type_is_ghost(llvmArgType)) // this argument is not actually passed theArg = ghostValue(argType); else if (llvmArgType->isAggregateType()) theArg = mark_julia_slot(AI++, argType); // this argument is by-pointer else - theArg = mark_julia_type(AI++, argType); + theArg = mark_julia_type(AI++, isboxed, argType); } else { Value *argPtr = builder.CreateGEP(argArray, ConstantInt::get(T_size, i)); - theArg = mark_julia_type(builder.CreateLoad(argPtr), vi.value.typ); + theArg = mark_julia_type(builder.CreateLoad(argPtr), true, vi.value.typ); } Value *lv = vi.memloc; @@ -4998,9 +5031,17 @@ static Function *emit_function(jl_lambda_info_t *lam) if (jl_is_expr(stmt) && ((jl_expr_t*)stmt)->head == return_sym) { jl_expr_t *ex = (jl_expr_t*)stmt; Value *retval; - Type *retty = ctx.sret ? f->getFunctionType()->getParamType(0)->getContainedType(0) : f->getReturnType(); - if (retty == jl_pvalue_llvmt) { - retval = boxed(emit_expr(jl_exprarg(ex,0), &ctx, true),&ctx,expr_type(stmt,&ctx)); + bool retboxed; + Type *retty; + if (specsig) { + retty = julia_type_to_llvm(jlrettype, &retboxed); + } + else { + retty = T_pjlvalue; + retboxed = true; + } + if (retboxed) { + retval = boxed(emit_expr(jl_exprarg(ex,0), &ctx, true), &ctx, expr_type(stmt, &ctx)); } else if (!type_is_ghost(retty)) { retval = emit_unbox(retty, @@ -5075,7 +5116,7 @@ static MDNode *tbaa_make_child( const char *name, MDNode *parent, bool isConstan static GlobalVariable *global_to_llvm(const std::string &cname, void *addr, Module *m) { GlobalVariable *gv = - new GlobalVariable(*m, jl_pvalue_llvmt, true, + new GlobalVariable(*m, T_pjlvalue, true, GlobalVariable::ExternalLinkage, NULL, cname); add_named_global(gv, addr); return gv; @@ -5102,9 +5143,17 @@ extern "C" void jl_fptr_to_llvm(void *fptr, jl_lambda_info_t *lam, int specsig) if (specsig) { // assumes !va std::vector fsig(0); jl_value_t *jlrettype = jl_ast_rettype(lam, (jl_value_t*)lam->ast); - Type *rt = (jlrettype == (jl_value_t*)jl_void_type ? T_void : julia_type_to_llvm(jlrettype)); + bool retboxed; + Type *rt; + if (jlrettype == (jl_value_t*)jl_void_type) { + rt = T_void; + retboxed = false; + } + else { + rt = julia_type_to_llvm(jlrettype, &retboxed); + } bool sret = false; - if (rt != jl_pvalue_llvmt && rt != T_void && deserves_sret(jlrettype, rt)) { + if (!retboxed && rt != T_void && deserves_sret(jlrettype, rt)) { sret = true; fsig.push_back(rt->getPointerTo()); rt = T_void; @@ -5174,6 +5223,8 @@ static void init_julia_llvm_env(Module *m) T_int1 = Type::getInt1Ty(getGlobalContext()); T_int8 = Type::getInt8Ty(getGlobalContext()); T_pint8 = PointerType::get(T_int8, 0); + T_ppint8 = PointerType::get(T_pint8, 0); + T_pppint8 = PointerType::get(T_ppint8, 0); T_int16 = Type::getInt16Ty(getGlobalContext()); T_pint16 = PointerType::get(T_int16, 0); T_int32 = Type::getInt32Ty(getGlobalContext()); @@ -5202,7 +5253,7 @@ static void init_julia_llvm_env(Module *m) Type *valueStructElts[1] = { PointerType::getUnqual(valueSt) }; ArrayRef vselts(valueStructElts); valueSt->setBody(vselts); - jl_value_llvmt = valueSt; + T_jlvalue = valueSt; DIBuilder dbuilder(*m); #ifdef LLVM37 @@ -5259,19 +5310,19 @@ static void init_julia_llvm_env(Module *m) dbuilder.getOrCreateArray(diargs)); #endif - jl_pvalue_llvmt = PointerType::get(jl_value_llvmt, 0); - jl_ppvalue_llvmt = PointerType::get(jl_pvalue_llvmt, 0); - two_pvalue_llvmt.push_back(jl_pvalue_llvmt); - two_pvalue_llvmt.push_back(jl_pvalue_llvmt); - three_pvalue_llvmt.push_back(jl_pvalue_llvmt); - three_pvalue_llvmt.push_back(jl_pvalue_llvmt); - three_pvalue_llvmt.push_back(jl_pvalue_llvmt); - V_null = Constant::getNullValue(jl_pvalue_llvmt); + T_pjlvalue = PointerType::get(T_jlvalue, 0); + T_ppjlvalue = PointerType::get(T_pjlvalue, 0); + two_pvalue_llvmt.push_back(T_pjlvalue); + two_pvalue_llvmt.push_back(T_pjlvalue); + three_pvalue_llvmt.push_back(T_pjlvalue); + three_pvalue_llvmt.push_back(T_pjlvalue); + three_pvalue_llvmt.push_back(T_pjlvalue); + V_null = Constant::getNullValue(T_pjlvalue); std::vector ftargs(0); - ftargs.push_back(jl_pvalue_llvmt); - ftargs.push_back(jl_ppvalue_llvmt); + ftargs.push_back(T_pjlvalue); + ftargs.push_back(T_ppjlvalue); ftargs.push_back(T_int32); - jl_func_sig = FunctionType::get(jl_pvalue_llvmt, ftargs, false); + jl_func_sig = FunctionType::get(T_pjlvalue, ftargs, false); assert(jl_func_sig != NULL); jl_pfptr_llvmt = PointerType::get(PointerType::get(jl_func_sig, 0), 0); @@ -5288,7 +5339,7 @@ static void init_julia_llvm_env(Module *m) jl_parray_llvmt = PointerType::get(jl_array_llvmt,0); jlpgcstack_var = - new GlobalVariable(*m, jl_ppvalue_llvmt, + new GlobalVariable(*m, T_ppjlvalue, false, GlobalVariable::ExternalLinkage, NULL, "jl_pgcstack"); add_named_global(jlpgcstack_var, (void*)&jl_pgcstack); @@ -5354,7 +5405,7 @@ static void init_julia_llvm_env(Module *m) add_named_global(jlerror_func, (void*)&jl_error); std::vector args1_(0); - args1_.push_back(jl_pvalue_llvmt); + args1_.push_back(T_pjlvalue); jlthrow_func = Function::Create(FunctionType::get(T_void, args1_, false), Function::ExternalLinkage, @@ -5370,7 +5421,7 @@ static void init_julia_llvm_env(Module *m) add_named_global(jlundefvarerror_func, (void*)&jl_undefined_var_error); std::vector args2_boundserrorv(0); - args2_boundserrorv.push_back(jl_pvalue_llvmt); + args2_boundserrorv.push_back(T_pjlvalue); args2_boundserrorv.push_back(T_psize); args2_boundserrorv.push_back(T_size); jlboundserrorv_func = @@ -5381,7 +5432,7 @@ static void init_julia_llvm_env(Module *m) add_named_global(jlboundserrorv_func, (void*)&jl_bounds_error_ints); std::vector args2_boundserror(0); - args2_boundserror.push_back(jl_pvalue_llvmt); + args2_boundserror.push_back(T_pjlvalue); args2_boundserror.push_back(T_size); jlboundserror_func = Function::Create(FunctionType::get(T_void, args2_boundserror, false), @@ -5391,7 +5442,7 @@ static void init_julia_llvm_env(Module *m) add_named_global(jlboundserror_func, (void*)&jl_bounds_error_int); std::vector args3_vboundserror(0); - args3_vboundserror.push_back(jl_ppvalue_llvmt); + args3_vboundserror.push_back(T_ppjlvalue); args3_vboundserror.push_back(T_size); args3_vboundserror.push_back(T_size); jlvboundserror_func = @@ -5403,7 +5454,7 @@ static void init_julia_llvm_env(Module *m) std::vector args3_uboundserror(0); args3_uboundserror.push_back(T_pint8); - args3_uboundserror.push_back(jl_pvalue_llvmt); + args3_uboundserror.push_back(T_pjlvalue); args3_uboundserror.push_back(T_size); jluboundserror_func = Function::Create(FunctionType::get(T_void, args3_uboundserror, false), @@ -5413,7 +5464,7 @@ static void init_julia_llvm_env(Module *m) add_named_global(jluboundserror_func, (void*)&jl_bounds_error_unboxed_int); std::vector args2_throw(0); - args2_throw.push_back(jl_pvalue_llvmt); + args2_throw.push_back(T_pjlvalue); args2_throw.push_back(T_int32); jlthrow_line_func = (Function*)m->getOrInsertFunction("jl_throw_with_superfluous_argument", @@ -5449,8 +5500,8 @@ static void init_julia_llvm_env(Module *m) std::vector te_args(0); te_args.push_back(T_pint8); te_args.push_back(T_pint8); - te_args.push_back(jl_pvalue_llvmt); - te_args.push_back(jl_pvalue_llvmt); + te_args.push_back(T_pjlvalue); + te_args.push_back(T_pjlvalue); te_args.push_back(T_int32); jltypeerror_func = Function::Create(FunctionType::get(T_void, te_args, false), @@ -5460,8 +5511,8 @@ static void init_julia_llvm_env(Module *m) add_named_global(jltypeerror_func, (void*)&jl_type_error_rt_line); std::vector args_2ptrs(0); - args_2ptrs.push_back(jl_pvalue_llvmt); - args_2ptrs.push_back(jl_pvalue_llvmt); + args_2ptrs.push_back(T_pjlvalue); + args_2ptrs.push_back(T_pjlvalue); jlcheckassign_func = Function::Create(FunctionType::get(T_void, args_2ptrs, false), Function::ExternalLinkage, @@ -5469,7 +5520,7 @@ static void init_julia_llvm_env(Module *m) add_named_global(jlcheckassign_func, (void*)&jl_checked_assignment); std::vector args_1ptr(0); - args_1ptr.push_back(jl_pvalue_llvmt); + args_1ptr.push_back(T_pjlvalue); jldeclareconst_func = Function::Create(FunctionType::get(T_void, args_1ptr, false), Function::ExternalLinkage, @@ -5477,7 +5528,7 @@ static void init_julia_llvm_env(Module *m) add_named_global(jldeclareconst_func, (void*)&jl_declare_constant); jlgetbindingorerror_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, args_2ptrs, false), + Function::Create(FunctionType::get(T_pjlvalue, args_2ptrs, false), Function::ExternalLinkage, "jl_get_binding_or_error", m); add_named_global(jlgetbindingorerror_func, (void*)&jl_get_binding_or_error); @@ -5518,8 +5569,8 @@ static void init_julia_llvm_env(Module *m) add_named_global(queuerootfun, (void*)&jl_gc_queue_root); std::vector wbargs(0); - wbargs.push_back(jl_pvalue_llvmt); - wbargs.push_back(jl_pvalue_llvmt); + wbargs.push_back(T_pjlvalue); + wbargs.push_back(T_pjlvalue); wbfunc = Function::Create(FunctionType::get(T_void, wbargs, false), Function::ExternalLinkage, "jl_gc_wb_slow", m); @@ -5530,31 +5581,31 @@ static void init_julia_llvm_env(Module *m) expect_func = Intrinsic::getDeclaration(m, Intrinsic::expect, exp_args); std::vector args3(0); - args3.push_back(jl_pvalue_llvmt); + args3.push_back(T_pjlvalue); jlbox_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, args3, false), + Function::Create(FunctionType::get(T_pjlvalue, args3, false), Function::ExternalLinkage, "jl_new_box", m); add_named_global(jlbox_func, (void*)&jl_new_box); jltopeval_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, args3, false), + Function::Create(FunctionType::get(T_pjlvalue, args3, false), Function::ExternalLinkage, "jl_toplevel_eval", m); add_named_global(jltopeval_func, (void*)&jl_toplevel_eval); jlcopyast_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, args3, false), + Function::Create(FunctionType::get(T_pjlvalue, args3, false), Function::ExternalLinkage, "jl_copy_ast", m); add_named_global(jlcopyast_func, (void*)&jl_copy_ast); std::vector args4(0); args4.push_back(T_pint8); - args4.push_back(jl_pvalue_llvmt); - args4.push_back(jl_pvalue_llvmt); + args4.push_back(T_pjlvalue); + args4.push_back(T_pjlvalue); jlclosure_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, args4, false), + Function::Create(FunctionType::get(T_pjlvalue, args4, false), Function::ExternalLinkage, "jl_new_closure", m); add_named_global(jlclosure_func, (void*)&jl_new_closure); @@ -5562,34 +5613,34 @@ static void init_julia_llvm_env(Module *m) std::vector args5(0); args5.push_back(T_size); jlnsvec_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, args5, true), + Function::Create(FunctionType::get(T_pjlvalue, args5, true), Function::ExternalLinkage, "jl_svec", m); add_named_global(jlnsvec_func, (void*)&jl_svec); std::vector mdargs(0); - mdargs.push_back(jl_pvalue_llvmt); - mdargs.push_back(jl_ppvalue_llvmt); - mdargs.push_back(jl_pvalue_llvmt); - mdargs.push_back(jl_pvalue_llvmt); - mdargs.push_back(jl_pvalue_llvmt); - mdargs.push_back(jl_pvalue_llvmt); - mdargs.push_back(jl_pvalue_llvmt); - mdargs.push_back(jl_pvalue_llvmt); + mdargs.push_back(T_pjlvalue); + mdargs.push_back(T_ppjlvalue); + mdargs.push_back(T_pjlvalue); + mdargs.push_back(T_pjlvalue); + mdargs.push_back(T_pjlvalue); + mdargs.push_back(T_pjlvalue); + mdargs.push_back(T_pjlvalue); + mdargs.push_back(T_pjlvalue); mdargs.push_back(T_int32); jlmethod_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, mdargs, false), + Function::Create(FunctionType::get(T_pjlvalue, mdargs, false), Function::ExternalLinkage, "jl_method_def", m); add_named_global(jlmethod_func, (void*)&jl_method_def); std::vector funcdefargs(0); - funcdefargs.push_back(jl_pvalue_llvmt); - funcdefargs.push_back(jl_ppvalue_llvmt); - funcdefargs.push_back(jl_pvalue_llvmt); - funcdefargs.push_back(jl_pvalue_llvmt); + funcdefargs.push_back(T_pjlvalue); + funcdefargs.push_back(T_ppjlvalue); + funcdefargs.push_back(T_pjlvalue); + funcdefargs.push_back(T_pjlvalue); jlgenericfunction_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, funcdefargs, false), + Function::Create(FunctionType::get(T_pjlvalue, funcdefargs, false), Function::ExternalLinkage, "jl_generic_function_def", m); add_named_global(jlgenericfunction_func, (void*)&jl_generic_function_def); @@ -5638,8 +5689,8 @@ static void init_julia_llvm_env(Module *m) add_named_global(jlleave_func, (void*)&jl_pop_handler); std::vector args_2vals(0); - args_2vals.push_back(jl_pvalue_llvmt); - args_2vals.push_back(jl_pvalue_llvmt); + args_2vals.push_back(T_pjlvalue); + args_2vals.push_back(T_pjlvalue); jlegal_func = Function::Create(FunctionType::get(T_int32, args_2vals, false), Function::ExternalLinkage, @@ -5647,8 +5698,8 @@ static void init_julia_llvm_env(Module *m) add_named_global(jlegal_func, (void*)&jl_egal); std::vector subt_args(0); - subt_args.push_back(jl_pvalue_llvmt); - subt_args.push_back(jl_pvalue_llvmt); + subt_args.push_back(T_pjlvalue); + subt_args.push_back(T_pjlvalue); subt_args.push_back(T_int32); jlsubtype_func = Function::Create(FunctionType::get(T_int32, subt_args, false), @@ -5659,26 +5710,26 @@ static void init_julia_llvm_env(Module *m) std::vector aoargs(0); aoargs.push_back(T_size); jlallocobj_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, aoargs, false), + Function::Create(FunctionType::get(T_pjlvalue, aoargs, false), Function::ExternalLinkage, "jl_gc_allocobj", m); add_named_global(jlallocobj_func, (void*)&jl_gc_allocobj); std::vector empty_args(0); jlalloc1w_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, empty_args, false), + Function::Create(FunctionType::get(T_pjlvalue, empty_args, false), Function::ExternalLinkage, "jl_gc_alloc_1w", m); add_named_global(jlalloc1w_func, (void*)&jl_gc_alloc_1w); jlalloc2w_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, empty_args, false), + Function::Create(FunctionType::get(T_pjlvalue, empty_args, false), Function::ExternalLinkage, "jl_gc_alloc_2w", m); add_named_global(jlalloc2w_func, (void*)&jl_gc_alloc_2w); jlalloc3w_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, empty_args, false), + Function::Create(FunctionType::get(T_pjlvalue, empty_args, false), Function::ExternalLinkage, "jl_gc_alloc_3w", m); add_named_global(jlalloc3w_func, (void*)&jl_gc_alloc_3w); @@ -5686,7 +5737,7 @@ static void init_julia_llvm_env(Module *m) std::vector atargs(0); atargs.push_back(T_size); jl_alloc_svec_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, atargs, false), + Function::Create(FunctionType::get(T_pjlvalue, atargs, false), Function::ExternalLinkage, "jl_alloc_svec", m); add_named_global(jl_alloc_svec_func, (void*)&jl_alloc_svec); @@ -5702,19 +5753,19 @@ static void init_julia_llvm_env(Module *m) add_named_global(jldlsym_func, (void*)&jl_load_and_lookup); std::vector newbits_args(0); - newbits_args.push_back(jl_pvalue_llvmt); + newbits_args.push_back(T_pjlvalue); newbits_args.push_back(T_pint8); jlnewbits_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, newbits_args, false), + Function::Create(FunctionType::get(T_pjlvalue, newbits_args, false), Function::ExternalLinkage, "jl_new_bits", m); add_named_global(jlnewbits_func, (void*)&jl_new_bits); std::vector getnthfld_args(0); - getnthfld_args.push_back(jl_pvalue_llvmt); + getnthfld_args.push_back(T_pjlvalue); getnthfld_args.push_back(T_size); jlgetnthfieldchecked_func = - Function::Create(FunctionType::get(jl_pvalue_llvmt, getnthfld_args, false), + Function::Create(FunctionType::get(T_pjlvalue, getnthfld_args, false), Function::ExternalLinkage, "jl_get_nth_field_checked", m); add_named_global(jlgetnthfieldchecked_func, (void*)*jl_get_nth_field_checked); @@ -6062,13 +6113,13 @@ extern "C" void jl_init_codegen(void) BOX_F(char,char); UBOX_F(gensym,size); - box8_func = boxfunc_llvm(ft2arg(jl_pvalue_llvmt, jl_pvalue_llvmt, T_int8), + box8_func = boxfunc_llvm(ft2arg(T_pjlvalue, T_pjlvalue, T_int8), "jl_box8", (void*)&jl_box8, m); - box16_func = boxfunc_llvm(ft2arg(jl_pvalue_llvmt, jl_pvalue_llvmt, T_int16), + box16_func = boxfunc_llvm(ft2arg(T_pjlvalue, T_pjlvalue, T_int16), "jl_box16", (void*)&jl_box16, m); - box32_func = boxfunc_llvm(ft2arg(jl_pvalue_llvmt, jl_pvalue_llvmt, T_int32), + box32_func = boxfunc_llvm(ft2arg(T_pjlvalue, T_pjlvalue, T_int32), "jl_box32", (void*)&jl_box32, m); - box64_func = boxfunc_llvm(ft2arg(jl_pvalue_llvmt, jl_pvalue_llvmt, T_int64), + box64_func = boxfunc_llvm(ft2arg(T_pjlvalue, T_pjlvalue, T_int64), "jl_box64", (void*)&jl_box64, m); } diff --git a/src/intrinsics.cpp b/src/intrinsics.cpp index d6231b77e5a76..552a1dd0cfdd2 100644 --- a/src/intrinsics.cpp +++ b/src/intrinsics.cpp @@ -231,7 +231,7 @@ static Constant *julia_const_to_llvm(jl_value_t *e, bool nested=false) } JL_GC_POP(); - Type *t = julia_struct_to_llvm(jt); + Type *t = julia_struct_to_llvm(jt, NULL); if (type_is_ghost(t)) return UndefValue::get(NoopType); if (t->isVectorTy()) @@ -254,7 +254,7 @@ static Constant *julia_const_to_llvm(jl_value_t *e, bool nested=false) static jl_cgval_t emit_unboxed(jl_value_t *e, jl_codectx_t *ctx) { Constant *c = julia_const_to_llvm(e); - if (c) return mark_julia_type(c, expr_type(e, ctx)); + if (c) return mark_julia_type(c, false, expr_type(e, ctx)); return emit_expr(e, ctx, false); } @@ -263,7 +263,7 @@ static jl_cgval_t ghostValue(jl_value_t *ty); // emit code to unpack a raw value from a box into registers static Value *emit_unbox(Type *to, const jl_cgval_t &x, jl_value_t *jt) { - assert(to != jl_pvalue_llvmt); + assert(to != T_pjlvalue); // TODO: fully validate that x.typ == jt? if (x.isghost) { if (type_is_ghost(to)) { @@ -325,8 +325,9 @@ static Value *auto_unbox(const jl_cgval_t &v, jl_codectx_t *ctx) emit_error("auto_unbox: unable to determine argument type", ctx); return UndefValue::get(T_void); } - Type *to = julia_type_to_llvm(v.typ); - if (to == NULL || to == jl_pvalue_llvmt) { + bool isboxed; + Type *to = julia_type_to_llvm(v.typ, &isboxed); + if (to == NULL || isboxed) { // might be some sort of incomplete (but valid) Ptr{T} type, for example unsigned int nb = jl_datatype_size(bt)*8; to = IntegerType::get(jl_LLVMContext, nb); @@ -372,8 +373,9 @@ static jl_value_t *staticeval_bitstype(jl_value_t *targ, const char *fname, jl_c static Type *staticeval_bitstype(jl_value_t *bt) { assert(jl_is_bitstype(bt)); - Type *to = julia_type_to_llvm(bt); - if (to == NULL || to == jl_pvalue_llvmt) { + bool isboxed; + Type *to = julia_type_to_llvm(bt, &isboxed); + if (to == NULL || isboxed) { unsigned int nb = jl_datatype_size(bt)*8; to = IntegerType::get(jl_LLVMContext, nb); } @@ -429,7 +431,7 @@ static jl_cgval_t generic_box(jl_value_t *targ, jl_value_t *x, jl_codectx_t *ctx builder.CreateAlignedStore(emit_unbox(llvmt, v, v.typ), builder.CreatePointerCast(newobj, llvmt->getPointerTo()), alignment); else builder.CreateMemCpy(newobj, builder.CreateBitCast(v.V, T_pint8), nb, alignment); - return mark_julia_type(newobj, bt ? bt : (jl_value_t*)jl_any_type); + return mark_julia_type(newobj, true, bt ? bt : (jl_value_t*)jl_any_type); } if (!jl_is_bitstype(bt)) { @@ -484,7 +486,7 @@ static jl_cgval_t generic_box(jl_value_t *targ, jl_value_t *x, jl_codectx_t *ctx vx = builder.CreateBitCast(vx, llvmt); } - return mark_julia_type(vx, bt); + return mark_julia_type(vx, false, bt); } // NOTE: signd (signed) only relevant if check == true @@ -502,7 +504,7 @@ static jl_cgval_t generic_trunc(jl_value_t *targ, jl_value_t *x, jl_codectx_t *c raise_exception_unless(builder.CreateICmpEQ(back, ix), prepare_global(jlinexacterr_var), ctx); } - return mark_julia_type(ans, jlto); + return mark_julia_type(ans, false, jlto); } static jl_cgval_t generic_sext(jl_value_t *targ, jl_value_t *x, jl_codectx_t *ctx) @@ -513,7 +515,7 @@ static jl_cgval_t generic_sext(jl_value_t *targ, jl_value_t *x, jl_codectx_t *ct Value *ix = JL_INT(auto_unbox(x, ctx)); if (ix->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error Value *ans = builder.CreateSExt(ix, to); - return mark_julia_type(ans, jlto); + return mark_julia_type(ans, false, jlto); } static jl_cgval_t generic_zext(jl_value_t *targ, jl_value_t *x, jl_codectx_t *ctx) @@ -524,7 +526,7 @@ static jl_cgval_t generic_zext(jl_value_t *targ, jl_value_t *x, jl_codectx_t *ct Value *ix = JL_INT(auto_unbox(x, ctx)); if (ix->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error Value *ans = builder.CreateZExt(ix, to); - return mark_julia_type(ans, jlto); + return mark_julia_type(ans, false, jlto); } static Value *emit_eqfsi(Value *x, Value *y) @@ -578,7 +580,7 @@ static jl_cgval_t emit_checked_fptosi(jl_value_t *targ, jl_value_t *x, jl_codect else { raise_exception_unless(emit_eqfsi(fx, ans), prepare_global(jlinexacterr_var), ctx); } - return mark_julia_type(ans, jlto); + return mark_julia_type(ans, false, jlto); } static jl_cgval_t emit_checked_fptoui(jl_value_t *targ, jl_value_t *x, jl_codectx_t *ctx) @@ -598,14 +600,14 @@ static jl_cgval_t emit_checked_fptoui(jl_value_t *targ, jl_value_t *x, jl_codect else { raise_exception_unless(emit_eqfui(fx, ans), prepare_global(jlinexacterr_var), ctx); } - return mark_julia_type(ans, jlto); + return mark_julia_type(ans, false, jlto); } static jl_cgval_t emit_runtime_pointerref(jl_value_t *e, jl_value_t *i, jl_codectx_t *ctx) { Value *preffunc = // TODO: move this to the codegen initialization code section jl_Module->getOrInsertFunction("jl_pointerref", - FunctionType::get(jl_pvalue_llvmt, two_pvalue_llvmt, false)); + FunctionType::get(T_pjlvalue, two_pvalue_llvmt, false)); int ldepth = ctx->gc.argDepth; jl_cgval_t parg = emit_boxed_rooted(e, ctx); Value *iarg = boxed(emit_expr(i, ctx), ctx); @@ -622,7 +624,7 @@ static jl_cgval_t emit_runtime_pointerref(jl_value_t *e, jl_value_t *i, jl_codec else { ety = (jl_value_t*)jl_any_type; } - return jl_cgval_t(ret, ety); + return mark_julia_type(ret, true, ety); } static jl_cgval_t emit_pointerref(jl_value_t *e, jl_value_t *i, jl_codectx_t *ctx) @@ -645,8 +647,9 @@ static jl_cgval_t emit_pointerref(jl_value_t *e, jl_value_t *i, jl_codectx_t *ct if (ety == (jl_value_t*)jl_any_type) return mark_julia_type( builder.CreateLoad(builder.CreateGEP( - builder.CreateBitCast(thePtr, jl_ppvalue_llvmt), + builder.CreateBitCast(thePtr, T_ppjlvalue), im1)), + true, ety); if (!jl_is_structtype(ety) || jl_is_array_type(ety) || !jl_is_leaf_type(ety)) { emit_error("pointerref: invalid pointer type", ctx); @@ -662,7 +665,7 @@ static jl_cgval_t emit_pointerref(jl_value_t *e, jl_value_t *i, jl_codectx_t *ct thePtr = builder.CreateGEP(builder.CreateBitCast(thePtr, T_pint8), im1); builder.CreateMemCpy(builder.CreateBitCast(strct, T_pint8), thePtr, size, 1); - return mark_julia_type(strct, ety); + return mark_julia_type(strct, true, ety); } // TODO: alignment? return typed_load(thePtr, im1, ety, ctx, tbaa_user, 1); @@ -736,7 +739,7 @@ static jl_cgval_t emit_pointerset(jl_value_t *e, jl_value_t *x, jl_value_t *i, j // TODO: alignment? typed_store(thePtr, im1, val, ety, ctx, tbaa_user, NULL, 1); } - return mark_julia_type(thePtr, aty); + return mark_julia_type(thePtr, false, aty); } static Value *emit_srem(Value *x, Value *den, jl_codectx_t *ctx) @@ -858,7 +861,7 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, int nb = get_bitstype_nbits(bt); Value *xi = JL_INT(auto_unbox(args[2],ctx)); if (xi->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error - return mark_julia_type(builder.CreateUIToFP(xi, FTnbits(nb)), bt); + return mark_julia_type(builder.CreateUIToFP(xi, FTnbits(nb)), false, bt); } HANDLE(sitofp,2) { @@ -867,7 +870,7 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, int nb = get_bitstype_nbits(bt); Value *xi = JL_INT(auto_unbox(args[2],ctx)); if (xi->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error - return mark_julia_type(builder.CreateSIToFP(xi, FTnbits(nb)), bt); + return mark_julia_type(builder.CreateSIToFP(xi, FTnbits(nb)), false, bt); } case fptoui: @@ -876,6 +879,7 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, if (x->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error return mark_julia_type( builder.CreateFPToUI(FP(x), JL_INTT(x->getType())), + false, JL_JLINTT(x->getType())); } else if (nargs == 2) { @@ -884,7 +888,7 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, int nb = get_bitstype_nbits(bt); Value *xf = FP(auto_unbox(args[2],ctx)); if (xf->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error - return mark_julia_type(builder.CreateFPToUI(xf, Type::getIntNTy(jl_LLVMContext, nb)), bt); + return mark_julia_type(builder.CreateFPToUI(xf, Type::getIntNTy(jl_LLVMContext, nb)), false, bt); } else { jl_error("fptoui: wrong number of arguments"); @@ -895,6 +899,7 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, Value *x = FP(auto_unbox(args[1], ctx)); return mark_julia_type( builder.CreateFPToSI(FP(x), JL_INTT(x->getType())), + false, JL_JLINTT(x->getType())); } else if (nargs == 2) { @@ -903,7 +908,7 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, int nb = get_bitstype_nbits(bt); Value *xf = FP(auto_unbox(args[2],ctx)); if (xf->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error - return mark_julia_type(builder.CreateFPToSI(xf, Type::getIntNTy(jl_LLVMContext, nb)), bt); + return mark_julia_type(builder.CreateFPToSI(xf, Type::getIntNTy(jl_LLVMContext, nb)), false, bt); } else { jl_error("fptosi: wrong number of arguments"); @@ -915,7 +920,7 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, int nb = get_bitstype_nbits(bt); Value *xf = FP(auto_unbox(args[2],ctx)); if (xf->getType() == T_void) return jl_cgval_t(); // auto_unbox threw an error - return mark_julia_type(builder.CreateFPTrunc(xf, FTnbits(nb)), bt); + return mark_julia_type(builder.CreateFPTrunc(xf, FTnbits(nb)), false, bt); } HANDLE(fpext,2) { @@ -934,18 +939,17 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, builder.CreateStore(FP(x), builder.CreateBitCast(prepare_global(jlfloattemp_var),FT(x->getType())->getPointerTo()), true); x = builder.CreateLoad(builder.CreateBitCast(prepare_global(jlfloattemp_var),FT(x->getType())->getPointerTo()), true); #endif - return mark_julia_type( - builder.CreateFPExt(x, FTnbits(nb)), - bt); + return mark_julia_type(builder.CreateFPExt(x, FTnbits(nb)), false, bt); } HANDLE(select_value,3) { Value *isfalse = emit_condition(args[1], "select_value", ctx); // emit the first argument jl_value_t *t1 = expr_type(args[2], ctx); jl_value_t *t2 = expr_type(args[3], ctx); - Type *llt1 = julia_type_to_llvm(t1); + bool isboxed; + Type *llt1 = julia_type_to_llvm(t1, &isboxed); Value *ifelse_result; - if (llt1 != jl_pvalue_llvmt && t1 == t2) { + if (!isboxed && t1 == t2) { // emit X and Y arguments jl_cgval_t x = emit_unboxed(args[2], ctx); jl_cgval_t y = emit_unboxed(args[3], ctx); @@ -965,18 +969,18 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, if (y.ispointer) // TODO: elid this load if unnecessary vy = builder.CreateLoad(builder.CreatePointerCast(vy, llt1->getPointerTo(0))); ifelse_result = builder.CreateSelect(isfalse, vy, vx); - return mark_julia_type(ifelse_result, t2); + return mark_julia_type(ifelse_result, false, t2); } else { int argStart = ctx->gc.argDepth; Value *arg1 = boxed(emit_expr(args[2],ctx,false), ctx, expr_type(args[2],ctx)); - // TODO: if (arg1->getType() != jl_pvalue_llvmt) + // TODO: if (!arg1.isboxed || arg1.needsgcroot) make_gcroot(arg1, ctx); Value *arg2 = boxed(emit_expr(args[3],ctx,false), ctx, expr_type(args[3],ctx)); ifelse_result = builder.CreateSelect(isfalse, arg2, arg1); ctx->gc.argDepth = argStart; jl_value_t *jt = (t1 == t2 ? t1 : (jl_value_t*)jl_any_type); - return mark_julia_type(ifelse_result, jt); + return mark_julia_type(ifelse_result, true, jt); } } @@ -1010,8 +1014,9 @@ static jl_cgval_t emit_intrinsic(intrinsic f, jl_value_t **args, size_t nargs, if (!newtyp && r->getType() != x->getType()) // cast back to the exact original type (e.g. float vs. int) before remarking as a julia type r = builder.CreateBitCast(r, x->getType()); - return mark_julia_type(r, newtyp ? newtyp : xinfo.typ); + return mark_julia_type(r, false, newtyp ? newtyp : xinfo.typ); } + } assert(0); } @@ -1438,7 +1443,7 @@ static FunctionType *ft2arg(Type *ret, Type *arg1, Type *arg2) } #define BOX_F(ct,jl_ct) \ - box_##ct##_func = boxfunc_llvm(ft1arg(jl_pvalue_llvmt, T_##jl_ct), \ + box_##ct##_func = boxfunc_llvm(ft1arg(T_pjlvalue, T_##jl_ct), \ "jl_box_"#ct, (void*)&jl_box_##ct, m); #define SBOX_F(ct,jl_ct) BOX_F(ct,jl_ct); box_##ct##_func->addAttribute(1, Attribute::SExt); From 8ad67575a913750aa2959e1fc8eed31523584cc2 Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Fri, 25 Sep 2015 17:36:02 -0400 Subject: [PATCH 3/6] fix missing assignment when passing a rooted, boxed, immutable value to another function (by-ref) and set !needsgcroot on constants and arguments --- src/codegen.cpp | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/codegen.cpp b/src/codegen.cpp index 12aa0358db5f5..7aacbb7803c73 100644 --- a/src/codegen.cpp +++ b/src/codegen.cpp @@ -399,7 +399,7 @@ struct jl_cgval_t { bool isimmutable; // V points to something that is definitely immutable (e.g. not stack allocated) //bool isstack; // points to stack-allocated memory //bool isarg; // derived from an argument - bool needsgcroot; // this value needs a gcroot + mutable bool needsgcroot; // this value needs a gcroot jl_cgval_t(Value *V, bool isboxed, jl_value_t *typ) : // general constructor (with pointer type auto-detect) V(V), // V is allowed to be NULL in a jl_varinfo_t context, but not during codegen contexts typ(typ), @@ -646,7 +646,9 @@ static inline jl_cgval_t mark_julia_const(jl_value_t *jv) if (type_is_ghost(julia_type_to_llvm(typ))) { return ghostValue(typ); } - return jl_cgval_t(literal_pointer_val(jv), true, typ); + jl_cgval_t constant(literal_pointer_val(jv), true, typ); + constant.needsgcroot = false; + return constant; } @@ -2795,7 +2797,7 @@ static jl_cgval_t emit_call_function_object(jl_function_t *f, Value *theF, Value // can lazy load on demand, no copy needed Value *argv = arg.V; if (argv->getType() != at) - builder.CreatePointerCast(argv, at); + argv = builder.CreatePointerCast(argv, at); argvals[idx] = argv; } else { @@ -4823,6 +4825,7 @@ static Function *emit_function(jl_lambda_info_t *lam) Value *argPtr = builder.CreateGEP(argArray, ConstantInt::get(T_size, i)); theArg = mark_julia_type(builder.CreateLoad(argPtr), true, vi.value.typ); } + theArg.needsgcroot = false; Value *lv = vi.memloc; if (lv == NULL) { From 60741878c1a59a24670c864cc6e152291f521812 Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Mon, 28 Sep 2015 19:24:09 -0400 Subject: [PATCH 4/6] correct isimmutable info and add needsgcroot annotations so that the runtime computation matches the result of is_stable_expr --- src/cgutils.cpp | 7 +++++-- src/codegen.cpp | 6 +++++- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/src/cgutils.cpp b/src/cgutils.cpp index 8deefc2cb9fef..7f0d2c5046bb3 100644 --- a/src/cgutils.cpp +++ b/src/cgutils.cpp @@ -1282,6 +1282,7 @@ static bool emit_getfield_unknownidx(jl_cgval_t *ret, const jl_cgval_t &strct, V if ((unsigned)stt->ninitialized != nfields) null_pointer_check(fld, ctx); *ret = mark_julia_type(fld, true, jl_any_type); + ret->needsgcroot = strct.needsgcroot || !strct.isimmutable; return true; } else if (is_tupletype_homogeneous(stt->types)) { @@ -1352,11 +1353,13 @@ static jl_cgval_t emit_getfield_knownidx(const jl_cgval_t &strct, unsigned idx, builder.CreateGEP(builder.CreateBitCast(strct.V, T_pint8), ConstantInt::get(T_size, jl_field_offset(jt,idx))); MDNode *tbaa = strct.isimmutable ? tbaa_immut : tbaa_user; - if (jl_field_isptr(jt,idx)) { + if (jl_field_isptr(jt, idx)) { Value *fldv = tbaa_decorate(tbaa, builder.CreateLoad(builder.CreateBitCast(addr, T_ppjlvalue))); if (idx >= (unsigned)jt->ninitialized) null_pointer_check(fldv, ctx); - return mark_julia_type(fldv, true, jfty); + jl_cgval_t ret = mark_julia_type(fldv, true, jfty); + ret.needsgcroot = strct.needsgcroot || !strct.isimmutable; + return ret; } else { int align = jl_field_offset(jt,idx); diff --git a/src/codegen.cpp b/src/codegen.cpp index 7aacbb7803c73..39561617a4263 100644 --- a/src/codegen.cpp +++ b/src/codegen.cpp @@ -731,6 +731,7 @@ static void alloc_local(jl_sym_t *s, jl_codectx_t *ctx) // CreateAlloca is OK here because alloc_local is only called during prologue setup Value *lv = builder.CreateAlloca(vtype, 0, s->name); vi.value = mark_julia_slot(lv, jt); + vi.value.isimmutable &= vi.isSA; // slot is not immutable if there are multiple assignments assert(vi.value.isboxed == false); #ifdef LLVM36 if (ctx->debug_enabled) { @@ -2012,7 +2013,9 @@ static jl_cgval_t emit_getfield(jl_value_t *expr, jl_sym_t *name, jl_codectx_t * ConstantInt::get(T_int32,2)); #endif ctx->gc.argDepth = argStart; - return mark_julia_type(result, true, jl_any_type); // (typ will be patched up by caller) + jl_cgval_t ret = mark_julia_type(result, true, jl_any_type); // (typ will be patched up by caller) + //ret.needsgcroot = arg1.needsgcroot || !arg1.isimmutable || !jl_is_leaf_type(arg1.typ) || !is_datatype_all_pointers((jl_datatype_t*)arg1.typ); + return ret; } static Value *emit_bits_compare(const jl_cgval_t &arg1, const jl_cgval_t &arg2, jl_codectx_t *ctx) @@ -2563,6 +2566,7 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, tbaa_decorate(tbaa_user, builder.CreateLoad(builder.CreateGEP(ctx->argArray, idx))), true, expr_type(expr, ctx)); + ret->needsgcroot = false; JL_GC_POP(); return true; } From 8b2ac7225792936f31012160d262a248005979e6 Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Fri, 2 Oct 2015 14:52:47 -0400 Subject: [PATCH 5/6] remove a usage of emit_boxed_rooted that could be incorrectly propagating a boxed ghost --- src/codegen.cpp | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/codegen.cpp b/src/codegen.cpp index 39561617a4263..365c6163f250f 100644 --- a/src/codegen.cpp +++ b/src/codegen.cpp @@ -2266,11 +2266,13 @@ static bool emit_known_call(jl_cgval_t *ret, jl_value_t *ff, FunctionType *ft = FunctionType::get(T_void, two_pvalue_llvmt, false); // TODO: move this to the codegen init section Value *typeassert = jl_Module->getOrInsertFunction("jl_typeassert", ft); int ldepth = ctx->gc.argDepth; - *ret = emit_boxed_rooted(args[1], ctx); + *ret = emit_expr(args[1], ctx); + Value *V = boxed(*ret, ctx); + make_gcroot(V, ctx); #ifdef LLVM37 - builder.CreateCall(prepare_call(typeassert), {ret->V, boxed(emit_expr(args[2], ctx),ctx)}); + builder.CreateCall(prepare_call(typeassert), {V, boxed(emit_expr(args[2], ctx),ctx)}); #else - builder.CreateCall2(prepare_call(typeassert), ret->V, boxed(emit_expr(args[2], ctx),ctx)); + builder.CreateCall2(prepare_call(typeassert), V, boxed(emit_expr(args[2], ctx),ctx)); #endif ctx->gc.argDepth = ldepth; JL_GC_POP(); From 39f818e29d9b3efab36e7ca5f5f46979e9e30571 Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Mon, 17 Aug 2015 15:40:04 -0400 Subject: [PATCH 6/6] more --compile=all related improvements --- base/essentials.jl | 2 +- src/gf.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/base/essentials.jl b/base/essentials.jl index 036a2b2bbdd0e..a02585ad1d6b8 100644 --- a/base/essentials.jl +++ b/base/essentials.jl @@ -111,7 +111,7 @@ cconvert{P<:Ptr}(::Type{P}, x) = x # but defer the conversion to Ptr to unsafe_c unsafe_convert{T}(::Type{T}, x::T) = x # unsafe_convert (like convert) defaults to assuming the convert occurred unsafe_convert{P<:Ptr}(::Type{P}, x::Ptr) = convert(P, x) -reinterpret{T,S}(::Type{T}, x::S) = box(T,unbox(S,x)) +reinterpret{T}(::Type{T}, x) = box(T, x) sizeof(x) = Core.sizeof(x) diff --git a/src/gf.c b/src/gf.c index 5b88689620f04..6b3421ed4a9ac 100644 --- a/src/gf.c +++ b/src/gf.c @@ -1617,6 +1617,7 @@ void jl_compile_all(void) htable_t h; htable_new(&h, 0); _compile_all(jl_main_module, &h); + htable_free(&h); } DLLEXPORT void jl_compile_hint(jl_function_t *f, jl_tupletype_t *types)