Make by-value struct passing work properly #7906

vtjnash · 2014-08-08T00:13:59Z

this is ~~a first cut at~~ (done) ~~rebasing #3466 onto master~~ (also done) ready-to-merge patch to improve ccall/cfunction compatibility. @Keno let me know if you see anything that looks wrong.

linux 32 (http://www.uclibc.org/docs/psABI-i386.pdf)
linux 64 (http://www.uclibc.org/docs/psABI-x86_64.pdf)
mac 64 (same as linux 64)
win 32 (https://msdn.microsoft.com/en-us/library/984x0h58.aspx)
win 64 (https://msdn.microsoft.com/en-us/library/zthk2dkh.aspx)

To keep track of issues (from comments) from the previous PR:

future work:

calling conventions for cfunction
types as C-structs #2818 (types as c-structs)
emit cfunction stubs for any jl_apply_generic call (possible sketch of implementation : https://gist.github.com/vtjnash/019b4d5d63b64a772a31)
more completely use tuples to emulate more stuff, like fixed size arrays
change cfunction(f, T, ...) to automatically call convert(T, ...)::T on the return value of f.
add convert(::Type{T}, x::Ref{T}) method
hide the typetag

~~NOTE: please DO NOT merge this until the cfunction API changes are finalized. (i don't want to need to update Gtk.jl twice)~~

Keno · 2014-08-08T00:44:18Z

Fixed size arrays can now be emulated by tuples. Anyway, I think we should merge this right after we tag 0.3. There is outstanding questions about the interface for cfunction, but since we intend 0.4 to be breaking anyway, we can figure that out later.

vtjnash · 2014-08-08T01:25:49Z

*mostly emulated by tuples – someone still should make that true for tuples inside of types. (hint, hint)

i meant to add the 0.4 tag, since i was intending to have this ready to merge this shortly into that...done now

julep proposal for the interface changes: https://gist.github.com/vtjnash/7296634

Keno · 2014-08-08T01:28:38Z

Yeah, we need to do that.

ihnorton · 2014-08-08T02:27:41Z

base/reflection.jl

@@ -182,3 +182,8 @@ function function_module(f::Function, types=(Any...))
    end
    m[1].func.code.module
 end
+
+#


a green X?

but yes, that's a good reminder that we need to scan through this and remove any debug code

vtjnash · 2014-08-08T03:21:49Z

base/reflection.jl

@@ -49,7 +49,7 @@ object_id(x::ANY) = ccall(:jl_object_id, Uint, (Any,), x)

 # type predicates
 const isimmutable = x->(isa(x,Tuple) || !typeof(x).mutable)
-isstructtype(t::DataType) = t.names!=() || (t.size==0 && !t.abstract)
+isstructtype(t::DataType) = (t.names!=() || t.size==0) && !t.abstract


@Keno which was correct? I suspect this wasn't supposed to change.

I'm not sure right now. I seem to recall this function also being implemented in C, so we should match whatever is done there.

julia/src/julia.h

Line 542 in 31eb8d4

STATIC_INLINE int jl_is_structtype(void *v)

seems to indicate this change was correct?

@JeffBezanson thoughts?

Yeah, that matches my recollection.

t.names!=() implies !t.abstract, so I believe the two versions are equivalent.

JeffBezanson · 2014-08-08T04:01:08Z

Nice!! +1

Keno · 2014-08-14T21:46:32Z

I'd like to go ahead and merge this very soon, so we have as much time as possible to encounter and fix any possible issues.

vtjnash · 2014-08-14T22:58:42Z

I assumed you would, which is why I wrote in big letters that more work needs to be done over the next few weeks to finish this (WIP tag added also)

Keno · 2014-08-14T23:01:02Z

Well, let's sit down via IRC this weekend and hammer out the cfunction changes then. That's what's been blocking this for about a year, so we should finally just do it.

vtjnash · 2014-08-14T23:03:53Z

i'm busy this weekend. either comment on the gist, propose your own, or wait for next weekend.

i don't think the API questions were blocking this, I just didn't want to steal your pull request to make the necessary additions, since I couldn't be sure if you were going to get back to it.

Keno · 2014-08-14T23:04:57Z

Well, the reason this wasn't merged a year ago is that the cfunction API wasn't done.

zenna · 2015-01-21T17:26:59Z

Any progress on this?

vtjnash · 2015-01-21T18:19:57Z

Lots. It's currently blocked by a number of other PRs that are needed first however.

johnmyleswhite · 2015-03-10T02:50:44Z

Awesome job improving the ccall docs, Jameson.

JeffBezanson · 2015-03-10T03:21:39Z

base/boot.jl

-    Bool, FloatingPoint, Float16, Float32, Float64, Number, Integer, Int, Int8, Int16,
-    Int32, Int64, Int128, Ref, Ptr, Real, Signed, UInt, UInt8, UInt16, UInt32,
-    UInt64, UInt128, Unsigned,
+    Real, Complex, Number, Integer, Bool, Ref, Ptr,


Complex shouldn't be here; it's not in this module.

i wonder if it should move here (or had at some points in the life of this PR), since it is now an influential type on ccall ABI generation

Not a big deal; the run time system already refers to some stuff in Base.

StefanKarpinski · 2015-03-10T03:52:39Z

This is epic. Especially love the wonderfully detailed documentation.

stevengj · 2015-03-10T16:04:37Z

doc/manual/calling-c-and-fortran-code.rst

-any expression, such as ``&0`` or ``&f(x)``.
+Notice that we have to be careful about the return type: ``qsort`` expects a function
+returning a C ``int``, so we must be sure to return ``Cint`` via a call to ``convert``
+and a ``typeassert``.


(I wish that cfunction would just call convert and add the typeassert for you.)

Can you edit the top post and add it as future work? It wouldn't actually be that hard to add anymore.

zenna · 2015-03-26T14:47:19Z

Typo?

The function cfunction generates the c-compatible function pointer for a call to a Julia Julia library function. Arguments to cfunction are as follows:

Addresses comments: #7906 (comment) #7906 (comment)

pao · 2015-03-31T17:16:43Z

@zenna fixed in 36ccc5b; thanks!

vtjnash added this to the 0.4 milestone Aug 8, 2014

ihnorton reviewed Aug 8, 2014
View reviewed changes

vtjnash changed the title ~~WIP: Make struct passing work properly #3466~~ Make struct passing work properly Aug 8, 2014

vtjnash reviewed Aug 8, 2014
View reviewed changes

JeffBezanson modified the milestones: 0.4, 0.4-projects Aug 10, 2014

vtjnash changed the title ~~Make struct passing work properly~~ WIP: Make struct passing work properly Aug 14, 2014

vtjnash mentioned this pull request Aug 21, 2014

Breaking compatibility with fortran-only BLAS interfaces JuliaLang/LinearAlgebra.jl#137

Closed

jiahao force-pushed the master branch 3 times, most recently from 6c7c7e3 to 1a4c02f Compare October 11, 2014 22:06

jiahao force-pushed the master branch from cdde4df to 7fdc860 Compare October 28, 2014 04:20

MikeInnes force-pushed the master branch from 5c60996 to b1c3df3 Compare November 14, 2014 17:07

vtjnash mentioned this pull request Jan 2, 2015

tryparse: parse string to Nullable #9487

Closed

vtjnash mentioned this pull request Jan 14, 2015

RFC: Make struct passing work properly #3466

Closed

tkelman mentioned this pull request Jan 19, 2015

Exception: EXCEPTION_ACCESS_VIOLATION at 0x7fff2bb07142 -- RegisterProcTableCallback - Windows #9817

Closed

vtjnash force-pushed the jn/ccall3 branch 3 times, most recently from b8db56e to 00756be Compare February 2, 2015 05:50

copy LDC license directly into abi implementation files

22490c1

vtjnash merged commit 22490c1 into master Mar 10, 2015

vtjnash deleted the jn/ccall3 branch March 10, 2015 02:36

JeffBezanson reviewed Mar 10, 2015
View reviewed changes

stevengj reviewed Mar 10, 2015
View reviewed changes

pao added a commit that referenced this pull request Mar 31, 2015

doc: Remove doubled "Julia" [av skip]

36ccc5b

Addresses comments: #7906 (comment) #7906 (comment)

tkelman mentioned this pull request Apr 19, 2015

ccall does not support complex return on 32 bit machines #2177

Closed

yuyichao mentioned this pull request Oct 14, 2015

Base.cconvert_gcroot is documented but does not exist. #13605

Closed

tkelman mentioned this pull request Oct 28, 2015

[release-0.3] backports for 0.3.12 #13718

Merged

ararslan mentioned this pull request Apr 24, 2017

Consistently apply code formatting to paths and file names #21500

Merged

simonbyrne mentioned this pull request Jul 13, 2020

EXCEPTION_ACCESS_VIOLATION on Windows JuliaIO/HDF5.jl#627

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make by-value struct passing work properly #7906

Make by-value struct passing work properly #7906

vtjnash commented Aug 8, 2014

Keno commented Aug 8, 2014

vtjnash commented Aug 8, 2014

Keno commented Aug 8, 2014

ihnorton Aug 8, 2014

vtjnash Aug 8, 2014

vtjnash Aug 8, 2014

Keno Aug 8, 2014

vtjnash Aug 8, 2014

vtjnash Aug 8, 2014

Keno Aug 8, 2014

JeffBezanson Aug 8, 2014

JeffBezanson commented Aug 8, 2014

Keno commented Aug 14, 2014

vtjnash commented Aug 14, 2014

Keno commented Aug 14, 2014

vtjnash commented Aug 14, 2014

Keno commented Aug 14, 2014

zenna commented Jan 21, 2015

vtjnash commented Jan 21, 2015

johnmyleswhite commented Mar 10, 2015

JeffBezanson Mar 10, 2015

vtjnash Mar 10, 2015

JeffBezanson Mar 10, 2015

StefanKarpinski commented Mar 10, 2015

stevengj Mar 10, 2015

vtjnash Mar 10, 2015

stevengj Mar 10, 2015

zenna commented Mar 26, 2015

pao commented Mar 31, 2015

Make by-value struct passing work properly #7906

Make by-value struct passing work properly #7906

Conversation

vtjnash commented Aug 8, 2014

Keno commented Aug 8, 2014

vtjnash commented Aug 8, 2014

Keno commented Aug 8, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JeffBezanson commented Aug 8, 2014

Keno commented Aug 14, 2014

vtjnash commented Aug 14, 2014

Keno commented Aug 14, 2014

vtjnash commented Aug 14, 2014

Keno commented Aug 14, 2014

zenna commented Jan 21, 2015

vtjnash commented Jan 21, 2015

johnmyleswhite commented Mar 10, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

StefanKarpinski commented Mar 10, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zenna commented Mar 26, 2015

pao commented Mar 31, 2015