WIP: generate types and methods using GObject introspection

[bfredl]

I've been experimenting a bit with auto-generating types and method wrapping using GObject-Introspection. This should fix many of the ccalls for us so that we can focus on create Julian wrappers. (is that the adjective for idiomatic julia?)

It is far from ready to merge yet (I have patched it in among the existing wrappers in quite an ugly way IMHO, and it currently leaks refs and strings like it's no tomorrow) but I'd like to discuss and settle on a consistent naming schemes. Should we have Gtk.Window or Gtk.GtkWindow, Clutter.Actor or ClutterActor? I tend to prefer the former as the primary names. Same questing with constants/Enums (not implemented yet) as Gobject-Introspection allows constants starting with numbers.

[lobingera]

Interesting work. I was about to propose using the GI some time ago, but the documentation left me wondering (even their java script example is not really readable).

For the naming, i'd also tend to Gtk.Window, Gtk.Frame.

[bfredl]

I was looking mostly into pygobject, but they "cheat" a bit and use g_function_info_invoke, which does the glib-side marshalling and pointer-handling. I think it's better to try generating native bindings directly; in principle all information necessary exist in the type database, but for instance the memory-management flags and bi-directional arguments seem a bit tricky.

[vtjnash]

I haven't gotten a chance to look through this, but on the subject of naming, I originally decided to export the 'official' Gtk names when you do using Gtk (e.g. Gtk.GtkWindow), but provide 'simpler' names when you do using Gtk.ShortNames (which are also available as Gtk.Name -- e.g. Gtk.Window)

Isn't GObject-Introspection an optional component? I initially ignored it because I expected that users would not have it installed (I don't have it).

[lobingera]

@vtjnash
afaics GI is/was introduced with Gtk3. The background is rather Gnome-ish.

[bfredl]

It is source-distributed inside gobject, but I guess many distributions split the binary packages. (I haven't thought about it, since gnome pulls in all libraries in the gobject family)

It makes sense for Gtk which is a short prefix, but code could be general for all GObject namespaces, which may have much longer names. The reason I ask is because

LongName(x) = x
Short = LongName
Short(x,y) = x

doesn't work, i e, each type must have a single name for adding constructors.

[vtjnash]

Not true. Just need to declare const Short = LongName -- see the ShortNames modules Gtk.jl for example implementation

[timholy]

What's the status of this on Windows/OSX? I would be concerned about introducing anything that jeopardizes cross-platform potential.

[vtjnash]

apt-get has this as gobject-introspection. MacPorts and Homebrew has this as gobject-introspection. I can't find libgirepository*.dll but I don't really know where to look / what to look for.

[bfredl]

Thanks, forgot to add const. Code generation is quite slow anyway, so one might ship pre-generated bindings like already done for properties using cindex.

[vtjnash]

Ah. In theory, cindex could be replaced by this, which should yield more precise methods.

[bfredl]

Probably we should split this out into it's own repo, to retain compability with gtk2 and systems without gintrospection. but then maybe the basic glib/gobject code should be split out too, so it could be shared. What do you think? (I just put everything inside Gtk.jl for now so I simply could reuse the already good gtypes/gobject bindings)

[vtjnash]

I would have no problem with that. In fact, it may solidify my idea that methods should be written against a parameterized abstract type, e.g. GObject{:GtkWindow} so that you could have libraries provide different implementations of GtkWindow, but still be able to inherit the same basic framework.

[bfredl]

Ok, but I think libgobject exposes enough runtime data alone (without needing libgintrospection) to generate the tree of Abstract types and also the handle wrapper types. So imho GObject.jl should generate these (if I'm correct), on first request by Gtk.jl / GI.jl / etc , and ensure that each gobject instance has exactly one julia wrapper object. I will look into it later. I will probably rename the auto-generated gtk_*_new constructors by GI.jl, so they don't conflict with those already present in Gtk.jl (there are a few clashes right now)

[vtjnash]

The documentation for this states that it is a GObject.

[vtjnash]

What is the performance & memory impact of computing the supertypes at runtime?

[bfredl]

just a quick check using tests.jl :
master:
@time include("Gtk.jl")
elapsed time: 5.832967949 seconds (74170972 bytes allocated)
@time include("../test/tests.jl")
elapsed time: 7.930535726 seconds (101171236 bytes allocated)
@time include("../test/tests.jl")
elapsed time: 1.145566246 seconds (2751804 bytes allocated)

supertypes at runtime:
@time include("Gtk.jl")
elapsed time: 6.323923368 seconds (82955720 bytes allocated
@time include("../test/tests.jl")
elapsed time: 7.836283733 seconds (100702332 bytes allocated)
@time include("../test/tests.jl")
elapsed time: 1.135036907 seconds (2645344 bytes allocated)

So module loading is slightly slower (this is consistent on repeated runs. It could be because I generate a few unnecessary wrappers right now).
Runtime performance/ memory overhead (also including warm-up method jit compilation) should be unaffected, as the generated types should be more or less identical. But probably a bit more testing is necessary to verify that :)

If you want I could rebase just the relevant changes (not including gi-introspection code) on master, that might make the changes easier to review and test.

[vtjnash]

That timing impact seems acceptable. Extracting just the changes that do not require GI would be good.

Note that Gtk does not guarantee that GTypes are stable identifiers. Since they are references to a hash function, they will mutate if the hash is updated. As such, they cannot be be stored or used as keys. The stable identifiers are either the name, or a call to a _get_type function.

I think the macro will need to take the following form, which specifies (a) the expected name (b) the library containing it's functions (c) the associated _get_type function:
@gtype GtkWidget libgtk gtk_widget

and should key the type dictionary using a symbol based on the name of the widget:
Dict{Symbol,Type{TypeVar(:T,GObject)}
(I'm not sure that a TypeVar is actually allowed there, but you can just use Type)

This form would also allow you to remove the call to gtk_test_register_all_types

[bfredl]

Ok, I thought GTypes were runtime stable, but then your suggestion seems the way to go. By the way, do you approve of splitting out the glib code into it's own namespace? Right now I do

module GLib
...
include("MutableTypes.jl")
using MutableTypes
include("gslist.jl")
include("gobject.jl") #gc code from gtktypes.jl
include("gvalues.jl")
include("gerror.jl")
include("signals.jl") #split out from events.jl
...
end

It much faster just [re]loading this than all of Gtk.jl.

[vtjnash]

yes

they are stable until the value of g_type_get_type_registration_serial changes

helpful note: Glib.bytestring is slightly different from Base.bytestring, in particular, you can quickly create a symbol from a pointer to a static string (such as returned by g_type_name by calling symbol(Glib.bytestring( <some pointer>, false)), where own=false is encoding gc information.

[bfredl]

There's still a bit of clean-up and testing to do before I'll do a new PR, but the new implementation of @gtype can be seen here:
bfredl@30a478b