Proposal: ObjC-style classes as a superset

[lerno]

This is a rather crazy suggestion, but I just wanted to make it so that it's all discussed. This is more of a 3.0 feature than 1.0, but I think it would be worthwhile to discuss, since it shapes how one thinks of structs. If they're the only tool, then it's necessary to add a lot of features, otherwise any dynamic feature might be caught by this proposal, if this is the way to go.

Objective-C in it's original version is essentially a pre-processor on top of C using the message passing paradigm of Smalltalk.

The only building blocks are:

1. Convert code of the format = Foo : Bar { unsigned int memoryAddress; } into a special type of struct. Add a registration of the class at startup.
2. Convert object methods of the format - name:part1:part2 { ... } into C functions with an implicit self pointer, and add registration of the function at startup
3. Convert class methods of the format + name:part1:part2 { ... } into C functions with implicit self pointer set to the class, and add registration of the function at startup.
4. Convert anything within [ ... ] into a send_msg call: eg [foo name:a part1:b part2:c] => send_msg(foo, "name:part1:part2", a, b, c)

Note that NeXT/Apple version added more bells and whistles to this, but the basic idea that Cox had was that this OO was large scale OO, and everything else was plain C. Unlike C++, which was an extension to C, this was more about making an in-language scripting language to C. A way to glue C code together easily. For C / C++, dynamic code is like pulling teeth, whereas for ObjC it's straightforward. What ObjC SHOULDN'T be, is for doing low level programming.

My suggestion is considering adding an OO message passing layer on top of Zig using a similar scheme as Brian Cox's original OO extension to C. For example:

class Foo : SuperObject
{
    x : i8;
    y : i8;
    fn bar(a: i8, b: i8) i8 { return (a - b) * self.x; }
}
[ ... ]
foo = Foo.new( ... ); // Class functions are known at compile time
// I introduce a wriggly pointer arrow here to make dynamic dispatch
// visually distinct. I avoid the [foo bar] syntax which is harder to 
// type in a C-like context.
i8 res = foo~>bar(4, 6); 
i8 res2 = foo~>bar(a: 5, b: 10);
foo~>retain();
foo~>release();

This would compile to something a bit similar to:

 const Foo = struct {
    [...] // object runtime information, like class pointer etc
    superob : i32, // Copied from SuperObject
    x : i8,
    y : i8,
 }  

 fn __Foo_bar(Foo *self, a: i8, b: i8)  { return (a - b) * self.x; }

 // Dynamically at runtime
 ... registerClass("Foo", ....)     
 ... register("bar", "Foo", __Foo_bar) ...

foo = __Foo_new( .... );
i8 res = send_msg(foo, "bar", 4, 6);
i8 res2 = send_msg_named(foo, "bar", "a", 5, "b", 10);
__Object_retain(foo); // Frequent functions can be compiled inline
__Object_release(foo); // Frequent functions can be compiled inline

Note that this is also the opposite of Swift's approach (unifying structs / dynamic classes)

P.S. Please overlook the poor examples here and try to consider it on a more general principle:
"How should the language handle cases where you want to add more abstraction?"

1. Say it's not a usecase for the language
2. Pile more functionality on structs (essentially the C++ route)
3. Use an embedded scripting language (lua etc)
4. Add a dynamic superset (like the proposal above)
5. Implement dynamic code manually on a case by case basis

[thejoshwolfe]

I don't understand the proposal or even the usecase. "add more abstraction" is pretty vague. How is message passing different from function calling?

I see OO inheritance in the examples. See #130 for the discussion about OOP in zig.

I see named parameters in the examples. See #479 for that proposal.

[lerno]

@thejoshwolfe c++/java basically use vtables for runtime dispatch methods where the exact method is not known at compile time. The Smalltalk/ObjC model the actual function to call cannot be uniquely determined during compile time and everything is resolved during runtime. Dynamic scripting languages like Ruby work like this as well (obviously).

I didn't want to drag the current crop of oo languages into the discussion, since they go all in on objects. The point of ObjC is that the OO part is only used as high-level glue, rather than an attempt to make everything OO. Message passing give the right idea: encapsulated domains inside which everything is low level, but can communicate with other domains using messages.

[0joshuaolson1]

So... function pointers minus stack issues publish/subscribe?

[thejoshwolfe]

vtables are discussed in #130.

encapsulated domains inside which everything is low level, but can communicate with other domains using messages.

What kinds of domains are you talking about? And how would message passing solve this problem differently from function calls?

[lerno]

A GUI library is an example where this really shines: the dynamic nature of message passing allows the generation of the UI to be partly/completely specified by a file without any recompilation.

Consider the problem of a button sending a callback to the logic when it is pushed. How would you solve this problem in C/C++?

With Java you can do some runtime reflection and ”sort of” get it to work.

With C/C++ either generate source code, use a scripting system or add reflection on top of everything with macros etc.

With ObjC it’s straightforward because you can say ”send a call to the owner of this object using the method with the name ’foo’” in a straightforward way. All object properties can be reflected during runtime etc.

This differs from C++/Java, where both languages use classes as very fine-grained components. C++ in particular even make them value object.

In ObjC they are more of a wrapper around C code, and doing ”objects all the way down” doesn’t really work well.

[0joshuaolson1]

Or dependency injection. Or a Qt-like build step.

[thejoshwolfe]

A GUI library is an example where this really shines

That was mentioned in #130. I don't understand what this discussion brings to the table that hasn't been covered in #130.

[lerno]

What would you mean is the difference between ”generating code” and a ”Qt-like build step”? Also, how would dep injection help in C/C++ unless you are doing some sort of pre-wiring? Which essentially would be some kind of reflection.

[lerno]

#130 is about inheritance / object hierachies in the context of a GUI. That is not what I’m talking about. I’m talking about the capability to hot-load/reload. It’s also about being able build a generic UI tool that does not know anything about the specific code but still is able to wire together and store the UI design, which then - during runtime - is assembled from a file.

That is not to say that this is not possible with, say C++, but you will end up with a lot of code just giving the UI components limited reflection capabilitied (typically: instantiating a class from name, accessing certain properties by name)

Again, it’s not to say that this is not possible with C/C++, it’s just so much more work since an ObjC style superset solves those issues without extra code.

[lerno]

For a smaller ObjC implementation than Apple’s, see Portable Object Compiler: http://users.telenet.be/stes/compiler.html
And also: https://mulle-objc.github.io/
Both have runtimes available to read through.

[thejoshwolfe]

see #68 for hot code swapping.

A WYSIWYG gui editor is certainly outside the scope of the compiler and probably outside the scope of the standard library. But we do want to make sure we don't make it impossible or unnecessarily hostile to implement such an editor. When we implement a GUI for #130, we can think about how that GUI could be created with an editor.

I don't see this as a language problem, but a userspace problem.

I still don't quite understand the proposal. What I've seen so far is:

• vtable abstraction
• hot code swapping
• wysiwyg code editor

These are either userspace problems or already have open discussions.

[isaachier]

@thejoshwolfe I had to read this section of the Wikipedia page on Smalltalk, but it clarified @lerno's points a lot: https://en.wikipedia.org/wiki/Smalltalk#Reflection.

Since the classes are themselves objects, they can be asked questions such as "what methods do you implement?" or "what fields/slots/instance variables do you define?". So objects can easily be inspected, copied, (de)serialized and so on with generic code that applies to any object in the system.
. . .
The Smalltalk compiler compiles textual source code into method objects, typically instances of CompiledMethod. These get added to classes by storing them in a class's method dictionary. The part of the class hierarchy that defines classes can add new classes to the system. The system is extended by running Smalltalk-80 code that creates or defines classes and methods. In this way a Smalltalk-80 system is a "living" system, carrying around the ability to extend itself at run time.
. . .
Smalltalk-80 also provides computational reflection, the ability to observe the computational state of the system. In languages derived from the original Smalltalk-80 the current activation of a method is accessible as an object named via a pseudo-variable (one of the six reserved words), thisContext. By sending messages to thisContext a method activation can ask questions like "who sent this message to me".

That said, I am not a fan of this in Zig, seeing as it almost guarantees a semi-interpreted or extremely slow language.

[lerno]

@thejoshwolfe No, the whole point is whether you make a system more ”objecty” like C->C with objects->C++ but still mostly static, or if you add a scripting-like optional dynamic, very loosely coupled OO system on top, or if one plans on doing dynamics through an embedded scripting language.

This is also the answer to @isaachier: like in ObjC the whole OO part is 100% optional and is used where you would use scripting or inject your own dynamic hooks. It’s the 5% glue, not the 95%. This is unlike Smalltalk where everything was an object. Again, the proposal is an ObjC solution, which is an optional smalltalk-like OO (not the Simula style OO of C++) as a superset of the normal language.

[thejoshwolfe]

I still don't understand the proposal. (Is this even a proposal, or just a discussion?) If the proposal is to add dynamic types to Zig, then that's probably not going to happen.

I'm not familiar with Smalltalk, but that wikipedia excerpt sounds to me like JavaScript's prototype mechanism. That's way out of scope for Zig to support as a first class language feature. Of course, you could accomplish that functionality in userspace by writing your own data structures, but dynamic typing goes against the Zen of Zig in many ways.

[lerno]

It seems very few here are familiar with Objective-C / Smalltalk.

Basically what is needed for an implementation is (as outlined before)

1. A syntax hook for defining ObjC-style objects, that are actually structs with a runtime specific start.
2. A syntax hook for invoking the "send_msg" function of the runtime
3. A runtime.

So basically think of this as the language allows a special kind of struct, that has a fixed initial part which points to a class definition. For this special kind of struct there is a special calling syntax which routes the call through a special runtime instead of executing a normal function call.

The runtime itself handles everything except the special struct parsing and converting the dynamic call syntax.

The point is that this makes the object system 100% optional and apart from the rest of the language. It is also extremely simple, and since the object is a struct, it's actually possible to treat is as one and inspect the memory and poke around into the internals.

This is extremely different from C++/Java where the OO is integrated into the language.

The reason why I'm lifting this is because unlike adding vtables etc (the C++ route), this requires a much more smaller change to the language and consequently if this OO solution is picked then:

1. Virtual dispatch and objecty structs can be rejected immediately
2. vtable abstraction of some kind. traits? oop? polymorphism? interfaces? #130 can be rejected
3. OO can be bolted on as an optional for a 2.0 of the language.

[lerno]

@isaachier I think you might misunderstand my motivation. Here I am just offering a different type of OO that would be a plug-in superset rather than something integrated in the language itself. The reason for raising this is simply that if this particular approach feels worthwhile (as opposed to the C++/Simula way) then one could skip the vtable approach entirely, and there would be no need to implement it in the core language.

This is not me suggesting that OO should be added to Zig. This is not me saying that an ObjC OO layer is the right fit for Zig. This is saying that there is an alternative approach to OO which should be investigated exactly because ObjC worked so flawlessly for adding dynamics to C. The classes in C++ add no dynamic to C, and is little more than a tool for abstraction. So once one makes the decision, the ObjC approach should have been evaluated as well.

Personally I see little value to adding C++ style OO to Zig, as the struct handling is mostly sufficient as is. An ObjC style OO would add dynamics, but Zig already covers a lot with the compile time code.

[isaachier]

Correct but there is no need to add either OO style IMO. The current Zig language doesn't use explicit vtables, but manages to implement dynamic dispatch using composition. I think this solution is complete as is.

[ghost]

dynamic dispatch using composition

how is that supposed to work?
composition does not imply dynamic dispatch capabilities at first, can you elaborate please?

[lerno]

@isaachier Can you elaborate on how you'd implement late bound dynamic dispatch in Zig?

[isaachier]

Sure. Essentially, you add the base class to the subclass as a field. This field serves as the type's vtable. It contains functions that must be set in the subclass initialization. Furthermore, these methods use a pointer to the base type as the self argument. Often, the subclass will implement those functions by taking the self argument and getting the subclass instance using the @fieldParentPtr built-in. See how FileStream implements InStream:

/// Implementation of InStream trait for File
pub const FileInStream = struct {
    file: *File,
    stream: Stream,

    pub const Error = @typeOf(File.read).ReturnType.ErrorSet;
    pub const Stream = InStream(Error);

    pub fn init(file: *File) FileInStream {
        return FileInStream{
            .file = file,
            .stream = Stream{ .readFn = readFn },
        };
    }

    fn readFn(in_stream: *Stream, buffer: []u8) Error!usize {
        const self = @fieldParentPtr(FileInStream, "stream", in_stream);
        return self.file.read(buffer);
    }
};
... 
pub fn InStream(comptime ReadError: type) type {
    return struct {
        const Self = this;
        pub const Error = ReadError;

        /// Return the number of bytes read. If the number read is smaller than buf.len, it
        /// means the stream reached the end. Reaching the end of a stream is not an error
        /// condition.
        readFn: fn (self: *Self, buffer: []u8) Error!usize,
...
    };
} 

[ghost]

sorry if that sounds rude but thats unusable/ unreadable IMO I mean I'd rather use cpp than wade through that kind of code to get stuff done.

[isaachier]

You get used to it. Also, it gives you more control over the OO behavior than C++ would. Anyway if you want a C++ replacement, why not use Rust? Zig is supposed to compete with C, which has no concept of inheritance/polymorphism. CPython uses a similar mechanism to implement inheritance in its source.

[BarabasGitHub]

I mean If you acknowledge that OO makes sense in some cases then the language should provide an adequate solution and not a hack.

I actually disagree here. Just because it makes sense in some cases doesn't mean you have to provide for it in your language. You shouldn't try to do everything. Then you quickly end up with something like C++ which does a little bit of everything, and it's basically a giant mess.

If it only makes sense in some cases people can easily work with it in inside the language.

[isaachier]

I'm not saying there should be no extension to the language to make inheritance simpler. However, I personally believe Zig would be complete whether or not it had those extensions. The difficulties you speak of @monouser7dig encourage the developer to use comptime instead of dynamic dispatch where possible, which translates into increased performance.

[ghost]

here is how I would implement a "getter" trait in zig at this point in time
only thing missing would be that I can actually declare that trait

const Foo = struct {
    a : f32,
    fn get(self : *const Foo) f32 {
        return self.a;
    }
};

const Bar = struct {
    a : f32,
    fn get(self : *const Bar) f32 {
        return self.a;
    }
};

fn getter(comptime T : type, instance : T) f32{
    return instance.get();
}

pub fn main() void {
    const bar : Bar = Bar {.a = 1};
    const foo : Foo = Foo {.a = 10};

    const val = getter(@typeOf(bar), bar);
    const val2 = getter(@typeOf(foo), foo);
}

You shouldn't try to do everything. Then you quickly end up with something like C++ which does a little bit of everything, and it's basically a giant mess.

I do not think it is just about how much you do but much more about HOW you do it.

I also think we just have to wait for Andrew to build the gui app and then we will have some traits.

[BarabasGitHub]

You can even do it without the explicit type.

fn getter(instance : var) f32{
    return instance.get();
}

of course you can also just call .get() :P I'm actually not sure what the function does for you tbh.

As for the C++ comment I meant that everyone has their own opinion of how to do stuff and so every C++ code looks quite different. While for example with C it's often quite similar, because you don't have that many (straightforward) options to do things.

[ghost]

because you don't have that many (straightforward) options to do things.

sure but the same is true for go although they have options for getting some OO into your code if you really want to

You can even do it without the explicit type.

thanks !

of course you can also just call .get()

getter was meant as an example of some connection of parts which has some interface which requires get and which is fed with different types

[isaachier]

@monouser7dig I think you are referring to type constraints. You would like a compile-time limitation on the type of T to be something that declares T.get(), which is fine, but different from polymorphism. If you wanted inheritance, it would look more like this at the moment:

const Gettable = struct {
    get: fn(*const Gettable) f64,
};

const Foo = struct {
    a : f32,
    gettable: Gettable,
    get: fn(gettable : *const Gettable) f64 {
        const self = @fieldParentPtr(gettable);
        return self.a;
    },
};

const Bar = struct {
    a : f32,
    gettable: Gettable,
    get: fn(gettable : *const Gettable) f64 {
        const self = @fieldParentPtr(gettable);
        return self.a;
    },
};

Instead of using the type Foo or Bar itself as the Gettable, you pass the gettable member. Note, this is exactly why you see things like stdout.stream, because stdout on its own is the subclass, stdout.stream is the base class member.

[BarabasGitHub]

@monouser7dig I see. I've never done any Go programming, so I wouldn't know about that.

[isaachier]

@BarabasGitHub @monouser7dig Go struct embedding is actually lifted from an extension in the Plan9 C compiler and I've love to see Zig use that. See more here: https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html.

[ghost]

@isaachier yes I may mix up polymorphism and interfaces here a bit, sorry

For me there really are two issues:

1. I have some object and want to use it's functional in another object, this can be done by inheritance or, which I find really appropriate, like in go by composition.
   https://medium.com/golangspec/promoted-fields-and-methods-in-go-4e8d7aefb3e3

type Person struct {
    name string
    age int32
}
func (p Person) IsAdult() bool {
    return p.age >= 18
}
type Employee struct {
    position string
}
func (e Employee) IsManager() bool {
    return e.position == "manager"
}
type Record struct {
    Person
    Employee
}
...
record := Record{}
record.name = "Michał"
record.age = 29

2. interfaces and as I think about what I just typed in zig I think this is actually very similar to what go does

fn getter(instance : var) f32{
    return instance.get();
}

is just like

type Getter interface {
  func get() float32
}

func getter(instance  Getter) f32{
    return instance.get();
}

the only difference is that once you have interfaces or traits or something, you can use that interface as a function parameter type instead of just var but you get a compile error in both cases.

So the main thing that interfaces could bring to zig is more readable and self documenting code.

[isaachier]

@monouser7dig sorry but I must stress interfaces are not type constraints either. The big difference between polymorphism and type constraints is whether it is a runtime or compile-time concept. Type constraints are in Haskell and to an extent in C++, and occur in Java as variance. They are compile-time constraints on the type T refers to. That way you can make a function available to certain type and not to others. Consider you want to implement modulo as a custom operator, but only for integer types, not all numbers. You need a way to express that comptime T should not include non-integers. That's where type constraints come up. They are strictly compile-time.

[0joshuaolson1]

And then one wonders about higher order types. This seems way too bikeshed-able for the current proposal.

A compare/contrast with alternatives like in Go's proposal process alternatives may help.

[ghost]

https://stackoverflow.com/a/1031385
So polymorphism is the ability (in programming) to present the same interface for differing underlying forms (data types).

Bar and Fooboth conform to the interface of having a get function
but both implement it potentially differently
but for a function (getter) that uses the get method they both are the same.

I mean the dynamic polymorphism you are talking about which can only be determined at runtime can only happen if you have inheritance which nobody wants to have.

Even a list of alternatives and why you'd use them where you use Zig may help.

#1205 (comment)

[0joshuaolson1]

You haven't shown how ugly the current Zig solution would be.

[ghost]

You haven't shown how ugly the current Zig solution would be.

isaachier has provided those examples

[0joshuaolson1]

What about GUIs and runtype type creation? It would be nice to see the discussion captured in the proposal comment. @lerno?

[isaachier]

According to @lerno, the Objective-C compiler does a lot of work to prettify its dispatch mechanism. The Zig compiler would need to add this new layer for him to provide an example.

[ghost]

runtype type creation

what kind of code should that be? I'm afraid that is absolutely not something that is desirable
compile time type creation is crazy enough already if you can do that which is probably going to happen as it seems

If you have a GUI and you have an Interface with a draw method then all your buttons and sliders can happily implement that draw method etc. and everything that wants to do work with those GUI elements just takes a Drawable or any other needed Interface or currently in zig a var.

[0joshuaolson1]

I mentioned runtime types because Smalltalk was brought into the conversation.

[lerno]

I won't rehash in detail how ObjC can work with an GUI. It's straightforward to look at some simple tutorial on UIKit.

The interesting part is how straightforward UIKit / Cocoa uses reflection and dynamic dispatch:

// Assume we want to create an UI component from a string name:
// theUIObjectClass = @"UIButton"
id uiObject = [[NSClassFromString(theUIObjectClass) alloc] init];
// "id" here is the "any" type.

// Reference a callback to run later:
// theCallbackName = @"open:"
Selector sel = NSSelectorFromString(theCallbackName);
// sel can now be stored, and called later:
[target performSelector:sel withObject:nil];
// the above runs the same code as [target open:nil]

If we compare this to C++ and possible solutions:

creation_map = { { "UIButton", [] { return new UIButton(); }, 
    { "UILabel",  [] { return new UILabel(); },
        /* ... */
    }

or:

UIObject createByName(const std::string &name) {
   if (name == "UIButton") return new UIButton();
   if (name == "UILabel") return new UILabel() 
   ....
}

Similar for callbacks, any names need to be registered somewhere, and obviously argument types etc needs to have some sort of way to encapsulate them etc.

Also notice how messaging to "id" allows duck typing:

void test(id someObject)
{
    printf("%d\n", [someObject count]);
}

... 

NSDictionary *foo = [NSDictionary dictionary];
NSArray *bar = [NSArray array];
NSNumber *baz = [NSObject new];
test(foo); // Works fine, NSDictionary has count
test(bar); // Works fine, NSArray has count
test(baz); 
// invokes [baz doesNotRecognizeSelector:@selector(count)]
// which throws an error by default.

Creating proxy objects is consequently also near trivial for ObjC since an object can pretend to be of a different class and dynamically respond to requests.

Note that much of this is possible in Java. The difference being that Java's reflection is x100 as heavy to use.

[kyle-github]

Zig has two things you can already leverage here:

1. Zig compiles all the code at once. There is no separate compilation unit with all the lost information as there is in C/C++ and many similar languages. So you do have complete type information for all points in the program (unless you use var?).
2. Zig has powerful compile time features. Remember you can run code at compile time.

@thejoshwolfe pointed out far above the discussion that in issue #130 there was a discussion of many implementation techniques for things like vtables that could be done now. That is just one way to implement dynamic dispatch. There is some sample code.

If you can figure out a way to support duck typing then I think you can cover most of what you are asking for. Mock/proxy objects are covered. The archetype used for OO inheritance discussion, the Shape object with the draw method can be covered (just make all the concrete shapes implement draw). Etc.

I suggest trying to see how far you can get. If there are small additions to Zig that would be necessary to implement most of this in library code, those would have a pretty good chance of getting accepted because they would have a good use case and would be minimal.

Look at the code in #130 and I think it may help spark some ideas.

[lerno]

@kyle-github was that directed towards me or someone else? The discussion is a bit hard to follow right now...

[kyle-github]

@lerno yes, that was toward you :-)

[lerno]

@kyle-github without syntax, the code would be rather boring:

 Class *class = register_class("Foo");
 register_ivar(class, "bar", f32);
 selector foo_sel = selector_from_string("foo");
 selector baz_sel = selector_from_string("baz:");
 register_method(class, foo_sel, function_foo);
 register_method(class, baz_sel, function_baz, i32);
 id object = new_object("Foo");
 send_msg(object, foo_sel);
 send_msg(object, selector_from_string("baz:"), 320);

[lerno]

I'm just close this one then.