Expose TokamakCore as a library (for GTK renderer)

[mortenbekditlevsen]

Expose TokamakCore as a library so that it can be used directly as an external dependency.

I tried building a GTK wrapper using TokamakCore and lack the possibility of referencing the library directly.

[MaxDesiatov]

My main worry about making TokamakCore public is that some APIs that are made public in it to expose them to renderers should be actually internal. But because Swift doesn't have packageprivate or something of the same effect, we have to keep them public. The DOM renderer is not complete yet, I'd expect some kind of breakage at least when Transaction and Animation types are introduced in #235. There could be even more breakage if we decide to fully support all underscored APIs in SwiftUI for whatever reason.

Would you be interested in submitting your GTK renderer as a PR to this repository directly? I'm not sure other maintainers would directly contribute to it, but at least with working CI we'd make efforts not to break this new renderer when changes are made to TokamakCore. And it also would get more exposure that way as one that's officially supported 🙂

[carson-katri]

I think putting the Renderer in the repo is the best choice here. I also have a working but unfinished GTK wrapper, and that’s how I did it, but would be very interested to see your implementation and possibly contribute.

[mortenbekditlevsen]

Hi @MaxDesiatov and @carson-katri ,
Certainly - I’d be happy to push my poc code to this repo.
It’s all very ad-hoc right now. I don’t know if I am implementing my intermediate view representations in the most logical way.
Furthermore I am currently using Dispatch for the single line scheduler, but if you have a good alternative for a Linux solution that can of course be changed.
It’s about 20 years since I used gtk at university, so that’s all quite foreign to me too. :-)
The build process for the gtk wrapper requires some scripts. I’ll see if that could be done without.
If you are curious to see the code, I pushed it here: https://github.com/mortenbekditlevsen/SwiftGTKUI
I just expanded on a sample gtk project, so the Readme is not updated - and I added Tokamak to my local version of an Xcode project, so it requires some tinkering to build.

[mortenbekditlevsen]

I added a screenshot to a topic on swift evolution if you’d like to see how it looks currently:
https://forums.swift.org/t/hacking-swift-runtime/42289/8

[MaxDesiatov]

Thank you so much, this is amazing work! 👏

[carson-katri]

Awesome, glad to see others getting involved in making renderers! I just uploaded my implementation to my fork if you wanted to take a look at how I approached it. I didn't use SwiftGTK and instead just called the GTK functions directly. You can run with make run and there's also a task for VSCode. I've only tested on CentOS 8, so it may not work on macOS.

[carson-katri]

And here's a screenshot:

[mortenbekditlevsen]

Hi @carson-katri ,

I agree that it is definitely the best strategy to use GTK directly instead of pulling in an extra dependency.

I attempted changing the Package.swift and TokamakShims to work on macOS too - and fixed some extraneous flags generated by pkg-config that caused an error.
Now I got the product setup to build, but I get a compiler error that I don't quite understand:

/Users/mortenbekditlevsen/PrivateProjects/Tokamak2/Sources/TokamakGTKDemo/main.swift:48:8: error: type 'TokamakGTKDemo' does not conform to protocol 'App'
struct TokamakGTKDemo : App {
       ^
/Users/mortenbekditlevsen/PrivateProjects/Tokamak2/Sources/TokamakGTK/App/App.swift:34:14: note: candidate has non-matching type 'AnyPublisher<ScenePhase, Never>' [with Body = some Scene]
  public var _phasePublisher: AnyPublisher<ScenePhase, Never> {
             ^
/Users/mortenbekditlevsen/PrivateProjects/Tokamak2/Sources/TokamakCore/App/App.swift:35:7: note: protocol requires property '_phasePublisher' with type 'AnyPublisher<ScenePhase, Never>'
  var _phasePublisher: AnyPublisher<ScenePhase, Never> { get }

I guess that the issue isn't actually with the _phasePublisher. That looks alright, so the problem is likely something else.

I am using the swift version bundled in Xcode-12.2:

Mortens-MBP:Tokamak2 mortenbekditlevsen$ swift --version
Apple Swift version 5.3.1 (swiftlang-1200.0.41 clang-1200.0.32.8)
Target: x86_64-apple-darwin20.1.0

I think that I can get the project building without using the TokamakGTKCHelpers, since the macros it uses expand to simple c function calls (when not compiled with gcc). I haven't checked if the casts I perform work, however, since I can't get the full project to compile. If the c helpers can be skipped then I assume that the makefile is not necessary, right? It's a shame spm can't use pkg-config for non-system-library targets - it would make sense to allow it for something like the c helpers, it seems...

I think I'll park my own experiments with the extra SwiftGTK wrapper library since that seems like it could be a dead end. My experiments were mainly educational/for fun anyway. :-)

But it was a lot of fun too! You have done an excellent job at abstracting out the renderer so it was quite easy to get started. Your renderer implementation guide is excellent too - and was very helpful to get started.

[mortenbekditlevsen]

Hi @carson-katri ,
Congrats to you all on the latest release.
I just wanted to hear if you agree that it's better to follow your path of integrating directly with 'CGTK' instead of going through an extra wrapper.
I'd like to help getting that to build on macOS, but unfortunately the compiler error above is keeping me back. Have you got an idea about what might be going on?

[carson-katri]

I haven’t looked into it yet but maybe it’s using OpenCombine instead of native Combine (via the shim) on macOS? I’ll see if I can get it working this weekend if that doesn’t help...

[carson-katri]

Got it running on macOS:

The updated code is on my fork, lmk if you have any problems @mortenbekditlevsen

[mortenbekditlevsen]

Thanks so much!
Looking forward to trying out!

[mortenbekditlevsen]

Hi @carson-katri ,
I got it working just fine. I had a small issue with the 'CRuntime' missing, but I temporarily hacked around it by changing Package.swift in one of the dependencies.

I started playing with it and experienced a crash - the closureboxes were released immediately in the handler, but the reference to them outlived the lifetime of the handler. As I gather from the GTK documentation, the correct place to release them is in the 'destroy_handler' GClosureNotify parameter of gtk_signal_connect.
As I mentioned, dealing with the gtk apis directly is a bit foreign to me, so I don't know if I did it correctly, but I created a PR with the changes:
carson-katri#1

[mortenbekditlevsen]

I also experimented with removing the TokamakGTKCHelpers dependency entirely - as well as the functionality in GtkContainer+forEach.swift, since it currently appears to be unused.
That allows me to compile (but not link) using swift build directly instead of using the Makefile.
I have a hard time grasping why the pkgConfig configuration in the systemLibrary target is not sufficient for linking, but I do get a warning before all the linker errors:

ld: warning: Could not find or use auto-linked library 'gtk-3'

so I guess that is related.
I tried using CGTK as a direct dependency of the executable, but that didn't make any difference.

[MaxDesiatov]

Thanks again for all your continued work on this! I would very much like to see this in the main Tokamak repository. Please feel free to submit PRs here, they don't have to have everything working at once. I'd be happy to review separate small PRs that introduce an empty GTK renderer module, and then making that work and fixing bugs bit by bit.

I know that maintaining forks is hard, so having that work here would help you avoid big merge conflicts that are difficult to resolve later.

[mortenbekditlevsen]

Hi @MaxDesiatov ,
I really have done very, very little. 😊
My main efforts were purely out of curiosity, and I think there's much sense in going with the work of @carson-katri .
If you're ok with adding this to the main project, @carson-katri , then I'd definitely like to contribute in the capacity that I can (which is probably not a whole lot).

[MaxDesiatov]

This was addressed to both of you 🙂

@carson-katri are you currently busy with #316? If so, would you mind if in the meantime I create a few PRs that merge GTK-related commits from your fork into this repo?

[carson-katri]

Yeah, that'd be good 👍

[MaxDesiatov]

Do any of you get fatal error: 'gtk/gtk.h' file not found on macOS? I have run brew install gtk+3 and it completed successfully.

I've also tried

swift run TokamakGTKDemo -Xswiftc -I/usr/local/opt/gtk+3/include/gtk-3.0 -Xcc -I/usr/local/opt/gtk+3/include/gtk-3.0

but that doesn't work...

[MaxDesiatov]

Ok, I need Makefile for this to work, got it.

[mortenbekditlevsen]

I have only made it build and run using the Makefile which runs pkgconf to get the compiler and linker flags.
If I skip the TokamakGTKCHelpers dependency (which is currently only used from a file with unused functions), then I can build directly using 'swift build'.
Unfortunately the linking of the executable still requires the Makefile (which I don't really get since I thought the pkgconfig parameter to CGTK in the Package.swift file would take care of supplying the linker options too).

[MaxDesiatov]

As far as I understand, pkgconfig integration with Homebrew is broken in SwiftPM. I had similar issues when I tried to make Binaryen bindings to work, and I still had to pass flags manually.

[mortenbekditlevsen]

Excellent, that would explain it. I'll see if I can find a bug tracking that issue.

[MaxDesiatov]

Closing as superseded by #333.