Add jank intro post

_posts/2024-12-20-jank.md

@@ -0,0 +1,235 @@
+---
+title: "The jank programming language"
+layout: post
+excerpt: "Learn about jank, the native Clojure dialect on LLVM,
+and how it uses Clang and CppInterOp to stand out from the pack.
+Discover how Clojure users develop and how jank brings that to
+the native world."
+sitemap: false
+author: Jeaye Wilkerson
+permalink: blogs/jank_intro/
+banner_image: /images/blog/jank_intro/logo.png
+date: 2024-12-20
+tags: jank clojure clang clang-repl cppinterop
+---
+
+# Intro
+Hi everyone! I'm [Jeaye Wilkerson](https://github.com/jeaye), creator of the
+[jank programming language](https://jank-lang.org). jank is a dialect of
+[Clojure](https://clojure.org) which I've been working on for around eight years
+now. To start with, jank was an exploration of what my ideal programming
+language would be. Coming from a background of systems programming and game
+development, I was very comfortable with C++, but I also was exploring the
+world of functional programming. All of this led me to try Clojure, which
+ultimately refined the direction of jank altogether.
+
+My work on jank is made possible by Clang, LLVM, and the tools coming out of the
+[Compiler Research Group](https://compiler-research.org/), including Cling/Clang
+REPL and [CppInterOp](https://github.com/compiler-research/CppInterOp).
+
+The past two years of jank development have been exciting as I've focused on
+building jank into a native Clojure dialect on LLVM. This marries the lispy
+world of REPL-driven, data-oriented, interactive programming and the C++ world
+of light, fast, native runtimes. jank isn't production ready yet, but it's come
+a long way in the past two years.
+
+
+<img src="/images/blog/jank_intro/star-history.png" style="display: block; margin-left: auto; margin-right: auto;" width="50%" />
+
+# What is Clojure?
+Clojure is a modern, practical take on Lisp. It's functional-first, has
+persistent, immutable data structures by default, but allows for adhoc side
+effects. As it's a lisp, it has a powerful macro system which allows devs to
+transform their own code using the same functions they use to transform any
+other data. Clojure is dynamically typed and also garbage collected.
+
+Clojure has a rich tooling story for interactive development. Clojure devs
+generally connect their text editor directly to their running programs, via a
+REPL (read, eval, print, loop) client/server. This allows us to send code from
+our editor to our program to be evaluated; the results come back as data which
+we can then continue to work on in our editor. This whole workflow allows any
+function to be redefined at any time during development and it allows for a
+development iteration loop like nothing else.
+
+Clojure is a hosted language, which means it has a symbiotic relationship with
+the host environment on which it runs. By default, Clojure is on the JVM.
+Clojure JVM enables seamless interop with other JVM languages like Java, Kotlin,
+Scala, etc. However, Clojure runs on many other hosts. For example,
+ClojureScript runs on JavaScript (browser or node). ClojureCLR runs on the CLR
+(where C#, F#, etc run). There's also ClojureDart, ClojErl (BEAM), and many
+others. Each of these dialects provides seamless interop with its host. Since
+Clojure is always hosted, things like socket libs, filesystem libs, etc are
+never included in Clojure; they just come from the host.
+
+Clojure on a native host, though, is where jank comes in.
+
+# What makes jank special?
+jank provides all of the interactive functionality of Clojure, which involves
+JIT compiling native code with LLVM, while also providing seamless interop with
+the native world. For those who are familiar, jank for Clojure is basically
+analogous to Clasp for Common Lisp. However, compared to Clasp, jank aims to
+provide even more seamless native interop which doesn't require making "bridge"
+files.
+
+## Interop features
+In jank, when you require a module (like including a header in C++ or importing
+a module in Python), that module may be backed by a jank file or by a C++ file.
+If it's backed by a C++ file, jank will JIT compile that file and then bootstrap
+it by invoking a well-known function. This allows you to vender C++ code
+alongside your jank code within the same project seamlessly.
+
+For example, take a look at this jank file which requires both a jank dependency
+and a C++ dependency. They're both required in the same way. When
+`clojure.pprint` is required, jank will find that the module is backed by a
+`.jank` file, so it will JIT compile that file. However, when `sleep-native` is
+required, jank will find that it's backed by a C++ file and it will JIT compile
+that using `clang::Interpreter`. In either of these cases, if a `.o` is present
+and up to date, jank will load that instead.
+
+```clojure
+(ns nap-time
+  (:require [clojure.pprint :as p] ; JIT loads a jank module
+            [sleep-native :as s])) ; JIT loads a C++ module
+
+(defn main []
+  (s/sleep 500)
+  (p/pprint "napped!"))
+```
+
+The `sleep-native` module can be backed by a C++ file which looks something like
+this. jank will look for a special `jank_load_sleep_native` function in the
+module, based on the name of the module itself. Note, the `-native` suffix is
+just a common pattern used for modules backed by C++ files.
+
+```cpp
+object_ptr jank_load_sleep_native()
+{
+  /* Create the sleep-native namespace so it's exposed to the jank world. */
+  auto const ns{ _rt_ctx->intern_ns("sleep-native") };
+  auto const intern_fn{ [=](native_persistent_string const &name, auto const fn) {
+    auto const boxed_fn{ make_box<obj::native_function_wrapper>(fn) };
+    ns->intern_var(name)->bind_root(boxed_fn);
+  } };
+
+  /* Create a var holding a pointer to the sleep function. */
+  intern_fn("sleep", [](object_ptr const ms) -> object_ptr {
+    auto const duration(std::chrono::milliseconds(to_int(ms)));
+    std::this_thread::sleep_for(duration);
+    return nil_const;
+  });
+
+  return nil_const;
+}
+```
+
+Overall, this automatic module support allows for arbitrarily complex C++ files
+to be loaded, which will generally be used to wrap some existing native code and
+use jank's C or C++ API to expose that code to the rest of the jank system.
+However, this is all still roughly equivalent to "bridge files" and I aim to do
+better.
+
+On top of that, jank will also support interop directly from jank code into C++
+land, allowing for the instantiation of native C++ objects as locals, calling
+native functions, and also instantiating C++ templates on the fly. This is where
+[CppInterOp](https://github.com/compiler-research/CppInterOp) comes in. At this
+point, it's only in design phase, but I aim to utilize CppInterOp to its fullest
+so that we can allow for arbitrary C++ values within jank, calling C++
+functions, and working with C++ data types. The working design looks something
+like this:
+
+```clojure
+; Feed some C++ into Clang so we can start working on it.
+; Including files can also be done in a similar way.
+(c++/declare "struct person{ std::string name; };")
+
+; `let` is a Clojure construct, but `c++/person.` creates a value
+; of the `person` struct we just defined above, in automatic memory.
+(let [s (c++/person. "sally siu")
+      ; We can then access structs using Clojure's normal interop syntax.
+      n (.-name s)
+      ; We can call member functions on native values, too.
+      ; Here we call std::string::size on the name member.
+      l (.size n)]
+ ; When we try to gives these native values to `println`, jank will
+ ; detect that they need boxing and will automatically find a
+ ; conversion function from their native type to jank's boxed
+ ; `object_ptr` type. If such a function doesn't exist, the
+ ; jank compiler fails with a type error.
+ (println n l))
+```
+
+All of this will work with RAII, allow auto-boxing, but also give complete
+access to C++ from within jank.
+
+## AOT features
+With jank, you can AOT compile your programs to two different runtimes:
+
+1. A dynamic runtime which enables REPL usage, further JIT compilation,
+   redefining functions, etc
+2. A static runtime which strips out all JIT capabilities, enables LTO, and
+   directly links functions instead of going through vars
+
+Static runtime binaries will also be able to be statically linked, providing a
+portable binary which will very easily run on any distro or in any docker
+container.
+
+# Why would people use jank?
+There are two main audiences for jank.
+
+## Clojure users
+jank's runtime is significantly lighter than the JVM. Since its runtime is
+specifically crafted for jank, rather than being a generic VM, it's also able to
+compete with the JVM in terms of runtime performance while keeping memory usage
+lower.
+
+On top of that, anyone who is using Clojure and wanting easier access to native
+libraries, tighter AOT compilation, etc, without sacrificing the fundamental
+Clojure interactive development workflows will choose jank.
+
+## Native users
+In game development, where my career has been focused, we build our engines and
+games in C++, generally. However, we very often include a scripting language
+like Lua in the engine so that we don't need to write all of our game logic in
+C++. jank is an excellent fit here, too. Not only does it provide seamless
+interop with the existing C++ engine/game code, but the REPL-driven workflows it
+supports will allow you to spin up your game once and then send code to it, get
+data back, and continue iterating on the code and data shapes without ever
+restarting your game.
+
+Naturally, this is a game development focused use case, but the same thing
+applies to any existing native code base which wants to utilize a higher level
+language for a tighter iteration loop.
+
+# Next steps
+jank is not yet released in production, but I aim to have it out the door in 2025.
+In fact, as of January 2025, I'll have quit my job at Electronic Arts to
+focus on jank full-time, unpaid aside from some small sponsors. I'm aiming to
+get jank released, gather initial feedback, and help existing Clojure users
+onboard jank into their systems.
+
+The three big tasks that are remaining before jank is released are:
+
+1. Improved error handling (following the recent work on Elm, Rust, and others)
+2. Finalized C++ interop support (using the CppInterOp library)
+3. Finalized AOT compilation support
+
+# Future plans
+In the long term, since I'm building my dream language, I won't stop at just a
+native Clojure dialect. jank will always be able to just be that, but I'll also
+be extending it to support gradual typing (maybe linear typing), explicit memory
+management, stronger pattern matching, value-based errors, and more. This will
+allow another axis of control, where some parts of the program can remain
+entirely dynamic and garbage collected while others are thoroughly controlled
+and better optimized. That's exactly the control I want when programming.
+
+# Thank you!
+For everyone who has worked on Cling, Clang REPL, and CppInterOp: Thank you!
+jank is possible because of the magic you folks have created. To Vassil, in
+particular, thanks for all of the support over the past couple of years as I
+onboarded with Cling, ported to Clang, and then ultimately picked up
+CppInterOp as a means to tackle seamless C++ interop.
+
+## Would you like to join in?
+1. Join the jank community on [Slack](https://clojurians.slack.com/archives/C03SRH97FDK)
+2. Join the jank design discussions or pick up a ticket on [GitHub](https://github.com/jank-lang/jank)
+3. Join the Compiler Research community on [Discord](https://discord.gg/Vkv3ne4zVK)