Skip to content

Commit

Permalink
Add basic string interpolation / string templates (#743)
Browse files Browse the repository at this point in the history
Fixes #722 and adds basic support for string templates. 

There are a few caveats to keep in mind with the current initial
implementation:
- All arguments already have to be strings. It is basically just
syntactic sugar so that you can write `"${a.show} and ${b.show}"`
instead of `a.show ++ " and " ++ b.show` where `a` and `b` are some
'showable' values. Internally, however, this is just how it is
desugared.
- String templates cannot be used to match string literals in pattern
matches
- The generated JS code looks a bit weird:
  ```
const v_r_897 = $effekt.println(((((((((("GET ") + (domain_0)))) +
(((("/users/") + (user_0))))))) + (((("/resource/") + (('' +
resourceId_0))))))));
  ```
+ If one forgets to call `show` on arguments which are not already of
type string, the error messages are hideous and possibly confusing:
<details>
<summary>error</summary>

```
-There are multiple overloads, which all fail to check:
-Possible overload: effekt::println of type Bool => Unit
-  Expected String but got Int.
-  Expected Bool but got String.
-
-Possible overload: list::println of type List[Bool] => Unit
-  Expected String but got Int.
-  Expected List[Bool] but got String.
-
-Possible overload: array::println of type Array[Int] => Unit
-  Expected String but got Int.
-  Expected Array[Int] but got String.
-
-Possible overload: effekt::println of type String => Unit
-  Expected String but got Int.
-
-Possible overload: list::println of type List[Int] => Unit
-  Expected String but got Int.
-  Expected List[Int] but got String.
-
-Possible overload: effekt::println of type Byte => Unit
-  Expected String but got Int.
-  Expected Byte but got String.
-
-Possible overload: list::println of type List[String] => Unit
-  Expected String but got Int.
-  Expected List[String] but got String.
-
-Possible overload: list::println of type List[Double] => Unit
-  Expected String but got Int.
-  Expected List[Double] but got String.
-
-Possible overload: array::println of type Array[String] => Unit
-  Expected String but got Int.
-  Expected Array[String] but got String.
-
-Possible overload: effekt::println of type Ordering => Unit
-  Expected String but got Int.
-  Expected Ordering but got String.
-
-Possible overload: array::println of type Array[Bool] => Unit
-  Expected String but got Int.
-  Expected Array[Bool] but got String.
-
-Possible overload: effekt::println of type Unit => Unit
-  Expected String but got Int.
-  Expected Unit but got String.
-
-Possible overload: option::println of type Option[Int] => Unit
-  Expected String but got Int.
-  Expected Option[Int] but got String.
-
-Possible overload: option::println of type Option[Double] => Unit
-  Expected String but got Int.
-  Expected Option[Double] but got String.
-
-Possible overload: array::println of type Array[Double] => Unit
-  Expected String but got Int.
-  Expected Array[Double] but got String.
-
-Possible overload: effekt::println of type Double => Unit
-  Expected String but got Int.
-  Expected Double but got String.
-
-Possible overload: effekt::println of type Int => Unit
-  Expected String but got Int.
-  Expected Int but got String.
-
-Possible overload: option::println of type Option[Bool] => Unit
-  Expected String but got Int.
-  Expected Option[Bool] but got String.
-
-  println("GET ${domain}/users/${user}/resource/${resourceId}")
```
</details>

Apart from that, the changes are really lightweight because of the early
desugaring in the parser.

I am looking forward to your feedback and ideas.

---------

Co-authored-by: Jiří Beneš <mail@jiribenes.com>
Co-authored-by: Marvin <git@marvinborner.de>
Co-authored-by: Philipp Schuster <philipp.schuster@uni-tuebingen.de>
Co-authored-by: Jonathan Brachthäuser <jonathan@b-studios.de>
  • Loading branch information
5 people authored Jan 28, 2025
1 parent f92d229 commit 6b2b191
Show file tree
Hide file tree
Showing 10 changed files with 180 additions and 17 deletions.
24 changes: 9 additions & 15 deletions effekt/jvm/src/main/scala/effekt/Runner.scala
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,15 @@ trait Runner[Executable] {
def includes(stdlibPath: File): List[File] = Nil

/**
* Modules this backend loads by default
* Modules this backend loads by default.
*
* Invariants:
* - All imports of prelude modules have to be in the prelude as well.
* - The order matters and should correspond to the topological ordering with respect to the imports, that is,
* if module A depends on module B, then B should come before A.
* - Furthermore, each module mentioned here must import the `effekt` module as its first import.
*/
def prelude: List[String] = List("effekt")
def prelude: List[String] = List("effekt", "option", "list", "result", "exception", "array", "string", "ref")

/**
* Creates a OS-specific script file that will execute the command when executed,
Expand Down Expand Up @@ -151,8 +157,6 @@ object JSNodeRunner extends Runner[String] {

override def includes(path: File): List[File] = List(path / ".." / "js")

override def prelude: List[String] = List("effekt", "option", "list", "result", "exception", "array", "string", "ref")

def checkSetup(): Either[String, Unit] =
if canRunExecutable("node", "--version") then Right(())
else Left("Cannot find nodejs. This is required to use the JavaScript backend.")
Expand Down Expand Up @@ -184,16 +188,13 @@ object JSNodeRunner extends Runner[String] {
}
}
object JSWebRunner extends Runner[String] {
import scala.sys.process.Process

val extension = "js"

def standardLibraryPath(root: File): File = root / "libraries" / "common"

override def includes(path: File): List[File] = List(path / ".." / "js")

override def prelude: List[String] = List("effekt", "option", "list", "result", "exception", "array", "string", "ref")

def checkSetup(): Either[String, Unit] =
Left("Running js-web code directly is not supported. Use `--compile` to generate a js file / `--build` to generate a html file.")

Expand All @@ -202,7 +203,6 @@ object JSWebRunner extends Runner[String] {
* and then errors out, printing it's path.
*/
def build(path: String)(using C: Context): String =
import java.nio.file.Path
val out = C.config.outputPath().getAbsolutePath
val jsFilePath = (out / path).unixPath
val jsFileName = path.unixPath.split("/").last
Expand Down Expand Up @@ -231,9 +231,7 @@ object JSWebRunner extends Runner[String] {
trait ChezRunner extends Runner[String] {
val extension = "ss"

def standardLibraryPath(root: File): File = root / "libraries" / "common"

override def prelude: List[String] = List("effekt", "option", "list", "result", "exception", "array", "string", "ref")
def standardLibraryPath(root: File): File = root / "libraries" / "common"

def checkSetup(): Either[String, Unit] =
if canRunExecutable("scheme", "--help") then Right(())
Expand Down Expand Up @@ -264,17 +262,13 @@ object ChezCallCCRunner extends ChezRunner {
}

object LLVMRunner extends Runner[String] {
import scala.sys.process.Process

val extension = "ll"

def standardLibraryPath(root: File): File = root / "libraries" / "common"

override def includes(path: File): List[File] = List(path / ".." / "llvm")

override def prelude: List[String] = List("effekt", "option", "list", "result", "exception", "string") // "array", "ref")


lazy val gccCmd = discoverExecutable(List("cc", "clang", "gcc"), List("--version"))
lazy val llcCmd = discoverExecutable(List("llc", "llc-15", "llc-16"), List("--version"))
lazy val optCmd = discoverExecutable(List("opt", "opt-15", "opt-16"), List("--version"))
Expand Down
40 changes: 39 additions & 1 deletion effekt/shared/src/main/scala/effekt/RecursiveDescent.scala
Original file line number Diff line number Diff line change
Expand Up @@ -974,8 +974,13 @@ class RecursiveDescent(positions: Positions, tokens: Seq[Token], source: Source)
case `fun` => funExpr()
case `new` => newExpr()
case `do` => doExpr()
case _ if isString => templateString()
case _ if isLiteral => literal()
case _ if isVariable => variable()
case _ if isVariable =>
peek(1).kind match {
case _: Str => templateString()
case _ => variable()
}
case _ if isHole => hole()
case _ if isTupleOrGroup => tupleOrGroup()
case _ if isListLiteral => listLiteral()
Expand Down Expand Up @@ -1016,6 +1021,33 @@ class RecursiveDescent(positions: Positions, tokens: Seq[Token], source: Source)
case `false` => true
case _ => isUnitLiteral
}

def isString: Boolean = peek.kind match {
case _: Str => true
case _ => false
}

def templateString(): Term =
nonterminal:
backtrack(idRef()) ~ template() match {
// We do not need to apply any transformation if there are no splices
case _ ~ Template(str :: Nil, Nil) => StringLit(str)
case _ ~ Template(strs, Nil) => fail("Cannot occur")
// s"a${x}b${y}" ~> s { do literal("a"); do splice(x); do literal("b"); do splice(y) }
case id ~ Template(strs, args) =>
val target = id.getOrElse(IdRef(Nil, "s"))
val doLits = strs.map { s =>
Do(None, IdRef(Nil, "literal"), Nil, List(StringLit(s)), Nil)
}
val doSplices = args.map { arg =>
Do(None, IdRef(Nil, "splice"), Nil, List(arg), Nil)
}
val body = interleave(doLits, doSplices)
.foldRight(Return(UnitLit())) { (term, acc) => ExprStmt(term, acc) }
val blk = BlockLiteral(Nil, Nil, Nil, body)
Call(IdTarget(target), Nil, Nil, List(blk))
}

def literal(): Literal =
nonterminal:
peek.kind match {
Expand Down Expand Up @@ -1272,6 +1304,12 @@ class RecursiveDescent(positions: Positions, tokens: Seq[Token], source: Source)
case Fail(_, _) => position = before; None
}

def interleave[A](xs: List[A], ys: List[A]): List[A] = (xs, ys) match {
case (x :: xs, y :: ys) => x :: y :: interleave(xs, ys)
case (Nil, ys) => ys
case (xs, Nil) => xs
}

/**
* Tiny combinator DSL to sequence parsers
*/
Expand Down
2 changes: 1 addition & 1 deletion examples/benchmarks/other/unify.effekt
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ module examples/benchmarks/unify
import examples/benchmarks/runner
import map
import result
import bytearray
import stream
import bytearray

type Type {
Var(name: String)
Expand Down
2 changes: 2 additions & 0 deletions examples/pos/string_interpolation.check
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
GET https://api.effekt-lang.org/users/effekt/resource/42
Fix point combinator: \ f -> (\ x -> f x x) \ x -> f x x
39 changes: 39 additions & 0 deletions examples/pos/string_interpolation.effekt
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import stringbuffer

type Expr {
Var(id: String)
Abs(param: String, body: Expr)
App(fn: Expr, arg: Expr)
}

def pretty { prog: () => Unit / {literal, splice[Expr]} }: String = {
with stringBuffer
try {
prog()
do flush()
} with literal { s =>
resume(do write(s))
} with splice[Expr] { expr =>
expr match {
case Var(id) =>
do write(id)
case App(Abs(param, body), arg) =>
do write(pretty"(${Abs(param, body)}) ${arg}")
case App(fn, arg) =>
do write(pretty"${fn} ${arg}")
case Abs(param, body) =>
do write(s"\\ ${param} -> " ++ pretty"${body}")
}
resume(())
}
}

def main() = {
val domain = "https://api.effekt-lang.org"
val user = "effekt"
val resourceId = 42
println("GET ${domain}/users/${user}/resource/${resourceId.show}")

val fixpoint = Abs("f", App(Abs("x", App(Var("f"), App(Var("x"), Var("x")))), Abs("x", App(Var("f"), App(Var("x"), Var("x"))))))
println(pretty"Fix point combinator: ${fixpoint}")
}
1 change: 1 addition & 0 deletions examples/stdlib/acme.effekt
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import seq
import set
import stream
import string
import stringbuffer
import test
import tty
import union_find
Expand Down
1 change: 1 addition & 0 deletions examples/stdlib/map/counter.effekt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import map

import bytearray

def counter(words: List[String]): Map[String, Int] = {
Expand Down
1 change: 1 addition & 0 deletions examples/stdlib/set/unique.effekt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import set

import bytearray

def unique(words: List[String]): Set[String] = {
Expand Down
22 changes: 22 additions & 0 deletions libraries/common/effekt.effekt
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,28 @@ def repeat(n: Int) { action: () => Unit } = each(0, n) { n => action() }

def repeat(n: Int) { action: () {Control} => Unit } = each(0, n) { (n) {l} => action() {l} }


// Splices
// =======
//
// The Effekt compiler translates the string interpolation
//
// "foo ${42} bar ${43}"
//
// to the following stream of `literal`s and `splice[Int]`s:
//
// do literal("foo "); do splice(42); do literal(" bar "); do splice(43)
//
// The stream is wrapped into a handler function, which defaults to `stringbuffer::s`
//
// s { ... }
//
// but can be specified by prefixing the string interpolation `custom"..."`.

effect literal(s: String): Unit
effect splice[A](x: A): Unit


namespace internal {
namespace boxing {
// For boxing polymorphic values
Expand Down
65 changes: 65 additions & 0 deletions libraries/common/stringbuffer.effekt
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
module stringbuffer

import bytearray

interface StringBuffer {
def write(str: String): Unit
def flush(): String
}

def stringBuffer[A] { prog: => A / StringBuffer }: A = {
val initialCapacity = 128
var buffer = bytearray::allocate(initialCapacity)
// next free index to write to
var pos = 0

def ensureCapacity(sizeToAdd: Int): Unit = {
val cap = buffer.size - pos + 1
if (sizeToAdd <= cap) ()
else {
// Double the capacity while ensuring the required capacity
val newSize = max(buffer.size * 2, buffer.size + sizeToAdd)
buffer = buffer.resize(newSize)
}
}

try { prog() }
with StringBuffer {
def write(str) = {
val bytes = fromString(str)
ensureCapacity(bytes.size)
bytes.foreach { b =>
buffer.unsafeSet(pos, b)
pos = pos + 1
}
resume(())
}
def flush() = {
// resize buffer to strip trailing zeros that otherwise would be converted into 0x00 characters
val str = bytearray::resize(buffer, pos).toString()
// after flushing, the stringbuffer should be empty again
buffer = bytearray::allocate(initialCapacity)
resume(str)
}
}
}

/// Handler for string interpolation using a string buffer
def s { prog: () => Unit / { literal, splice[String] } }: String =
stringBuffer {
try { prog(); do flush() }
with splice[String] { x => resume(do write(x)) }
with literal { s => resume(do write(s)) }
}

namespace examples {
def main() = {
with stringBuffer
do write("hello")
do write(", world")
// prints `hello, world`
println(do flush())
// prints the empty string
println(do flush())
}
}

0 comments on commit 6b2b191

Please sign in to comment.