Support for eval-less template execution

[legrego]

One of the ways Kibana leverages Handlebars is via user-supplied templates, which are then executed in the browser.

We recently, finally, removed script-src 'unsafe-eval' from our Content Security Policy. The most challenging part of this exercise was finding a way to get Handlebars to execute templates without the need for dynamic code generation (in other words, without eval).

Inspired by @nknapp's comment in #1443, we took a stab at executing templates by walking the Handlebars-generated AST. We've had our implementation running in production for a little while now, and it's working well for us.

I won't go into too many details about our approach here, but I'll instead refer you to our implementation, which includes a descriptive README:
https://github.com/elastic/kibana/tree/main/packages/kbn-handlebars

If there is a community interest for this, we would be happy to contribute our work to the Handlebars project. Our approach would live alongside the existing approach, rather than replace it. The performance tradeoff would not be acceptable for all users.

So, with all that said:

1. Is there interest in having Handlebars natively support eval-less execution?
2. If so, what would be the best way to incorporate our work into the project? What sort of API would you like to see?

---

aside: I am just the person asking the questions, credit for this work goes to @watson and @thomheymann

[salmin89]

1. Is there interest in having Handlebars natively support eval-less execution?
2. If so, what would be the best way to incorporate our work into the project? What sort of API would you like to see?

aside: I am just the person asking the questions, credit for this work goes to @watson and @thomheymann

1. Yes, we use handlebars and have hit a roadblock due to the origin of this issue. Your fix might be just what we need. Will test it out this week.
2. I’m not sure if you want to maintain a fork or what the best approach is. I’ll let other people decide on this but I did want to raise appreciation for the solution provided.

[legrego]

Thanks for expressing your interest, @salmin89, and do let me know how you make out with your test!

[salmin89]

@legrego your solution worked perfectly. I will copy the package for now, but I'd love to see it natively supported by handlebars.

If you don't get any response, are there any plans on publishing your package as a standalone solution?

[legrego]

@salmin89, happy to hear that.

If you don't get any response, are there any plans on publishing your package as a standalone solution?

We are discussing this as an alternative here: elastic/kibana#150522. My personal preference is to have this become a part of the official distribution, so that we don't have to deal with compatibility changes across the various versions of Handlebars

[davidfant]

For those looking, I created a kibana fork to publish kbn-handlebars: https://www.npmjs.com/package/kbn-handlebars
davidfant/kibana-handlebars@e3e925c

[spandl]

This is great and working for many cases.

I came across a use case that isn't supported: nested expressions.

Example, the first block is working, but the second block isn't omitting the nested CUSTOM_HELPER

{{#if (CUSTOM_HELPER "parameter")}}
        Text
{{else}}
         Alternative text
{{/if}}

{{#if (CUSTOM_HELPER "parameter")}}
        Text with {{CUSTOM_HELPER_2 "parameter"}}
{{else}}
         Alternative text
{{/if}}

Chances this could make it into the library?
Thanks!

[Georift]

Cheers for the library, and sharing the work with the open source community, so great to have a path forward on this while handlebars upstream is blocked.

Just out of interest I wonder if another solution to this probably could be shipping the unsafe-eval work into a worker. And posting messages to and from to do the rendering work. Broadly following something like this gist describes

@legrego @thomheymann @watson am I missing something that you might have found in the trenches? Would this not actually solve the issue for some reason or another?

[nknapp]

I forgot to mention in my comment, that I had started a project https://handlebars-ng.knappi.org a while ago. It should also create a language spec.

I stopped because there seemed to be no interest and I didn't need it personally.

Its a bit stale now, but I would invite everybody who wants to contribute. Although this might be too late now...

[nknapp]

⁵

Cheers for the library, and sharing the work with the open source community, so great to have a path forward on this while handlebars upstream is blocked.

Just out of interest I wonder if another solution to this probably could be shipping the unsafe-eval work into a worker. And posting messages to and from to do the rendering work. Broadly following something like this gist describes

@legrego @thomheymann @watson am I missing something that you might have found in the trenches? Would this not actually solve the issue for some reason or another?

The documentation at handlebars.js uses workers to run templates in the playground. I did this to prevent the main thread from being stuck in endless loops.

[TotalTechGeek]

Hey folks, I'm currently working on a write-up / an alternative engine (both compiler and interpreter) for executing the Handlebars templates. It also supports async. It seems to get pretty decent perf!

It's not quite a drop in replacement at this point in time, I'd guesstimate it's about 90-95% compatible with most Handlebars templates in the wild.

Essentially, I drafted a quick PEG Grammar spec to convert Handlebars Templates into JSON Logic as its AST representation, and I'm using JSON Logic as my execution back-end.

(Benchmarks, 1M Iterations, https://github.com/TotalTechGeek/handlebars-jle/blob/main/bench/index.js)

Handlebars Simple: 2301.95ms | 1.00x
Elastic Simple: 1549.88ms | 1.49x
JLE Interpreted Simple: 90.60ms | 25.41x
JLE Async Simple: 94.90ms | 24.26x
JLE Simple: 4.63ms | 497.68x
---
Handlebars SimpleEach: 2543.65ms | 1.00x
Elastic SimpleEach: 3609.35ms | 0.70x
JLE Interpreted SimpleEach: 408.20ms | 6.23x
JLE Async SimpleEach: 165.64ms | 15.36x
JLE SimpleEach: 54.00ms | 47.10x
---
Handlebars SimpleIf: 2295.44ms | 1.00x
Elastic SimpleIf: 1586.04ms | 1.45x
JLE Interpreted SimpleIf: 247.09ms | 9.29x
JLE Async SimpleIf: 80.97ms | 28.35x
JLE SimpleIf: 15.10ms | 151.97x
---
Handlebars SimpleIf (False): 2332.97ms | 1.00x
Elastic SimpleIf (False): 1666.56ms | 1.40x
JLE Interpreted SimpleIf (False): 185.57ms | 12.57x
JLE Async SimpleIf (False): 78.06ms | 29.89x
JLE SimpleIf (False): 19.42ms | 120.13x
---
Handlebars NestedIf: 2359.29ms | 1.00x
Elastic NestedIf: 1685.26ms | 1.40x
JLE Interpreted NestedIf: 187.97ms | 12.55x
JLE Async NestedIf: 74.92ms | 31.49x
JLE NestedIf: 16.38ms | 144.08x
---
Handlebars NestedIf (False): 2811.23ms | 1.00x
Elastic NestedIf (False): 2519.05ms | 1.12x
JLE Interpreted NestedIf (False): 292.77ms | 9.60x
JLE Async NestedIf (False): 85.16ms | 33.01x
JLE NestedIf (False): 19.03ms | 147.74x
---
Handlebars NestedIf (18): 2345.10ms | 1.00x
Elastic NestedIf (18): 1641.50ms | 1.43x
JLE Interpreted NestedIf (18): 194.29ms | 12.07x
JLE Async NestedIf (18): 81.54ms | 28.76x
JLE NestedIf (18): 17.98ms | 130.45x
---
Handlebars EachStatic: 2765.14ms | 1.00x
Elastic EachStatic: 5663.27ms | 0.49x
JLE Interpreted EachStatic: 33.74ms | 81.95x
JLE Async EachStatic: 72.76ms | 38.00x
JLE EachStatic: 13.07ms | 211.63x
---
Handlebars AddExampleWithTraversal: 4170.25ms | 1.00x
Elastic AddExampleWithTraversal: 6626.68ms | 0.63x
JLE Interpreted AddExampleWithTraversal: 1229.93ms | 3.39x
JLE Async AddExampleWithTraversal: 480.27ms | 8.68x
JLE AddExampleWithTraversal: 377.17ms | 11.06x
---
Handlebars Example 1: 3529.97ms | 1.00x
Elastic Example 1: 13298.24ms | 0.27x
JLE Interpreted Example 1: 2956.29ms | 1.19x
JLE Async Example 1: 267.85ms | 13.18x
JLE Example 1: 202.44ms | 17.44x
---
Handlebars YAML: 7379.91ms | 1.00x
Elastic YAML: 21563.89ms | 0.34x
JLE Interpreted YAML: 3411.66ms | 2.16x
JLE Async YAML: 450.53ms | 16.38x
JLE YAML: 399.06ms | 18.49x

In many cases, the interpreter with the optimizer enabled can perform on par with mainline compiler, and the compiler itself seems to be able to eek out an advantage.

For our real-world templates, we were seeing 20-40x performance improvements.

I hope to add more benchmarks and tests, the lib is still in early stages of fleshing out compat, but I was hoping I might be able to solicit feedback / guidance on if this might be helpful to the HBS ecosystem.

---

To add color some color as to why the "interpreter" is performing well, I've designed JSON Logic to optimize logic it's seen before by creating closures to bind methods together directly, to avoid the overhead of traversing the AST. This allows the JIT to kick in and optimize this further.

For the record, EachStatic is "cheating", because JSON Logic is able to see that the template is deterministic and static, and precompute the result so it never has to process it again. This does not apply to the other templates.

[TotalTechGeek]

A small update, I've been working on improving compatibility with the mainline project,

Some of the missing prior features:

• Whitespace Control
• Partial Syntax
• Implicit Iterator and If Support {{#kids}}...{{/kids}}
• BlockParams (as |num| syntax)
• A handful of alternative ways to reference variables ./ vs this. and {{^}} instead of {{else}}
• Recursive Lookup Support / Enabling & Disabling Escaping
• Some of the stuff Handlebars has matured to support, like iterating over Sets and Maps.

The additions nearly doubled my SLoC count 😅.

The bench suite needs some more additions, but I'd estimate the overall compatibility to truly be a lot closer to 95-97%, with most use cases being a fairly drop-in substitution. Fortunately the performance seems to have ended up in about the same place where it started.

Handlebars Simple: 2203.67ms | 1.00x
Elastic Simple: 1454.22ms | 1.52x
JLE Interpreted Simple: 80.78ms | 27.28x
JLE Async Simple: 91.65ms | 24.04x
JLE Simple: 3.64ms | 605.62x
---
Handlebars SimpleEach: 2400.94ms | 1.00x
Elastic SimpleEach: 3537.01ms | 0.68x
JLE Interpreted SimpleEach: 432.64ms | 5.55x
JLE Async SimpleEach: 159.89ms | 15.02x
JLE SimpleEach: 76.91ms | 31.22x
---
Handlebars SimpleIf: 2257.92ms | 1.00x
Elastic SimpleIf: 1543.33ms | 1.46x
JLE Interpreted SimpleIf: 182.51ms | 12.37x
JLE Async SimpleIf: 75.44ms | 29.93x
JLE SimpleIf: 14.38ms | 157.03x
---
Handlebars SimpleIf (False): 2267.63ms | 1.00x
Elastic SimpleIf (False): 1554.01ms | 1.46x
JLE Interpreted SimpleIf (False): 183.25ms | 12.37x
JLE Async SimpleIf (False): 76.76ms | 29.54x
JLE SimpleIf (False): 17.89ms | 126.76x
---
Handlebars NestedIf: 2278.28ms | 1.00x
Elastic NestedIf: 1555.28ms | 1.46x
JLE Interpreted NestedIf: 198.17ms | 11.50x
JLE Async NestedIf: 72.06ms | 31.62x
JLE NestedIf: 17.41ms | 130.89x
---
Handlebars NestedIf (False): 2670.69ms | 1.00x
Elastic NestedIf (False): 2515.64ms | 1.06x
JLE Interpreted NestedIf (False): 283.52ms | 9.42x
JLE Async NestedIf (False): 84.36ms | 31.66x
JLE NestedIf (False): 19.69ms | 135.63x
---
Handlebars NestedIf (18): 2284.74ms | 1.00x
Elastic NestedIf (18): 1548.73ms | 1.48x
JLE Interpreted NestedIf (18): 182.23ms | 12.54x
JLE Async NestedIf (18): 77.30ms | 29.56x
JLE NestedIf (18): 17.52ms | 130.40x
---
Handlebars EachStatic: 2708.79ms | 1.00x
Elastic EachStatic: 5529.97ms | 0.49x
JLE Interpreted EachStatic: 31.62ms | 85.66x
JLE Async EachStatic: 70.96ms | 38.17x
JLE EachStatic: 13.54ms | 200.05x
---
Handlebars AddExampleWithTraversal: 4107.33ms | 1.00x
Elastic AddExampleWithTraversal: 6588.83ms | 0.62x
JLE Interpreted AddExampleWithTraversal: 1056.67ms | 3.89x
JLE Async AddExampleWithTraversal: 482.76ms | 8.51x
JLE AddExampleWithTraversal: 424.28ms | 9.68x
---
Handlebars Example 1: 3496.99ms | 1.00x
Elastic Example 1: 13478.98ms | 0.26x
JLE Interpreted Example 1: 1738.79ms | 2.01x
JLE Async Example 1: 294.65ms | 11.87x
JLE Example 1: 231.39ms | 15.11x
---
Handlebars YAML: 7409.85ms | 1.00x
Elastic YAML: 21396.63ms | 0.35x
JLE Interpreted YAML: 3168.22ms | 2.34x
JLE Async YAML: 388.12ms | 19.09x
JLE YAML: 326.41ms | 22.70x
---
Handlebars Partials: 5241.32ms | 1.00x
Elastic Partials: 23968.83ms | 0.22x
JLE Interpreted Partials: 655.47ms | 8.00x
JLE Async Partials: 116.11ms | 45.14x
JLE Partials: 56.31ms | 93.07x

Some current drawbacks / notable issues:

• The engine's truthiness is configurable; right now, the default is set to JLE's default, which is JavaScript's defaults for truthiness, I'll probably override this soon.
• No support for deprecated features, like decorators or this/syntax/for/paths. Inline is just a block helper.
• Creating new block helpers is undocumented.
• "SafeString" is not a thing for returning from helpers, the behavior is inverted where it expects you to escape if needed. If this ends up being an issue, I may provide an alternative way to register the helpers.
• I have a heck ton of built-in helpers, which opinionates HBS a bit. (A bunch of arithmetic and comparison and string manipulation stuff).
• No Raw Blocks (I don't fully understand their use case; is the point to be able to make it easier to embed handlebars as text?)

But I think I'm about ready to start gathering templates and aiming for more rigorous compatibility.