JIT's handling of polymorphism isn't always working

[Fidget-Spinner]

Thanks to @markshannon 's work on dumping executors on graphviz, I can finally inspect how polymorphism is handled in our benchmarks. I have come to the conclusion that the trace trees we produce are not handling polymorphism in certain cases. Here's why:

The problem

Here's the graphviz output from bm_raytrace, with tracing through allocations on my branch, and visualization: https://gist.github.com/Fidget-Spinner/a30b59dd2c7a8353b3d96edd4eb6d6b8

Place it into a visualizer and you'll notice two things:

A series of 3 additional side exits point out from _GUARD_TYPE_VERSION, but their type versions are unchanged:

This means the guard type version for polymorphism actually continuously fails!

There's also another part, with _GUARD_BOTH_FLOAT side exits not changing:

In this case, they should become _GUARD_BOTH_INT or just generic _BINARY_OP

What's happening?

Basically when the trace trees re-project themselves, the specialized bytecodes have not updated yet (or unfortunately, updated to the same value). This causes the trace trees to re-project the exact same trace. Thus polymorphism is not being handled at all.

One fix is to reduce the specializing interpreter's backoff counter only when it has an executor. Indeed on my branch, this fixes polymorphism.

CC @brandtbucher

[Fidget-Spinner]

One fix is to reduce the specializing interpreter's backoff counter only when it has an executor. Indeed on my branch, this fixes polymorphism.

This fixes polymorphism but slows down the specializing interpreter quite a bit. So it seems it's not a viable fix until jit execution proportion is significantly greater than interpreter execution.

[Fidget-Spinner]

An alternative fix that I tried that seems promising: drop to the default instruction when encountering unstable ones. This could be tuned better, but it already produces a 3% speedup on bm_raytrace.

Before vs after executor graph for bm_raytrace:
https://gist.github.com/Fidget-Spinner/05fe1a97ff30580e8d09d95596888955