Other interpreter-related things that would be nice to have hookable for a JIT

[kmod]

There are a few interpreter-related features that it would be nice for a JIT to be able to replace: currently a JIT has to implement these features the exact way the interpreter does (or break them), but a JIT can have a more efficient implementation.

The ones I can think of are:

• PyFrame_GetLineNumber(): a JIT doesn't need to keep f_lasti up to date except for this feature, since it can compute this on-demand using a mapping table
• PyEval_SetProfile / PyEval_SetTrace -- the interpreter implements these by polling them, but it'd be nice to be able to speculate that they're not called and deoptimize if they are called
• GIL handling -- I haven't thought through what the exact implementation would be, but it'd be nice to not have to poll eval_breaker like the interpreter does
• PyFrame_LocalsToFast -- we want to be able to speculate that the fastlocals array didn't change in unexpected ways, so it'd be nice to hook this function and deoptimize if it's called. On a related note, Pyston doesn't currently need this but it seems to make sense to also make PyFrame_FastToLocals hookable as well in order to support different ways of representing local variables.

@carljm @itamaro @undingen

[markshannon]

• PyFrame_GetLineNumber(). You will need to keep the necessary metadata somewhere, wouldn't it be as cheap (or cheaper) to write the f_lasti value whenever execution escapes the jitted code?

• PyEval_SetProfile, PyEval_SetTrace I expect that these will be implemented using instrumentation, see PEP 669. Even if PEP 669 isn't accepted, we will use something like it internally to implement tracing and profiling.

• PyFrame_LocalsToFast and PyFrame_FastToLocals. Hopefully these will become obsolete, if either PEP 558 or PEP 667 are accepted. Although you might want to hook into the frame proxy objects instead.

[undingen]

Concerning PyFrame_GetLineNumber / f_lasti:

Many months a go I wrote a AMD64 POC for removing this f_lasti updates inside Pyston. This is how it worked:

• idea was that only a call instruction (cpu not bytecode) can user visible modify f_lasti (for a rare case where this is not true because it's completely inlined it needs to manually update f_lasti)
• on entry into a JIT compiled func: store the CPUs framepointer inside the Python frame object.
• on exit of a JIT compiled func: update f_lasti and clear the stored framepointer
• this makes is easy and fast to retrieve the IP of the next instruction by just looking at the fixed stack location which contains the IP of the next instruction after the call (this will not work for arm64 because it stores the return address in a register) without requiring unwinding info.
• during JIT compilation we generate a table which maps from the CPUs IP to f_lasti.
• when somebody retrieves f_lasti we can calculate it from the IP -> f_lasti table.
  This removed all of the f_lasti updates with a single store of the cpus frame pointer on entry. But made retrieving the f_lasti much slower.
  In the end it turned out to not help perf because the retrieval was very frequent and the implementation not optimized and having the code full of stores to f_lasti
  (everywhere where the JIT calls into a external function but this happens for nearly all bytecodes (and likely all when taking into account that decrefing a value can call a finalizer) has low execution time overhead.

Now that 3.11* replaced f_lasti with a pointer to the bytecode instruction this became more important because our JIT generated code does not want to permanently store a 64bit pointer to a memory address (loading a 64bit pointer immediate into a register on arm64 requires in many cases 4 instructions + 1 store (20bytes). If we switch to just loading it once and than updating via difference it would require 1load + 1add/sub + 1store instruction (=12bytes) which also sounds not good for something which needs to be done all the time)

So I think it makes sense to look at a optimization like the one outlined above so having a hook would be nice.

*I have not had a deep look yet at 3.11 so maybe this is not necessary.

[ericsnowcurrently]

• GIL handling -- I haven't thought through what the exact implementation would be, but it'd be nice to not have to poll eval_breaker like the interpreter does

Note that eval_breaker applies to more than just the GIL, at least in the main thread of the main interpreter.