Exponential backoff in counters

[markshannon]

Mentioned here #332 (comment)

The idea is to split the 16 bit counter for specialized instructions into a 12 bit (high bits) counter and 4 bit backoff counter (low bits).
Each time we fail to specialize an instruction the backoff counter is increased by one (up to 12), and the counter set to (1<<backoff)-1
If we successfully specialize something, we reset the backoff to its starting value.

Currently when we set the counter to ADAPTIVE_CACHE_BACKOFF which is 64.
Given that we now have backoff, we could start with a lower value of 32, i.e. a backoff of 5.

If we hit an instruction that cannot specialize a 10000 times, then currently we attempt to specialize it 160 times.
With exponential backoff we would only attempt to specialize it 10 times.

We didn't implement this in the original PEP 659, as we only had 8 bit counters.
Now we have 16 bit counters, we should do this.

I'd like to get this into 3.11 to reduce slowdowns where we fail to specialize, but that's ultimately up to @pablogsal

[markshannon]

python/cpython#93355