Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-95913: Edit, expand & format Bytecode sect in 3.11 WhatsNew #98559

Merged
merged 13 commits into from
Oct 24, 2022

Conversation

CAM-Gerlach
Copy link
Member

@CAM-Gerlach CAM-Gerlach commented Oct 22, 2022

Part of #95913

This PR expands, edits and formats the CPython Bytecode Changes section in the What's New in Python 3.11 document.

On the editing side, like with the other PRs, it:

  • Uses clear, consistent phrasing on all bytecode entries, making them easier to scan and read
  • Orders them more consistently (Added, Replaced, Removed, Changed, roughly alphabetical by opcode name within each, aside from special cases), for the same reasons
  • Fixes and improves the Sphinx syntax and semantics

Likewise, it adds some missing additional opcodes that were added, replaced or removed in 3.11:

  • Added COPY_FREE_VARS, MAKE_CELL, PUSH_EXC_INFO, PREP_RE_RAISE_STAR, RETURN_GENERATOR, SEND and ASYNC_GEN_WRAP
  • Replaced SETUP_WITH and SETUP_ASYNC_WITH with BEFORE_WITH
  • Removed POP_BLOCK, SETUP_FINALLY and YIELD_FROM

Finally, it consolidates the sets of replaced opcodes into a much quicker and easier to navigate table.

Rendered Preview

--> Rendered Preview (click to expand) <---

image

@bedevere-bot bedevere-bot added awaiting review docs Documentation in the Doc dir skip news labels Oct 22, 2022
@CAM-Gerlach CAM-Gerlach added 3.11 only security fixes needs backport to 3.11 only security fixes labels Oct 22, 2022
Comment on lines 1518 to 1522
Changed Opcodes
---------------

* Changed :opcode:`JUMP_IF_TRUE_OR_POP` and :opcode:`JUMP_IF_FALSE_OR_POP`
to now be relative rather than absolute.
Copy link
Member Author

@CAM-Gerlach CAM-Gerlach Oct 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left the Changes section as it was for now (aside from minor technical edits), but there were a handful of additional Opcode changes of what seemed to be roughly comparable magnitude to those mentioned that weren't noted here, mostly grouped by a few specific common alterations, that we could include in a final short table (with columns Opcode(s) and Change), if you feel its worth including.

Here are the additional changes that could be mentioned:

List of additional changes

Added where oparg

  • GET_AWAITABLE

Exception representation on the stack now consist of one, not three, items

  • END_ASYNC_FOR
  • POP_EXCEPT
  • RERAISE
  • WITH_EXCEPT_START

The exit function is in position 4 of the stack rather than 7.

  • WITH_EXCEPT_START

If the low bit of namei is set, then a NULL is pushed to the stack before the global variable.

  • LOAD_GLOBAL

i is no longer offset by the length of co_varnames

  • LOAD_CLOSURE
  • LOAD_DEREF
  • LOAD_CLASSDEREF
  • STORE_DEREF
  • DELETE_DEREF

@CAM-Gerlach CAM-Gerlach mentioned this pull request Oct 22, 2022
33 tasks
@pablogsal
Copy link
Member

This should be also checked with @markshannon, @brandtbucher, @iritkatriel and @ericsnowcurrently.

If this doesn't land today, it won't be included in 3.11.0

@CAM-Gerlach CAM-Gerlach requested review from iritkatriel, markshannon and ericsnowcurrently and removed request for JelleZijlstra October 23, 2022 03:25

* :opcode:`JUMP_IF_TRUE_OR_POP` and :opcode:`JUMP_IF_FALSE_OR_POP` are now
relative rather than absolute.
.. _whatsnew311-added-opcodes:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list we have now has all the jump related changes together, but if you split it into a "added" table and a "replaced" table then they are no longer together. I don't know if that's an improvement.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created an alternative PR -#98566. It tries to consolidate related items into a high level descriptions, rather than enumerate all the changes. Maybe this is more useful for whatsnew. The full details, if you need them, as in the dis doc.

I added an item about the exc-info related changes. Didn't do anything about these:

GET_AWAITABLE

WITH_EXCEPT_START
If the low bit of namei is set, then a NULL is pushed to the stack before the global variable.

LOAD_GLOBAL
i is no longer offset by the length of co_varnames

LOAD_CLOSURE
LOAD_DEREF
LOAD_CLASSDEREF
STORE_DEREF
DELETE_DEREF

Copy link
Member Author

@CAM-Gerlach CAM-Gerlach Oct 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great points, thanks. I've incorporated the changes suggested via your PR, specifically:

  • Consolidated the jump opcode additions, replacements and changes (except for the NO_INTERRUPT opcode addition, which didn't appear to be closely related to the others) into a single Replaced table entry with a linked note
  • Just linked the high-level jump opcode name patterns, instead of every opcode.
  • Added a "Changed" note about the exception stack representation
  • (EXC/EG changes were already consolidated)

Beyond that, I've given this PR a major revamp based on your feedback above:

  • Converted the "Added" section back to (much shorter) bullets
  • Inlined the previous separate notes into the Replaced table
  • Combined the previous "Removed" and "Changed" sections into one, condensing the Removed items into a single bullet to take up much less space
  • Made additional textual refinements, particularly in the Changed entries, to be both more clear and more concise

In total, this retains nearly the same space consumption of the previous dense wall of flat bullets format (once allowing for the missing substantive additions, replacements and removals added here), while being much easier for the reader to quickly navigate, find what they're looking for and jump to more information in the dis docs.

Flat Bullets Initial PR Revised PR
image image image

Doc/whatsnew/3.11.rst Outdated Show resolved Hide resolved
Doc/whatsnew/3.11.rst Outdated Show resolved Hide resolved
@iritkatriel
Copy link
Member

Yeah this is better! Just CHECK_EG_MATCH is also new for PEP 654.

@pablogsal
Copy link
Member

Heads up: please land this before 3Pm GMT+0 if you want it to be in 3.11.0

@miss-islington
Copy link
Contributor

Thanks @CAM-Gerlach for the PR, and @iritkatriel for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 24, 2022
…ythonGH-98559)

(cherry picked from commit 22739a0)

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
@bedevere-bot
Copy link

GH-98597 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Oct 24, 2022
miss-islington added a commit that referenced this pull request Oct 24, 2022
)

(cherry picked from commit 22739a0)

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
@iritkatriel
Copy link
Member

Heads up: please land this before 3Pm GMT+0 if you want it to be in 3.11.0

It's merged and backported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes docs Documentation in the Doc dir skip news
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants