Merge micropython v1.19.1

[dhalbert]

Merging MicroPython v1.19.1

Just before this PR was pushed, my manuals tests showed that the silabs port fails when compiling, and some tests fail when make test is done in port/unix. The other ports are compiling. I did some smoke tests on a CPX: a program that played Jingle Bells and made NeoPixel animations happened to be on that CPX, so that tested a bunch of things including frozen modules and generators. Also I did some simple gc testing by creating large bytearrays.

More failures may appear when the CI runs.

Major changes from upstream

• New macro mp_obj_malloc() allocates an object and sets its type. This was straightforward to incorporate.
• The REGISTER_MODULE mechanism has changed extensively; took changes, causing many changes in makemoduledefs.py, mkrules.mk, and other places.
• MICROPY_MODULE_ATTR_DELEGATION features added. Lots of churn in objmodule.c.
• qstr storage is different. QENUM is now QDEF. Qstr lengths and hashes were factored out into separate arrays, and there's a lot of qstr churn in the .mk files.

Major changes made that affect CircuitPython-only features

• Long-lived object allocation removed. fun_bc objects now have a context field, which is const. This caused problems when long-lived objects were moved in the course of recursive long-living. I have some ideas about doing long-lived more explicitly only on compiled objects when they are allocated, instead of moving them later. This would require passing down "please long-live objects you create" flags.
• Removed Fix two watchdog crashes #5272 feature (CIRCUITPY_PRECOMPUTE_QSTR_ATTR) due to revamping of qstrs.
• Track first free atbs for multiple block sizes instead of just 1 #2614 removed.
• Undo Some flash size optimizations related to string0.c (implementation of str/mem functions) #6397 for now, because it was enmeshed in in Track first free atbs for multiple block sizes instead of just 1 #2614 changes.
• gc_auto_collect_enabled was uint16_t in MicroPython. We changed it to bool. I changed it back for less churn but why is it a uint16_t in MicroPython?
• translations: compression.generated.h renamed to compressed_translations.h, to be clearer. MicroPython now also has compression of messages, but not translation, and it's done in a different way. We might look at it and see which scheme works out better, including for non-ASCII languages.

Tooling

• make V= is now any combination of these space-separated keywords: "steps commands rules" instead of the old 0, 1, or 2. rules is new and gives you the makefile line that is doing the step.

Attempts made to get closer to MicroPython sources

• I tried to remove gratuitous changes we had made to make the sources, especially in py/ be closer to MicroPython. For instance, "SPDX" copyright lines were restored to the original.
• I restored '#if'dcode from upstream that we will never use, such asdupterm` features that we always have turned off. This makes for fewer diffs between files.

For v1.20 merge

• Our PR gc.c: Ensure a gap of one byte before the finaliser table. #5245 made a bunch of changes in gc.c. These were incorporated, and will be in the v1.20 merge.
• Watch for lib/re1.5 changes after v1.19.1. We have changes as well, so this merge may be complicated.
• I incorporated @jepler's type object space-savings ("extended fields"). This was done in a somewhat different way in v1.20, and I think we are just going to take that way when we get there.
• In 'ulab, CircuitPython-specific toggle for module registration can be removed.

Things still to fix

• Silabs: The silabs port does not build. The port has make-rule differences from the other ports, to accommodate idiosyncrasies in the silabs SDK. For instance, their ECHO is different from ours. These seem to be breaking dependency ordering for the genhdr generated files. This causes genhdr/mpversion.h not to be built in time. I tried to fix, but didn't succeed yet.
• The dependency ordering for qstr generated and genhdr is kind of a mess in general. his is partly inherited from upstream. There are many order dependencies, maybe too many, and I think this maybe could be cleaned up. See tricky FORCE usage, and SRC_QSTR and OBJ_EXTRA_ORDER_DEPS usage. I had to add OBJ_EXTRA_ORDER_DEPS to get some things build in the right order. I think it could be simpler.
  I think there may still be some non-determinancy, which might be what is breaking the silabs build. Try building -j1 vs -j<n>.
• Tests failed locally. The viper tests are running, though I have tried to turn them off (and they were turned off before in our builds).
• I don't know if we care about the "native" code generation enough to try to make it work. We made changes in generators and tracebacks which may need extra work in the native code implementation to work.

Aspirations

• I would like to see fewer differences between what we have and upstream, especially in py/ and extmod/.
• Let's try to PR our changes upstream when possible, including minor things like spelling fixes. Let's try to keep in sync more. If upstream takes our changes, but with edits, let's incorporate those edits. Major things we might contribute back:
  • Our fancier tracebacks
  • generator improvments
  • async/await impl, which includes an explicit __await__()
  • module attributes or some property stuff -- I forget the details.

Notes

• I made one pass with git mergetool, and then did another pass with manual merging to re-check things, because the auto-merging was sometimes wrong, and sometimes I wanted to revert more things to what was in upstream (like copyright lines.)
  This little alias was useful. adafruit/circuitpython is set to main, circuitpython is where I was merging, and micropython was set to v1.19.1.

3meld() {
    pushd ~/repos
    meld adafruit/circuitpython/$1 circuitpython/$1 micropython/$1 &
    popd
}

[jepler]

yay we've finally gotten to building boards during CI :) 🚀

[dhalbert]

Tested on a bunch of representative boards. Not tested on broadcom or silabs. Spresense failed, but I think that may be an SDK version skew problem, not necessarily related to this PR. All others passed a simple smoke test: random program running or microcontroller.cpu.temperature.

[jepler]

It's ready! .. for others to test

[dhalbert]

The merge doc build is failing in a weird way: https://github.com/adafruit/circuitpython/actions/runs/5932484043/job/16088305509#step:8:1

[bill88t]

So far I have found 0 breaking changes (other than the need to regen all .mpy's).

.mpy sizes reduced a good bit:
mpy-cross-8.1.0: 73.3kb
mpy-cross-8.2.0-87-ge08ad22ef (current master): 68.0kb

Speed to have increased a bit, but I still haven't made benchmarks.