Expand pyatomic coverage

[colesbury]

Feature or enhancement

Implementing PEP 703 requires use of atomic operations on more data types than provided by pycore_atomic.h. Additionally, pycore_atomic.h is only usable from Py_BUILD_CORE modules; it can't be used in public headers. PEP 703 will require atomic operations in object.h for Py_INCREF/DECREF, for example.

I propose adding a new header to wrap platform/C11 provided atomic operations:

• The functions will be exposed in the public C-API but prefixed with an underscore to indicate they are private. They need to be exposed because they will be used by other public inline functions.
• The functions operate on plain data types (i.e. int instead of _Py_atomic_int). This allows use of atomic operations on existing data types where we don't want to change the types of contained fields (i.e, the fields of PyTypeObject.) and avoids some issues regarding mixing C and C++.
• Use of GCC built-in atomics when available. These are available in GCC 4.8+. They do not require passing -std=gnu11 or -std=c11 to the compiler, which avoids breaking third-party extensions.
• The memory order is part of the function name instead of an argument. This means there are more function definitions (e.g., both _Py_atomic_load_int and _Py_atomic_load_int_relaxed) but keeps the implementation of each function simpler, particularly for MSVC. This reduces boiler-plate code when stepping through debug builds.

Linked PRs

• gh-108337: Add pyatomic.h header #108338
• gh-108337: Add pyatomic.h header #108701

[scoder]

The leading underscore marks these as "private". However, with the advent of a GIL-free Python, these will become increasingly useful for third-party code. Why consider them private?

[colesbury]

There is ongoing discussion about whether they should be made public, but currently no consensus. Some of the reasons to keep them private:

• When MSVC finally adds support C11 atomics, we may want to use C11 atomics directly or at least simplify the API.
• Many extensions have better options, such as Rust atomics or C11 or C++11 atomics
• Some maintainers believe that it is out of scope for CPython: basically that CPython shouldn't be providing cross-platform wrappers for OS or compiler APIs, but limited to focusing on APIs that interact with CPython.

If we maintained a similar header-only atomics library outside this repo, would that be useful for Cython? Basically, something like https://github.com/python/pythoncapi-compat.