Add Python 3.11 support #27264
Add Python 3.11 support #27264
Conversation
boring-cyborg
bot
added
area:dev-tools
area:production-image
potiuk
requested review from
eladkal,
ashb,
BasPH,
dstandish,
ephraimbuddy,
feluelle,
ferruzzi,
Fokko,
jedcunningham,
kaxil,
mik-laj,
o-nikolas,
uranusjr,
XD-DENG and
xinbinhuang
|
There you go.
|
|
Really cool. Unfortunately pyarrow is currently not compatible with 3.11 😐 |
|
Related apache/arrow#14499 |
potiuk
force-pushed
the
python-3-11
branch
2 times, most recently
from
2371ed5
to
0642e35
Compare
|
All is ready. All tests are passing, everything looks good. Now, the only thing we need is, we need to wait for the #31322 to be released. Currently it is impossible to find the right "provider" packages to install by airflow and generate constraints for "pypi" installation - that's why I had to cancel the constraint generation that run for ~ 1hr - because some of our providers (transitively) are not compatible with Python 3.11. Once the providers get released, they will not yet have the So Python 3.11 support is now in the hands of those who test and vote on providers (cc: @eladkal) |
| from airflow import PY311 | ||
|
|
||
| if PY311: | ||
| pytest.skip( | ||
| "The tests are skipped because Apache Hive provider is not supported on Python 3.11", | ||
| allow_module_level=True, | ||
| ) |
There was a problem hiding this comment.
Should this use pytestmark = pytest.mark.skipif(...) instead?
|
|
||
| with pytest.raises(ParamValidationError, match=error_pattern): | ||
| Param("21 May 1975", type="string", format="date").resolve() | ||
| @pytest.mark.parametrize( |
potiuk
force-pushed
the
python-3-11
branch
2 times, most recently
from
41dceba
to
2b75487
Compare
Python 3.11 has been released as scheduled on October 25, 2022 and this is the first attempt to see how far Airflow (mostly dependencies) are from being ready to officially support 3.11.
|
FINALLY GREEN... MERGING |
| | MSSQL | 2017(\*), 2019(\*) | 2017(\*), 2019(\*) | | ||
| | | Main version (dev) | Stable version (2.6.1) | | ||
| |------------|------------------------------|---------------------------| | ||
| | Python | 3.7, 3.8, 3.9, 3.10, 3.11 | 3.7, 3.8, 3.9, 3.10, 3.11 | |
There was a problem hiding this comment.
Are we supporting Python 3.11 in 2.6.1?
Add Python 3.11 support Python 3.11 has been released as scheduled on October 25, 2022 and finally, after all dependencies got upgraded - we can support it. --------- Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> (cherry picked from commit c5597d1)
The official Python image for 3.11 had a brief period when the date `20120503` has been an acceptable date, but it is not accepted as of 3.11.4 release on 5th of July. This PR reverts the changes in tests implemented in apache#27264 to accomodate for it and brings back the date as error case.
The official Python image for 3.11 had a brief period when the date `20120503` has been an acceptable date, but it is not accepted as of 3.11.4 release on 5th of July. This PR reverts the changes in tests implemented in #27264 to accomodate for it and brings back the date as error case.
Python 3.11 has been released as scheduled on October 25, 2022 and
this is the first attempt to see how far Airflow (mostly dependencies)
are from being ready to officially support 3.11.
The fio
Support for Python 3.11 databricks/databricks-sql-python#59
and Support for Python 3.11 Azure/azure-sdk-for-python#27066
We might decide to release Airflow in 3.11 with those providers
disabled in case they are lagging behind eventually, but for the
moment we want to work with all the projects in concert to be
able to release all providers (Google Provider requires quite
a lot of work and likely Google Team stepping up and community helping
with migration to latest Goofle cloud libraries)
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.