[Core] Initial port of Ray to Python 3.13 #47984

pcmoritz · 2024-10-10T23:07:56Z

Why are these changes needed?

This is the first step towards #47933

It is not very tested at the moment (on Python 3.13), but it compiles locally (with pip install -e . --verbose) and can execute a simple workload like

>>> import ray
>>> ray.init()
2024-10-10 16:03:31,857	INFO worker.py:1799 -- Started a local Ray instance.
RayContext(dashboard_url='', python_version='3.13.0', ray_version='3.0.0.dev0', ray_commit='{{RAY_COMMIT_SHA}}')
>>> @ray.remote
... def f():
...     return 42
...     
>>> ray.get(f.remote())
42
>>>

(and similar for actors).

The main thing that needed to change to make Ray work on Python 3.13 was to upgrade Cython to 3.0.11 which seems to be the first version of Cython to support Python 3.13. Unfortunately it has a compiler bug cython/cython#3235 (the fix is not released yet) that I had to work around.

I also had to work around cython/cython#5750 by changing some typing from float to int | float.

Related issue number

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

aslonnie

(skipping review for now)

btakita · 2024-12-01T07:36:02Z

It looks like cython/cython#3235 was merged. Looking forward to 3.13 & 3.13t support soon!

btakita · 2025-01-18T21:57:33Z

Just checking in on how this is going. Are there any blockers other than doing the work to upgrade to 3.13t? Just curious if this is on the priority list & when 3.13t support will be available.

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

pcmoritz · 2025-02-04T08:24:45Z

This is hitting cython/cython#5750, since Cython unfortunately doesn't seem to fix it, we will have to work around it.

edoakes

Looks good to me. Can we leave a comment in unique_ids.pxd to undo the workarounds whenever it's possible?

python/ray/_raylet.pxd

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

## Why are these changes needed? After #47984 we can now build the wheels. Note that these are not tested yet, since unfortunately conda doesn't currently support Python 3.13 as the base environment (conda/conda#14353) and changing everything to run in a custom conda environment seems to require a decent amount of changes. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :(

This reverts commit d2004b6.

This reverts commit d2004b6. Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

## Why are these changes needed? After ray-project#47984 we can now build the wheels. Note that these are not tested yet, since unfortunately conda doesn't currently support Python 3.13 as the base environment (conda/conda#14353) and changing everything to run in a custom conda environment seems to require a decent amount of changes. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( Signed-off-by: 400Ping <43886578+400Ping@users.noreply.github.com>

## Why are these changes needed? This is the first step towards #47933 It is not very tested at the moment (on Python 3.13), but it compiles locally (with `pip install -e . --verbose`) and can execute a simple workload like ``` >>> import ray >>> ray.init() 2024-10-10 16:03:31,857 INFO worker.py:1799 -- Started a local Ray instance. RayContext(dashboard_url='', python_version='3.13.0', ray_version='3.0.0.dev0', ray_commit='{{RAY_COMMIT_SHA}}') >>> @ray.remote ... def f(): ... return 42 ... >>> ray.get(f.remote()) 42 >>> ``` (and similar for actors). The main thing that needed to change to make Ray work on Python 3.13 was to upgrade Cython to 3.0.11 which seems to be the first version of Cython to support Python 3.13. Unfortunately it has a compiler bug cython/cython#3235 (the fix is not released yet) that I had to work around. I also had to work around cython/cython#5750 by changing some typing from `float` to `int | float`. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Philipp Moritz <pcmoritz@gmail.com> Co-authored-by: pcmoritz <pcmoritz@anyscale.com> Co-authored-by: srinathk10 <68668616+srinathk10@users.noreply.github.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

## Why are these changes needed? After #47984 we can now build the wheels. Note that these are not tested yet, since unfortunately conda doesn't currently support Python 3.13 as the base environment (conda/conda#14353) and changing everything to run in a custom conda environment seems to require a decent amount of changes. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :(

## Why are these changes needed? This is the first step towards ray-project#47933 It is not very tested at the moment (on Python 3.13), but it compiles locally (with `pip install -e . --verbose`) and can execute a simple workload like ``` >>> import ray >>> ray.init() 2024-10-10 16:03:31,857 INFO worker.py:1799 -- Started a local Ray instance. RayContext(dashboard_url='', python_version='3.13.0', ray_version='3.0.0.dev0', ray_commit='{{RAY_COMMIT_SHA}}') >>> @ray.remote ... def f(): ... return 42 ... >>> ray.get(f.remote()) 42 >>> ``` (and similar for actors). The main thing that needed to change to make Ray work on Python 3.13 was to upgrade Cython to 3.0.11 which seems to be the first version of Cython to support Python 3.13. Unfortunately it has a compiler bug cython/cython#3235 (the fix is not released yet) that I had to work around. I also had to work around cython/cython#5750 by changing some typing from `float` to `int | float`. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Philipp Moritz <pcmoritz@gmail.com> Co-authored-by: pcmoritz <pcmoritz@anyscale.com> Co-authored-by: srinathk10 <68668616+srinathk10@users.noreply.github.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

## Why are these changes needed? After ray-project#47984 we can now build the wheels. Note that these are not tested yet, since unfortunately conda doesn't currently support Python 3.13 as the base environment (conda/conda#14353) and changing everything to run in a custom conda environment seems to require a decent amount of changes. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :(

## Why are these changes needed? This is the first step towards ray-project#47933 It is not very tested at the moment (on Python 3.13), but it compiles locally (with `pip install -e . --verbose`) and can execute a simple workload like ``` >>> import ray >>> ray.init() 2024-10-10 16:03:31,857 INFO worker.py:1799 -- Started a local Ray instance. RayContext(dashboard_url='', python_version='3.13.0', ray_version='3.0.0.dev0', ray_commit='{{RAY_COMMIT_SHA}}') >>> @ray.remote ... def f(): ... return 42 ... >>> ray.get(f.remote()) 42 >>> ``` (and similar for actors). The main thing that needed to change to make Ray work on Python 3.13 was to upgrade Cython to 3.0.11 which seems to be the first version of Cython to support Python 3.13. Unfortunately it has a compiler bug cython/cython#3235 (the fix is not released yet) that I had to work around. I also had to work around cython/cython#5750 by changing some typing from `float` to `int | float`. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Philipp Moritz <pcmoritz@gmail.com> Co-authored-by: pcmoritz <pcmoritz@anyscale.com> Co-authored-by: srinathk10 <68668616+srinathk10@users.noreply.github.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

## Why are these changes needed? After ray-project#47984 we can now build the wheels. Note that these are not tested yet, since unfortunately conda doesn't currently support Python 3.13 as the base environment (conda/conda#14353) and changing everything to run in a custom conda environment seems to require a decent amount of changes. ## Related issue number  ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :(

Initial port of Ray to Python 3.13

7c6fa31

pcmoritz requested review from richardliaw, edoakes and aslonnie as code owners October 10, 2024 23:07

pcmoritz mentioned this pull request Oct 10, 2024

[Core] Experimental support for no-GIL build and JIT in Python 3.13 #47933

Open

cleanup

57f5b62

aslonnie reviewed Oct 11, 2024

View reviewed changes

anyscalesam added triage Needs triage (eg: priority, bug/not-bug, and owning component) core Issues that should be addressed in Ray Core labels Oct 16, 2024

jjyao removed the triage Needs triage (eg: priority, bug/not-bug, and owning component) label Oct 28, 2024

ecederstrand mentioned this pull request Nov 25, 2024

[Feature] Core: Python 3.12 wheel #40211

Closed

pcmoritz and others added 4 commits January 22, 2025 16:58

Merge branch 'master' into py313

1d16140

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

update

f76f5ae

Merge branch 'py313' of github.com:ray-project/ray into py313

f1bcd5a

update

500ffc0

pcmoritz requested a review from a team as a code owner January 23, 2025 02:51

fix

b0299b4

pcmoritz added the go add ONLY when ready to merge, run all tests label Jan 23, 2025

pcmoritz and others added 4 commits January 22, 2025 21:09

update

e9aae5f

update

806d4e1

Merge branch 'master' into py313

401ade4

Merge branch 'master' into py313

5c8637d

update

05f7185

pcmoritz requested review from hongchaodeng and a team as code owners February 4, 2025 08:25

pcmoritz added 2 commits February 4, 2025 10:40

update

79ab618

update

ad3f194

pcmoritz added 4 commits February 4, 2025 17:08

fix

d53c19a

clean up

8e9dcc1

update

2620c0e

cleanup

768e8c7

pcmoritz changed the title ~~[WIP] Initial port of Ray to Python 3.13~~ Initial port of Ray to Python 3.13 Feb 5, 2025

pcmoritz changed the title ~~Initial port of Ray to Python 3.13~~ [Core] Initial port of Ray to Python 3.13 Feb 5, 2025

pcmoritz assigned jjyao and MengjinYan Feb 5, 2025

edoakes reviewed Feb 12, 2025

View reviewed changes

python/ray/_raylet.pxd Outdated Show resolved Hide resolved

edoakes approved these changes Feb 12, 2025

View reviewed changes

pcmoritz and others added 4 commits February 12, 2025 18:22

Update python/ray/_raylet.pxd

a6c1696

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>

Merge branch 'master' into py313

8a58d3f

update cython

b9bd91c

add comment

01f5211

pcmoritz enabled auto-merge (squash) February 13, 2025 07:00

pcmoritz merged commit d2004b6 into master Feb 13, 2025
6 checks passed

pcmoritz deleted the py313 branch February 13, 2025 08:08

pcmoritz mentioned this pull request Feb 13, 2025

Build Python 3.13 wheels #50531

Merged

8 tasks

edoakes added a commit to edoakes/ray that referenced this pull request Feb 19, 2025

Revert "[Core] Initial port of Ray to Python 3.13 (ray-project#47984)"

d848cb3

This reverts commit d2004b6.

edoakes added a commit to edoakes/ray that referenced this pull request Feb 19, 2025

Revert "[Core] Initial port of Ray to Python 3.13 (ray-project#47984)"

205bd35

This reverts commit d2004b6.

edoakes added a commit to edoakes/ray that referenced this pull request Feb 19, 2025

Revert "[Core] Initial port of Ray to Python 3.13 (ray-project#47984)"

94cefe2

This reverts commit d2004b6. Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>

chenkins mentioned this pull request Feb 21, 2025

116 Add support for py 3.13 flatland-association/flatland-rl#121

Draft

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Core] Initial port of Ray to Python 3.13 #47984

[Core] Initial port of Ray to Python 3.13 #47984

pcmoritz commented Oct 10, 2024 •

edited

Loading

aslonnie left a comment

btakita commented Dec 1, 2024 •

edited

Loading

btakita commented Jan 18, 2025

pcmoritz commented Feb 4, 2025 •

edited

Loading

edoakes left a comment

[Core] Initial port of Ray to Python 3.13 #47984

[Core] Initial port of Ray to Python 3.13 #47984

Conversation

pcmoritz commented Oct 10, 2024 • edited Loading

Why are these changes needed?

Related issue number

Checks

aslonnie left a comment

Choose a reason for hiding this comment

btakita commented Dec 1, 2024 • edited Loading

btakita commented Jan 18, 2025

pcmoritz commented Feb 4, 2025 • edited Loading

edoakes left a comment

Choose a reason for hiding this comment

pcmoritz commented Oct 10, 2024 •

edited

Loading

btakita commented Dec 1, 2024 •

edited

Loading

pcmoritz commented Feb 4, 2025 •

edited

Loading