[RELEASE-ONLY CHANGES] Update release/0.2 to pin and test against PyTorch release/2.3

[guangy10]

• CI: Pin to latest commit in release/2.3 (somehow directly pinning release/2.3 doesn't work)
• DevX: Update install_requirements.sh to install prebuilt torch-2.3 and torchvision/torchaudio.
• Fix ExecuTorch downstream to use the old API

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/2818

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 2a2df5e with merge base e5a8de0 ():

NEW FAILURES - The following jobs have failed:

• Apple / test-demo-ios / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 65
• pull / test-models-linux (cmake, llava_encoder, portable, linux.4xlarge, 90) / linux-job (gh)
  RuntimeError: Command docker exec -t da50fe65adbc896e03cf1631a2a03b647bc2bf709d7d11c259bcd5b247ac6180 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[guangy10]

Curious if this will be the final url to fetch torch2.3? Maybe it's where RCs are, but not for the final release?

[huydhn]

All the RCs are there on that link. Once it's promoted to prod, the final prod binary will come from https://download.pytorch.org/whl/cpu or PyPI

[huydhn]

Per my investigation in T184361451, I think we should avoid using -I as in pip install bitsandbytes -I as it seems to confuse pip list

[huydhn]

Could we have a python -c 'import torch; print(torch.__version__)' to confirm the torch version in used here? I'm curious

[huydhn]

Maybe python -c 'import torchvision; print(torchvision.__version__)' and python -c 'import torchaudio; print(torchaudio.__version__)' too. From what I try to reproduce locally, their versions looks correct (trying on ET main)

(executorch) [huydo@devgpu018.odn1 ~/local/github/executorch (main)]$ python -c 'import torchvision; print(torchvision.__version__)'
0.19.0.dev20240324+cpu
(executorch) [huydo@devgpu018.odn1 ~/local/github/executorch (main)]$ python -c 'import torchaudio; print(torchaudio.__version__)'
2.2.0.dev20240324+cpu

[guangy10]

Added the debug log and removed -I. However, I don't even see torch version = is printed hmm

[huydhn]

I could see it in the log though, i.e. https://github.com/pytorch/executorch/actions/runs/8542428080/job/23404038664?pr=2818#step:11:6208

[guangy10]

I could see it in the log though, i.e. https://github.com/pytorch/executorch/actions/runs/8542428080/job/23404038664?pr=2818#step:11:6208

Okay, now I can see the version seems correct now and the job is going further, though it fails the failure is due to memory limits. I think Martin was asking about this in a thread and there is solution for it?

[guangy10]

Will deal with T184361451 later.

[guangy10]

Updated third-party/pytorch to point to release/2.3, otherwise build from source will use the wrong version of torch.

[guangy10]

test-demo-ios / macos-job is flaky in all branches, it's irrelevant to this PR and the issue is tracked in #2877