Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable ARM Windows builds in release pipeline #1141

Closed
charliermarsh opened this issue Jan 27, 2024 · 19 comments · Fixed by #10885
Closed

Re-enable ARM Windows builds in release pipeline #1141

charliermarsh opened this issue Jan 27, 2024 · 19 comments · Fixed by #10885
Labels
wish Not on the immediate roadmap

Comments

@charliermarsh
Copy link
Member

charliermarsh commented Jan 27, 2024

See: https://github.com/astral-sh/puffin/actions/runs/7673412766 (failure to build libgit2).

See: https://github.com/astral-sh/puffin/actions/runs/7673603363 (failure to build libgit2, even after enabling vendoring).

@charliermarsh charliermarsh added the bug Something isn't working label Jan 27, 2024
@konstin
Copy link
Member

konstin commented Jan 27, 2024

Does this mean we want support aarch64 windows? Then we need e.g. launcher binaries and some testing that puffin actually works.

@charliermarsh
Copy link
Member Author

I'll turn this into a more general issue. This was just about getting CI building but that makes sense. I assume I should similarly disable the i686 Windows then?

@charliermarsh charliermarsh added wish Not on the immediate roadmap and removed bug Something isn't working labels Jan 29, 2024
@konstin
Copy link
Member

konstin commented Jan 30, 2024

I don't understand which of the errors was the fatal one. On linux with cargo xwin build --target x86_64-pc-windows-msvc, libgit2 compile but then the linking fails with

lld-link: error: could not open 'msvcrtd.lib': No such file or directory

@messense
Copy link
Contributor

messense commented Mar 28, 2024

@konstin This is because cargo-xwin does not keep debug libs by default, you can pass --xwin-include-debug-libs to keep them, or build in release mode instead.

FYI, msvcrtd.dll is a "debug"-version of the file msvcrt.dll.

@konstin
Copy link
Member

konstin commented Mar 28, 2024

Thank you, release mode indeed works!

@henryiii
Copy link
Contributor

Any plans here? Did it just need to be built in release mode? Windows ARM is another platform cibuildwheel supports. I also have a Windows ARM dev box that I'd like to be able to use uv on. :)

@konstin
Copy link
Member

konstin commented May 14, 2024

The trampolines are committed and ready. Last time we were investigating arm windows we were told that not even microsoft themselves properly support it, so we just haven't made CI builds for a priority.

@matterhorn103
Copy link

If you guys would be helped at all by someone testing or compiling on a Windows ARM machine, I have one and am happy to help!

@darthtrevino
Copy link

So with the advent of the Snapdragon X chips, we're seeing Windows ARM machines become more commonplace. Older ARM chips were pretty lame, but they're dramatically improving, and dev ecosystem support is too. I have one team member who uses Windows ARM as their personal dev machine.

@zanieb
Copy link
Member

zanieb commented Oct 17, 2024

If someone wants to put some effort into getting an ARM build working I'm all for it. I think we could start with something low impact like the #8269 build then move to distributing releases as a secondary goal.

@kalradivyanshu
Copy link
Contributor

kalradivyanshu commented Jan 4, 2025

@zanieb can you give me some guidance on how to go about it? I have a snapdragon laptop and heavily use UV, would love to contribute.

@zanieb
Copy link
Member

zanieb commented Jan 4, 2025

@kalradivyanshu

I added a runner called github-windows-11-aarch64-4 to the org. I think the idea would basically be to copy the following job with the aarch64 runner and call it build-binary-windows-aarch64 (as we do for macOS)

uv/.github/workflows/ci.yml

Lines 529 to 560 in 9d417da

build-binary-windows:
needs: determine_changes
timeout-minutes: 10
if: ${{ github.repository == 'astral-sh/uv' && !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
runs-on:
labels: windows-latest-large
name: "build binary | windows"
steps:
- uses: actions/checkout@v4
- name: Create Dev Drive using ReFS
run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1
# actions/checkout does not let us clone into anywhere outside ${{ github.workspace }}, so we have to copy the clone...
- name: Copy Git Repo to Dev Drive
run: |
Copy-Item -Path "${{ github.workspace }}" -Destination "${{ env.UV_WORKSPACE }}" -Recurse
- uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ env.UV_WORKSPACE }}
- name: "Build"
working-directory: ${{ env.UV_WORKSPACE }}
run: cargo build
- name: "Upload binary"
uses: actions/upload-artifact@v4
with:
name: uv-windows-${{ github.sha }}
path: ${{ env.UV_WORKSPACE }}/target/debug/uv.exe
retention-days: 1

@kalradivyanshu
Copy link
Contributor

So @zanieb basically, I need to clone the uv repo, run the steps to build it on my local machine, fix any bugs, and then write:

uv/.github/workflows/ci.yml

Lines 529 to 560 in 9d417da

build-binary-windows:
needs: determine_changes
timeout-minutes: 10
if: ${{ github.repository == 'astral-sh/uv' && !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
runs-on:
labels: windows-latest-large
name: "build binary | windows"
steps:
- uses: actions/checkout@v4
- name: Create Dev Drive using ReFS
run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1
# actions/checkout does not let us clone into anywhere outside ${{ github.workspace }}, so we have to copy the clone...
- name: Copy Git Repo to Dev Drive
run: |
Copy-Item -Path "${{ github.workspace }}" -Destination "${{ env.UV_WORKSPACE }}" -Recurse
- uses: Swatinem/rust-cache@v2
with:
workspaces: ${{ env.UV_WORKSPACE }}
- name: "Build"
working-directory: ${{ env.UV_WORKSPACE }}
run: cargo build
- name: "Upload binary"
uses: actions/upload-artifact@v4
with:
name: uv-windows-${{ github.sha }}
path: ${{ env.UV_WORKSPACE }}/target/debug/uv.exe
retention-days: 1
for windows arm?

Is this the right approach?

@zanieb
Copy link
Member

zanieb commented Jan 5, 2025

That sounds correct to me, yep!

@kalradivyanshu
Copy link
Contributor

Ok @zanieb I was able to get uv installed for arm, and then after downloading python for arm windows, everything seems to be working fine, all I had to do was get clang, arm visual studio installed -> rust installed -> clone uv -> cargo build --release --target aarch64-pc-windows-msvc

It worked without any bugs/hiccups.

Do you think:

 build-binary-windows-arm:
    needs: determine_changes
    timeout-minutes: 10
    if: ${{ github.repository == 'astral-sh/uv' && !contains(github.event.pull_request.labels.*.name, 'no-test') && (needs.determine_changes.outputs.code == 'true' || github.ref == 'refs/heads/main') }}
    runs-on:
      labels: windows-latest-large
    name: "build binary | windows"
    steps:
      - uses: actions/checkout@v4

      - name: Create Dev Drive using ReFS
        run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1

      # actions/checkout does not let us clone into anywhere outside ${{ github.workspace }}, so we have to copy the clone...
      - name: Copy Git Repo to Dev Drive
        run: |
          Copy-Item -Path "${{ github.workspace }}" -Destination "${{ env.UV_WORKSPACE }}" -Recurse

      - uses: Swatinem/rust-cache@v2
        with:
          workspaces: ${{ env.UV_WORKSPACE }}

      - name: "Build"
        working-directory: ${{ env.UV_WORKSPACE }}
        run: cargo build --release --target aarch64-pc-windows-msvc

      - name: "Upload binary"
        uses: actions/upload-artifact@v4
        with:
          name: uv-windows-${{ github.sha }}
          path: ${{ env.UV_WORKSPACE }}/target/debug/uv.exe
          retention-days: 1

will work? Will run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1 have rust for windows arm, or will that need to be modified?

@zanieb
Copy link
Member

zanieb commented Jan 5, 2025

Feel free to just post the PR and we can see if CI works.

    runs-on:
      labels: windows-latest-large
    name: "build binary | windows"

You need to change the runs-on to the arm runner and add aarch64 to the name. We prefer aarch64 over arm, as it's the official terminology.

Will run: ${{ github.workspace }}/.github/workflows/setup-dev-drive.ps1 have rust for windows arm, or will that need to be modified?

Presumably it will work — if not we can just drop the dev-drive.

@kalradivyanshu
Copy link
Contributor

@zanieb Can you tell me the label for windows aarch64 runner? According to this https://github.blog/news-insights/product-news/arm64-on-github-actions-powering-faster-more-efficient-build-systems/#get-started-using-arm-hosted-runners-today and https://www.youtube.com/watch?v=cgI6SBP8pEM it is only available to enterprise github accounts, does uv have access?

@zanieb
Copy link
Member

zanieb commented Jan 5, 2025

I set one up, see #1141 (comment)

@kalradivyanshu
Copy link
Contributor

@zanieb , can you take a look at this #10306?

zanieb added a commit that referenced this issue Jan 23, 2025
Based on discussion in #10306, this
adds building aarch64 exe using x86 runner. See:
#10402 (comment)

Addresses: #1141

---------

Co-authored-by: Zanie Blue <contact@zanie.dev>
zanieb added a commit that referenced this issue Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wish Not on the immediate roadmap
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants