Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Fix windows tests disk space failure #3122

Merged
merged 1 commit into from
Apr 12, 2022

Conversation

cnpryer
Copy link
Contributor

@cnpryer cnpryer commented Apr 11, 2022

Fixes: #2783

Problem

The windows tests were failing due to insufficient disk space because of cargo management on the D: drive. Unlike other OS options, the windows GHA option uses the D: drive which has less capacity than the C: drive.

Solution

Use Swatinem/rust-cache@v1 or similar logic to manage caching deps

NOTE: I modified the on to trigger this GHA in order to demonstrate the passing test. Before merging I can pull this out.

Other

While looking into this I noticed:

  • windows-2019 is often associated with random issues (changed to windows-latest in this PR)
  • The first pass I submitted used a hacky workaround based on info from the issue I linked here

I'm curious how well this will hold up over time since nightly is used here. In case there's a need to fallback to the workaround, something along these lines (+ some cleaning logic) could probably be used:

jobs:
  test-rust:
    runs-on: windows-latest
    steps:
      - name: Install latest Rust nightly
        uses: actions-rs/toolchain@v1
        with:
          toolchain: nightly-2022-04-01
          override: true
      - name: Checkout branch
        run: |
          git clone ${{ github.event.pull_request.head.repo.full_name }} C:/polars --depth 1
      - uses: Swatinem/rust-cache@v1
        with:
          working-directory: C:/polars
      - name: Run tests
        working-directory: C:/polars
        run: |
          rustup override set nightly-2022-04-01
          git checkout ${{ github.event.pull_request.head.sha }}
          set RUSTFLAGS="-C debuginfo=0"
          cd polars && make test && make integration-tests

Note the cloning and checkout need to be further validated.

Other Resources:

@cnpryer cnpryer marked this pull request as draft April 12, 2022 00:13
@cnpryer cnpryer force-pushed the gha-windows-failure branch 5 times, most recently from 26e54c9 to 2996f9d Compare April 12, 2022 01:17
@cnpryer cnpryer marked this pull request as ready for review April 12, 2022 02:10
@cnpryer cnpryer force-pushed the gha-windows-failure branch from 2996f9d to b9a978b Compare April 12, 2022 02:13
@cnpryer cnpryer force-pushed the gha-windows-failure branch from b9a978b to d04c327 Compare April 12, 2022 02:48
@ritchie46
Copy link
Member

Thanks a lot! So as I understand it, a hack was copying to C:/ and after updating to latest it wasn't needed anymore?

@ritchie46 ritchie46 merged commit 13040f9 into pola-rs:master Apr 12, 2022
@cnpryer
Copy link
Contributor Author

cnpryer commented Apr 12, 2022

The workaround fixed it but probably only because the c drive had more capacity. Upgrading to windows latest wasn't the fix, I threw that in there because I kept seeing 2019 be related to other issues (this hopefully prevents problems in future).

Swatinem/rust-cache@v1 looks like it's the more robust fix.

The Swatinem/rust-cache@v1 action cleans up cargo management by adding "smart caching". So we no longer hit capacity on The d drive (we'll see how long this lasts).

A bunch of projects use it for the same problem that we saw here. The only catch I think is that with nightly the caches will be dropped with each new version that's used. So I'll keep my eye on this. Not sure if it cleans dropped caches.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: Windows runner fail because we are out of disk space.
2 participants