ci: update structure of wheel storage

[raunakab]

Overview

Previously, wheels were stored in AWS S3 as: s3://bucket/builds/$COMMIT_HASH/$WHEEL_NAME. The wheel name would be some name following the PEP 427 standard. You can have multiple wheels stored inside of /builds/$COMMIT_HASH.

Having multiple wheels stored inside of the /builds/$COMMIT_HASH meant that you would need to employ some additional logic to figure out:

• the wheel name for the platform on which you want to run on
• which version of python you want to run with
• which ABI you want
• etc..

I.e., I'm running on Ubuntu 22.04 with glibc-v2.34, on an x86 machine. Therefore, I need to find a wheel inside of /builds/$COMMIT_HASH that has x86 and manylinux_2_34 in it.). This logic can be cumbersome.

This PR proposes a new structure: s3://bucket-name/builds/$COMMIT_HASH/$GHA_RUN_ID_$GHA_RUN_ATTEMPT/$WHEEL_NAME. Inside of /builds/$COMMIT_HASH/$GHA_RUN_ID_$GHU_RUN_ATTEMPT, there will always be at most 1 wheel. Using this new structure, you will not need to know the wheel-name; if you know the $COMMIT_HASH, the $GHA_RUN_ID and the $GHA_RUN_ATTEMPT, you will always be able to locate the wheel (without having to perform any wheel-name logic).

Note

This is a non-breaking change. Other workflows only grab the URL of a wheel, they don't observe the structure. Therefore, updating the structure should not affect the other workflows.

[raunakab]

@jaychia This is just a proposition. The new structure may be a bit too "centered" around GitHub Actions.

Maybe the old structure is nice enough?

[codspeed-hq]

CodSpeed Performance Report

Merging #3591 will degrade performances by 34.62%

_{Comparing ci/build-commit (52345aa) with main (47f5897)}

Summary

⚡ 1 improvements
❌ 1 regressions
✅ 25 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	ci/build-commit	Change
⚡	test_iter_rows_first_row[100 Small Files]	215 ms	159.9 ms	+34.42%
❌	test_show[100 Small Files]	15.5 ms	23.7 ms	-34.62%

[jaychia]

Seems fine to me, I'm guessing this is a prerequisite to doing the if wheel !exists { build_wheel() } else { skip() } logic from run-cluster?

I feel like there should just be a canonical way to represent the wheel names that we're missing though. I.e. we shouldn't need to invent our own naming convention