Skip to content

Commit

Permalink
Round up sdxl golden dispatch sizes and mi250 times by 10%. (iree-org…
Browse files Browse the repository at this point in the history
…#17879)

See [this discussion on
Discord](https://discord.com/channels/689900678990135345/689957613152239638/1261059802214305913).

These are change detector tests with _some_ use, but exact checking is
too noisy and is causing significant churn across PRs and postsubmit CI
runs. We could change the test script to allow a certain error range,
but for now just increase the thresholds used in `<=` checks by about
10% (then drop a few digits to 0 so we don't give the appearance of high
precision).

ci-exactly: build_packages,regression_test
  • Loading branch information
ScottTodd authored Jul 12, 2024
1 parent 65a7bd0 commit f07c96c
Showing 1 changed file with 13 additions and 10 deletions.
23 changes: 13 additions & 10 deletions .github/workflows/pkgci_regression_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -307,21 +307,24 @@ jobs:
--durations=0 \
--config-files=${SDXL_VAE_CONFIG_FILE_PATH}
# Note: mi250 benchmark times are more lenient than mi300 (allowing about
# 10% deviation from observed averages), since the mi250 runners we use
# are more unstable and we care most about peak performance on mi300.
- name: "Running SDXL rocm pipeline benchmark (mi250)"
if: contains(matrix.name, 'rocm_mi250_gfx90a')
run: |
source ${VENV_DIR}/bin/activate
pytest SHARK-TestSuite/iree_tests/benchmarks/sdxl/benchmark_sdxl_rocm.py \
--goldentime-rocm-e2e-ms 1340.0 \
--goldentime-rocm-unet-ms 342.0 \
--goldentime-rocm-clip-ms 17.5 \
--goldentime-rocm-vae-ms 300.0 \
--goldentime-rocm-e2e-ms 1450.0 \
--goldentime-rocm-unet-ms 370.0 \
--goldentime-rocm-clip-ms 18.5 \
--goldentime-rocm-vae-ms 315.0 \
--goldendispatch-rocm-unet 1714 \
--goldendispatch-rocm-clip 1569 \
--goldendispatch-rocm-vae 248 \
--goldensize-rocm-unet-bytes 2073609 \
--goldensize-rocm-clip-bytes 783848 \
--goldensize-rocm-vae-bytes 764909 \
--goldensize-rocm-unet-bytes 2280000 \
--goldensize-rocm-clip-bytes 860000 \
--goldensize-rocm-vae-bytes 840000 \
--gpu-number 6 \
--rocm-chip gfx90a \
--log-cli-level=info \
Expand All @@ -341,9 +344,9 @@ jobs:
--goldendispatch-rocm-unet 1714 \
--goldendispatch-rocm-clip 1569 \
--goldendispatch-rocm-vae 248 \
--goldensize-rocm-unet-bytes 2065289 \
--goldensize-rocm-clip-bytes 783848 \
--goldensize-rocm-vae-bytes 765869 \
--goldensize-rocm-unet-bytes 2270000 \
--goldensize-rocm-clip-bytes 860000 \
--goldensize-rocm-vae-bytes 840000 \
--gpu-number 0 \
--rocm-chip gfx942 \
--log-cli-level=info \
Expand Down

0 comments on commit f07c96c

Please sign in to comment.