Skip to content

Commit

Permalink
Disable Android benchmarks while device is offline. (iree-org#17910)
Browse files Browse the repository at this point in the history
We have two pixel-6-pro devices/runners, one labeled for `presubmit` one
labeled for `postsubmit`. The presubmit runner has been offline for
multiple days, causing benchmark runs on LLVM integrate PRs to time out
waiting for a runner (and blocking other benchmark results from getting
reported as PR comments).

Some options:
* iree-org#17896 
* Switch only Android presubmit benchmarks to also use postsubmit
benchmark runners
* Turn off Android benchmarks on presubmit (or just for LLVM integrate
PRs)
* Switch benchmarks from pixel-6-pro to pixel-8-pro (we tried this
before but pixel 6 was more stable than pixel 8... I'm not sure how much
that level of stability is useful at this stage though, need to keep
rolling forward with testing on recent hardware)
* (THIS PR) Turn off Android benchmarks entirely

See also [this discussion on
Discord](https://discord.com/channels/689900678990135345/1080178290188374049/1260359410035724402).

benchmark-extra: default
  • Loading branch information
ScottTodd authored Jul 15, 2024
1 parent f89a7da commit 2eea916
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions build_tools/github_actions/configure_ci.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,15 @@ def contains(cls, val):
for preset in benchmark_presets.DEFAULT_PRESETS
# RISC-V benchmarks haven't been supported in CI workflow.
if preset not in [benchmark_presets.RISCV]
# Disabled while iree-pixel-6-pro-1 lab Android presubmit device is offline.
# The iree-pixel-6-pro-2 postsubmit device may still be online, so this
# could be adapted to keep the Android benchmarks enabled on postsubmit.
and preset
not in [
benchmark_presets.ANDROID_CPU,
benchmark_presets.ANDROID_CPU_DT_ONLY,
benchmark_presets.ANDROID_GPU,
]
] + ["comp-stats"]
DEFAULT_BENCHMARK_PRESET = "default"
LARGE_BENCHMARK_PRESET_GROUP = benchmark_presets.LARGE_PRESETS
Expand Down

0 comments on commit 2eea916

Please sign in to comment.