Skip to content

Commit

Permalink
Use stable tree timer rather than Nightly scheduler for refleak builds (
Browse files Browse the repository at this point in the history
#515)

* Replace 'Nightly' schedulers

Instead of once per day, they now wait for 1 hour after any change
before starting a build (that is, if there is a change made and 59
minutes elapse before another change, the scheduler will wait another
hour before starting a build containing both changes.

The hope is that this will do a few things for us:
- Avoid blocking workers that do refleak builds in addition to other
  builds:
    - Don't build branches that haven't changed (this could also be
      accomplished by adding 'onlyIfChanged=True' to the existing
      'Nightly' schedulers)
    - Non-refleak builds should have time to finish up before refleak
      build(s) start (caveat: a refleak build on one branch may still
      block builds of other branches)
- Associate refleak builds with actual changes

* Remove superflous DAILYBUILDERS list

They were exclusively refleak builders, which are now detected by tag.
  • Loading branch information
zware committed Aug 19, 2024
1 parent 56db932 commit b23e2ff
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 36 deletions.
26 changes: 0 additions & 26 deletions master/custom/builders.py
Original file line number Diff line number Diff line change
Expand Up @@ -335,32 +335,6 @@ def get_builders(settings):
return all_builders


DAILYBUILDERS = [
"AMD64 Windows11 Refleaks",
"AMD64 Fedora Rawhide Refleaks",
"AMD64 Fedora Stable Refleaks",
"AMD64 RHEL8 Refleaks",
"AMD64 CentOS9 Refleaks",
"AMD64 Fedora Rawhide NoGIL refleaks",
# Linux PPC64LE
"PPC64LE Fedora Rawhide Refleaks",
"PPC64LE Fedora Stable Refleaks",
"PPC64LE RHEL8 Refleaks",
"PPC64LE CentOS9 Refleaks",
"PPC64LE Fedora Rawhide NoGIL refleaks",
# Linux s390x
"s390x Fedora Rawhide Refleaks",
"s390x Fedora Refleaks",
"s390x RHEL9 Refleaks",
"s390x RHEL8 Refleaks",
# Linux aarch64
"aarch64 Fedora Rawhide Refleaks",
"aarch64 Fedora Stable Refleaks",
"aarch64 RHEL8 Refleaks",
"aarch64 CentOS9 Refleaks",
"aarch64 Fedora Rawhide NoGIL refleaks",
]

# Match builder name (excluding the branch name) of builders that should only
# run on the main and "custom" branches.
ONLY_MAIN_BRANCH = (
Expand Down
23 changes: 13 additions & 10 deletions master/master.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ from custom.release_dashboard import get_release_status_app # noqa: E402
from custom.builders import ( # noqa: E402
get_builders,
STABLE,
DAILYBUILDERS,
ONLY_MAIN_BRANCH,
TIER_1,
TIER_2,
Expand Down Expand Up @@ -207,7 +206,7 @@ mail_status_builders = []

for branch_num, (git_url, branchname, git_branch) in enumerate(git_branches):
buildernames = []
dailybuildernames = []
refleakbuildernames = []
for name, worker_name, buildfactory, stability, tier in BUILDERS:
if any(
pattern in name for pattern in ONLY_MAIN_BRANCH
Expand Down Expand Up @@ -245,8 +244,8 @@ for branch_num, (git_url, branchname, git_branch) in enumerate(git_branches):
elif "nondebug" not in tags and branchname in {"3.11", "3.12"}:
continue

if name in DAILYBUILDERS:
dailybuildernames.append(buildername)
if 'refleak' in tags:
refleakbuildernames.append(buildername)
else:
buildernames.append(buildername)
if all(pattern not in buildername for pattern in NO_NOTIFICATION):
Expand Down Expand Up @@ -277,14 +276,18 @@ for branch_num, (git_url, branchname, git_branch) in enumerate(git_branches):
fileIsImportant=is_important_change,
)
)
if dailybuildernames:
if refleakbuildernames:
c["schedulers"].append(
schedulers.Nightly(
name=branchname + "-daily",
hour=int(branch_num / (len(git_branches) - 1) * 23),
minute=0,
schedulers.SingleBranchScheduler(
name=branchname + "-refleak",
change_filter=util.ChangeFilter(branch=git_branch),
builderNames=dailybuildernames,
# Wait this many seconds for no commits before starting a build
# NB: During extremely busy times, this can cause the builders
# to never actually fire. The current expectation is that it
# won't ever actually be that busy, but we need to keep an eye
# on that.
treeStableTimer=1 * 60 * 60, # h * m * s
builderNames=refleakbuildernames,
)
)

Expand Down

0 comments on commit b23e2ff

Please sign in to comment.