[Tracking] Consolidate and automate creation of sig-release owned testgrid dashboards

[mariantalla]

(Following up from the last sig-release catchup (notes))

Context: At the end of the 1.13 release, we rearranged sig-release owned dashboards into blocking and informing, based on criteria discussed in kubernetes/sig-release#347.

What should be cleaned up or changed:

• Consolidate master- dashboards in master-blocking and master-informing, by moving master-upgrade jobs into one of them (Move master-upgrade jobs to master-informing #12441 Move master-upgrade jobs to master-informing #12511)
• Shape 1.14-blocking to contain exact equivalents of master-blocking, by moving all other 1.14- jobs into 1.14-informing (add issue/PR here)
• Resolve the 1.14-informing - master-informing diff, by
  • auditing the diff
  • for jobs that exist in one but not the other dashboard, deleting or porting over accordingly
    (add issue/PR here)
• Create tooling to automate creation of <non-master>- dashboards and their contents (e.g. prow jobs), using master- dashboards as input (Add tooling to automate release-based job forking #12443 and followups)

Related/Complementary, but out of scope for this work:

• Upgrade tests should move to use a community supported way of upgrading clusters

cc @alejandrox1 @spiffxp @claurence @neolit123 @tpepper @calebamiles @jberkus
/sig release

[mariantalla]

Re: consolidating master- dashboards, we spoke about moving them to -informing rather than -blocking, because they use /cluster (=non-community-supporting) scripts to upgrade clusters. For 1.14, CI signal treated them as equally blocking as master-blocking. They're also currently the only way we have to get signal on how stable upgrades are, so I feel there is a good argument for them to move into blocking too.

@alejandrox1 @claurence thoughts?

[spiffxp]

/assign @spiffxp @mariantalla
I would like to help push this with maria (whom I've assigned because she's started pushing this in #12441)

/assign @alejandrox1
CI Signal for v1.15

FYI @claurence as 1.15 release lead

FYI @Katharine given you are trying to help simply forking of the jobs that end up on these dashboards (ref: #12443)

[spiffxp]

/milestone v1.15

[spiffxp]

Create tooling to automate creation of - dashboards and their contents (e.g. prow jobs), using master- dashboards as input (add issue/PR here)

/assign @Katharine
Now that I'm reading a little more closely, this is what #12443 is for, so assigning to make sure we agree this is done

[spiffxp]

I am, FWIW, trying to link anything related to "clean up release dashboards" or "clean up release jobs" or "make these things more consistent" etc, because this seems like the most relevant umbrella

[spiffxp]

/area config
/area testgrid

[spiffxp]

#11977 is adding -informing dashboards for all branches

[spiffxp]

We are now in an intermediary state I would like to get out of:

• release-x.y-blocking
• release-x.y-informing
• release-x.y-all

I want us to get rid of -all

[spiffxp]

/milestone v1.16

[spiffxp]

ref: #13883

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[justaugustus]

Hopping on this since I've done recent-ish work to spruce these boards up:

• Create sig-release-job-config-errors dashboard: [1.17] releng: Update release branch jobs for 1.17 #15023
• Signal potentially misconfigured jobs and delete sig-release-x.y-all dashboards: Signal potentially misconfigured jobs and delete sig-release-x.y-all dashboards #15039
• sig-release: Rearrange dash configs and prune some jobs from sig-release-job-config-errors: sig-release: Rearrange dash configs and prune some jobs from sig-release-job-config-errors #15068

What remains for us to do to close this out?

/assign
/remove-lifecycle stale
/lifecycle frozen

[spiffxp]

/close
I can't think of anything off the top of my head so I think we should close. If there's something specific that comes up, use followup issues.

[k8s-ci-robot]

@spiffxp: Closing this issue.

In response to this:

/close
I can't think of anything off the top of my head so I think we should close. If there's something specific that comes up, use followup issues.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.