Take inventory of Google-owned cluster-based k8s-infra

[spiffxp]

Google has a number of clusters floating around running different pieces of infra for the project. What's running on them

[spiffxp]

/assign

[spiffxp]

First pass. Based on a survey of internal clusters, with some educated guesses about which services/deployments/statefulsets are still relevant. I can provide something that's more raw if desired, but it will take time to scrub out potentially sensitive info.

k8s-gubernator/g8r (#789)

• kettle: https://github.com/kubernetes/test-infra/tree/master/kettle (ref: Migrate kettle to k8s-infra #787)

k8s-mungegithub/mungegithub (#788)

• perfdash: https://github.com/kubernetes/perf-tests/tree/master/perfdash - in flight (ref: Move perf-dash to community owned infra #549)

k8s-prow/prow

• prow: https://github.com/kubernetes/test-infra/tree/master/prow/cluster (Develop a prow migration plan #752)

k8s-prow-builds/prow

• boskos: https://github.com/kubernetes/test-infra/tree/master/boskos (Develop a prow migration plan #752)
• greenhouse: https://github.com/kubernetes/test-infra/tree/master/greenhouse (Develop a prow migration plan #752)

kubernetes-site/utilicluster (#754)

• cert-manager: https://github.com/kubernetes/k8s.io/tree/master/cert-manager - running in aaa
• gcsweb: https://github.com/kubernetes/test-infra/tree/master/gcsweb - running in aaa (Moving Gcsweb to CNCF Infrastructure #204)
• k8s-io: https://github.com/kubernetes/k8s.io/tree/master/k8s.io - running in aaa
• node-perf-dash: https://github.com/kubernetes-retired/contrib/tree/master/node-perf-dash (Migrate node-perf-dash to aaa cluster (or retire) #753)
• velodrome: https://github.com/kubernetes/test-infra/tree/master/velodrome - turned down due to security issues, needs rewrite

kubernetes-tools/tools-cluster (#790)

• slack-event-log: https://github.com/kubernetes-sigs/slack-infra/tree/master/slack-event-log (Migrate slack-infra services to aaa cluster #751)
• slack-moderator: https://github.com/kubernetes-sigs/slack-infra/tree/master/slack-moderator (Migrate slack-infra services to aaa cluster #751)
• slack-welcomer: https://github.com/kubernetes-sigs/slack-infra/tree/master/slack-welcomer (Migrate slack-infra services to aaa cluster #751)
• slackin: https://github.com/kubernetes-sigs/slack-infra/tree/master/cluster/slackin (Migrate slack-infra services to aaa cluster #751)

non-google:

• publishing-bot: https://github.com/kubernetes/publishing-bot - running in dev2, to be run in aaa (Move Publishing bot from development2 to aaa #503)
• oncall-issue-filer: https://github.com/philips/oncall-issue-filer - replaced with HackerOne

[ameukam]

I noticed velodrome use very old versions of prometheus and grafana. Do we plan to upgrade those tools during the migration ?

[philips]

Is there an estimate on when services will be ready to migrate? I need to decide whether to stand up a new cluster for the oncall-issue-filer or do something else. Thanks!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[ameukam]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[ameukam]

/remove-lifecycle stale

[thockin]

/lifecycle frozen
/remove-lifecycle stale
/remove-lifecycle rotten

[thockin]

The new cluster is up and is viable for migrations. A small number of things, including k8s.io, have moved.

@philips do you have docs or info on your thing?

[philips]

@thockin I just decommissioned my thing on Tuesday because it has been replaced by HackerOne 😂

[spiffxp]

To close this out I'd like to create or link a corresponding issue for each piece of cluster infra #240 (comment)

[bartsmykla]

As perfdash is currently running at the aaa and the last bits are updating the docs (kubernetes/perf-tests#1184 and #778) I'm happy to write we can destroy the k8s-mungegithub/mungegithub cluster and mark perfdash as done. :-)

[spiffxp]

/close
OK, I've stopped short of creating "turn down" issues for the k8s-prow and k8s-prow-build clusters because those may stick around for other purposes. I've created issues for everything else here.

Next is non-cluster infra, such as BigQuery, App Engine, etc. which can lead to turn down of GCP projects, not just clusters. That's #241

[k8s-ci-robot]

@spiffxp: Closing this issue.

In response to this:

/close
OK, I've stopped short of creating "turn down" issues for the k8s-prow and k8s-prow-build clusters because those may stick around for other purposes. I've created issues for everything else here.

Next is non-cluster infra, such as BigQuery, App Engine, etc. which can lead to turn down of GCP projects, not just clusters. That's #241

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.