Move away from Google Container registry, and cloud build system for image builds

[HumairAK]

As per the last KFP call on 2024-09-11

It was decided that the following pieces that are part of the release process are still highly dependent on google infra:

• Buildling mechanism, currently we’re using cloud build mechanism
• container registry, technically you can use the github container registry, but not sure if there’s a ratelimiting

Suggestion:

This looks like we can easily resolve this dependency by utilizing Github Actions as the build mechanism, and using Github Container Registry. With KF as a CNCF project we should have enterprise access to GitHub Container Registry.

[HumairAK]

cc @zijianjoy / @chensun

[HumairAK]

@rimolive will contact CNCF and follow up

[rimolive]

/assign @rimolive

[HumairAK]

/assign @HumairAK

[HumairAK]

cc @chensun / @zijianjoy I've added a PR for a workflow that will build and push images to the kubeflow/pipelines github container registry

how does this sound to you? based on their pricing it seems it's free for public repos: https://github.com/pricing

If there's anything missing that we need from enterprise, we can reach out to CNCF for enterprise access (however that works)

[HumairAK]

I've also started some slack discussions in the CNCF kubeflow slack channels here and here to discuss ghcr as a candidate

Edit: Currently it seems there is no consensus on the standard image registry for KF to use, some are using docker hub, and some are using GHCR. My suggestion is that we go with ghcr for now as the integration is a bit easier, the github workflow already has built in credentials via $GITHUB_TOKEN, and their pull rate limit seems a lot more generous. We may need to revisit once there's a finalized location for all of KF to host its images.