feat(api): Support custom MCAD container image

[astefanutti]

Fixes #114.

[astefanutti]

/assign @jbusche @KPostOffice @anishasthana

[anishasthana]

/lgtm
Changes look good to me. I haven't run through with testing + deploying it, however.

[jbusche]

OK, great, let me see if I can test it, see if it works as expected...

[jbusche]

OK - I'm having one small snag, but seeing it also with the main branch, so it's not part of this PR that I can see.
Problem: When I do a make install It's not properly replacing the ${MCAD_CRD} version in the file [kustomization.yaml ](https://github.com/project-codeflare/codeflare-operator/blob/main/config/crd/mcad/kustomization.yaml#L4) so I'm seeing this error when running make install:`

Error: accumulating resources: accumulation err='accumulating resources from 'mcad': '/Users/jamesbusche/projects/CODEFLARE/ANTONIN/codeflare-operator/config/crd/mcad' must resolve to a file': recursed accumulation of path '/Users/jamesbusche/projects/CODEFLARE/ANTONIN/codeflare-operator/config/crd/mcad': accumulating resources: accumulation err='accumulating resources from 'github.com/project-codeflare/multi-cluster-app-dispatcher/config/crd?ref=release-v0.0.0': evalsymlink failure on '/Users/jamesbusche/projects/CODEFLARE/ANTONIN/codeflare-operator/config/crd/mcad/github.com/project-codeflare/multi-cluster-app-dispatcher/config/crd?ref=release-v0.0.0' : lstat /Users/jamesbusche/projects/CODEFLARE/ANTONIN/codeflare-operator/config/crd/mcad/github.com: no such file or directory': git cmd = '/opt/homebrew/bin/git fetch --depth=1 origin release-v0.0.0': exit status 128

My temporary fix is to edit vi ./config/crd/mcad/kustomization.yaml and change the file from:

- github.com/project-codeflare/multi-cluster-app-dispatcher/config/crd?ref=release-v0.0.0  # kpt-set: ${MCAD_CRD}
to
- github.com/project-codeflare/multi-cluster-app-dispatcher/config/crd?ref=release-v1.31.0

[jbusche]

The testing I've done is this:
On my laptop

oc login …. --insecure-skip-tls-verify=true

cd /Users/jamesbusche/projects/CODEFLARE/ANTONIN/
git clone https://github.com/astefanutti/codeflare-operator.git
cd codeflare-operator
git checkout pr-10
make manifests
make install (installs the new CRDS)
# Note, this is having trouble. To fix it I had to do this:
vi ./config/crd/mcad/kustomization.yaml
changing:
- github.com/project-codeflare/multi-cluster-app-dispatcher/config/crd?ref=release-v0.0.0  # kpt-set: ${MCAD_CRD}
to
- github.com/project-codeflare/multi-cluster-app-dispatcher/config/crd?ref=release-v1.31.0  # kpt-set: ${MCAD_CRD}

make run  (runs the operator locally, you'll see the operator logs…

Now in another window, let's deploy the regular kfdef
oc apply -f https://raw.githubusercontent.com/opendatahub-io/distributed-workloads/main/codeflare-stack-kfdef.yaml -n opendatahub

and then edit the mcad cr
oc edit mcad mcad
and under spec: add something like this:
spec:
  controllerImage: quay.io/project-codeflare/mcad-controller:main-v1.30.0

and then watch the pod restart from v1.31.0 to v1.30.0

oc describe pod mcad-controller-mcad-5678f66fdd-mv98x |grep Image:
    Image:         quay.io/project-codeflare/mcad-controller:main-v1.30.0

[jbusche]

LGTM
Tested out fine - I'll open an issue on the make install problem, that seems unrelated to this PR.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jbusche

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jbusche]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment