fix: fail templating gracefully

[HumairAK]

Currently we panic and crash the operator if templating fails, this is not ideal., the controller should be resilient to errors and forward them up the call stack and log them.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from humairak. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dsp-developers]

A new image has been built to help with testing out this PR: quay.io/opendatahub/data-science-pipelines-operator:pr-609
An OCP cluster where you are logged in as cluster admin is required.

To use this image run the following:

cd $(mktemp -d)
git clone git@github.com:opendatahub-io/data-science-pipelines-operator.git
cd data-science-pipelines-operator/
git fetch origin pull/609/head
git checkout -b pullrequest 0b02edc412a799385fc3968cfcaba4b889a714fd
oc new-project opendatahub
make deploy IMG="quay.io/opendatahub/data-science-pipelines-operator:pr-609"

More instructions here on how to deploy and test a Data Science Pipelines Application.

[dsp-developers]

Change to PR detected. A new PR build was completed.
A new image has been built to help with testing out this PR: quay.io/opendatahub/data-science-pipelines-operator:pr-609

[gmfrasca]

/lgtm

[gregsheremeta]

/lgtm