Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: always enable server config and make it customizable #605

Merged
merged 2 commits into from
Mar 15, 2024

Conversation

HumairAK
Copy link
Contributor

@HumairAK HumairAK commented Mar 14, 2024

Description of your changes:

We should always mount the server config, regadless of whether we have a custom cabundle or not, so that we don't end up in a situation where based on the cabundle we are using different configs, as that may result in different behaviors if somehow the default config in the dsp server image ends up diverging from the one we mount.

To prevent future issues with server config, this pr also adds the ability for users to provide a custom sever config, in case the default options are not sufficient, the user can customize their own server config in such a case, for instance changing the path in s3 where pipelines are stored.

The name for the default server config is also renamed to be consistent with the rest of the dspa's resource naming pattern "ds-pipelines-*"

Testing instructions

  1. Deploy simple dspa
  2. confirm server-config is mounted, in the pipeline server pod you should see the following:
         volumeMounts:
            - name: server-config
              mountPath: /config/config.json
              subPath: config.json
  1. confirm pipeline runs successfully
  2. In a separate namespace, deploy another dspa but with a custom server config, for example:
kind: ConfigMap
apiVersion: v1
metadata:
  name: custom-script
  namespace: dspa1
  labels:
    app: ds-pipeline-test
    component: data-science-pipelines
data:
  blah: |
    {
      "DBConfig": {
        "MySQLConfig": {
          "ExtraParams": {"tls":"false"},
          "GroupConcatMaxLen": "4194304"
         },
        "PostgreSQLConfig": {},
        "ConMaxLifeTime": "120s"
      },
      "DBDriverName": "mysql",
      "InitConnectionTimeout": "6m",
      "CacheEnabled": "false"
    }

and in dspa:

apiVersion: datasciencepipelinesapplications.opendatahub.io/v1alpha1
kind: DataSciencePipelinesApplication
metadata:
  name: test
spec:
  apiServer:
    customServerConfigMap:
      name: custom-script
      key: blah
  dspVersion: v2
  objectStorage:

    minio:
      deploy: true
      image: 'quay.io/opendatahub/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance'
  mlpipelineUI:
    image: quay.io/opendatahub/ds-pipelines-frontend:latest

then inspect the apiserver mounts:

 volumeMounts:
    - name: server-config
      mountPath: /config/config.json
      subPath: blah

Checklist

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

@dsp-developers
Copy link
Contributor

A new image has been built to help with testing out this PR: quay.io/opendatahub/data-science-pipelines-operator:pr-605
An OCP cluster where you are logged in as cluster admin is required.

To use this image run the following:

cd $(mktemp -d)
git clone git@github.com:opendatahub-io/data-science-pipelines-operator.git
cd data-science-pipelines-operator/
git fetch origin pull/605/head
git checkout -b pullrequest a3ceacb47f3fd4c0bb0d5f9acdf49b904394a39b
oc new-project opendatahub
make deploy IMG="quay.io/opendatahub/data-science-pipelines-operator:pr-605"

More instructions here on how to deploy and test a Data Science Pipelines Application.

@HumairAK HumairAK force-pushed the always_enable_server_config branch from a3ceacb to 75a814b Compare March 14, 2024 21:27
@dsp-developers
Copy link
Contributor

Change to PR detected. A new PR build was completed.
A new image has been built to help with testing out this PR: quay.io/opendatahub/data-science-pipelines-operator:pr-605

@HumairAK HumairAK changed the title chore: always enable server_config Always enable server config and make it customizable Mar 14, 2024
@dsp-developers
Copy link
Contributor

Change to PR detected. A new PR build was completed.
A new image has been built to help with testing out this PR: quay.io/opendatahub/data-science-pipelines-operator:pr-605

@HumairAK HumairAK changed the title Always enable server config and make it customizable feat: always enable server config and make it customizable Mar 14, 2024
Signed-off-by: Humair Khan <HumairAK@users.noreply.github.com>
@HumairAK HumairAK force-pushed the always_enable_server_config branch 2 times, most recently from 3f01f4c to 0028363 Compare March 15, 2024 14:29
Signed-off-by: Humair Khan <HumairAK@users.noreply.github.com>
@HumairAK HumairAK force-pushed the always_enable_server_config branch from 0028363 to 46306ea Compare March 15, 2024 14:30
@dsp-developers
Copy link
Contributor

Change to PR detected. A new PR build was completed.
A new image has been built to help with testing out this PR: quay.io/opendatahub/data-science-pipelines-operator:pr-605

Copy link
Contributor

@amadhusu amadhusu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worked as suggested with the default server config and customized one as well. Pipelines ran successfully as well.

default server config

Screenshot from 2024-03-15 20-09-13

Screenshot from 2024-03-15 20-10-24

custom server config

Screenshot from 2024-03-15 20-19-22

Screenshot from 2024-03-15 20-20-21

@rimolive
Copy link
Contributor

/approve

Copy link
Contributor

openshift-ci bot commented Mar 15, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amadhusu, rimolive

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@HumairAK HumairAK merged commit ad47dda into opendatahub-io:main Mar 15, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants