Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a configmap generator for the spawner_ui_config.yaml #1013

Merged
merged 4 commits into from
Mar 27, 2020

Conversation

jlewi
Copy link
Contributor

@jlewi jlewi commented Mar 16, 2020

  • Using a configmap generator is a bit cleaner because we can store
    the configuration file in a external file rather than inlining it in
    the config map.

  • Using a configmap generator also has the advantage that kustomize will
    generate a unique name based on a hash of the content.

    • This has the advantage that if a user updates the config, the hash
      would change so deployments would automatically be redeployed in order
      to pick up the config changes.
  • Also update the default image used in the configmap to fix jupyter web app config is defaulting to an old image #1010


This change is Reviewable

@jlewi
Copy link
Contributor Author

jlewi commented Mar 16, 2020

/assign @kimwnasptd

@kimwnasptd There are two issues with the tests

#1012 presubmits are failing because the workflow names are too long.

The unittests are not passing after regenerating the unittests based on this change. It looks like a bug in how the generate scripts are generating the expected value for the configmap.

   ---                                                                                        ---
   apiVersion: v1                                                                             apiVersion: v1
   data:                                                                                      data:
X    spawner_ui_config.yaml: |2-                                                                spawner_ui_config.yaml: |-

The left hand side is the expected value. I don't know why its

 spawner_ui_config.yaml: |2- 

I'm not going to try to fix that until after fixing #306. The fix to #306 should simplify the logic for generating the expected value. In particular we will run kustomize build -o ... to generate the expected value. Running kustomize build is generating correct output so that should fix the problem.

In the meantime it would be helpful if you could take a look at this PR so once those issues are fixed we could go ahead and merge it.

@kimwnasptd
Copy link
Member

Thanks for the PR @jlewi! to make sure I'm in the same page, until we resolve #306 if I would like to make some change in the configmap would I also need to change the hardcoded expected value in the go files?

Or until #306 is resolved we expect the unit tests for the config map to not pass?
Is my understanding correct or am I missing something?

@jlewi
Copy link
Contributor Author

jlewi commented Mar 17, 2020

@kimwnasptd #306 will be fixed by #1016; my plan is to merge that first; fix the tests and then merge this.

In the meantime you can review this to see if there are any issues with switching to a generator for the configmap.

Jeremy Lewi added 2 commits March 24, 2020 20:48
* Using a configmap generator is a bit cleaner because we can store
  the configuration file in a external file rather than inlining it in
  the config map.

* Using a configmap generator also has the advantage that kustomize will
  generate a unique name based on a hash of the content.

  * This has the advantage that if a user updates the config, the hash
    would change so deployments would automatically be redeployed in order
    to pick up the config changes.

* Also update the default image used in the configmap to fix kubeflow#1010
@jlewi
Copy link
Contributor Author

jlewi commented Mar 25, 2020

@kimwnasptd #1016 was merged and I rebased and updated the tests. I ran the tests locally and they passed so I expect presubmits will pass.

Could you PTAL?

  the jupyter web app config map without the prefix/suffix.
@kimwnasptd
Copy link
Member

The tests also pass on my local machine.
Two things I noticed:

  1. After running kustomize build jupyter/jupyter-web-app/overlays/application the generated deployment's ENV Vars USERID_{HEADER,PREFIX} are set to "". Are they substituted later on with kfctl?
  2. After running kustomize build jupyter/jupyter-web-app/overlays/istio the destination field in the virtual service's spec has $(namespace) in the generated YAML. After applying the yaml I get the error
Error from server: error when creating "STDIN": admission webhook "pilot.validation.istio.io" denied the request: configuration is invalid: domain name "$(namespace).svc.$(clusterDomain)" invalid (label "$(namespace)" invalid)

@jlewi
Copy link
Contributor Author

jlewi commented Mar 26, 2020

I think my most recent PR broke the unittests because I changed the config map but failed to regenerate the tests.
https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubeflow_manifests/1013/kubeflow-manifests-presubmit/1242889171895324672/

Prior to that PR the E2E tests were failing because I wasn't properly setting the name of the config map so that the value in the deployment matched the original name of the generated config map.

So hopeful my latest fix will fix this.

@jlewi
Copy link
Contributor Author

jlewi commented Mar 26, 2020

@kimwnasptd tests passed could you LGTm please?

@kimwnasptd
Copy link
Member

/lgtm

@jlewi
Copy link
Contributor Author

jlewi commented Mar 27, 2020

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jlewi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit bba06d9 into kubeflow:master Mar 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

jupyter web app config is defaulting to an old image
4 participants