Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Persist pod logs to permanent storage #439

Closed
IronPan opened this issue Dec 3, 2018 · 2 comments
Closed

Persist pod logs to permanent storage #439

IronPan opened this issue Dec 3, 2018 · 2 comments
Labels
priority/p0

Comments

@IronPan
Copy link
Member

IronPan commented Dec 3, 2018

For clusters such as auto scaling clusters, the pod logs will be lost if the cluster is scaled.
We should store the pod log so UI can still get it.

The change includes

  • Create a storage for storing log - Either database table, or object storage.
  • Storing the log in permanent storage using persist agent
  • Surface the log through API
  • Update UI to read log from the API
@jlewi
Copy link
Contributor

jlewi commented Dec 3, 2018

Most clusters will already be configured to have cluster level logging that stores pods in permanent storage (e.g. StackdDriver, DataDog, etc...).

Can the UI just be configured to fetch the logs from one of those locations?

@IronPan
Copy link
Member Author

IronPan commented Dec 3, 2018

As long as it's consistent for both onprem and gcp, I'm good with it.

@IronPan IronPan mentioned this issue Dec 6, 2018
Closed
@IronPan IronPan closed this as completed Dec 6, 2018
magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this issue Oct 22, 2023
@yuzisun @k8s-ci-robot
Fix missing alibi key in explainers configmap (kubeflow#439)
9a38524 
* Fix missing alibi key in configmap

* Remove hardcode alibi image name

* Add alibi explainer default test

* Add validation/default tests for explainer

* Add alibi explainer container creation test

* Fix format

* Fix alibiexplainer image registry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
priority/p0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jlewi @IronPan