-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: feat(api/sdk/backend): implement PipelineConfig and setting ttl on pipelines. Fixes: #10899 #10942
WIP: feat(api/sdk/backend): implement PipelineConfig and setting ttl on pipelines. Fixes: #10899 #10942
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Large diffs are not rendered by default.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -679,7 +679,8 @@ message ValueOrRuntimeParameter { | |
} | ||
|
||
// The definition of the deployment config of the pipeline. It contains the | ||
// the platform specific executor configs for KFP OSS. | ||
// the platform specific executor configs for KFP OSS, as well as platform level config | ||
// such as settings to pass to an Argo Workflows Workflow CR. | ||
message PipelineDeploymentConfig { | ||
// The specification on a container invocation. | ||
// The string fields of the message support string based placeholder contract | ||
|
@@ -852,6 +853,11 @@ message PipelineDeploymentConfig { | |
} | ||
// Map from executor label to executor spec. | ||
map<string, ExecutorSpec> executors = 1; | ||
|
||
// begin platform level / workflow level config | ||
|
||
// duration in seconds after which the pipeline platform will do garbage collection | ||
optional int32 completed_pipeline_ttl = 2; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. would suggest changing this type to |
||
} | ||
|
||
// Value is the value of the field. | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -28,6 +28,7 @@ | |
from kfp.dsl import component_factory | ||
from kfp.dsl import for_loop | ||
from kfp.dsl import pipeline_channel | ||
from kfp.dsl import pipeline_config | ||
from kfp.dsl import pipeline_context | ||
from kfp.dsl import pipeline_task | ||
from kfp.dsl import placeholders | ||
|
@@ -1850,13 +1851,15 @@ def create_pipeline_spec( | |
pipeline: pipeline_context.Pipeline, | ||
component_spec: structures.ComponentSpec, | ||
pipeline_outputs: Optional[Any] = None, | ||
pipeline_config: pipeline_config.PipelineConfig = None, | ||
) -> Tuple[pipeline_spec_pb2.PipelineSpec, pipeline_spec_pb2.PlatformSpec]: | ||
"""Creates a pipeline spec object. | ||
|
||
Args: | ||
pipeline: The instantiated pipeline object. | ||
component_spec: The component spec structures. | ||
pipeline_outputs: The pipeline outputs via return. | ||
pipeline_config: The pipeline config object. | ||
|
||
Returns: | ||
A PipelineSpec proto representing the compiled pipeline. | ||
|
@@ -1874,6 +1877,10 @@ def create_pipeline_spec( | |
# Schema version 2.1.0 is required for kfp-pipeline-spec>0.1.13 | ||
pipeline_spec.schema_version = '2.1.0' | ||
|
||
# pipeline-level config options | ||
if pipeline_config.completed_pipeline_ttl_seconds is not None and pipeline_config.completed_pipeline_ttl_seconds > 0: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. May we raise an error if <= 0? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good point. I think the behavior should be: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Some error handling message would be nice to have here. For instance,
|
||
deployment_config.completed_pipeline_ttl = pipeline_config.completed_pipeline_ttl_seconds | ||
|
||
pipeline_spec.root.CopyFrom( | ||
_build_component_spec_from_component_spec_structure(component_spec)) | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# Copyright 2024 The Kubeflow Authors | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
"""Pipeline-level config options.""" | ||
|
||
|
||
class PipelineConfig(): | ||
gregsheremeta marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"""PipelineConfig contains pipeline-level config options.""" | ||
|
||
def __init__(self): | ||
self.completed_pipeline_ttl_seconds = None | ||
|
||
def set_completed_pipeline_ttl_seconds(self, seconds: int): | ||
"""Configures the duration in seconds after which the pipeline platform will | ||
do garbage collection. This is dependent on the platform's implementation, but | ||
typically involves deleting pods and higher level resources associated with the | ||
pods. | ||
|
||
Args: | ||
seconds: number of seconds for the platform to do garbage collection after | ||
the pipeline run is completed. | ||
""" | ||
self.completed_pipeline_ttl_seconds = seconds | ||
return self |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,6 +17,7 @@ | |
from typing import Callable, Optional | ||
|
||
from kfp.dsl import component_factory | ||
from kfp.dsl import pipeline_config | ||
from kfp.dsl import pipeline_task | ||
from kfp.dsl import tasks_group | ||
from kfp.dsl import utils | ||
|
@@ -27,7 +28,8 @@ def pipeline(func: Optional[Callable] = None, | |
name: Optional[str] = None, | ||
description: Optional[str] = None, | ||
pipeline_root: Optional[str] = None, | ||
display_name: Optional[str] = None) -> Callable: | ||
display_name: Optional[str] = None, | ||
pipeline_config: pipeline_config.PipelineConfig = None) -> Callable: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Optional nitpick: Add the new There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
||
"""Decorator used to construct a pipeline. | ||
|
||
Example | ||
|
@@ -57,6 +59,7 @@ def my_pipeline(a: str, b: int): | |
description=description, | ||
pipeline_root=pipeline_root, | ||
display_name=display_name, | ||
pipeline_config=pipeline_config, | ||
) | ||
|
||
if pipeline_root: | ||
|
@@ -67,6 +70,7 @@ def my_pipeline(a: str, b: int): | |
name=name, | ||
description=description, | ||
display_name=display_name, | ||
pipeline_config=pipeline_config, | ||
) | ||
|
||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as per @droctothorpe comment from yesterday's meeting, what do we think about providing a comment here about logging? something a long the lines of:
"Note: when persisted logging is unavailable, any tasks that are garbage collected will result in the loss of the associated logs for those tasks in the UI."
adding this here will surface them in api docs, fyi
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note to self: this should be a Duration
ref: https://cloud.google.com/apis/design/naming_convention#time_and_duration