Skip to content

Commit

Permalink
SDK - DSL - Deprecated output_artifact_paths parameter in ContainerOp…
Browse files Browse the repository at this point in the history
… constructor

The users should switch to file_outputs instead.
Previously `file_outputs` only supported small data outputs, but now it supports big files.
  • Loading branch information
Ark-kun committed Jul 9, 2020
1 parent c6ac83f commit fbf1967
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 9 deletions.
3 changes: 1 addition & 2 deletions sdk/python/kfp/compiler/_op_to_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -181,8 +181,7 @@ def _op_to_template(op: BaseOp):
processed_op = _process_base_ops(op)

if isinstance(op, dsl.ContainerOp):
# default output artifacts
output_artifact_paths = OrderedDict(op.output_artifact_paths)
output_artifact_paths = OrderedDict()
# This should have been as easy as output_artifact_paths.update(op.file_outputs), but the _outputs_to_json function changes the output names and we must do the same here, so that the names are the same
output_artifact_paths.update(sorted(((param.full_name, processed_op.file_outputs[param.name]) for param in processed_op.outputs.values()), key=lambda x: x[0]))

Expand Down
15 changes: 8 additions & 7 deletions sdk/python/kfp/dsl/_container_op.py
Original file line number Diff line number Diff line change
Expand Up @@ -996,13 +996,11 @@ def __init__(
At pipeline run time, the value of the artifact argument is saved to a local file with specified path.
This parameter is only needed when the input file paths are hard-coded in the program.
Otherwise it's better to pass input artifact placement paths by including artifact arguments in the command-line using the InputArgumentPath class instances.
file_outputs: Maps output labels to local file paths. At pipeline run time,
the value of a PipelineParam is saved to its corresponding local file. It's
one way for outside world to receive outputs of the container.
output_artifact_paths: Maps output artifact labels to local artifact file paths.
It has the following default artifact paths during compile time.
{'mlpipeline-ui-metadata': '/mlpipeline-ui-metadata.json',
'mlpipeline-metrics': '/mlpipeline-metrics.json'}
file_outputs: Maps output names to container local output file paths.
The system will take the data from those files and will make it available for passing to downstream tasks.
For each output in the file_outputs map there will be a corresponding output reference available in the task.outputs dictionary.
These output references can be passed to the other tasks as arguments.
output_artifact_paths: Deprecated. Maps output artifact labels to local artifact file paths. Deprecated: Use file_outputs instead. It now supports big data outputs.
is_exit_handler: Deprecated. This is no longer needed.
pvolumes: Dictionary for the user to match a path on the op's fs with a
V1Volume or it inherited type.
Expand Down Expand Up @@ -1085,6 +1083,9 @@ def _decorated(*args, **kwargs):
self.artifact_arguments = artifact_arguments
self.file_outputs = file_outputs
self.output_artifact_paths = output_artifact_paths or {}
if output_artifact_paths:
file_outputs.update(output_artifact_paths)
warnings.warn('The output_artifact_paths parameter is deprecated since SDK v0.1.32. Use the file_outputs parameter instead. file_outputs now supports outputting big data.', DeprecationWarning)

self._metadata = None

Expand Down

0 comments on commit fbf1967

Please sign in to comment.