Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give generated exit-handler a more special name #78

Closed
rileyjbauer opened this issue Nov 6, 2018 · 4 comments
Closed

Give generated exit-handler a more special name #78

rileyjbauer opened this issue Nov 6, 2018 · 4 comments

Comments

@rileyjbauer
Copy link
Contributor

If an exit handler is specified in the DSL, the compiler currently uses a bit of a hack to ensure that it is always called before the pipeline terminates: an extra DAG is added to the compiled yaml that is named something like exit-handler-1 which is the only task of the entrypoint DAG, and wraps all other steps within the pipeline (they are tasks within the exit handler's DAG).

This works alright as a workaround, but it clutters the UI in a rather unhelpful way, so we currently hide this node by checking if the name starts with "exit-handler".

This is potentially problematic as it's not the most unlikely name for a user to pick, so maybe something like "__exit-handler" would be better until a better solution is found to the exit handler problem.

@vicaire
Copy link
Contributor

vicaire commented Mar 26, 2019

The Argo yaml should have the information describing what the exit handlers are. Could the UI extract this information from the Argo yaml instead of relying on some special convention with the DSL? Since users will be able to specify the yaml without the DSL, this seems needed.

@rileyjbauer
Copy link
Contributor Author

IIRC, the problem is that the DSL and Argo don't have exactly the same conception of exit handlers. Argo supports exit handlers via the onExit key which just points to a template to run after the workflow has otherwise ended, but as I mentioned above, the DSL explicitly wraps the entire pipeline in an exit handler in the form of a DAG template. I do not know exactly why this is done

@vicaire
Copy link
Contributor

vicaire commented Mar 27, 2019

Thank you for the clarification @rileyjbauer.

@rileyjbauer
Copy link
Contributor Author

@gaoning777 can you comment on why the DSL implements exit-handling like this?

magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this issue Oct 22, 2023
magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this issue Oct 22, 2023
This reverts commit b6ec6e65440e19c99637f923ac099ce716a0bceb.
HumairAK added a commit to HumairAK/data-science-pipelines that referenced this issue Oct 10, 2024
UPSTREAM: <carry>: Rebase code to kfp 2.2.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants