TFX-KFP: created experiement which can't be deleted and stop upload another one

[rmgogogo]

What steps did you take:

tfx pipeline create --engine kubeflow ...

It will create/upload a KFP pipeline to KPF server. Meanwhile it also create an experiment with the same name of the pipeline.

I delete the pipeline so that "tfx pipeline create" can create a newer one.

What happened:

I can't delete the Experiment from the UI.
Run "tfx pipeline create" gave me error msg: pipeline already exist.

What did you expect to happen:

1. I can delete Experience
2. "tfx pipeline create" dont' create experiment

Environment:

How did you deploy Kubeflow Pipelines (KFP)?

KFP version:

KFP SDK version:

Anything else you would like to add:

[Miscellaneous information that will assist in solving the issue.]

/kind bug

[rmgogogo]

@jingzhang36 is this issue already been processing by you and Jiyong?
Remember we mentioned similar case before but I didn't expect it also create Experiment.

[numerology]

@rmgogogo
I think pipeline already exists error is triggered because the local pipeline file was not cleaned up. It's not sufficient to delete the pipeline in KFP server but also some local pipeline files need to be cleaned up as well.

You can refer to the code at https://github.com/tensorflow/tfx/blob/master/tfx/tools/cli/handler/base_handler.py#L186

BTW, the 'blessed way' of updating a pipeline should be invoking tfx pipeline update IMO

[Bobgy]

@rmgogogo From the error message, I cannot find any mention of experiment. Did you misread the message?

[rmgogogo]

@rmgogogo From the error message, I cannot find any mention of experiment. Did you misread the message?

"tfx pipeline create --engine kubeflow ..." will give you a new experiment (not default) with the same name of the pipeline and I can't find a way to delete it.

[rmgogogo]

I expect "tfx pipeline create --engine kubeflow ..." just create pipeline in default experiment.
Or allow me to delete the auto-created experiment.

[rmgogogo]

@jingzhang36 believe it has some BE API issue related with this path.

[Bobgy]

I see, added area/backend label too

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.