[feature] Mounting emptyDir option on kfp v2 workflow-task using kfp-kubernetes

[harshad16]

Feature Area

/area sdk

What feature would you like to see?

In kfp-kubernetes SDK can we include add_volume function.
as in the kfp dsl, we had the option in kfp v1

pipelines/sdk/python/kfp/deprecated/dsl/_container_op.py

Line 934 in f80bfd4

def add_volume(self, volume):

Use case is to have an option to volumes to the workflow tasks.

What is the use case or pain point?

In (kfp v2) kfp-kubernetes only provides the add_pvc and mount_pvc options.
As currently , there is no option to mount , emptyDir
secretes and configmaps can be mounted separately.

https://github.com/kubeflow/pipelines/blob/master/kubernetes_platform/python/kfp/kubernetes/volume.py

Is there a workaround currently?

No.
use older deprecated version.

---

Love this idea? Give it a 👍.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[gregsheremeta]

not stale ... i'm working on it

[thesuperzapper]

It looks like there is a PR to add this important feature in #10913, but I am not sure about its status.

Can one of @gregsheremeta @hbelmiro @droctothorpe @rimolive confirm what we need to do to get that merged?

---

If you are wondering why this is important, it's because PyTorch expects there to be a tempfs mounted at /dev/shm, and the only way to ensure this in a Kubernetes Pod is to mount an emptyDir with medium="Memory" to it.

[gregsheremeta]

confirm what we need to do to get that merged?

We need one of the Google maintainers to review it. Chen is out until July 22. It doesn't appear that Connor works on this project anymore.

[haiminh2001]

Hi all, can I ask when will these feature be released ? And I expect that this feature only requires me to update my python KFP SDK, is that true?

[gregsheremeta]

The change requires both a backend update and an SDK update. The project doesn't currently have a fixed release schedule, but I'll ask about the next release at this week's KFP community meeting (tomorrow, Wednesday August 28 at 1PM US Eastern Time, agenda)

[haiminh2001]

@gregsheremeta great! Thank you.

[gregsheremeta]

Update from the meeting: @chensun is ready to to both backend and SDK releases. There's currently a failing test blocking him, but as soon as that's fixed, they should be out. I don't have an ETA but shouldn't be too long.

[gregsheremeta]

I don't have an ETA but shouldn't be too long.

#11148

[haiminh2001]

@gregsheremeta thank you for your update.