[backend] Executor v2 : incorrect use of 0644 file mask when creating local dirs while retrieving artifacts from object storage ?

[francoisserra]

Environment

• How did you deploy Kubeflow Pipelines (KFP)?
  deployed Kubeflow on a GKE cluster

• KFP version:

kubeflow v1.4 branch

• KFP SDK version:

n/a

Steps to reproduce

We discovered that when local copying Minio artifacts, the Kubeflow Executor V2 creates local dirs with a 0644 mask, preventing the non-root users to access artifact dirs - and that's the case when using Kaniko to build some Docker image generated by the BentoML Framework (see bentoml/BentoML#2199)

Is is a required behaviour or is it possible to use the 0744 mask for dirs ?

pipelines/v2/objectstore/object_store.go

Line 266 in 627b37c

if err := os.MkdirAll(localDir, 0644); err != nil {

more details: When launching a pipeline step in k8s that uses Kaniko the /kaniko/executor command is wrapped by /kfp-launcher/launch command (i.e. the executor V2). This command processes the args to resolve pipeline artifacts (namely thoses begining with minio:// ) i.e. it copy from the minio bucket to the local FS of the container (under /minio/ directory. Note it is not a mount point/k8s volume) . It turns out that the dir is in drw-r--r-- mode (0644), so preventing to perform chown operations. modifying it to drwxr--r-- before launching kaniko fix the bug w/o tweaking the Dockerfile.

Expected result

use 0744 file mask when creating local dirs while downloading artifacts from minio storage.

Materials and Reference

---

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

[chensun]

Fixed by the above PR.