403 Insufficient Permission error when writing to GCS

[SinaChavoshi]

When attempting to copy files to GCS I get an AccessDenied error however the kubeflow user service account has write access to the bucket,

sample component is as follows:

def copy_to_gcs_op(step_name='load data'):
return dsl.ContainerOp(
name = step_name,
image = GCLOUD_SDK,
command=['sh', '-c'],
arguments = [''gsutil -m cp -R gs:// "]
).apply(use_gcp_secret('user-gcp-sa'))

results in:

Copying file:///mnt/vol/retrain_logs/validation/events.out.tfevents.1547924733.tfhub-image-classifier-bjpjx-2847653555 [Content-Type=application/octet-stream]...
/ [0/2 files][ 0.0 B/ 4.6 MiB] 0% Done
AccessDeniedException: 403 Insufficient Permission
CommandException: 1 files/objects could not be transferred.

[SinaChavoshi]

Seems like this issue is now resolved as of pipelines 1.7. Just in case, you can add the secret using .apply(use_gcp_secret('')), which will mount the json auth file under '/secret/gcp-credentials/.json'

example:

def some_op(step_name='some op'):
    return dsl.ContainerOp(
        name = step_name,
        image = <image>,
        command=['sh', '-c'],
        arguments = ['''gcloud auth activate-service-account --key-file '/secret/gcp-credentials/user-gcp-sa.json' && gsutil  <some_command> ''']).apply(use_gcp_secret('user-gcp-sa'))