set securityContext on each driver and pod

[cometta]

May i know is it possible to customize security Context for each driver and pod spun up by lighter? i use below on lighter session config, but does not work

"spark.kubernetes.driver.podTemplateFile": "s3a://buckert/pod_template.yml",
"spark.kubernetes.executor.podTemplateFile": "s3a://bucket/pod_template.yml",

[pdambrauskas]

We always override those to

"spark.kubernetes.driver.podTemplateFile": "/home/app/k8s/driver_pod_template.yaml",
"spark.kubernetes.executor.podTemplateFile": "/home/app/k8s/executor_pod_template.yaml",

We'll try make those configurable on next Lighter release.

As a workaround you can create a configmap for those files and mount them to the /home/app/k8s/ path on your lighter pods.