pass ServiceAccountName in predictor to prepackaged servers initContainer

[enissay14]

To use the the new fine-grained AWS IAM roles for service accounts feature with the default pre-packaged servers SeldomDeployment example, the ServiceAccountName ( defining a service account that have the new eks.amazonaws.com/role-arn annotation like defined here https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/) needs to be passed to the generated initContainer (or any of the container in the pod).

The eks annotation doesn't work if the SeldonDeployment is defined like in the documentation here:

apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: tfserving
  namespace: testseldon
spec:
  name: mnist
  predictors:
  - componentSpecs:
    graph:
      children: []
      implementation: TENSORFLOW_SERVER
      modelUri: s3://9y167l-eks-ml-data/mnist/tf_saved_model
      serviceAccountName: iam-s3-sa
      name: mnist-model
      parameters:
        - name: signature_name
          type: STRING
          value: predict_images
        - name: model_name
          type: STRING
          value: mnist-model
    name: default
    replicas: 1lt
    replicas: 1

This is because the serviceAccountName is used to in model_initializer_injector.go controller to get the secrets defined in the Service Account only and doesn't attach it to the containers itself so that EKS can do its magic...

The workaround for now is to explicitly define all the containers and use the v0.3.0 or above of gcr.io/kfserving/storage-initializer (example below). It would be nice to keep using the pre-packaged servers with this new feature.

{
    "apiVersion": "machinelearning.seldon.io/v1alpha2",
    "kind": "SeldonDeployment",
    "metadata": {
        "labels": {
            "app": "seldon"
        },
        "name": "tfserving-mnist",
        "namespace": "testseldon"
    },
    "spec": {
        "name": "tf-mnist",
        "predictors": [
            {
                "componentSpecs": [{
                    "spec": {
                        "serviceAccountName": "iam-s3-sa",
                        "volumes": [
                			{
                				"name": "podinfo",
                				"downwardAPI": {
                					"items": [
                						{
                							"path": "annotations",
                							"fieldRef": {
                								"apiVersion": "v1",
                								"fieldPath": "metadata.annotations"
                							}
                						}
                					],
                					"defaultMode": 420
                				}
                			},
                			{
                				"name": "tfserving-provision-location",
                				"emptyDir": {}
                			}
                		],
                        "initContainers": [
                            {
                                "image": "gcr.io/kfserving/storage-initializer:latest",
                                "name": "storage-initializer",
                                "args": [
                                    "s3://9y167l-eks-ml-data/mnist/tf_saved_model", 
                                    "/mnt/models"
                                ],
                                "resources": {
                					"limits": {
                						"cpu": "1",
                						"memory": "1Gi"
                					},
                					"requests": {
                						"cpu": "100m",
                						"memory": "100Mi"
                					}
                				},
                                "volumeMounts": [
                					{
                						"name": "tfserving-provision-location",
                						"mountPath": "/mnt/models"
                					}
                				],
                				"terminationMessagePath": "/dev/termination-log",
                				"terminationMessagePolicy": "File",
                				"imagePullPolicy": "IfNotPresent"
                            }
                        ],
                        "containers": [
                            {
                                "image": "seldonio/tfserving-proxy:0.1",
                                "name": "tfserving-proxy"
                            },
                            {
                                "args": [
                                    "/usr/bin/tensorflow_model_server",
                                    "--port=7001",
                                    "--model_name=mnist-model",
                                    "--model_base_path=/mnt/models"
                                ],
                                "volumeMounts": [
                					{
                						"name": "tfserving-provision-location",
                						"readOnly": true,
                						"mountPath": "/mnt/models"
                					}
                				],
                                "image": "gcr.io/kubeflow-images-public/tensorflow-serving-1.7:v20180604-0da89b8a",
                                "name": "mnist-model",
                                "ports": [
                                    {
                                        "containerPort": 7001,
                                        "protocol": "TCP"
                                    }
                                ],
                                "resources": {
                                    "limits": {
                                        "cpu": "4",
                                        "memory": "4Gi"
                                    },
                                    "requests": {
                                        "cpu": "1",
                                        "memory": "1Gi"
                                    }
                                },
                                "securityContext": {
                                    "runAsUser": 1000
                                }
                            }
                        ],
                        "terminationGracePeriodSeconds": 1
                    }
                }],
                "graph": {
                    "name": "tfserving-proxy",
                    "endpoint": { "type" : "REST" },
                    "type": "MODEL",
                    "children": [],
                    "parameters":
                    [
                        {
                            "name":"grpc_endpoint",
                            "type":"STRING",
                            "value":"localhost:7001"
                        },
                        {
                            "name":"model_name",
                            "type":"STRING",
                            "value":"mnist-model"
                        },
                        {
                            "name":"model_output",
                            "type":"STRING",
                            "value":"scores"
                        },
                        {
                            "name":"model_input",
                            "type":"STRING",
                            "value":"images"
                        },
                        {
                            "name":"signature_name",
                            "type":"STRING",
                            "value":"predict_images"
                        }
                    ]
                },
                "name": "mnist-tfserving",
                "replicas": 1
            }
        ]
    }
}