add terraform script to auto deploy TiDB cluster on AWS

[tennix]

This PR adds initial support for one-click deploying Cloud TiDB on AWS. PTAL @gregwebs @onlymellb @weekface

[gregwebs]

This looks great! Rather than having it under a directory cloud/ I think it will make more sense to have a directory deploy/: that name would also make sense for an on-prem deploy.

[c4pt0r]

Awesome! AWS finally got some love!

[c4pt0r]

Block at creating tiller.

...

null_resource.setup-env: Still creating... (4m40s elapsed)
null_resource.setup-env: Still creating... (4m50s elapsed)
null_resource.setup-env: Still creating... (5m0s elapsed)
null_resource.setup-env (local-exec): Error: tiller was not found. polling deadline exceeded

Error: Error applying plan:

1 error(s) occurred:

* null_resource.setup-env: Error running command 'kubectl apply -f manifests/crd.yaml
kubectl apply -f manifests/local-volume-provisioner.yaml
kubectl apply -f manifests/gp2-storageclass.yaml
kubectl apply -f manifests/tiller-rbac.yaml
helm init --service-account tiller --upgrade --wait
': exit status 1. Output: customresourcedefinition.apiextensions.k8s.io/tidbclusters.pingcap.com unchanged
storageclass.storage.k8s.io/local-storage unchanged
configmap/local-provisioner-config unchanged
daemonset.extensions/local-volume-provisioner unchanged
serviceaccount/local-storage-admin unchanged
clusterrolebinding.rbac.authorization.k8s.io/local-storage-provisioner-pv-binding unchanged
clusterrole.rbac.authorization.k8s.io/local-storage-provisioner-node-clusterrole unchanged
clusterrolebinding.rbac.authorization.k8s.io/local-storage-provisioner-node-binding unchanged
storageclass.storage.k8s.io/ebs-gp2 unchanged
serviceaccount/tiller unchanged
clusterrolebinding.rbac.authorization.k8s.io/tiller-clusterrolebinding configured
$HELM_HOME has been configured at /Users/dongxu/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Error: tiller was not found. polling deadline exceeded


Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

[gregwebs]

This should be integrated with the existing documentation for deploying to AWS EKS: https://github.com/pingcap/tidb-operator/blob/master/docs/aws-eks-tutorial.md

[c4pt0r]

Yes, it should be. But this PR still needs a lot of polish.

[tennix]

@c4pt0r Did you follow the readme instructions to install aws-iam-authenticator?

[tennix]

@gregwebs I've updated the code to move terraform scripts to deploy directory as you suggested.

[c4pt0r]

null_resource.setup-env (local-exec): Happy Helming!
null_resource.setup-env: Creation complete after 1m28s (ID: 1960644279543843491)
helm_release.tidb-operator: Creating...
  chart:            "" => "/home/dongxu/tidb-operator/deploy/aws/charts/tidb-operator"
  disable_webhooks: "" => "false"
  force_update:     "" => "false"
  metadata.#:       "" => "<computed>"
  name:             "" => "tidb-operator"
  namespace:        "" => "tidb-admin"
  recreate_pods:    "" => "false"
  reuse:            "" => "false"
  reuse_values:     "" => "false"
  status:           "" => "DEPLOYED"
  timeout:          "" => "300"
  verify:           "" => "false"
  version:          "" => "0.1.0"
  wait:             "" => "true"

Error: Error applying plan:

1 error(s) occurred:

* helm_release.tidb-operator: 1 error(s) occurred:

* helm_release.tidb-operator: error installing: the server could not find the requested resource (post deployments.extensions)

[c4pt0r]

I'm sure I successfully installed aws-iam-authenticator

[tennix]

@c4pt0r Could you try to run terraform apply again? The helm install may fail in the first time, I'm still figuring out how to prevent it.

[c4pt0r]

Ok....but it really confused me...why 2 times?

[gregwebs]

I left a comment on #436 wondering about our approach to templating the helm values file.

[c4pt0r]

LGTM.

[gregwebs]

I think we should template only the values necessary (e.g. cluster version is not) and let the user change the helm values directly. They need to know how to do that anyways.

[tennix]

@gregwebs The customize section already documents about that. Because we must use the specified storage class for each component, we do not want to let users modify that.

[gregwebs]

@tennix yes. But why do you expose tidb_version? It seems like this is not essential but instead making up for a UX issue with our helm values?

It seems like you are figuring out how to query K8s for information to put into terraform: could we eventually do that for the pd_count, etc also?

[tennix]

The current deployment is for single TiDB cluster. We need another setup and documentation for deploying multiple TiDB clusters in the same EKS cluster.