clean nodegroup deletion

[errordeveloper]

As mentioned earlier in #523 (comment), we need to devise a better strategy for nodegroup deletion, which should also help us with #103 and #274 (as well as other related issues).

So we should try implementing the following strategy:

• for all nodegroups
  • taint the node and drain pods
• delete all daemonsets
• for all nodegroups
  • scale nodes to zero
  • delete dangling ENIs
  • look for ELBs, ALBs and EBS volumes and delete those
    • also ALBs created through Ingress
• delete each nodegroup stack
• delete cluster stack

We may do something different for deleting individual nodegroups, e.g. only scale drain, taint and down, but we do need to implement this first, as deletion is an important aspect of user experience.

related: #2278, #2172

[errordeveloper]

Maybe related to aws/amazon-vpc-cni-k8s#69.

[errordeveloper]

Fixing this would pave the road towards fixing other issue with deletions.

[gemagomez]

@2opremio is this finished?

[2opremio]

Nope, I fixed #103 in a different way and haven't tackled #274 yet for which we may follow a different approach.

[nikimanoledaki]

What was left to do was draining the nodegroups and deleting the daemonsets. This was addressed so this issue can be closed now.