[EC2] name ec2 instances and security groups consistently

[nchammas]

Security groups created by spark-ec2 do not prepend “spark-“ to the
name.

Since naming the instances themselves is new to spark-ec2, it’s better
to change that pattern to match the existing naming pattern for the
security groups, rather than the other way around.

[AmplabJenkins]

Can one of the admins verify this patch?

[nchammas]

For what it's worth, I think spark-ec2 should somehow tag all the instances it launches with the word "Spark" so that they are easy to find in the EC2 console.

Perhaps a separate tag on each instance would suffice? Or is it OK to prepend spark- to the security group names (as well as the instance names)?

[pwendell]

Hey @nchammas - If users want to select all spark clusters, can't they just name their clusters spark-XXX? The cluster name is only used for these tags (IIRC) so it's not like there is some other place they might want to have a different name. I would worry a bit about changing the naming now because I know people automate things around these ec2 scripts and this would introduce a change that is not backwards compatible since we'd coerce spark- into the name.

I definitely think making the tagging the same everywhere is good, i.e. I like this change as it currently stands!

[nchammas]

If users want to select all spark clusters, can't they just name their clusters spark-XXX?

Absolutely, and I think that's fine.

I would worry a bit about changing the naming now because I know people automate things around these ec2 scripts and this would introduce a change that is not backwards compatible since we'd coerce spark- into the name.

Yes, that is my concern too, which is why I opted for removing the spark- from the instance tags as opposed to adding it in to the security group names.

I originally made the change to add instance name tagging in, and it's not in the 1.0.0 release, so I think it's safe to change the instance name tags as long as it makes it in to the next release.

One way or the other, the next release will be the first where spark-ec2 tags instances with names.

[nchammas]

@pwendell - Should I specifically mention you when responding, or is that not necessary? Just making sure you got my response.

Anyway, in summary, I think it is safe to merge this in for the 1.0.1 release, since instance name tagging does not exist at all in 1.0.0.

[pwendell]

Okay sounds good, I was actually okay to change the instance tags anyways because it was a very new feature, but even better if it was never in a release. So let's just merge this as-is.

BTW - this will end up in 1.1.0 which is the next feature release. 1.0.1 is just a bug-fix release that actually has already finished voting and will be packaging soon.

No need to mention me, I check all of the PR's that I've commented on if someone updates them!

[pwendell]

Jenkins, test this please.

[pwendell]

LGTM pending tests.

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

QA tests have started for PR 1344. This patch merges cleanly.
View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16504/consoleFull

[nchammas]

OK, sounds good to me!

[SparkQA]

QA results for PR 1344:
- This patch PASSES unit tests.
- This patch merges cleanly
- This patch adds no public classes

For more information see test ouptut:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16504/consoleFull

[AmplabJenkins]

Merged build finished. All automated tests passed.

[AmplabJenkins]

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16504/