Problems with specs2 test filters and test xml

[chaoren]

bazelbuild/intellij#155 (comment)

If I create a failing test at Foo#bar::baz, Foo#bar:: and Foo# succeeds because they're running an empty suite.

bazelbuild/intellij#155 (comment)

The specs2 test.xml seems to generate incorrect test suites right now:

com.example.TestClass#foo should::do bar
Produces:

• test suite TestClass (should be fully qualified com.example.TestClass)
• test suite foo should (should be fully qualified com.example.TestClass#foo should or something else instead of #)
• test case foo should::do bar with class name com.example.TestClass (this one seems correct)

bazelbuild/intellij#155 (comment)

If you have multiple test scopes with tests of the same name, e.g.,

TestClass
"foo" should "do blah"
"bar" should "do blah"
"baz" should "do blah"

running any of them (e.g., "--filter=TestClass#foo::do blah", "--filter=TestClass#bar::do blah", "--filter=TestClass#baz::do blah"), will only run the first one (i.e., "foo" should "do blah").

[chaoren]

@hmemcpy, I'll put further bugs in the specs2 test runner here.

[chaoren]

Correction, for the third issue, it looks like you run tests from all scopes with the same name, but only report the first one, and whichever one has an error in the test.xml.

So

If you have:

TestClass
"foo" should "do blah" pass
"bar" should "do blah" pass
"baz" should "do blah" fail
"asdf" should "do nothing" fail

And you run --test_filter=TestClass#bar should::do blah the test runner will run all of these:

"foo" should "do blah"
"bar" should "do blah"
"baz" should "do blah"

and report these in the test.xml

"foo" should "do blah" passed
"baz" should "do blah" failed

[chaoren]

The reporting problem looks like it's due to: bazelbuild/bazel#3201