No tests are discovered if feature starts with comments followed by description

[fslev]

👓 What did you see?

It seems that (at least) the junit-platform-suite fails to discover tests if any feature starts with a comment followed by a description

✅ What did you expect to see?

Tests should be discovered since syntax is correct

📦 Which tool/library version are you using?

Cucumber for java 7.8.1

🔬 How could we reproduce it?

Steps to reproduce the behavior:

1. A feature which starts with a comment followed by a description

Feature: Feature 2
  # some comment
  some description

  Scenario: Scenario 1
    * Wait 3s

3. Run feature, either from Maven or from the IDE. You may also try to execute the whole suite of tests containing this Feature. No test will be executed.
4. See error from maven surefire plugin:

[ERROR] There was an error in the forked process
[ERROR] TestEngine with ID 'junit-platform-suite' failed to discover tests
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR] TestEngine with ID 'junit-platform-suite' failed to discover tests
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:701)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:311)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:268)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1334)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1167)

You can reproduce this issue by building the following cucumber-showcase project:
https://github.com/fslev/cucumber-showcase.git

mvn clean test

[fslev]

If feature starts with description followed by comment, or if it starts only with a comment (without description), then everything works fine

Feature: Feature 2
  some description
  # some comment

works !

Feature: Feature 2
 # some comment

also works !

[mpkorstanje]

Run feature, either from Maven or from the IDE. You may also try to execute the whole suite of tests containing this Feature. No test will be executed.

These are very different execution paths.

With the @Suite I see:

Caused by: io.cucumber.core.gherkin.FeatureParserException: Failed to parse resource at: classpath:io/cucumber/skeleton/belly.feature
(3:3): expected: #EOF, #Comment, #BackgroundLine, #TagLine, #ScenarioLine, #RuleLine, #Empty, got 'some description'
	at io.cucumber.core.gherkin.messages.GherkinMessagesFeatureParser.parse(GherkinMessagesFeatureParser.java:43)

Though with the CLI the parse error is silently dropped.

[fslev]

So, after all this is an invalid syntax:

Feature: Some feature
# some comment
With some description

, but this is valid ?

Feature: Some feature
# some comment

[mpkorstanje]

Gherkin has a grammar that explicitly includes comments and comments are not allowed before the description.

https://github.com/cucumber/common/blob/main/gherkin/gherkin.berp#L37

DescriptionHelper := #Empty* Description? #Comment*

You may want to create an issue in https://github.com/cucumber/common to address that.

[fslev]

By upgrading the cucumber-showcase project from above to 7.9.0 and building it via Maven, it seems the feature parse error is still not displayed

mvn clean test

[ERROR] TestEngine with ID 'junit-platform-suite' failed to discover tests
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR] TestEngine with ID 'junit-platform-suite' failed to discover tests

However, using the -e option , the error is now displayed

mvn clean test -e

TestEngine with ID 'junit-platform-suite' failed to discover tests
org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-platform-suite' failed to discover tests

Caused by: io.cucumber.core.gherkin.FeatureParserException: Failed to parse resource at: classpath:io/example/Feature2.feature
(3:3): expected: #EOF, #Comment, #BackgroundLine, #TagLine, #ScenarioLine, #RuleLine, #Empty, got 'some description'

Though, this might be expected..

[mpkorstanje]

Yeah, that's between JUnit 5 and Surefire. You could petition the JUnit 5 team to include more information in the JUnitException thrown when discovery fails.