Let "cucumber" fail early when @Solo features fail (closes #68)

makandra · Nov 15, 2019 · b8ca5a5 · b8ca5a5
1 parent b455263
commit b8ca5a5
Show file tree

Hide file tree

Showing 6 changed files with 50 additions and 18 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,12 +5,16 @@ This project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html
 
 
 ## Unreleased
-
 ### Compatible changes
-
 ### Breaking changes
 
 
+## Unreleased
+
+### Compatible changes
+- Fixed [#68](https://github.com/makandra/geordi/issues/68): The "cucumber" command now fails early when @solo features fail.
+
+
 ## 2.6.0 2019-11-04
 
 ### Compatible changes

diff --git a/features/cucumber.feature b/features/cucumber.feature
@@ -18,6 +18,7 @@ Feature: The cucumber command
     When I run `geordi cucumber features/single.feature`
     Then the output should contain "# Running features"
       And the output should contain "> Only: features/single.feature"
+      And the output should contain "Features green."
     But the output should not contain "parallel"
 
 
@@ -60,6 +61,7 @@ Feature: The cucumber command
       And the output should contain "Rerunning failed scenarios"
       And the output should contain "Using the rerun profile"
       And the exit status should be 1
+      And the output should contain "Features failed."
 
 
   Scenario: Running all features in a given subfolder
@@ -96,10 +98,10 @@ Feature: The cucumber command
     """
 
     When I run `geordi cucumber --verbose features`
-    Then the output should contain "# Running features"
-      And the output should match /^> .*cucumber .*--tags \"~@solo\"/
-      And the output should contain "# Running @solo features"
+    Then the output should contain "# Running @solo features"
       And the output should match /^> .*cucumber .*--tags @solo/
+      And the output should contain "# Running features"
+      And the output should match /^> .*cucumber .*--tags \"~@solo\"/
 
 
   Scenario: When there are no scenarios tagged @solo, the extra run is skipped
@@ -146,6 +148,28 @@ Feature: The cucumber command
       And the output should not contain "No such file or directory"
 
 
+  Scenario: It does not start the full test run when the @solo run fails
+    Given a file named "features/step_definitions/test_steps.rb" with:
+    """
+    Given 'this test fails' do
+      raise
+    end
+    """
+    And a file named "features/failing.feature" with:
+    """
+    Feature: Failing feature
+      @solo
+      Scenario: Failing scenario
+        And this test fails
+      Scenario: Other scenario
+    """
+
+    When I run `geordi cucumber`
+    Then the output should contain "# Running @solo features"
+      And the output should contain "Features failed."
+    But the output should not contain "# Running features"
+
+
   Scenario: Specifying a firefox version to use
     Given a file named "features/sub/one.feature" with:
     """

diff --git a/features/firefox.feature b/features/firefox.feature
@@ -6,7 +6,7 @@ Feature: The firefox/chrome command
     Given a file named "testfile" with "testcontent"
 
     When I run `geordi firefox cat testfile`
-    Then the output should contain "> VNC is ready to hold Selenium test browsers. Use `geordi vnc` to view them."
+    Then the output should contain "> Run `geordi vnc` to view the Selenium test browsers"
       And the output should contain "testcontent"
     But the output should not contain "Firefox for Selenium"
 
@@ -15,7 +15,7 @@ Feature: The firefox/chrome command
     Given a file named "testfile" with "testcontent"
 
     When I run `geordi chrome cat testfile`
-    Then the output should contain "> VNC is ready to hold Selenium test browsers. Use `geordi vnc` to view them."
+    Then the output should contain "> Run `geordi vnc` to view the Selenium test browsers"
       And the output should contain "testcontent"
 
 
@@ -48,7 +48,7 @@ Feature: The firefox/chrome command
       And a file named "testfile" with "testcontent"
 
     When I run `geordi firefox cat testfile`
-    Then the output should contain "> VNC is ready"
+    Then the output should contain "> Run `geordi vnc`"
       And the output should contain "testcontent"
     But the output should not contain "Firefox for Selenium"
 

diff --git a/lib/geordi/commands/cucumber.rb b/lib/geordi/commands/cucumber.rb
@@ -54,9 +54,7 @@ def cucumber(*args)
     cmd_opts, files = args.partition { |f| f.start_with? '-' }
     cmd_opts << '--format' << 'pretty' << '--backtrace' if options.debug
 
-    announce 'Running features'
-
-    # Serial run of @solo scenarios
+    # Serial run of @solo scenarios ############################################
     if files.any? { |f| f.include? ':' }
       note '@solo run skipped when called with line numbers' if options.verbose
     else
@@ -73,23 +71,29 @@ def cucumber(*args)
         solo_cmd_opts << '--tags' << '@solo'
 
         announce 'Running @solo features'
-        Geordi::Cucumber.new.run files, solo_cmd_opts, :verbose => options.verbose, :parallel => false
+        solo_success = Geordi::Cucumber.new.run files, solo_cmd_opts, :verbose => options.verbose, :parallel => false
+        solo_success or fail 'Features failed.'
       end
     end
 
-    # Normal run
-    unless Geordi::Cucumber.new.run(files, cmd_opts, :verbose => options.verbose)
+    # Parallel run of all given features + reruns ##############################
+    announce 'Running features'
+    normal_run_successful = Geordi::Cucumber.new.run(files, cmd_opts, :verbose => options.verbose)
+
+    unless normal_run_successful
       cmd_opts << '--profile' << 'rerun'
 
       # Reruns
-      (1 + options.rerun).times do |i|
+      (options.rerun + 1).times do |i|
         fail 'Features failed.' if (i == options.rerun) # All reruns done?
 
         announce "Rerun ##{ i + 1 } of #{ options.rerun }"
         break if Geordi::Cucumber.new.run(files, cmd_opts, :verbose => options.verbose, :parallel => false)
       end
     end
 
+    success 'Features green.'
+
   else
     note 'Cucumber not employed.'
   end

diff --git a/lib/geordi/commands/tests.rb b/lib/geordi/commands/tests.rb
@@ -2,9 +2,9 @@
 def tests
   rake_result = invoke_cmd 'with_rake'
 
+  # Since `rake` usually is configured to run all tests, only run them if `rake`
+  # did not perform
   if rake_result == :did_not_perform
-    # Since `rake` usually runs all tests, only run them here if `rake` did not
-    # perform
     invoke_cmd 'unit'
     invoke_cmd 'rspec'
     invoke_cmd 'cucumber'

diff --git a/lib/geordi/cucumber.rb b/lib/geordi/cucumber.rb
@@ -60,7 +60,7 @@ def setup_vnc
         ENV["BROWSER"] = ENV["LAUNCHY_BROWSER"] = File.expand_path('../../../bin/launchy_browser', __FILE__)
         ENV["DISPLAY"] = VNC_DISPLAY
 
-        note 'VNC is ready to hold Selenium test browsers. Use `geordi vnc` to view them.'
+        note 'Run `geordi vnc` to view the Selenium test browsers'
       end
     end