add assertion error formatter

resolves #386
cucumber · Dec 22, 2016 · a455741 · a455741
1 parent e973f03
commit a455741
Show file tree

Hide file tree

Showing 48 changed files with 295 additions and 247 deletions.
diff --git a/features/ambiguous_step.feature b/features/ambiguous_step.feature
@@ -17,7 +17,7 @@ Feature: Ambiguous Steps
       })
       """
     When I run cucumber.js with `-f progress`
-    Then it outputs this text:
+    Then it outputs the text:
       """
       A
 
@@ -34,4 +34,4 @@ Feature: Ambiguous Steps
       1 step (1 ambiguous)
       <duration-stat>
       """
-    And the exit status should be 1
+    And it fails
diff --git a/features/background.feature b/features/background.feature
@@ -17,7 +17,8 @@ Feature: Background
           When a scenario step
       """
     When I run cucumber.js
-    Then the scenario "a scenario" has the steps
+    Then it fails
+    And the scenario "a scenario" has the steps
       | IDENTIFIER              |
       | Given a background step |
       | When a scenario step    |
@@ -36,11 +37,12 @@ Feature: Background
           When another scenario step
       """
     When I run cucumber.js
-    Then the scenario "a scenario" has the steps
+    Then it fails
+    And the scenario "a scenario" has the steps
       | IDENTIFIER              |
       | Given a background step |
       | When a scenario step    |
-    Then the scenario "another scenario" has the steps
+    And the scenario "another scenario" has the steps
       | IDENTIFIER                 |
       | Given a background step    |
       | When another scenario step |
diff --git a/features/cli.feature b/features/cli.feature
@@ -19,7 +19,6 @@ Feature: Command line interface
       })
       """
     When I run cucumber.js with `-r step_definitions/cucumber_steps.js`
-    And the exit status should be 0
 
   Scenario: run feature with step definitions in required directory (-r option)
     Given a file named "features/a.feature" with:
@@ -37,14 +36,11 @@ Feature: Command line interface
       })
       """
     When I run cucumber.js with `-r step_definitions`
-    And the exit status should be 0
 
   Scenario: display Cucumber version
     When I run cucumber.js with `--version`
     Then I see the version of Cucumber
-    And the exit status should be 0
 
   Scenario: display help
     When I run cucumber.js with `--help`
-    Then I see the help of Cucumber
-    And the exit status should be 0
+    Then I see the help text for Cucumber
diff --git a/features/custom_formatter.feature b/features/custom_formatter.feature
@@ -35,7 +35,8 @@ Feature: custom formatter
       module.exports = SimpleFormatter
       """
     When I run cucumber-js with `--format ./simple_formatter.js`
-    Then it outputs this text:
+    Then it fails
+    And it outputs the text:
       """
       a feature / a scenario
         Given an undefined step - undefined
@@ -72,7 +73,8 @@ Feature: custom formatter
       module.exports = SimpleFormatter
       """
     When I run cucumber-js with `--format ./simple_formatter.js`
-    Then it outputs this text:
+    Then it fails
+    And it outputs the text:
       """
       a feature / a scenario
         Given an undefined step - undefined

diff --git a/features/custom_stack_trace.feature b/features/custom_stack_trace.feature
@@ -21,7 +21,7 @@ Feature: Custom stack trace
       })
       """
     When I run cucumber.js
-    Then it outputs this text:
+    Then it outputs the text:
       """
       Feature: Some feature
 
@@ -32,4 +32,3 @@ Feature: Custom stack trace
       1 step (1 passed)
       <duration-stat>
       """
-    And the exit status should be 0
diff --git a/features/dryrun_mode.feature b/features/dryrun_mode.feature
@@ -33,8 +33,10 @@ Feature: Dryrun mode
       })
       """
     When I run cucumber.js with `--dry-run`
-    Then the step "a step" has status "ambiguous"
+    Then it fails
+    And the step "a step" has status "ambiguous"
 
   Scenario: undefined step
     When I run cucumber.js with `--dry-run`
-    Then the step "a step" has status "undefined"
+    Then it fails
+    And the step "a step" has status "undefined"
diff --git a/features/fail_fast.feature b/features/fail_fast.feature
@@ -23,4 +23,4 @@ Feature: Fail fast
       """
     When I run cucumber.js with `--fail-fast`
     Then it runs the scenario "Failing"
-    And the exit status should be 1
+    And it fails
diff --git a/features/failed_scenarios.feature b/features/failed_scenarios.feature
@@ -18,7 +18,7 @@ Feature: Failed scenarios
       })
       """
     When I run cucumber.js with `-f progress`
-    Then it outputs this text:
+    Then it outputs the text:
       """
       F
 
@@ -34,4 +34,4 @@ Feature: Failed scenarios
       1 step (1 failed)
       <duration-stat>
       """
-    And the exit status should be 1
+    And it fails
diff --git a/features/failing_steps.feature b/features/failing_steps.feature
@@ -18,7 +18,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       function has 0 arguments, should have 1 (if synchronous or returning a promise) or 2 (if accepting a callback)
       """
@@ -33,7 +34,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       function has 2 arguments, should have 0 (if synchronous or returning a promise) or 1 (if accepting a callback)
       """
@@ -50,7 +52,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       my error
       """
@@ -69,7 +72,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       my error
       """
@@ -88,7 +92,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       my error
       """
@@ -107,7 +112,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       function uses multiple asynchronous interfaces: callback and promise
       """
@@ -130,7 +136,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       my error
       """
@@ -153,7 +160,8 @@ Feature: Failing steps
       })
       """
     When I run cucumber.js
-    Then the step "a failing step" failed with:
+    Then it fails
+    And the step "a failing step" failed with:
       """
       my error
       """
diff --git a/features/generator_step_definitions.feature b/features/generator_step_definitions.feature
@@ -33,8 +33,8 @@ Feature: Generator Step Definitions
 
   Scenario: without generator function runner
     When I run cucumber-js
-    Then the exit status should be 1
-    And the error output contains the text:
+    Then it fails
+    And the output contains the text:
       """
       The following hook/step definitions use generator functions:
 
@@ -61,4 +61,4 @@ Feature: Generator Step Definitions
       })
       """
     When I run cucumber-js
-    Then the exit status should be 0
+    Then it passes
diff --git a/features/handling_step_errors.feature b/features/handling_step_errors.feature
@@ -26,4 +26,4 @@ Feature: Handling step errors
       """
       { member: [Circular] }
       """
-    And the exit status should be 1
+    And it fails
diff --git a/features/hook_interface.feature b/features/hook_interface.feature
@@ -28,7 +28,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
     And the output contains the text:
       """
       function has 3 arguments, should have 0 or 1 (if synchronous or returning a promise) or 2 (if accepting a callback)
@@ -50,7 +50,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 0
+    Then it passes
 
     Examples:
       | TYPE   |
@@ -69,7 +69,7 @@ Feature: After hook interface
       }
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
 
     Examples:
       | TYPE   |
@@ -89,7 +89,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 0
+    Then it passes
 
     Examples:
       | TYPE   |
@@ -110,7 +110,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
 
     Examples:
       | TYPE   |
@@ -131,7 +131,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
 
     Examples:
       | TYPE   |
@@ -152,7 +152,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
     And the output contains the text:
       """
       function uses multiple asynchronous interfaces: callback and promise
@@ -180,7 +180,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 0
+    Then it passes
 
     Examples:
       | TYPE   |
@@ -205,7 +205,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
 
     Examples:
       | TYPE   |
@@ -230,7 +230,7 @@ Feature: After hook interface
       module.exports = hooks
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
 
     Examples:
       | TYPE   |
@@ -255,7 +255,7 @@ Feature: After hook interface
       })
       """
     When I run cucumber.js
-    And the exit status should be 1
+    Then it fails
 
     Examples:
       | TYPE   |

diff --git a/features/hook_timeouts.feature b/features/hook_timeouts.feature
@@ -29,11 +29,11 @@ Feature: Step definition timeouts
           Given a passing step
       """
     When I run cucumber.js
-    Then the output contains the text:
+    Then it fails
+    And the output contains the text:
       """
       function timed out after 500 milliseconds
       """
-    And the exit status should be 1
 
 
   Scenario: slow hooks can increase their timeout
@@ -45,7 +45,7 @@ Feature: Step definition timeouts
           Given a passing step
       """
     When I run cucumber.js
-    Then the exit status should be 0
+    Then it passes
 
 
   Scenario: changing hooks timeouts does not effect other hooks
@@ -58,8 +58,8 @@ Feature: Step definition timeouts
           Given a passing step
       """
     When I run cucumber.js
-    Then the output contains the text:
+    Then it fails
+    And the output contains the text:
       """
       function timed out after 500 milliseconds
       """
-    Then the exit status should be 1