New Integration tests

[JalilArfaoui]

This PR is an attempt to modernize our integration tests, with the following expected benefits :

• use BDD written feature so minimize the gap between domain experts and developers
• CLI only launch (no need to touch the browser)
• As close as possible to real world
• (Later) Embedded in CI pipeline so that regression check is done all the time

It's based on cucumber, puppeteer and expect.

For those who are not so familiar with Cucumber, it works by feeding the folder ./features with specifications by example with a natural language called Gherkin. I currently made one really simple :

To run in, start with yarn start as usual and then yarn test:e2e in a second terminal. The extension is automatically loaded (an refreshed) in the browser.

Feature: Matching context
  In order to read relevant notice
  As a user
  I want to see notices that match my current tab context

  Scenario: Open Pixmania homepage
    When I open the url "https://pixmania.fr"
    Then I see the notification within 10 seconds
      And The first notice has text "Que choisir signale que de nombreux clients mécontents"

If all scenario's steps are already defined, nothing more to do.
It not, then we have to define missing steps in ./features/step_definitions

BREAKING:
Now yarn test launches unit tests + integration tests
To launch only unit tests, use yarn test:unit
To launch only integration tests, use yarn test:e2e

If you find this useful and going in the right direction, the next steps are:

• Add a yarn test:ci that runs the tests headless (without Chrome showing)
• Migrate all existing tests in ./test/integration to this new format
• Use page objects/helpers to simplify step_definitions writing/maintainance
• Improve reporter/progress
• Begin every new feature or rule change by the corresponding scenario :-)

[bmenant]

I like the general idea, as it would greatly enhance/simplify this QA checklist: https://docs.google.com/spreadsheets/d/1x1URoKD8Hl5BbphKhF_xabcP7YtI2XqT7ffhMFM8zL8/edit?pli=1#gid=0

If all scenario's steps are already defined, nothing more to do.
If not, then we have to define missing steps in ./features/step_definitions

[...]

• Use page objects/helpers to simplify step_definitions writing/maintainance

Can you be more specific?
I guess it’s related to the fact there’re many scenario steps that are repeated over many cases...

[JalilArfaoui]

Can you be more specific?
I guess it’s related to the fact there’re many scenario steps that are repeated over many cases...

Exactly. Consider the following existing and working scenarios:

Scenario: Open Pixmania homepage
  When I open the url "https://pixmania.fr"
  Then I see the notification within 10 seconds
    And The first notice has text "Que choisir signale que de nombreux clients mécontents"

Scenario: Open Fnac homepage
  When I open the url "https://fnac.com"
  Then I don't see the notification

This suppose that the following steps are defined :

  When(/^I open the url "(.+)"$/, async function(url: string) => { ... })
  Then(/^I see the notification within (\d+) seconds$/, async function(delay: number) => { ... })
  Then(/^I don't see the notification$/, async function() => { ... })
  Then(/^The first notice has text "(.+)"$/, async function(text: string) => { ... })

This means that you can add the following scenario without having to write or change any step definition:

Scenario: Open bmenant test page
  When I open the url "http://tests.menant-benjamin.fr/"
  Then I see the notification within 10 seconds
    And The first notice has text "Le tribunal dira mardi s'il confirme des ordonnances prises"

But if you want to add :

Scenario: Open Pixmania homepage
  When I open the url "http://tests.menant-benjamin.fr/"
  Then I see the notification within 10 seconds
    And The first notice has intention approval

Then we have to write a new step definition :

Then(/^The first notice has intention (approval|disapproval|...)$/, async function(intention: Intention) => {...})

This is the re-usability advantage of Cucumber.

[lutangar]

Looks promising!
Just an overall feedback, I'm not sure Bulle should be translated to Bubble. @bmenant

[lutangar]

Should be rebased and refresh a little bit.

[JalilArfaoui]

Totally ! I’ll try to get that done in the next weeks …

[christpet]

@JalilArfaoui is this still in progress?

[MaartenLMEM]

page to test : https://classic.yarnpkg.com/en/docs/pnp/
instead of pixmania
Has to appear if you follow Maarten (who relays Jalil's bubble)

[JalilArfaoui]

2 years after the first draft, this is a major revamp :

• Adapt to code changes
• Add subscriptions mechanism
• Support profiles app
• Use Page Objects
• Make it work in Docker/CI <-- This was hard