Simplify template hierarchy - enable multiple versions

[vincent-psarga]

Note: this is work in progress and the goal is to get ideas.

TLDR: it's a pain to support a new version of a test framework without breaking code for the users. Let's tackle that, but you'll have to read stuff :D

The problem

So, one of the current problem of hiptest-publisher today is the difficulty to handle multiple versions of a given language/framework.
For example, for all Cucumber frameworks, we do not yet use cucumber-expressions and still generate regular expressions. Same thing, we don't support newer Gherkin syntax.

If we look at hps-* repositories, we have a good idea of the problem, we're testing with old versions of the frameworks as we can not really update them ...

Of course we could do major releases every now and then and stop supporting old frameworks (only the latest one) but that does not really look like a good idea.

The proposed solution

The idea here is first to stop having weird hierarchy of templates (for example gherkin/java/ or cucumber/java/actionwords/ for example).

There are now three main folders in the templates:

• common: this is for common things (boolean, strings, empty output etc). That does not change much
• languages: should generate code for a language (java, python etc) without taking care of the test framework (so it will not generate tests, folders etc). special case (of course): Gherkin is a language (as is Robot framework syntax) which generates tests
• frameworks: should generate the code specific to the given framework. For example, cucumber-ruby and cucumber-java are now two different frameworks.

Now, inside a folder (for a language or a framework, principles is the same), we'll find templates for each version, eg:

 - ruby/
    - version-2.3
       - actionword.hbs
       - call.hbs
    - version-2.4
      - call.hbs
   ...

Currently, we always pick the first one, but the idea on the long term will be to have the possibility to specify a version and hiptest-publisher will find the template that is needed.
If a template is not found for the asked version, we'll try previous versions.

In the previous example, if we ask to render a call with ruby version 2.4, it'll use ruby/version-2.4/call.hbs.
If we ask to render an action word, it'll use ruby/version-2.3/actionword.hbs.

So logically we should not have to copy-paste templates between each versions, only override with the changes.

Now, two things a bit particular:

flavors:

Languages can have flavors, basically a different way of writing things but not a different version. Currently, we have two examples of those flavors in gherkin:

 - gherkin
   - version-3.0
      - inlined_uid_flavor
         _scenario_header.hbs
     - no_empty_scenario_flavor
        scenario.hbs
     scenario.hbs

cucumber-java uses those two flavors which makes the Gherkin rendering a bit different from the one in cucumber-ruby

specific rendering for group languages

When exporting, we have different groups: feature, step_definitions and actionwords for Gherkin based frameworks, tests and actionwords for the other ones.

When the rendering for the same node needs to be different based on the group (usual case is for gherkin based framework: we render each actionword once as a function, once with the regex to match the feature file), it is possible to specify it in sub-folders.
For example for cucumber-java framework:

 - cucumber-java
   - version-1.2
      - actionwords
         actionword.hbs # Will render the action word as a function
      - step_definition
        actionword.hbs # Will render a call to the actionwords class and the regex to match the text

What need to be done

So far, this approach has only been applied to ruby (RSpec, minitest and cucumber) and cucumber-java.
If multiple versions of a language/framework exist, fallback to previous versions if the template does not exist.

I still need to figure out some priorities when it comes to selecting the template (when we'll have multiple versions + flavors for example).

We need a way to ease migration for users who currently override templates (we've started a discussion with @cbliard about that, I think it'll be another PR soon ;) )

We need a way to select the language/framework version directly from CLI (template_finder supports it, but we just pick the first one now)

What it will change

Lots of things hopefully. Mainly it should simplify finding which templates does what and, of course, be able to use multiple versions for a language and/or framework.

We should also be able to fine tune the exports (for example by letting users select the flavors they want)
The languageand framework options should disappear to use a single option. That will remove weird stuff (like --language=cucumber --framework=java should be framework=cucumber-java) and make things simpler.

Also the config files for the different languages should be easier to read:

[features]
template_dirs = gherkin, common
gherkin_flavors = inlined_uids, no_empty_scenarios

[step_definitions]
template_dirs = cucumber-java, java, common

[actionwords]
template_dirs = cucumber-java, java, common

Most of the time, it'll just be framework-name, language-name, commonand eventually some flavors for languages.

[TimotheeVille]

Is it really important to have no duplicates in templates? Actually I found it harder to read. Let say actionword for ruby is same since 2 version and step only from previous one, we should have 3 version opened to have a global view of how it is rendered for the last version. And we should also search what is the last version that have the template rendered.

[vincent-psarga]

Is it really important to have no duplicates in templates? Actually I found it harder to read. Let say actionword for ruby is same since 2 version and step only from previous one, we should have 3 version opened to have a global view of how it is rendered for the last version. And we should also search what is the last version that have the template rendered.

That's a good question :)
One of the things we've discussed with @cbliard and that may (should ?) be integrated in this PR is an easy way to find/copy templates used during the generation process.

For the duplication, I'd rather avoid it as much as possible for maintenance purposes. Also much of the templates won't change over time, I don't think we'll change more than two or three template from version to version.

[cbliard]

Hi @vincent-psarga,

Is this PR still relevant?

[benjamin-ltr]

approved so it can disappear from my daily notifications :p