Test matrix of ruby/rails versions on different Databases #4666

elia · 2022-10-10T11:57:23Z

Summary

Continuing from #4652, most of the work done by @cpfergus1, I'm just taking this through the finish line

parameterize the main job
add matrixes to better cover what we officially support
try to be efficient and avoid combinatorial explosion (i.e. skip combos sufficiently covered by other jobs)
this doesn't consider ruby versions older than 2.7 although the gemspec allows installing solidus with 2.5

Probably @solidusio/core-team should decide on how to interpret semver with regards to EOL dependencies such as Ruby and Rails, my suggestion would be to automatically drop support for versions that are no longer maintained.

Open to suggestions on how to streamline the matrixes 👂

ℹ️ The higher amount of runs increases the chance to incur into a flakey, this is not a fault of more coverage per-se but something to be aware of, and possibly motivation to fix them

Checklist

Check out our PR guidelines for more details.

The following are mandatory for all PRs:

I have written a thorough PR description.
I have kept my commits small and atomic.
I have used clear, explanatory commit messages.

The following are not always needed (~~cross them out~~ if they are not):

I have added automated tests to cover my changes.
I have attached screenshots to demo visual changes.
I have opened a PR to update the guides.
I have updated the readme to account for my changes.

We're seeing weird errors on ruby 3 which might come from specs stored with a slightly different and older YAML format. ``` Fetching https://github.com/jejacks0n/teaspoon.git Warning: Permanently added the ECDSA host key for IP address '140.82.114.3' to the list of known hosts. Fetching gem metadata from https://rubygems.org/............ --- ERROR REPORT TEMPLATE ------------------------------------------------------- ``` ArgumentError: undefined class/module YAML:: /usr/local/lib/ruby/site_ruby/3.0.0/rubygems/specification.rb:1263:in `load' /usr/local/lib/ruby/site_ruby/3.0.0/rubygems/specification.rb:1263:in `_load' /usr/local/lib/ruby/site_ruby/3.0.0/bundler.rb:563:in `load' /usr/local/lib/ruby/site_ruby/3.0.0/bundler.rb:563:in `load_marshal' ```

kennyadsl · 2022-10-12T05:53:35Z

A duration of 9m 38s for all those combinations is definitely acceptable.

We will discuss the EOL thing during the next core team meeting @elia, thanks!

From solidusio#4666 all CI jobs have been running with the legacy event system. We were defaulting the `legacy` parameter to `0`, which we copied over to the `USE_LEGACY_EVENTS` environment variable. However, the test suite was checking for the mere existence of the environment variable to select the legacy adapter and not for an actual `1` value. We are now: - Renaming the `legacy` parameter to `legacy_events` to be clearer. - Making the parameter a boolean one, defaulting to false. - Checking that `USE_LEGACY_EVENTS` is the string `"true"` before selecting the legacy adapter. - Using the legacy adapter for the job on Ruby 2.7 & Rails 6.0.

From solidusio#4666 all CI jobs have been running with the legacy event system. We were defaulting the `legacy` parameter to `0`, which we copied over to the `USE_LEGACY_EVENTS` environment variable. However, the test suite was checking for the mere existence of the environment variable to select the legacy adapter and not for an actual `1` value. We are now: - Renaming the `legacy` parameter to `legacy_events` to be clearer. - Making the parameter a boolean one, defaulting to false. - Checking that `USE_LEGACY_EVENTS` is the string `"true"` before selecting the legacy adapter. - Using the legacy adapter for the job on Ruby 2.7 & Rails 6.0. - Adding `-legacy_events` to the job name when selected.

From #4666 all CI jobs have been running with the legacy event system. We were defaulting the `legacy` parameter to `0`, which we copied over to the `USE_LEGACY_EVENTS` environment variable. However, the test suite was checking for the mere existence of the environment variable to select the legacy adapter and not for an actual `1` value. We are now: - Renaming the `legacy` parameter to `legacy_events` to be clearer. - Making the parameter a boolean one, defaulting to false. - Checking that `USE_LEGACY_EVENTS` is the string `"true"` before selecting the legacy adapter. - Using the legacy adapter for the job on Ruby 2.7 & Rails 6.0. - Adding `-legacy_events` to the job name when selected. (cherry picked from commit f109039)

From solidusio#4666 all CI jobs have been running with the legacy event system. We were defaulting the `legacy` parameter to `0`, which we copied over to the `USE_LEGACY_EVENTS` environment variable. However, the test suite was checking for the mere existence of the environment variable to select the legacy adapter and not for an actual `1` value. We are now: - Renaming the `legacy` parameter to `legacy_events` to be clearer. - Making the parameter a boolean one, defaulting to false. - Checking that `USE_LEGACY_EVENTS` is the string `"true"` before selecting the legacy adapter. - Using the legacy adapter for the job on Ruby 2.7 & Rails 6.0. - Adding `-legacy_events` to the job name when selected.

Since solidusio#4666, all jobs have been running with the paperclip adapter. The reason is that when the `paperclip` parameter was `false` on CircleCI, that translated into the `DISABLE_ACTIVE_STORAGE` env var being the string `"false"`. As we were checking for the mere presence of `DISABLE_ACTIVE_STORAGE`, it always evaluated to `true`. Fixes solidusio#4901

Since #4666, all jobs have been running with the paperclip adapter. The reason is that when the `paperclip` parameter was `false` on CircleCI, that translated into the `DISABLE_ACTIVE_STORAGE` env var being the string `"false"`. As we were checking for the mere presence of `DISABLE_ACTIVE_STORAGE`, it always evaluated to `true`. Fixes #4901 (cherry picked from commit af5904b)

Use a matrix to test ruby/rails/database versions

a134be3

elia force-pushed the elia+cpfergus1/matrix-ci branch from f412a98 to 6ac152c Compare October 10, 2022 12:09

elia mentioned this pull request Oct 10, 2022

Test matrix of ruby/rails versions on different Databases #4652

Closed

7 tasks

elia force-pushed the elia+cpfergus1/matrix-ci branch from 6ac152c to 5254c38 Compare October 10, 2022 12:59

jarednorman approved these changes Oct 11, 2022

View reviewed changes

adammathys approved these changes Oct 11, 2022

View reviewed changes

kennyadsl merged commit d4a0a39 into solidusio:master Oct 12, 2022

kennyadsl deleted the elia+cpfergus1/matrix-ci branch October 12, 2022 11:08

waiting-for-dev added the changelog:repository Changes to the repository not within any gem label Dec 13, 2022

kennyadsl mentioned this pull request Dec 20, 2022

[v3.2] Fix CircleCI Config #4793

Merged

7 tasks

waiting-for-dev mentioned this pull request Jan 26, 2023

Fix CI only testing with the legacy event system adapter #4880

Merged

3 tasks

waiting-for-dev mentioned this pull request Jan 30, 2023

[v3.2] Fix CI only testing with the legacy event system adapter #4888

Merged

3 tasks

waiting-for-dev mentioned this pull request Feb 6, 2023

Fix CI only testing with the paperclip adapter #4905

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test matrix of ruby/rails versions on different Databases #4666

Test matrix of ruby/rails versions on different Databases #4666

elia commented Oct 10, 2022 •

edited

Loading

kennyadsl commented Oct 12, 2022

Test matrix of ruby/rails versions on different Databases #4666

Test matrix of ruby/rails versions on different Databases #4666

Conversation

elia commented Oct 10, 2022 • edited Loading

Summary

Checklist

kennyadsl commented Oct 12, 2022

elia commented Oct 10, 2022 •

edited

Loading