Ornery++

[eileenmcnaughton]

Overview

This flags some particularly slow tests in the apiv3SyntaxConfirmanceTest so they do not run on PR runs. These tests used to be a good little canaries but have grown soft & fat as we improved our code & no longer have the stamina to run several times a day in a timely manner

Before

On PRs submitted about the same time

After

On this PR apiv3 time taken

& with the second ornery

Technical Details

There are a couple of other tests we could do this to but with this change the apiv3 tests are faster than the crm test so there might not be much to gain from further apiv3 test speed improvements

Comments

[civibot]

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers...

• ➡️ PR Demo site (can be rebuilt with the command test this please).
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

[eileenmcnaughton]

test this please

[colemanw]

@civicrm-builder retest this please

[colemanw]

api_v3_SystemCheckTest::testSystemCheckBasic with data set "APIv3" (3)
Exception: CRM_Core_Exception: "checkDefaultMailbox not in results"

@eileenmcnaughton I suspect the test failure has to do with the component not being enabled. Probably the syntax conformance tests would enable all components & then other tests would just blithely assume they are always enabled.

[eileenmcnaughton]

@colemanw that makes sense - I had concluded the test was not improving things as much as I hoped (the screenshots above are not consistent) & that the one that creates the custom data sets might be a bigger win - but then all these tests .....

[eileenmcnaughton]

@colemanw I got it!