Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5.18 #15237

Merged
merged 8 commits into from
Sep 7, 2019
Merged

5.18 #15237

merged 8 commits into from
Sep 7, 2019

Conversation

seamuslee001
Copy link
Contributor

No description provided.

seamuslee001 and others added 6 commits September 7, 2019 08:11
…also ensure that goal_general is not html encoded as it is a wysiwyg field
…QL 5.5

Overview
--------

This disables a particularly flaky test-scenario.

Technical Details
-----------------

The `SyntaxConformanceTest::testSqlOperators()` has been rather flaky for a
while.  (I think this goes back either to the creation of the test or the
expansion of Dedupe API -- in other words, the test of this scenario has
always been flaky.)

When the test fails, it looks like this:

```
api_v3_SyntaxConformanceTest::testSqlOperators with data set #28 ('Dedupe')
incorrect count returned from Dedupe getcount
Failed asserting that 0 matches expected 2.

/home/jenkins/bknix-min/build/build-1/web/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php:191
/home/jenkins/bknix-min/build/build-1/web/sites/all/modules/civicrm/tests/phpunit/api/v3/SyntaxConformanceTest.php:1131
/home/jenkins/bknix-min/build/build-1/web/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:215
/home/jenkins/bknix-min/civicrm-buildkit/extern/phpunit6/phpunit6.phar:570
```

I've spent a chunk of time investigating the flakiness and found:

1. The test usually (possibly always?) passes on MySQL 5.7.
2. The test often (but not always) fails on MySQL 5.5.
3. The version of PHP (php70 vs php72) does not make much difference.
4. The test never seems to fail when run by itself. The smallest
   scenario which seemd to reliably produce the failure was:
   ```
    env
      SYNTAX_CONFORMANCE_ENTITIES='ContactType Country Dedupe Domain StateProvince' \
      CIVICRM_UF=UnitTests \
      phpunit5 \
      --stop-on-failure --tap \
      tests/phpunit/api/v3/SyntaxConformanceTest.php \
      --filter '(testCustomDataGet|testLimit|testSqlOperators)'
    ```
5. The reproducibility differed on two laptops.  The laptops were largely
   identical wrt PHP/MySQL binaries+configuration (macOS,
   bknix@master-loco, `min` cfg). They differed in OS version
   and hardware specs. The slower machine (low-wattage 2-core CPU; 8gb; Mojave)
   produced test-failures much more reliably than the faster machine
   (high-wattage 4-core CPU; 16gb; Sierra).

Rationalizations for why it's OK to sip:

* The test will continue to be monitored on MySQL 5.7+.
* The test *does* pass reliably when run by itself on MySQL 5.5.
* [Civi doesn't officially support MySQL 5.5](https://docs.civicrm.org/sysadmin/en/latest/requirements/#mysql-version),
so we're not required to faff-about over a failure that only occurs on MySQL 5.5.

See also: https://chat.civicrm.org/civicrm/pl/s7nbkro8yjfgzk5z4epo8g9jph
I think it makes sense to cast here rather than require a string
dev/core#1233 Fix deleting of campaigns and saving of custom data for campaign and …
SyntaxConformanceTest::testSqlOperators - Skip "Dedupe" entity on MySQL 5.5
dev/core#1236 fix miscasting on setContactSubType
@civibot
Copy link

civibot bot commented Sep 7, 2019

(Standard links)

@civibot civibot bot added the master label Sep 7, 2019
yashodha and others added 2 commits September 7, 2019 12:45
…tandalone/tab context for create mode

fix for create grant from tab
(dev/core#1065) fix condition so that the contact field appears for s…
@eileenmcnaughton eileenmcnaughton merged commit 8d6badf into master Sep 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants