Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
SyntaxConformanceTest::testSqlOperators - Skip "Dedupe" entity on MyS…
…QL 5.5 Overview -------- This disables a particularly flaky test-scenario. Technical Details ----------------- The `SyntaxConformanceTest::testSqlOperators()` has been rather flaky for a while. (I think this goes back either to the creation of the test or the expansion of Dedupe API -- in other words, the test of this scenario has always been flaky.) When the test fails, it looks like this: ``` api_v3_SyntaxConformanceTest::testSqlOperators with data set #28 ('Dedupe') incorrect count returned from Dedupe getcount Failed asserting that 0 matches expected 2. /home/jenkins/bknix-min/build/build-1/web/sites/all/modules/civicrm/Civi/Test/Api3TestTrait.php:191 /home/jenkins/bknix-min/build/build-1/web/sites/all/modules/civicrm/tests/phpunit/api/v3/SyntaxConformanceTest.php:1131 /home/jenkins/bknix-min/build/build-1/web/sites/all/modules/civicrm/tests/phpunit/CiviTest/CiviUnitTestCase.php:215 /home/jenkins/bknix-min/civicrm-buildkit/extern/phpunit6/phpunit6.phar:570 ``` I've spent a chunk of time investigating the flakiness and found: 1. The test usually (possibly always?) passes on MySQL 5.7. 2. The test often (but not always) fails on MySQL 5.5. 3. The version of PHP (php70 vs php72) does not make much difference. 4. The test never seems to fail when run by itself. The smallest scenario which seemd to reliably produce the failure was: ``` env SYNTAX_CONFORMANCE_ENTITIES='ContactType Country Dedupe Domain StateProvince' \ CIVICRM_UF=UnitTests \ phpunit5 \ --stop-on-failure --tap \ tests/phpunit/api/v3/SyntaxConformanceTest.php \ --filter '(testCustomDataGet|testLimit|testSqlOperators)' ``` 5. The reproducibility differed on two laptops. The laptops were largely identical wrt PHP/MySQL binaries+configuration (macOS, bknix@master-loco, `min` cfg). They differed in OS version and hardware specs. The slower machine (low-wattage 2-core CPU; 8gb; Mojave) produced test-failures much more reliably than the faster machine (high-wattage 4-core CPU; 16gb; Sierra). Rationalizations for why it's OK to sip: * The test will continue to be monitored on MySQL 5.7+. * The test *does* pass reliably when run by itself on MySQL 5.5. * [Civi doesn't officially support MySQL 5.5](https://docs.civicrm.org/sysadmin/en/latest/requirements/#mysql-version), so we're not required to faff-about over a failure that only occurs on MySQL 5.5. See also: https://chat.civicrm.org/civicrm/pl/s7nbkro8yjfgzk5z4epo8g9jph
- Loading branch information