dev/core#423 MySQL 5.7 complains about 4.7.19 upgrade script

[kenwest]

MySQL 5.7 may bork when comparing datetime columns to '0000-00-00 00:00:00' so cast the column to a CHAR(20) when comparing

Overview

MySQL 5.7 complains "Incorrect datetime value: '0000-00-00 00:00:00' for column 'trxn_date'"

The error is raised by 2 lines of SQL in civicrm/CRM/Upgrade/Incremental/sql/4.7.19.mysql.tpl ...

UPDATE civicrm_financial_trxn SET trxn_date = NULL WHERE trxn_date = '0000-00-00 00:00:00';
UPDATE civicrm_contribution SET receive_date = NULL WHERE receive_date = '0000-00-00 00:00:00';

Before

MySQL 5.7 complains during the upgrade

After

Upgrade successful

Technical Details

See https://stackoverflow.com/questions/35565128/mysql-incorrect-datetime-value-0000-00-00-000000/37780259#37780259

Comments

I haven't tested this change on other versions of MySQL

[civibot]

(Standard links)

• If this is your first pull-request for CiviCRM, please browse CONTRIBUTING.md for information about the development and testing processes.
• If you are reviewing this pull-request, you may wish to consult the test sites and the Review Standards (long template, short template).

[seamuslee001]

@kenwest this seems strange given that for a good portion of this year PR tests have been against MySQL 5.7 and the upgrade tests have all been passing. AFAIK they don’t alter any of the standard sql modes

[kenwest]

@seamuslee001,

As far as I'm aware I have a standard install of MySQL on Ubuntu 16.04. The Ubuntu package is 5.7.23-0ubuntu0.16.04.1 and when I run mysql --version it says mysql Ver 14.14 Distrib 5.7.23, for Linux (x86_64) using EditLine wrapper

Here are my SQL modes which come straight out of the box ...

mysql> select @@GLOBAL.sql_mode;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| @@GLOBAL.sql_mode                                                                                                                         |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> select @@SESSION.sql_mode;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| @@SESSION.sql_mode                                                                                                                        |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

[kenwest]

@seamuslee001, the SQL manual at https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sql-mode-setting tells me that the modes I have are the defaults.

BTW I have no other issues running MySQL 5.7 in development or production with these settings.

[eileenmcnaughton]

@seamuslee001 I think we should merge this - I tested on my mysql setting the value & running

UPDATE civicrm_contribution SET receive_date = NULL WHERE CAST(receive_date AS CHAR(20)) = '0000-00-00 00:00:00';

So my feeling is that still works & fixes the problem in some cases. I think we would ideally do a bigger fix & ensure that NO_ZERO_IN_DATE,NO_ZERO_DATE are disabled & enable those modes on our mysql boxes (@totten ).

However, I think it would be valid to just merge this as an extremely limited fix and leave further changes out of scope for now.

[seamuslee001]

@eileenmcnaughton agreed as long as it has passed jenkins

Just noting for posterity AFAIK these are the sql modes in play when unit tests are run

mysql> select @@GLOBAL.sql_mode;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| @@GLOBAL.sql_mode                                                                                                                         |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql>  select @@SESSION.sql_mode;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| @@SESSION.sql_mode                                                                                                                        |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql>

[seamuslee001]

Test failures are un-related merging given Eileen's review on the matter

[eileenmcnaughton]

@seamuslee001 it occurs to me this could be slow on large installs.But I think I'm still OK with it - as creating less rather than more support issues - esp since it's an older version update