dev/core#748 Move UPPER() from sql to php domain

[mattwire]

Overview

Extracted from #13713

Using UPPER()/LOWER() in SQL queries causes the query to stop using indexes. The query for the
the first, slowest, query when doing a search is just to generate any extended characters for the alphabet pager across the top of the search results!

Before

Very slow query for complex smartgroups / larger datasets.

After

Removed both "UPPER()" functions from the "alphabet" query (as that stops it using indexes) and moved that to the PHP level. Query much faster and no functional change.

Technical Details

Combined with further work in #13713 to reduce left joins this reduced a number of queries from infinity to < 1 second on a site with a number of complex smartgroups.

Comments

There is precedent for removing UPPER/LOWER from SQL as it is known to cause performance issues.

@eileenmcnaughton I've extracted alphabetQuery() in this too.

[civibot]

(Standard links)

• If this is your first pull-request for CiviCRM, please browse CONTRIBUTING.md for information about the development and testing processes.
• If you are reviewing this pull-request, you may wish to consult the test sites and the Review Standards (long template, short template).

[eileenmcnaughton]

I did some quick tests and this works fine. Code is simpler & I know there is some test coverage of alphabet queries - merge -on-pass

[seamuslee001]

Merging as per the tag