Skip to content

Commit

Permalink
(dev/core#217) PrevNext - Use more consistent cache-keys while adjust…
Browse files Browse the repository at this point in the history
…ing filters

Suppose you run a search ("Find Contact", "Advanced Search", "Custom Search", etc). The result screen includes
several elements (which we'll reference below):

1. Standard pagination (Previous/Next; First/Last; Jump-To)
2. Numerical option for page-size
3. Sortable columns
4. An alphabetical filter
5. Checkboxes

As you work with these options, the content of the `civicrm_prevnext_cache` table may change. This patch does
not substantively change what's in that cache, but makes the column `cacheKey` simpler and more consistent.

Both Before and After (Unchanged)
---------------------------------
* The form's qfKey identifies the current screen/filters/cache.
* If you navigate to the next/previous page (`#1`) or adjust the page-size (`#2`), the content in `civicrm_prevnext_cache` remains the same (for the given qfKey).
* If you change the sort column (`#3`) or alphabetic filter (`#4`), the content in `civicrm_prevnext_cache` is deleted and repopulated (for the given qfKey).
* If you toggle a checkbox, the `civicrm_prevnext_cache.is_selected` property updates accordingly. These selections are retained when changing pages (`#1`/`#2`),
  but they're reset if you use sort or alphabet options (`#3`/`#4`).

Before
------
* The content of `civicrm_prevnext_cache.cacheKey` takes one of two forms, depending on whether you're using an alphabetic filter (`#4`).
    * `civicrm search {qfKey}` (typical, without any alphabetic filter)
    * `civicrm search {qfKey}_alphabet` (less common, with an alphabetic filter)
* The queries which read or delete the query-cache use a prefix+wildcard, i.e. `WHERE cacheKey LIKE 'civicrm search {qfKey}%'`.

After
-----
* The content of `civicrm_prevnext_cache.cacheKey` takes only one form
    * `civicrm search {qfKey}`
* The queries which read or delete the query-cache use an exact match, i.e. `WHERE cacheKey = 'civicrm search {qfKey}'`.`
* The text `_alphabet` does not appear in the PHP source folders (CRM, Civi, bin, api, extern, tests).

Comments
--------
In theory, one can imagine that it's desireable to keep the cached results for each of the sorted/filtered variants of the query.
That might allow the user to quickly switch among different sortings and different alphabetic-filters, or it might
allow some kind of clever management of the selections. But this is not so. As we see (both before and after), the substance
of the cache is deleted whenever the user changes `#3`/`#4`. In reality, one user browsing a search screen corresponds to exactly
one query-cache. As near as I can tell, the old code made the names change for no real reason at all.

To observe the behavior empirically, I would twiddle the UI widgets and concurrently inspect the content of the cache tables.  For example:

```
mysql> select group_name, path, FROM_BASE64(data), expired_date  from civicrm_cache where path like 'civicrm search%';
select 'Total records in' as label, cacheKey, count(*), min(id), max(id) from civicrm_prevnext_cache group by cacheKey
union select 'Selected records in ', cacheKey, count(*), min(id), max(id) from civicrm_prevnext_cache where is_selected=1 group by cacheKey;
+------------------------------+------------------------------------------------------+--------------------------------------------------------------+--------------+
| group_name                   | path                                                 | FROM_BASE64(data)                                            | expired_date |
+------------------------------+------------------------------------------------------+--------------------------------------------------------------+--------------+
| CiviCRM Search PrevNextCache | civicrm search a8ed1e2039241c41457a88f65aa8a8ee_7845 | s:52:"civicrm search a8ed1e2039241c41457a88f65aa8a8ee_7845"; | NULL         |
+------------------------------+------------------------------------------------------+--------------------------------------------------------------+--------------+
1 row in set (0.00 sec)

+----------------------+------------------------------------------------------+----------+---------+---------+
| label                | cacheKey                                             | count(*) | min(id) | max(id) |
+----------------------+------------------------------------------------------+----------+---------+---------+
| Total records in     | civicrm search a8ed1e2039241c41457a88f65aa8a8ee_7845 |        6 |     787 |     792 |
| Selected records in  | civicrm search a8ed1e2039241c41457a88f65aa8a8ee_7845 |        1 |     789 |     789 |
+----------------------+------------------------------------------------------+----------+---------+---------+
2 rows in set (0.01 sec)
```
  • Loading branch information
totten committed Aug 14, 2018
1 parent 3422e9b commit 15df34f
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 13 deletions.
3 changes: 0 additions & 3 deletions CRM/Contact/Form/Task.php
Original file line number Diff line number Diff line change
Expand Up @@ -165,9 +165,6 @@ public static function preProcessCommon(&$form) {
if ((CRM_Utils_Array::value('radio_ts', self::$_searchFormValues) == 'ts_all') ||
($form->_task == CRM_Contact_Task::SAVE_SEARCH)
) {
$sortByCharacter = $form->get('sortByCharacter');
$cacheKey = ($sortByCharacter && $sortByCharacter != 'all') ? "{$cacheKey}_alphabet" : $cacheKey;

// since we don't store all contacts in prevnextcache, when user selects "all" use query to retrieve contacts
// rather than prevnext cache table for most of the task actions except export where we rebuild query to fetch
// final result set
Expand Down
1 change: 0 additions & 1 deletion CRM/Contact/Selector.php
Original file line number Diff line number Diff line change
Expand Up @@ -903,7 +903,6 @@ public function buildPrevNextCache($sort) {
$countRow = Civi::service('prevnext')->getCount($cacheKey);
// $sortByCharacter triggers a refresh in the prevNext cache
if ($sortByCharacter && $sortByCharacter != 'all') {
$cacheKey .= "_alphabet";
$this->fillupPrevNextCache($sort, $cacheKey, 0, max(self::CACHE_SIZE, $pageSize));
}
elseif (($firstRecord + $pageSize) >= $countRow) {
Expand Down
30 changes: 21 additions & 9 deletions CRM/Core/PrevNextCache/Sql.php
Original file line number Diff line number Diff line change
Expand Up @@ -97,37 +97,37 @@ public function markSelection($cacheKey, $action, $cIds = NULL) {
if (is_array($cIds)) {
$cIdFilter = "(" . implode(',', $cIds) . ")";
$whereClause = "
WHERE cacheKey LIKE %1
WHERE cacheKey = %1
AND (entity_id1 IN {$cIdFilter} OR entity_id2 IN {$cIdFilter})
";
}
else {
$whereClause = "
WHERE cacheKey LIKE %1
WHERE cacheKey = %1
AND (entity_id1 = %2 OR entity_id2 = %2)
";
$params[2] = array("{$cIds}", 'Integer');
}
if ($action == 'select') {
$whereClause .= "AND is_selected = 0";
$sql = "UPDATE civicrm_prevnext_cache SET is_selected = 1 {$whereClause} {$entity_whereClause}";
$params[1] = array("{$cacheKey}%", 'String');
$params[1] = array($cacheKey, 'String');
}
elseif ($action == 'unselect') {
$whereClause .= "AND is_selected = 1";
$sql = "UPDATE civicrm_prevnext_cache SET is_selected = 0 {$whereClause} {$entity_whereClause}";
$params[1] = array("%{$cacheKey}%", 'String');
$params[1] = array($cacheKey, 'String');
}
// default action is reseting
}
elseif (!$cIds && $cacheKey && $action == 'unselect') {
$sql = "
UPDATE civicrm_prevnext_cache
SET is_selected = 0
WHERE cacheKey LIKE %1 AND is_selected = 1
WHERE cacheKey = %1 AND is_selected = 1
{$entity_whereClause}
";
$params[1] = array("{$cacheKey}%", 'String');
$params[1] = array($cacheKey, 'String');
}
CRM_Core_DAO::executeQuery($sql, $params);
}
Expand Down Expand Up @@ -157,12 +157,12 @@ public function getSelection($cacheKey, $action = 'get') {
$actionGet = ($action == "get") ? " AND is_selected = 1 " : "";
$sql = "
SELECT entity_id1, entity_id2 FROM civicrm_prevnext_cache
WHERE cacheKey LIKE %1
WHERE cacheKey = %1
$actionGet
$entity_whereClause
ORDER BY id
";
$params[1] = array("{$cacheKey}%", 'String');
$params[1] = array($cacheKey, 'String');

$contactIds = array($cacheKey => array());
$cIdDao = CRM_Core_DAO::executeQuery($sql, $params);
Expand Down Expand Up @@ -199,7 +199,19 @@ public function getPositions($cacheKey, $id1, $id2) {
* @param string $entityTable
*/
public function deleteItem($id = NULL, $cacheKey = NULL, $entityTable = 'civicrm_contact') {
CRM_Core_BAO_PrevNextCache::deleteItem($id, $cacheKey, $entityTable);
$sql = "DELETE FROM civicrm_prevnext_cache WHERE entity_table = %1";
$params = array(1 => array($entityTable, 'String'));

if (is_numeric($id)) {
$sql .= " AND ( entity_id1 = %2 OR entity_id2 = %2 )";
$params[2] = array($id, 'Integer');
}

if (isset($cacheKey)) {
$sql .= " AND cacheKey = %3";
$params[3] = array($cacheKey, 'String');
}
CRM_Core_DAO::executeQuery($sql, $params);
}

/**
Expand Down

0 comments on commit 15df34f

Please sign in to comment.