(dev/core#635) Implement local array-cache for use with Redis/Memcache

[totten]

Overview

Consumers of Redis or Memcached currently trigger I/O anytime they read a value from the cache -- even if the value has been read before. This is OK if the caller is organized to do one read. It can be even good in some edge-cases of inter-process communication. However, if the cache is read frequently, this can lead to a lot of redundant reads; and there are certainly use-cases where we have redundant reads.

This PR allows one to define a cache-hierarchy -- e.g. combining a thread-local array with an external cache (like Redis, Memcache, SQL, or a on-disk file).

Context

This is an off-shoot/subset of dev/core#635 and #13489. For dev/core#635, we want to minimize unnecessary SQL writes (e.g. by directing all caches to a non-SQL cache-service). The examples I've got all use CRM_Core_BAO_Cache, which is hard-coded with 2-tier cache-hierarchy (thread-local array-cache plus SQL-cache).

Concurrently, there've been critiques that some of the existing Redis/Memcache consumers should be using 2-tier cache-hierarchy.

In subsequent PRs, we'll want to create a "cache-hierarchy". With these utilities, it can be done with only 1-2 lines of extra code, and it will be PSR-16 compliant.

This depends on #13500.

Before

• CRM_Utils_Cache::create(...type => *memory*...) returns an instance for direct-access to the underlying cache.
• If you want extra caching of the results, you have to write your own.

After

• CRM_Utils_Cache::create() accepts an option withArray => FALSE|TRUE|fast, which allows you to request a Redis/Memcache instance which uses a thread-local array.
• The utility class CRM_Utils_Cache_ArrayDecorator implements a PSR-16 compliant wrapper. It takes any cache and puts a local-array in front of it.
• The utility class CRM_Utils_Cache_FastArrayDecorator does the same thing, but it's not PSR-16 compliant. It sacrifices some correctness in order to improve performance.
• The utility class CRM_Utils_Cache_Tiered allows construction of arbitrary N-tier cache hierarchies. Whereas ArrayDecorator allows two specific tiers (e.g. array => $delegate), Tiered can be used with stacks (like array => redis => sql or redis => sql => file).

Technical Details / Comments

I originally drafted the implementation of Tiered because it was a more flexible design. Then I ran a naive benchmark to compare the 2-tier hierarchy (array=>redis) against direct redis and found... it was about the same, and sometimes slower!

There were two basic reasons for this:

1. Usage patterns: Tiering isn't a universal good; it depends on usage patterns. If you just write a record, read that record one time, and then repeat with different records... then tiers suck. There's no gain from avoiding reads, and writes are more expensive. But if you re-read data several times, then it helps a lot. I needed to change the benchmark to compare performance in that usage pattern.

2. PSR-16 Compliance: PSR-16 standardizes certain edge-cases, esp: (a) TTL/expiration, which leads to an extra cost for enforcing consistent expiration times and (b) mutability of objects, which leads to an extra cost for copying in-memory objects ($copy=deserialize(serialize($original))).

The revised benchmark script lets us specify the #read operations. The ArrayDecorator and FastArrayDecorator were attempts to squeeze out more performance. And the benchmarks improved. In the figures below, we measure the write-time (writing 1000 items) and read-time (reading 200 items). Note how it checks performance with different readPerItem values (i.e. each item is read once; or read 10 times; or read 150 times).

[[ trialCount=3 writeItems=1000 readItems=200 numTimesToReadEachItem=1 ]]

Mean values across all trials:

direct          write=0.1050s   read=0.0092s   readPerItem=0.00922100s
tiered          write=0.1270s   read=0.0141s   readPerItem=0.01405501s
arrdec          write=0.1122s   read=0.0108s   readPerItem=0.01083700s
fastdec         write=0.1049s   read=0.0101s   readPerItem=0.01009822s

[[ trialCount=3 writeItems=1000 readItems=200 numTimesToReadEachItem=10 ]]

Mean values across all trials:

direct          write=0.1075s   read=0.0903s   readPerItem=0.00902683s
tiered          write=0.1233s   read=0.0363s   readPerItem=0.00362550s
arrdec          write=0.1133s   read=0.0179s   readPerItem=0.00179152s
fastdec         write=0.1036s   read=0.0129s   readPerItem=0.00128693s

[[ trialCount=3 writeItems=1000 readItems=200 numTimesToReadEachItem=150 ]]

Mean values across all trials:

direct          write=0.1012s   read=1.3051s   readPerItem=0.00870067s
tiered          write=0.1202s   read=0.3467s   readPerItem=0.00231102s
arrdec          write=0.1059s   read=0.1356s   readPerItem=0.00090372s
fastdec         write=0.1028s   read=0.0611s   readPerItem=0.00040723s

If there's only 1 read (no re-reading), then direct-access is best; all others have overhead. The overhead diminishes as you go from Tiered to ArrayDecorator to FastArrayDecorator.

Why include all three -- why not just do one? Well, (1) they're already written, and the parent unit-test goes a long way to showing the correctness of each, so it's not much difference cost-wise. (2) The best one actually depends on the situation -- why you're using a cache, how big the data-records are, how frequently the records are read. That means the decision about which mechanism to use (in which use-cases) should not be in this PR (which just provides the utility). The decision should be made when we use the cache-hierarchy (in subsequent PRs).

Reviewer Tips

Some things that might help in evaluating this:

• Each of the new classes has a corresponding test (like E2E_Cache_TieredTest), and the test classes are derived from E2E_Cache_CacheTestCase and \Cache\IntegrationTests\LegacySimpleCacheTest. That class originates with the php-cache project and provides pretty good test-coverage for functions like get($key, $default=NULL), set($key,$value,$ttl=NULL), and has($key).
• You can play with caches manually via cv cli. You may not have Redis or Memcache locally, but you can pick any cache-driver (CRM_Utils_Cache_SqlGroup, CRM_Utils_Cache_ArrayCache).
  • Step 1: Create some underlying caches ($a, $b, ...) and an overarching cache ($z), like one of these:
    • $a=new CRM_Utils_Cache_ArrayCache([]); $z=new CRM_Utils_Cache_ArrayDecorator($a);
    • $a=new CRM_Utils_Cache_ArrayCache([]); $b=new CRM_Utils_Cache_ArrayCache([]); $z = new CRM_Utils_Cache_Tiered([$a, $b]);
    • $a=CRM_Utils_Cache::create(['name'=>'foo-mem', 'type'=>['*memory*']]); $b=CRM_Utils_Cache::create(['name'=>'foo-sql', 'type'=>['SqlGroup']]); $z=new CRM_Utils_Cache_Tiered([$a,$b])
  • Step 2: Interact with these cache objects, e.g.
    • $z->set('foo',123);
    • $z->get('foo')
    • $a->get('foo')
    • print_r($z);
    • print_r(['a'=>$a, 'b'=>$b, 'z'=>$z])

[civibot]

(Standard links)

• If this is your first pull-request for CiviCRM, please browse CONTRIBUTING.md for information about the development and testing processes.
• If you are reviewing this pull-request, you may wish to consult the test sites and the Review Standards (long template, short template).

[totten]

Note: I've taken some bits that were originally copied by each driver (for handling "nack"s). Those have now been moved to a separate PR, #13500. #13496 is rebased and depends on #13500.

[eileenmcnaughton]

I've done some testing on this.

Using Redis with or without this PR a 'normal' page load on a site with Shoreditch installed hits the Redis cache 26 times to get the key for the extension container - in this case
/017185808f04d7829d5f9aaf191b7ee5/full

With this PR I was, however, able to make the below one-liner to get it to use the ArrayDecorator and as expected it only hit Redis once, getting the remaining 25 instances from memory.

In previous testing switching to Redis didn't give a speed performance until I addressed a bunch of places which didn't have array caching & were falling back on Redis all the time (but had been using array caching in conjunction with mysql per Tim's comments above) . At the time I fixed through Civi::statics - however, this makes it much more possible to replace that with something more correct.

I got similar results with the fast array caching but didn't dig into the tiered caching as it's not currently in use & felt a bit obscure

diff --git a/CRM/Extension/System.php b/CRM/Extension/System.php
index 20d171d131..102ca29867 100644
--- a/CRM/Extension/System.php
+++ b/CRM/Extension/System.php
@@ -260,6 +260,7 @@ class CRM_Extension_System {
         'name' => $cacheGroup,
         'type' => array('*memory*', 'SqlGroup', 'ArrayCache'),
         'prefetch' => TRUE,
+        'withArray' => TRUE,
       ));
     }
     return $this->cache;

[eileenmcnaughton]

Note that I think we should follow this up with a change to caching on the Extension System cache!

[totten]

Agree! There's ~4 that I think we should change.