Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify caching of status checks #17817

Merged
merged 1 commit into from
Jul 27, 2020

Conversation

colemanw
Copy link
Member

@colemanw colemanw commented Jul 13, 2020

Overview

Simplifies code and improves performance during system status checks.

Before

  • 'domain_id' was missing from the api params, which would have caused unexpected results and possibly crashes in multi-site setups.
  • Civi::cache was being read but never written to.
  • Non-static object caching did not persist in memory across different check classes and since there was no data in Civi::cache this caused multiple redundant queries (one per CRM_Utils_Check_Component_* class).
  • If we did write to Civi::cache there is no mechanism for clearing that cache when data changes.

After

Solution: use simple array cache a-la Civi::$statics

@civibot
Copy link

civibot bot commented Jul 13, 2020

(Standard links)

@civibot civibot bot added the master label Jul 13, 2020
@colemanw
Copy link
Member Author

Ping @eileenmcnaughton

@eileenmcnaughton
Copy link
Contributor

@colemanw I can see it was broken but I don't think I get why we ditched civi::cache:

"If we did write to Civi::cache there is no mechanism for clearing that cache when data changes."

Isn't the point that it is cached for a period of time & then it ages out? This doesn't seem more change prone than 'anything else'

@eileenmcnaughton
Copy link
Contributor

I think testing caching is crazy difficult and at this level it doesn't make sense

@colemanw
Copy link
Member Author

I'm not sure I can be said to be ditching it here as it wasn't actually getting used in the first place.

We pretty much use CIvi::$statics for everything so I'm not sure what makes this particular case special that it needs extra caching?

@eileenmcnaughton
Copy link
Contributor

@colemanw since the cache system got improved we've been using it more - using a 'fastArray' cache means a static cache sits in front of 'the other' cache - so for Redis there is no DB query at all & once retrieved you still get the benefits of array caching. Generally I have been using civi::cache('metadata') for all not-constantly-changing stuff - contact types, membership types, price sets etc

@colemanw
Copy link
Member Author

I'd be worried about time-based caching causing bugs here, as these settings can be changed & then the cached data would be wrong. IMO if we're going to use a cache we'd also need to clear it in the StatusPref BAO for every write operation.

@colemanw colemanw force-pushed the getChecksConfig branch 2 times, most recently from 138e525 to 580ae08 Compare July 14, 2020 16:58
@colemanw
Copy link
Member Author

@eileenmcnaughton looking at this again, I found an actual bug: domain_id' was missing from the api params, which would have caused unexpected results and possibly crashes in multi-site setups.

So now that this PR fixes a real bug and simplifies things in general, can we go ahead and merge this first and then take a look at the caching issue second

@colemanw
Copy link
Member Author

Test fail is unrelated

@demeritcowboy
Copy link
Contributor

@colemanw - File conflicts

Fixes a few apparent problems:
- domain_id was ommitted causing unexpected results
- Civi::cache was being read but never written to
- Object caching did not persist in memory across different check classes
- If we did write to Civi::cache there is no mechanism for clearing that cache when data changes

Solution: use simple array cache in Civi::$statics
@eileenmcnaughton
Copy link
Contributor

OK - I'm not convinced about reducing our caching - but this should be a cheap call

@eileenmcnaughton eileenmcnaughton merged commit 6010fe3 into civicrm:master Jul 27, 2020
@colemanw colemanw deleted the getChecksConfig branch July 27, 2020 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants