CRM-20600 - AngularJS - Allowing hooking and static-caching

[totten]

In AngularJS, the most visible policy decisions are made by arranging/enhancing a series of HTML fragments (aka "partials"). Partials are traditionally developed by hand then packaged/distributed as a static asset. This PR introduces an extra step where server-side PHP logic can programmatically manipulate the HTML fragments before Angular initializes. It's intended to allow extensions to alter UIs (a step toward OHUPA-2).

Allowing runtime alteration of AngularJS has implications for performance. To alleviate this concern, the PR also changes the caching strategy: instead of building the full JS/JSON/HTML assets on every request, we build the assets and store them as static, web-readable assets (under [civicrm.files]). For typical users, this should remove 500-1000ms from pageload.

Example

function example_civicrm_alterAngular($angular) {
  $angular->add(\Civi\Angular\ChangeSet::create('mychanges')
    ->alterHtml('~/crmMailing/EditMailingCtrl/2step.html', function(phpQueryObject $doc) {
      $doc->find('[ng-form="crmMailingSubform"]')->attr('cat-stevens', 'ts(\'wild world\')');
    })
  );
}

Summary of changes

1. Add phpQuery: This server-side library allows you to manipulate HTML documents the same way you do in jQuery.
2. Purify HTML: Organic HTML documents have small discrepancies in notation (e.g. <div/> vs <div></div>; <img foo-bar> vs <img foo-bar="">). PHP's DOMDocument (and therefore phpQuery) don't always preserve these details exactly. To ensure consistent behavior, this PR adds a consistency test (PartialSyntaxTest).
3. Add Civi\Angular\ChangeSet: This provides a way to represent changes to Angular HTML.
4. Add hook_civicrm_alterAngular: This exposes Civi\Angular\Manager and Civi\Angular\ChangeSet so that you can modify any HTML file using phpQuery.
5. Perform translations after the hook: Since the hook is able to change the content of the HTML, the list of strings remains open until the hook finishes.
6. Store the computed data in a static file (eg /sites/default/files/civicrm/persist/contribute/dyn/angular-modules.12341234.json): There are a large number of partials. Doing hooks and translations creates more overhead, so this PR uses a new subsystem called AssetBuilder. On my local, this saves ~800ms when a typical user opens the Angular base page.
7. Add setting assetCache: This allows you to opt-in or out of using cache files. By default (auto), caching is enabled production sites and disabled for debug sites.
8. Related: cv #22 adds a command-line for browsing/inspecting Angular HTML. Ex: if you want to see how the file ~/crmMailing/BlockSummary.html has been altered, run cv ang:html:show --diff crmMailing/BlockSummary.html

Possible followups

These are non-blocker concerns that could merit follow-up:

• Allow more hooking into client-side JS services, such as crmMailingMgr. Currently, you need to write directives which either take advantage of existing data-flow -- like CiviMail's autosave -- or setup strictly independent dataflow -- like an independent AJAX call. There's no in-between.

Related PRs

• CRM-20600 - Purify Angular HTML #10381 - Purify Angular HTML
• CRM-20600 - Add chapters for AngularJS and AssetBuilder civicrm-dev-docs#172 - Add developer docs for AngularJS and AssetBuilder
• CRM-19303 - Drupal - Fix computation of civicrm.files on multisite #10511 - Fix bug with accessing [civicrm.files] on D7 multisite

---

• CRM-20600: Expose AngularJS screens to hooks

[eileenmcnaughton]

@totten wanna rebase this?

[totten]

I've rebased, continued the work, and updated the description to reflect current status.

[totten]

I've been doing some evaluation, and HtmlCollection needs to be rewritten. The current interface works at OHUPA-2, but to reach OHUPA-4 we'll have use-cases with inspection, conflict-resolution, etc. The prospects of forward-compatibility seem better if we can itemize the alterations as different things.

For example, a minimal formulation might be to put each alteration in a callback:

// Pseudocode
function example_civicrm_angularPartials(HtmlCollection $html) {
  $html->alter('~/crmMailing/EditMailingCtrl/2step.html', function(phpQueryObject $doc) {
    $doc->find('[ng-form="crmMailingSubform"]')->attr('cat-stevens', 'ts(\'wild world\')');
   });
}

[totten]

The previous commit using HtmlCollection was 2200c27. I've replaced it with a new commit, 0d6d2c6, which uses HtmlFilters.

[totten]

jenkins, test this please

[totten]

Some updates:

• Reworked HtmlFilters as ChangeSet.
• The ChangeSet can be used to modify other things (more than just HTML) -- specifically, the requires list
• Altering the general data of CRM_Core_Resources::getSettings() is not a bad idea, but it's not sufficient for modifying the Angular requires list. There a few different processes which feed off that list, and some don't rely on CRM_Core_Resources::getSettings(). So instead ChangeSet has explicit support for requires(...).

[totten]

Rebased. Resolved conflicts. Added setting assetCache.

This has been tested implicitly for a week or so during civicase dev. Brian has done some testing and says NYSS will do more. Guanhuan also says that Compucorp will be testing Angular interfaces during RC.

[totten]

Test failures are common false-negatives.