Automatic localisation

[mottosso]

Goal

Take setup and maintenance out of localisation.

Motivation

Developers don't want to be localising packages. Artists want their tools to run fast. What is one to do? Automation!

Implementation

Find some way to monitor packages involved in requests, and tally up their frequency along with where they're coming from.

I really wanted to leverage the existing memcached setup for this, considering it's already being spammed with requests and must have some notion of where and when they came in. Alas, it does not.. So my next port to call is to have a look at AMQP for this, which is also integrated although I haven't yet investigated to which extent it is, and whether it's sufficient for this purpose. Absolute worst case we'll have to introduce an additional mechanism to transmit requests.

[mottosso]

There are at least two ways of listening and making use of what you hear.

1. Pull - Each user listens to his own context resolves, and localises packages relevant to themselves.
2. Push - A single workstation listens for context resolves across many workstations, and triggers localisation remotely.

Prerequisites

• A running RabbitMQ instance
• A REZ_CONTEXT_TRACKING_HOST value
• A REZ_CONTEXT_TRACKING_AMQP value

Listening locally

Per default, listening merely repeats what it hears.

$ rez env localz -- listen
# Listening for context resolves @ localhost..
b'{"action": "created", "host": "toy", "user": "manima", "context": {"status": "solved", "implicit_packages": ["~platform==windows", "~arch==AMD64", "~os==windows-10"], "timestamp": 1561724216, "resolved_packages": [{"variables": {"index": null, "version": "1.0.1", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "pika"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "windows", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "platform"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "AMD64", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "arch"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "2.32.1", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "rez"}, "key": "filesystem.variant"}, {"variables": {"index": 0, "version": "3.7.3", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "python"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "0.2.0", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "localz"}, "key": "filesystem.variant"}], "from_cache": false, "package_requests": ["pika", "localz"], "load_time": 0.0, "solve_time": 0.023999929428100586}}'
b'{"action": "created", "host": "toy", "user": "manima", "context": {"status": "solved", "implicit_packages": ["~platform==windows", "~arch==AMD64", "~os==windows-10"], "timestamp": 1561724242, "resolved_packages": [{"variables": {"index": null, "version": "1.0.1", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "pika"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "windows", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "platform"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "AMD64", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "arch"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "2.32.1", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "rez"}, "key": "filesystem.variant"}, {"variables": {"index": 0, "version": "3.7.3", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "python"}, "key": "filesystem.variant"}, {"variables": {"index": null, "version": "0.2.0", "repository_type": "filesystem", "location": "C:\\\\Users\\\\manima\\\\packages", "name": "localz"}, "key": "filesystem.variant"}], "from_cache": false, "package_requests": ["pika", "localz"], "load_time": 0.0, "solve_time": 0.023999929428100586}}'

Pass --file to save the results to a JSON document.

$ rez env localz -- listen --file ~/localz.json
 [*] Saving messages to C:\Users\marcus\localz.json
 [*] Listening for context resolves @ localhost

The resulting file looks like this.

{
  "toy": {
    "manima": {
      "PySide2-5.12.3": {
        "firstUsed": 1561717066,
        "lastUsed": 1561717161
      },
      "arch-AMD64": {
        "firstUsed": 1561717048,
        "lastUsed": 1561724533
      },
      "localz-0.2.0": {
        "firstUsed": 1561717048,
        "lastUsed": 1561724524
      },
      "pika-1.0.1": {
        "firstUsed": 1561717048,
        "lastUsed": 1561724524
      },
}

Which facilitates:

1. Multiple machines
2. Multiple users per machine
3. Individual resolved packages
   1. Their first use date
   2. Their last use date