[PR] Configurable state persistence methods

[kopf-archiver]

A pull request by nolar at 2020-03-25 00:20:31+00:00
Original URL: zalando-incubator/kopf#331
Merged by nolar at 2020-03-30 09:36:29+00:00

What do these changes do?

Add flexible methods of persisting the handlers' state between handling cycles. Defaults to backward-compatible approach with status stanza smoothly transitioning to annotations.

Description

Previously, the handlers' states were stored in .status.kopf.progress, and it was hard-coded.

This led to issues with strict CRD schemas, when arbitrary fields are not accepted (and actually just lost without errors), and with built-in resources (e.g. pods), where the schema cannot be hacked with workarounds.

Specifically, in operators with multiple handlers, only the 1st one was executed. In case of errors, no retries were performed. All of this is because .status patches were silently ignored by K8s API.

With this PR, the handlers' state is migrating to annotations. But the migration is smooth, so that both old status fields and new annotations are supported — for easier version upgrades of Kopf.

---

As an advanced use-case and side-effect, the state persistence can now be controlled on a very fine-grained level: which status keys to use, which annotation prefixes, should they be persisted after handling or not, etc.

It also becomes possible to write custom state persistence engines, e.g. in external databases (not going to be part of the framework itself, as there is no foreseeable need).

import kopf

@kopf.on.startup()
def configure_state_persistence(config: kopf.OperatorConfig, **_):
    config.persistence.progress_storage = kopf.AnnotationsProgressStorage(prefix='my-op.example.com')
    config.persistence.progress_storage = kopf.StatusProgressStorage(field='status.my-op')
    config.persistence.progress_storage = kopf.SmartProgressStorage(prefix='my-op.example.com', field='status.my-op')

The change is (supposed to be) fully backward compatible, including the version upgrade of Kopf in Kopf-based operators.

---

The same logic is applied to storage of diff-base (aka "last-handled-configuration") — which is used for calculating the diffs since the last handling. The setting is settings.persistence.diffbase_storage.

See configuration.rst in this PR for more details. Preview:

• https://kopf-nolars-fork.readthedocs.io/en/flexible-states/configuration.html#handling-progress
• https://kopf-nolars-fork.readthedocs.io/en/flexible-states/configuration.html#change-detection

As usually, reviewing individual commits can make it easier.

Issues/PRs

Issues: #308 #23 #283 #321

Type of changes

• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)

Checklist

• The code addresses only the mentioned problem, and this problem only
• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes
• If you provide code modification, please add yourself to CONTRIBUTORS.txt