Support key rotation for Encrypted Saved Objects

[peterschretlen]

Encrypted Saved Objects make use of a key specified in config
xpack.encrypted_saved_objects.encryptionKey to encrypt and decrypt properties. This is primarily used for

• alerting, to encrypt the API keys used to authorize alert checks
• actions, to encrypt sensitive properties like credentials

If you change the encryptionKey, at the moment there is no mechanism to update saved objects that rely on it. Alerts and actions will stop working, and you have two options: recreate the alerts and actions using the new key, or revert back to the old key.

In addition, with multiple Kibana instances it's possible to end up with different keys on each instance. When this occurs alerts & actions will fail when they run/decrypt on a different instance than the one that encrypted the data. There is no way to fix this problem when it occurs.

A mechanism is needed to move data to a new key, and retire existing keys (decrypt only) so they can eventually be removed.

Related: #56448

In the encrypted saved objects RFC, key rotation was briefly discussed and could be used as a starting point for this issue: #33740 (comment)

• having a primary key, and list of secondary keys (used for decryption only)
• having a privileged API that can re-encrypt existing data ( decrypt only objects using secondary keys, and re-encrypt with the primary key)

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[kobelb]

I think adding this functionality would be great. However, I wanted to make sure that we were all on the same page that adding this would likely not prevent data-loss when a temporary and randomly generated encryption key is being used. Normally, users don't realize that they're using a temporary encryption key until Kibana restarts and a new encryption key is generated. When an encryption key is "lost", we'll never be able to decrypt the data.

[peterschretlen]

However, I wanted to make sure that we were all on the same page that adding this would likely not prevent data-loss when a temporary and randomly generated encryption key is being used.

@kobelb agreed, we're also in the process of disabling the alerting and actions APIs and clients when using a generated key - #56420 - to address that aspect (at least for alerts and actions)