Sync option to GC old ConfigMaps & Secrets

[jessesuen]

Spawned from: #1629

It may be desirable for Argo CD to have GC controls on ConfigMaps/Secrets which are no longer referenced, to complement the feature in #1629, which ignores "extra" resources in a namespace.

One proposal is to introduce a new sync option: --prune-unreferenced-config at the API and spec level:

The CLI to enable this command might be something like:

argocd app sync --prune-unreferenced-config

For declarative app specs which desire to have this happen automatically as part of auto-sync, the spec addition would be:

spec:
  syncPolicy:
    automated:
      pruneUnreferencedConfig: true

The way pruneUnreferencedConfig would work is, when set to true:

1. Before performing sync, iterate all live configmaps/secrets which are part of the application which are candidates for pruning (i.e. no longer defined in git)
2. For each configmap/secret which is a pruning candidate, iterate all pods of the corresponding namespace.
3. If any pod is referencing the configmap or secret, do not prune, otherwise proceed to perform pruning.

[jessesuen]

Need syntax to support all of the following cases during argocd app sync:

1. don't prune any extra resources
2. don't prune any extra resources except for unreferenced configs
3. prune all extra resources except for referenced configs
4. prune all extra resources

[jessesuen]

Here is the proposed syntax. We would introduce a syncOptions feature to support this. The first option would be ConfigGC=true|false

1. don't prune any extra resources

CLI:

argocd app sync

Spec:

spec:
  syncPolicy:
    automated: {}

2. don't prune any extra resources except for unreferenced configs

CLI:

argocd app sync -o ConfigGC=true

Spec:

spec:
  syncPolicy:
    automated:
      syncOptions:
      - ConfigGC=true

3. prune all extra resources except for referenced configs

CLI:

argocd app sync --prune -o ConfigGC=false

Spec:

spec:
  syncPolicy:
    automated:
      prune: true
      syncOptions:
      - ConfigGC=false

4. prune all extra resources

CLI:

argocd app sync --prune

Spec:

spec:
  syncPolicy:
    automated:
      prune: true

[alexec]

Related to #1629

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alexec]

Did we fix this with sync-options? https://argoproj.github.io/argo-cd/user-guide/sync-options/#no-prune-resources

[kplimack]

👍

[Myafq]

Is there any workaround for such garbage collecting?

[kahirokunn]

Is there any progress?

[adrian-gierakowski]

would you accept a PR for this?

[andrii-korotkov-verkada]

I think these should be managed by changing the manifests. If something was deleted, enabling application austosync with prune would clean this up.