Skip to content

Commit

Permalink
Added some comments about sequencing for implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
i344628 committed Mar 15, 2019
1 parent 075ba05 commit 5b44e84
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions doc/controller.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ This means that co-ordination is required across nodes for some (or all) tasks s
Some of that co-ordination could be done with the current sidecar approach.
But many of them, especially, restoration and scaling are done better with a controller.

For backups, a better option might be to separate the backup sidecar as a separate pod for the whole etcd cluster rather than a sidecar container for each etcd instance in the etcd cluster.
This could be optionally used only in the multi-node scenario and we can perhaps continue to use the sidecar container approach for the single-node scenarion.
This could be the first step and we can think of introducing the controller for co-ordination for other life-cycle operations.

## Database Restoration

Database restoration is also currently done on startup (or a restart) (if database verification fails) within the same backup-restore sidecar's main process.
Expand All @@ -51,6 +55,8 @@ Introducing a controller enables the option to perform database restoration as a
The main advantage of this approach is to decouple the memory requirement of a database restoration from the regular backup (full and delta) tasks.
This could be especially of interest because the delta snapshot restoration requires an embedded `etcd` instance which might mean that the memory requirement for database restoration is almost certain to be proportionate to the database size. However, the memory requirement for backup (full and delta) need not be proportionate to the database size at all. In fact, it is very realistic to expect that the memory requirement for backup be more or less independent of the database size.

This could be a second step because we can introduce only a lean controller for restoration and this might even be possible to do this without introducing a CRD.

## Database Incremental Backup Compaction

Incremental/continuous backup is used for finer granularity backup (in the order of minutes) with full snapshots being taken at a much larger intervals (in the order of hours). This makes the backup efficient both in terms of disk, network bandwidth and backup storage space utilization as well as compute resource utilization during backup.
Expand All @@ -59,6 +65,8 @@ However, if the proportion of changes in the incremental backup is large then th

A controller can be used to periodically perform compaction of the incremental backups in the backup storage. This can optimize both the restoration times as well as the backup storage space utilization while not affecting the regular backup performance because such compaction would be done asynchronously.

This feature could be implemented as an enhancement of the [database restoration](#database-restoration) functionality.

## Database Verification

Database verification is currently done on startup (or a restart) within the same backup-restore sidecar's main process.
Expand Down

0 comments on commit 5b44e84

Please sign in to comment.