Add tool script to run backup

[JerryHue]

Issue This PR Addresses

Fixes #2822

Type of Change

• Bugfix: Change which fixes an issue
• New Feature: Change which adds functionality
• Documentation Update: Change which improves documentation
• UI: Change which improves UI
• Development Tools: Change which improves development tools

Description

This PR adds a small Docker container that will run a backup script as a cron job, it also bundles up another script to run a restoration using one of the available database dumps.

The scripts in question are:

• create-backup.sh, which creates a dump file by using pg_dump. The dump file will list all data definitions and data values from all tables in a specific database).
• restore-backup.sh, which will use a specific dump file to restore a database with pg_restore. You are supposed to use the dump file generated by the create-backup file.

Steps to test the PR

1. Start the services.
2. Enter some information into a few tables.
3. Login into the pg-backup-cron-job container (docker exec -it pg-backup-cron-job sh).
4. Instead of waiting for the cron job to be executed, you may execute the file with the following command: /etc/periodic/daily/create-backup. After the command has been executed, there should be a file in /var/opt/, similar to 2022-04-08-02231.dump, make a note of its name.
5. After getting the backup file, modify the database by deleting those tables with data.
6. To restore those tables, run the restore-backup file with the following command: ./restore-backup /var/opt/2022-04-08-02231.dump, for example.

Checklist

• Quality: This PR builds and passes our npm test and works locally
• Tests: This PR includes thorough tests or an explanation of why it does not
• Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not (if applicable)
• Documentation: This PR includes updated/added documentation to user exposed functionality or configuration variables are added/changed or an explanation of why it does not(if applicable)

[gitpod-io]

[humphd]

This is going to be challenging to use in production, since we don't expose the database port. We'll have to run it in a container that has access to the postgres instance via Docker DNS, and maybe as a cron job or something.

[JerryHue]

So, your suggestion is to have a docker container that runs the create-backup script as a cron job, and outputs that file into a volume, correct?

Where should we place this docker container's Dockerfile?

[humphd]

It could live in the docker/supabase/* folder. We do something like this now with how nginx re-reads its config every 6 hours, see https://github.com/Seneca-CDOT/telescope/blob/master/docker/production.yml#L152.

So this script could mount a volume like what we do with postgres in Supabase (see https://github.com/Seneca-CDOT/telescope/blob/master/docker/supabase/supabase-production.yml#L10), and dump the backup there. Later we can move that to a remote machine, but having it at all is good for now.

[humphd]

I was looking at something unrelated today, and saw this:

  initializer:
    image: "defectdojo/defectdojo-django:${DJANGO_VERSION:-latest}"
    profiles: 
      - mysql-rabbitmq
      - mysql-redis
      - postgres-rabbitmq
      - postgres-redis
    depends_on:
      - mysql
      - postgres
    entrypoint: ['/wait-for-it.sh', '${DD_DATABASE_HOST}:${DD_DATABASE_PORT}', '--', '/entrypoint-initializer.sh']
    environment:
      DD_DATABASE_URL: ${DD_DATABASE_URL}
      DD_ADMIN_USER: "${DD_ADMIN_USER:-admin}"
      DD_ADMIN_MAIL: "${DD_ADMIN_USER:-admin@defectdojo.local}"
      DD_ADMIN_FIRST_NAME: "${DD_ADMIN_FIRST_NAME:-Admin}"
      DD_ADMIN_LAST_NAME: "${DD_ADMIN_LAST_NAME:-User}"
      DD_INITIALIZE: "${DD_INITIALIZE:-true}"
      DD_SECRET_KEY: "${DD_SECRET_KEY:-hhZCp@D28z!n@NED*yB!ROMt+WzsY*iq}"
      DD_CREDENTIAL_AES_256_KEY: "${DD_CREDENTIAL_AES_256_KEY:-&91a*agLqesc*0DJ+2*bAbsUZfR*4nLw}"
    volumes:
        - type: bind
          source: ./docker/extra_settings
          target: /app/docker/extra_settings

This is an example of a container that is running a script on a db

[JerryHue]

I was looking at something unrelated today, and saw this:

  initializer:
    image: "defectdojo/defectdojo-django:${DJANGO_VERSION:-latest}"
    profiles: 
      - mysql-rabbitmq
      - mysql-redis
      - postgres-rabbitmq
      - postgres-redis
    depends_on:
      - mysql
      - postgres
    entrypoint: ['/wait-for-it.sh', '${DD_DATABASE_HOST}:${DD_DATABASE_PORT}', '--', '/entrypoint-initializer.sh']
    environment:
      DD_DATABASE_URL: ${DD_DATABASE_URL}
      DD_ADMIN_USER: "${DD_ADMIN_USER:-admin}"
      DD_ADMIN_MAIL: "${DD_ADMIN_USER:-admin@defectdojo.local}"
      DD_ADMIN_FIRST_NAME: "${DD_ADMIN_FIRST_NAME:-Admin}"
      DD_ADMIN_LAST_NAME: "${DD_ADMIN_LAST_NAME:-User}"
      DD_INITIALIZE: "${DD_INITIALIZE:-true}"
      DD_SECRET_KEY: "${DD_SECRET_KEY:-hhZCp@D28z!n@NED*yB!ROMt+WzsY*iq}"
      DD_CREDENTIAL_AES_256_KEY: "${DD_CREDENTIAL_AES_256_KEY:-&91a*agLqesc*0DJ+2*bAbsUZfR*4nLw}"
    volumes:
        - type: bind
          source: ./docker/extra_settings
          target: /app/docker/extra_settings

This is an example of a container that is running a script on a db

So, a way I would approach this is by running the script in the container that is running the database. Since it will run as a cron job, it shouldn't interfere with the database, anyway. Would that be fine?

[humphd]

Do you need any help with this?

[JerryHue]

@humphd, it took me a little, but I found a nice website that gave me the perfect solution. I changed this PR so now it includes the docker container. I also updated the instructions in the original PR message.

[humphd]

This is awesome! A few comments.