gitlab-project-export

Simple Python project for exporting gitlab projects with Export Project feature in GitLab API.

Primarily useful for remote backup of projects in GitLab.com to private storage server.

Breaking Changes

05-2020

Code was modified to work with Python3, not longer compatible with Python2.

Prerequisite

• Configured Gitlab API Token, https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html

Install

Simply install via pip:

pip install gitlab-project-export

or

pip install git+https://github.com/rvojcik/gitlab-project-export

or clone the project and install manually:

git clone https://github.com/rvojcik/gitlab-project-export
cd gitlab-project-export/
sudo python3 setup.py install

or use it without installing to your environment (install only requirements):

git clone https://github.com/rvojcik/gitlab-project-export
cd gitlab-project-export/
pip install -f requirements.txt

Usage

usage: gitlab-project-export.py [-h] [-c CONFIG] [-d] [-f]

optional arguments:
  -h, --help  show this help message and exit
  -c CONFIG   config file
  -d          Debug mode
  -f          Force mode - overwrite backup file if exists
  -n          Only print what would be done, without doing it

usage: gitlab-project-import.py [-h] [-c CONFIG] [-f FILEPATH] [-p PROJECT_PATH] [-d]

optional arguments:
  -h, --help       show this help message and exit
  -c CONFIG        config file
  -f FILEPATH      Path to gitlab exported project file
  -p PROJECT_PATH  Project path
  -d               Debug mode

Prepare and edit your config file

mv config-example.yml config.yml

Simply run the script with optional config parameter

./gitlab-project-export.py -c /path/to/config.yml

Configuration

System uses simple yaml file as configuration.

Example below

gitlab:                                                   - gitlab configuration
  access:
    gitlab_url: "https://gitlab.com"                      - GitLab url, official or your instance
    token: "MY_PERSONAL_SECRET_TOKEN"                     - personal access token
  projects:                                               - list of projects to export
    - rvojcik/example-project

backup:                                                   - backup configuration
  destination: "/data/backup"                             - base backup dir
  project_dirs: True                                      - store projects in separate directories
  backup_name: "gitlab-com-{PROJECT_NAME}-{TIME}.tar.gz"  - backup file template
  backup_time_format: "%Y%m%d"                            - TIME template, use whatever compatible with
                                                            python datetime - date.strftime()
  retention_period: 3                                     - purge files in the destination older than the specified value (in days)

Backup use-case in cron

Create cron file in /etc/cron.d/gitlab-backup

With following content

MAILTO=your_email@here.tld

0 1 * * * root /path/to/cloned-repo/gitlab-project-export.py -c /etc/gitlab-export/config.yml

Migration use-case

First create two config files

config1.yml for exporting our project from gitlab.com

gitlab:                                                   - gitlab configuration
  access:
    gitlab_url: "https://gitlab.com"                      - GitLab url, official or your instance
    token: "MY_PERSONAL_SECRET_TOKEN"                     - personal access token
  projects:                                               - list of projects to export
    - rvojcik/project1
    - rvojcik/project2

backup:                                                   - backup configuration
  destination: "/data/export-dir"                         - base backup dir
  backup_name: "gitlab-com-{PROJECT_NAME}-{TIME}.tar.gz"  - backup file template
  backup_time_format: "%Y%m%d"                            - TIME template, use whatever compatible with
                                                            python datetime - date.strftime()

and config2.yml where we need only gitlab access part for importing projects to private gitlab instance

gitlab:                                                   - gitlab configuration
  access:
    gitlab_url: "https://gitlab.privatedomain.tld"        - GitLab url, official or your instance
    token: "MY_PERSONAL_SECRET_TOKEN"                     - personal access token

Now it's time to export our projects

./gitlab-project-export.py -c ./config1.yml -d

Your projects are now exported in /data/export-dir

After that we use gitlab-project-import.py with config2.yml for importing into our pricate gitlab instance.

./gitlab-project-import.py -c ./config2.yml -f ./gitlab-com-rvojcik-project1-20181224.tar.gz -p "rvojcik/project1"
./gitlab-project-import.py -c ./config2.yml -f ./gitlab-com-rvojcik-project2-20181224.tar.gz -p "rvojcik/project2"

Done ;)