From fdb9eaff41516dc62ef568715c7675d074a63a2b Mon Sep 17 00:00:00 2001 From: santong Date: Tue, 18 May 2021 10:28:45 +0800 Subject: [PATCH] docs of dragonfly2.0 Signed-off-by: santong --- docs/en/quick_start/README.md | 117 ++++++++++++ docs/en/user_guide/download_files.md | 131 ++++++++++++++ docs/en/user_guide/install_cdn.md | 167 ++++++++++++++++++ docs/en/user_guide/install_client.md | 136 ++++++++++++++ docs/en/user_guide/install_manager.md | 167 ++++++++++++++++++ docs/en/user_guide/install_scheduler.md | 137 ++++++++++++++ .../user_guide/multi_machines_deployment.md | 118 +++++++++++++ .../proxy/docker.md | 0 .../registry/cri-containerd.md | 0 .../registry/cri-o.md | 0 10 files changed, 973 insertions(+) create mode 100644 docs/en/quick_start/README.md create mode 100644 docs/en/user_guide/download_files.md create mode 100644 docs/en/user_guide/install_cdn.md create mode 100644 docs/en/user_guide/install_client.md create mode 100644 docs/en/user_guide/install_manager.md create mode 100644 docs/en/user_guide/install_scheduler.md create mode 100644 docs/en/user_guide/multi_machines_deployment.md rename docs/en/{user-guide => user_guide}/proxy/docker.md (100%) rename docs/en/{user-guide => user_guide}/registry/cri-containerd.md (100%) rename docs/en/{user-guide => user_guide}/registry/cri-o.md (100%) diff --git a/docs/en/quick_start/README.md b/docs/en/quick_start/README.md new file mode 100644 index 00000000000..3113d0d7ceb --- /dev/null +++ b/docs/en/quick_start/README.md @@ -0,0 +1,117 @@ +# Dragonfly Quick Start + +Dragonfly Quick Start document aims to help you to quick start Dragonfly journey. This experiment is quite easy and simplified. + +If you are using Dragonfly in your **production environment** to handle production image distribution, please refer to supernode and dfget's detailed production parameter configuration. + +## Prerequisites + +All steps in this document is doing on the same machine using the docker container, so make sure the docker container engine installed and started on your machine. You can also refer to the documentation: [multi-machine deployment](../user_guide/multi_machines_deployment.md) to experience Dragonfly. + +## Step 1: Deploy Dragonfly Manager Server + +```bash +docker run -d --name supernode \ + --restart=always \ + -p 8001:8001 \ + -p 8002:8002 \ + -v /home/admin/supernode:/home/admin/supernode \ + dragonflyoss/supernode:1.0.2 +``` + +## Step 2: Deploy Dragonfly CDN Server + +```bash +docker run -d --name supernode \ + --restart=always \ + -p 8001:8001 \ + -p 8002:8002 \ + -v /home/admin/supernode:/home/admin/supernode \ + dragonflyoss/supernode:1.0.2 +``` + +## Step 3: Deploy Dragonfly Scheduler Server + +```bash +docker run -d --name supernode \ + --restart=always \ + -p 8001:8001 \ + -p 8002:8002 \ + -v /home/admin/supernode:/home/admin/supernode \ + dragonflyoss/supernode:1.0.2 +``` + +## Step 2: Deploy Dragonfly Client + +```bash +SUPERNODE_IP=`docker inspect supernode -f '{{.NetworkSettings.Networks.bridge.IPAddress}}'` +docker run -d --name dfclient \ + --restart=always \ + -p 65001:65001 \ + -v $HOME/.small-dragonfly:/root/.small-dragonfly \ + dragonflyoss/dfclient:1.0.2 --registry https://index.docker.io --node $SUPERNODE_IP +``` + +**NOTE**: + +- The `--registry` parameter specifies the mirrored image registry address, and `https://index.docker.io` is the address of official image registry, you can also set it to the other **non-https image registries**. +- The `--node` parameter specifies the supernode's address in the format of **HOST:IP**. And the default value `8002` will be used if the port is not specified. Here we use `docker inspect` to get the ip of supernode container as the host value. Since the supernode container exposes its ports, you can specify this parameter to node ip address as well. + +## Step 3. Configure Docker Daemon + +We need to modify the Docker Daemon configuration to use the Dragonfly as a pull through registry. + +1. Add or update the configuration item `registry-mirrors` in the configuration file`/etc/docker/daemon.json`. + +```json +{ + "registry-mirrors": ["http://127.0.0.1:65001"] +} +``` + +**Tip:** For more information on `/etc/docker/daemon.json`, see [Docker documentation](https://docs.docker.com/registry/recipes/mirror/#configure-the-cache). + +2. Restart Docker Daemon. + +```bash +systemctl restart docker +``` + +## Step 4: Pull images with Dragonfly + +Through the above steps, we can start to validate if Dragonfly works as expected. + +And you can pull the image as usual, for example: + +```bash +docker pull nginx:latest +``` + +## Step 5: Validate Dragonfly + +You can execute the following command to check if the nginx image is distributed via Dragonfly. + +```bash +docker exec dfclient grep 'downloading piece' /root/.small-dragonfly/logs/dfclient.log +``` + +If the output of command above has content like + +``` +2019-03-29 15:49:53.913 INFO sign:96027-1553845785.119 : downloading piece:{"taskID":"00a0503ea12457638ebbef5d0bfae51f9e8e0a0a349312c211f26f53beb93cdc","superNode":"127.0.0.1","dstCid":"127.0.0.1-95953-1553845720.488","range":"67108864-71303167","result":503,"status":701,"pieceSize":4194304,"pieceNum":16} +``` + +then Dragonfly works successfully. + +## SEE ALSO + +- [multi machines deployment](../user_guide/multi_machines_deployment.md) - experience Dragonfly on multiple machines +- [install manager](../user_guide/install_manager.md) - how to install the Dragonfly manager +- [install cdn](../user_guide/install_cdn.md) - how to install the Dragonfly cdn +- [install scheduler](../user_guide/install_scheduler.md) - how to install the Dragonfly scheduler +- [install client](../user_guide/install_client.md) - how to install the Dragonfly client +- [proxy](../user_guide/proxy/docker.md) - make Dragonfly as an HTTP proxy for docker daemon +- [download files](../user_guide/download_files.md) - download files with Dragonfly +- Container Runtimes + - [cri-o mirror](../user_guide/registry/cri-o.md) - make Dragonfly as Registry Mirror for CRIO daemon + - [cri-containerd_mirror](../user_guide/registry/cri-containerd.md) - make Dragonfly as Registry Mirror for containerd daemon diff --git a/docs/en/user_guide/download_files.md b/docs/en/user_guide/download_files.md new file mode 100644 index 00000000000..e0c4c8b41e4 --- /dev/null +++ b/docs/en/user_guide/download_files.md @@ -0,0 +1,131 @@ +# Downloading Files with Dragonfly + +Things are done differently when you download container images and download general files with Dragonfly. + +## Prerequisites + +- You are using Linux operating system. +- The supernode service is started. + + **Tip:** For more information on the dfget command, see [dfget](../cli_reference/dfget.md). For more information on the installation of supernodes, see [Installing Server](./install_server.md). + +## Downloading container images + +1. Config the supernodes with the configuration file. + + ```shell + cat < /etc/dragonfly/dfget.yml + nodes: + - supernode01:port + - supernode02:port + - supernode03:port + EOD + ``` + +2. Start the dfget proxy (dfdaemon). + + ```sh + # Start dfdaemon and specify the image repo URL. The default port is `65001`. + dfdaemon --registry https://xxx.xx.x + # Review dfdaemon logs + tailf ~/.small-dragonfly/logs/dfdaemon.log + ``` + + **Tip:** To list all available parameters for dfdaemon, run `dfdaemon -h`. + +3. Configure the Docker daemon. + + a. Modify the configuration file `/etc/docker/daemon.json`. + + ```sh + vi /etc/docker/daemon.json + ``` + + **Tip:** For more information on `/etc/docker/daemon.json`, see [Docker documentation](https://docs.docker.com/registry/recipes/mirror/#configure-the-cache). + + b. Add or update the configuration item `registry-mirrors` in the configuration file. + + ```sh + "registry-mirrors": ["http://127.0.0.1:65001"] + ``` + + c. Restart Docker daemon. + + ```bash + systemctl restart docker + ``` + + d. Add authentication info for the private docker registry in `~/.docker/config.json` if the registry is configured with auth. + + ```json + { + "auths": { + "https://index.docker.io/v1/": { + "auth": "${auth_value}" + } + } + } + ``` + + The ${auth_value} is `base64("${username}:${password}")`. + + ```bash + echo "${username}:${password}" | base64 + ``` + +4. Download an image with Dragonfly. + + ```bash + docker pull {imageName} + ``` + + **Note:** Don't include the image repo URL in {imageName}, because the repo URL has been specified with the `registry` parameter when starting dfdaemon. + +## Downloading General Files + +1. Specify the supernodes in one of the following ways. + + - Specifying with the configuration file. + + ```sh + cat < /etc/dragonfly/dfget.yml + nodes: + - supernode01:port + - supernode02:port + - supernode03:port + EOD + ``` + + - Specifying with the parameter in the command line. + + ```sh + dfget -u "http://www.taobao.com" -o /tmp/test.html --node supernode01:port,supernode02:port,supernode03:port + ``` + + **Note:** When using this method, you must add the `node` parameter whenever you run the dfget command. And the parameter in the command line takes precedence over the configuration file. + +2. Download general files with Dragonfly in one of the following ways. + + - Download files with the default `/etc/dragonfly/dfget.yml` configuration. + + ```sh + dfget --url "http://xxx.xx.x" + ``` + + **Tip:** To list all available parameters for dfget, run `dfget -h`. + + - Download files with your specified supernodes. + + ```sh + dfget --url "http://xxx.xx.x" --node "127.0.0.1:8002" + ``` + + - Download files to your specified output file. + + ```sh + dfget --url "http://xxx.xx.x" -o a.txt + ``` + +## After this Task + +To review the downloading log, run `less ~/.small-dragonfly/logs/dfclient.log`. diff --git a/docs/en/user_guide/install_cdn.md b/docs/en/user_guide/install_cdn.md new file mode 100644 index 00000000000..db4cb1a0c5a --- /dev/null +++ b/docs/en/user_guide/install_cdn.md @@ -0,0 +1,167 @@ +# Installing Dragonfly CDN Server + +This topic explains how to install the Dragonfly cdn server. + +## Context + +Install CDN in one of the following ways: + +- Deploying with Docker. +- Deploying with physical machines: Recommended for production usage. + +## Prerequisites + +When deploying with Docker, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Docker|1.12.0+ + +When deploying with physical machines, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Golang|1.12.x +Nginx|0.8+ + +## Procedure - When Deploying with Docker + +### Get cdn image + +You can get it from [DockerHub](https://hub.docker.com/) directly. + +1. Obtain the latest Docker image of the cdn. + + ```sh + docker pull d7yio/cdn + ``` + +Or you can build your own cdn image. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly2 + ``` + +3. Build the Docker image. + + ```sh + TAG="1.0.0" + make docker-build-cdn D7Y_VERSION=$TAG + ``` + +4. Obtain the latest Docker image ID of the cdn. + + ```sh + docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 + ``` + +### Start cdn + +**NOTE:** Replace ${cdnDockerImageId} with the ID obtained at the previous step. + +```sh +docker run -d --name cdn --restart=always -p 8001:8001 -p 8003:8003 -v /home/admin/cdn:/home/admin/cdn ${cdnDockerImageId} +--download-port=8001 +``` + +## Procedure - When Deploying with Physical Machines + +### Get cdn executable file + +1. Download a binary package of the cdn. You can download one of the latest builds for Dragonfly on the [github releases page](https://github. + com/dragonflyoss/Dragonfly2/releases). + + ```sh + version=1.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` + +2. Unzip the package. + + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_1.0.0_linux_amd64.tar.gz -C xxx + ``` + +3. Move the `cdn` to your `PATH` environment variable to make sure you can directly use `cdn` command. + +Or you can build your own cdn executable file. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly2 + ``` + +3. Compile the source code. + + ```sh + make build-cdn && make install-cdn + ``` + +### Start cdn + +```sh +cdnHomeDir=/home/admin/cdn +cdnDownloadPort=8001 +cdn --home-dir=$cdnHomeDir --port=8003 --download-port=$cdnDownloadPort +``` + +### Start file server + +You can start a file server in any way. However, the following conditions must be met: + +- It must be rooted at `${cdnHomeDir}/repo` which is defined in the previous step. +- It must listen on the port `cdnDownloadPort` which is defined in the previous step. + +Let's take nginx as an example. + +1. Add the following configuration items to the Nginx configuration file. + + ```conf + server { + # Must be ${cdnDownloadPort} + listen 8001; + location / { + # Must be ${cdnHomeDir}/repo + root /home/admin/cdn/repo; + } + } + ``` + +2. Start Nginx. + + ```sh + sudo nginx + ``` + +## After this Task + +- After cdn is installed, run the following commands to verify if Nginx and **cdn** are started, and if Port `8001` and `8003` are available. + + ```sh + telnet 127.0.0.1 8001 + telnet 127.0.0.1 8003 + ``` + +- [Install the Dragonfly client](./install_client.md) and test if the downloading works. + + ```sh + dfget --url "http://${resourceUrl}" --output ./resource.png --supernode "127.0.0.1:8002=1" + ``` diff --git a/docs/en/user_guide/install_client.md b/docs/en/user_guide/install_client.md new file mode 100644 index 00000000000..387434c7137 --- /dev/null +++ b/docs/en/user_guide/install_client.md @@ -0,0 +1,136 @@ +# Installing Dragonfly Client + +This topic explains how to install the Dragonfly `dfclient`. + +## Context + +Install the `dfclient` in one of the following ways: + +- Deploying with Docker. +- Deploying with physical machines. + +## Prerequisites + +When deploying with Docker, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Docker|1.12.0+ + +When deploying with physical machines, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Golang|1.12.x + +## Procedure - When Deploying with Docker + +### Get dfclient image + +You can get it from [DockerHub](https://hub.docker.com/) directly. + +1. Obtain the latest Docker image ID of the SuperNode. + + ```sh + docker pull dragonflyoss/dfclient:1.0.0 + ``` + +Or you can build your own dfclient image. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly + ``` + +3. Build the Docker image. + + ```sh + TAG="1.0.0" + make docker-build-client DF_VERSION=$TAG + ``` + +4. Obtain the latest Docker image ID of the `dfclient`. + + ```sh + docker image ls | grep 'dfclient' | awk '{print $3}' | head -n1 + ``` + +### Start the dfdaemon + +**NOTE:** You should prepare the [config files](../config) which should locate under `/etc/dragonfly` by default. + +```sh +version=1.0.0 +# Replace ${supernode} with your own supernode node with format `ip:port=weight`. +SUPERNODE=$supernode +docker run -d --name dfclient --restart=always -p 65001:65001 -v $HOME/.small-dragonfly:/root/.small-dragonfly -v /etc/dragonfly:/etc/dragonfly dragonflyoss/dfclient:$version --node $SUPERNODE +``` + +## Procedure - When Deploying with Physical Machines + +### Get dfclient executable file + +1. Download a binary package of the SuperNode. You can download one of the latest builds for Dragonfly on the [github releases page](https://github.com/dragonflyoss/Dragonfly/releases). + + ```sh + version=1.0.0 + wget https://github.com/dragonflyoss/Dragonfly/releases/download/v$version/Dragonfly_$version_linux_amd64.tar.gz + ``` + +2. Unzip the package. + + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly_1.0.0_linux_amd64.tar.gz -C xxx + ``` + +3. Move the `dfget` and `dfdaemon` to your `PATH` environment variable to make sure you can directly use `dfget` and `dfdaemon` command. + +Or you can build your own dfclient executable files. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly + ``` + +3. Build `dfdaemon` and `dfget`. + + ```sh + make build-client && make install-client + ``` + +### Start the dfdaemon + +**NOTE:** You can ignore this step when using only dfget for file distribution . + +```sh +# Replace ${supernode} with your own supernode node with format `ip:port=weight`. +SUPERNODE=$supernode +dfdaemon --node $SUPERNODE +``` + +## After this Task + +Test if the downloading works. + +```sh +dfget --url "http://${resourceUrl}" --output ./resource.png --node "127.0.0.1:8002" +``` + +And test dfdaemon by [pulling an image](./download_files.md). diff --git a/docs/en/user_guide/install_manager.md b/docs/en/user_guide/install_manager.md new file mode 100644 index 00000000000..db4cb1a0c5a --- /dev/null +++ b/docs/en/user_guide/install_manager.md @@ -0,0 +1,167 @@ +# Installing Dragonfly CDN Server + +This topic explains how to install the Dragonfly cdn server. + +## Context + +Install CDN in one of the following ways: + +- Deploying with Docker. +- Deploying with physical machines: Recommended for production usage. + +## Prerequisites + +When deploying with Docker, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Docker|1.12.0+ + +When deploying with physical machines, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Golang|1.12.x +Nginx|0.8+ + +## Procedure - When Deploying with Docker + +### Get cdn image + +You can get it from [DockerHub](https://hub.docker.com/) directly. + +1. Obtain the latest Docker image of the cdn. + + ```sh + docker pull d7yio/cdn + ``` + +Or you can build your own cdn image. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly2 + ``` + +3. Build the Docker image. + + ```sh + TAG="1.0.0" + make docker-build-cdn D7Y_VERSION=$TAG + ``` + +4. Obtain the latest Docker image ID of the cdn. + + ```sh + docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 + ``` + +### Start cdn + +**NOTE:** Replace ${cdnDockerImageId} with the ID obtained at the previous step. + +```sh +docker run -d --name cdn --restart=always -p 8001:8001 -p 8003:8003 -v /home/admin/cdn:/home/admin/cdn ${cdnDockerImageId} +--download-port=8001 +``` + +## Procedure - When Deploying with Physical Machines + +### Get cdn executable file + +1. Download a binary package of the cdn. You can download one of the latest builds for Dragonfly on the [github releases page](https://github. + com/dragonflyoss/Dragonfly2/releases). + + ```sh + version=1.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` + +2. Unzip the package. + + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_1.0.0_linux_amd64.tar.gz -C xxx + ``` + +3. Move the `cdn` to your `PATH` environment variable to make sure you can directly use `cdn` command. + +Or you can build your own cdn executable file. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly2 + ``` + +3. Compile the source code. + + ```sh + make build-cdn && make install-cdn + ``` + +### Start cdn + +```sh +cdnHomeDir=/home/admin/cdn +cdnDownloadPort=8001 +cdn --home-dir=$cdnHomeDir --port=8003 --download-port=$cdnDownloadPort +``` + +### Start file server + +You can start a file server in any way. However, the following conditions must be met: + +- It must be rooted at `${cdnHomeDir}/repo` which is defined in the previous step. +- It must listen on the port `cdnDownloadPort` which is defined in the previous step. + +Let's take nginx as an example. + +1. Add the following configuration items to the Nginx configuration file. + + ```conf + server { + # Must be ${cdnDownloadPort} + listen 8001; + location / { + # Must be ${cdnHomeDir}/repo + root /home/admin/cdn/repo; + } + } + ``` + +2. Start Nginx. + + ```sh + sudo nginx + ``` + +## After this Task + +- After cdn is installed, run the following commands to verify if Nginx and **cdn** are started, and if Port `8001` and `8003` are available. + + ```sh + telnet 127.0.0.1 8001 + telnet 127.0.0.1 8003 + ``` + +- [Install the Dragonfly client](./install_client.md) and test if the downloading works. + + ```sh + dfget --url "http://${resourceUrl}" --output ./resource.png --supernode "127.0.0.1:8002=1" + ``` diff --git a/docs/en/user_guide/install_scheduler.md b/docs/en/user_guide/install_scheduler.md new file mode 100644 index 00000000000..cac3edacbd7 --- /dev/null +++ b/docs/en/user_guide/install_scheduler.md @@ -0,0 +1,137 @@ +# Installing Dragonfly Scheduler Server + +This topic explains how to install the Dragonfly scheduler server. + +## Context + +Install scheduler in one of the following ways: + +- Deploying with Docker. +- Deploying with physical machines. + +## Prerequisites + +When deploying with Docker, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Docker|1.12.0+ + +When deploying with physical machines, the following conditions must be met. + +Required Software | Version Limit +---|--- +Git|1.9.1+ +Golang|1.12.x +Nginx|0.8+ + +## Procedure - When Deploying with Docker + +### Get scheduler image + +You can get it from [DockerHub](https://hub.docker.com/) directly. + +1. Obtain the latest Docker image of the scheduler. + + ```sh + docker pull d7yio/scheduler + ``` + +Or you can build your own scheduler image. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly2 + ``` + +3. Build the Docker image. + + ```sh + TAG="1.0.0" + make docker-build-scheduler D7Y_VERSION=$TAG + ``` + +4. Obtain the latest Docker image ID of the scheduler. + + ```sh + docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1 + ``` + +### Start scheduler + +**NOTE:** Replace ${schedulerDockerImageId} with the ID obtained at the previous step. + +```sh +docker run -d --name scheduler --restart=always -p 8002 -v /home/admin/scheduler:/home/admin/scheduler ${schedulerDockerImageId} +--download-port=8001 +``` + +## Procedure - When Deploying with Physical Machines + +### Get scheduler executable file + +1. Download a binary package of the scheduler. You can download one of the latest builds for Dragonfly on the [github releases page](https://github. + com/dragonflyoss/Dragonfly2/releases). + + ```sh + version=1.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` + +2. Unzip the package. + + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_1.0.0_linux_amd64.tar.gz -C xxx + ``` + +3. Move the `scheduler` to your `PATH` environment variable to make sure you can directly use `scheduler` command. + +Or you can build your own scheduler executable file. + +1. Obtain the source code of Dragonfly. + + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` + +2. Enter the project directory. + + ```sh + cd Dragonfly2 + ``` + +3. Compile the source code. + + ```sh + make build-scheduler && make install-scheduler + ``` + +### Start scheduler + +```sh +schedulerHomeDir=/home/admin/scheduler +cdn --home-dir=$cdnHomeDir --port=8003 --download-port=$cdnDownloadPort +``` +## After this Task + +- After cdn is installed, run the following commands to verify if Nginx and **cdn** are started, and if Port `8001` and `8003` are available. + + ```sh + telnet 127.0.0.1 8001 + telnet 127.0.0.1 8003 + ``` + +- [Install the Dragonfly client](./install_client.md) and test if the downloading works. + + ```sh + dfget --url "http://${resourceUrl}" --output ./resource.png --supernode "127.0.0.1:8002=1" + ``` diff --git a/docs/en/user_guide/multi_machines_deployment.md b/docs/en/user_guide/multi_machines_deployment.md new file mode 100644 index 00000000000..6c24ad43d3f --- /dev/null +++ b/docs/en/user_guide/multi_machines_deployment.md @@ -0,0 +1,118 @@ +# Dragonfly multi-machines deployment + +The multi-machines deployment documentation is designed to help you fully experience Dragonfly on multiple machines. + +If you are using Dragonfly in your production environment to handle production image distribution, please refer to supernode and dfget's detailed production parameter configuration. + +## Prerequisites + +Assuming that experiment requires us to prepare three host machines, one to play a role of supernode, and the other two for dfclient. Then the topology of the three nodes cluster is like the following: + +![quick start cluster topology](../images/quick-start-topo.png) + +Then, we must provide: + +1. three host nodes in a LAN, and we assume that 3 machine IPs are replaced by the following names. + + - **dfsupernode**: Dragonfly server + - **dfclient0**: Dragonfly client one + - **dfclient1**: Dragonfly client two + +2. every node has deployed docker daemon + +## Step 1: Deploy Dragonfly Server (SuperNode) + +Deploy the Dragonfly server (Supernode) on the machine `dfsupernode`. + +```bash +docker run -d --name supernode \ + --restart=always \ + -p 8001:8001 \ + -p 8002:8002 \ + -v /home/admin/supernode:/home/admin/supernode \ + dragonflyoss/supernode:1.0.2 --download-port=8001 +``` + +## Step 2: Deploy Dragonfly Client (dfclient) + +The following operations should be performed both on the client machine `dfclient0`, `dfclient1`. + +### Prepare the configuration file + +Dragonfly's configuration file is located in the `/etc/dragonfly` directory by default. When using the container to deploy the client, you need to mount the configuration file to the container. + +Configure the Dragonfly Supernode address for the client: + +```bash +cat < /etc/dragonfly/dfget.yml +nodes: + - dfsupernode +EOD +``` + +### Start Dragonfly Client + +```bash +docker run -d --name dfclient \ + --restart=always \ + -p 65001:65001 \ + -v /etc/dragonfly:/etc/dragonfly \ + -v $HOME/.small-dragonfly:/root/.small-dragonfly \ + dragonflyoss/dfclient:1.0.2 --registry https://index.docker.io +``` + +**NOTE**: The `--registry` parameter specifies the mirrored image registry address, and `https://index.docker.io` is the address of official image registry, you can also set it to the others. + +## Step 3. Configure Docker Daemon + +We need to modify the Docker Daemon configuration to use the Dragonfly as a pull through registry both on the client machine `dfclient0`, `dfclient1`. + +1. Add or update the configuration item `registry-mirrors` in the configuration file`/etc/docker/daemon.json`. + +```json +{ + "registry-mirrors": ["http://127.0.0.1:65001"] +} +``` + +**Tip:** For more information on `/etc/docker/daemon.json`, see [Docker documentation](https://docs.docker.com/registry/recipes/mirror/#configure-the-cache). + +2. Restart Docker Daemon. + +```bash +systemctl restart docker +``` + +## Step 4: Pull images with Dragonfly + +Through the above steps, we can start to validate if Dragonfly works as expected. + +And you can pull the image as usual on either `dfclient0` or `dfclient1`, for example: + +```bash +docker pull nginx:latest +``` + +## Step 5: Validate Dragonfly + +You can execute the following command to check if the nginx image is distributed via Dragonfly. + +```bash +docker exec dfclient grep 'downloading piece' /root/.small-dragonfly/logs/dfclient.log +``` + +If the output of command above has content like + +``` +2019-03-29 15:49:53.913 INFO sign:96027-1553845785.119 : downloading piece:{"taskID":"00a0503ea12457638ebbef5d0bfae51f9e8e0a0a349312c211f26f53beb93cdc","superNode":"127.0.0.1","dstCid":"127.0.0.1-95953-1553845720.488","range":"67108864-71303167","result":503,"status":701,"pieceSize":4194304,"pieceNum":16} +``` + +that means that the image download is done by Dragonfly. + +If you need to ensure that if the image is transferred through other peer nodes, you can execute the following command: + +```bash +docker exec dfclient grep 'downloading piece' /root/.small-dragonfly/logs/dfclient.log | grep -v cdnnode +``` + +If the above command does not output the result, the mirror does not complete the transmission through other peer nodes. Otherwise, the transmission is completed through other peer nodes. diff --git a/docs/en/user-guide/proxy/docker.md b/docs/en/user_guide/proxy/docker.md similarity index 100% rename from docs/en/user-guide/proxy/docker.md rename to docs/en/user_guide/proxy/docker.md diff --git a/docs/en/user-guide/registry/cri-containerd.md b/docs/en/user_guide/registry/cri-containerd.md similarity index 100% rename from docs/en/user-guide/registry/cri-containerd.md rename to docs/en/user_guide/registry/cri-containerd.md diff --git a/docs/en/user-guide/registry/cri-o.md b/docs/en/user_guide/registry/cri-o.md similarity index 100% rename from docs/en/user-guide/registry/cri-o.md rename to docs/en/user_guide/registry/cri-o.md