chore: add docs for dragonfly2.0 (#234)

* cdn error definition Signed-off-by: santong <weipeng.swp@alibaba-inc.com> * docs of dragonfly2.0 Signed-off-by: santong <weipeng.swp@alibaba-inc.com> * docs of dragonfly2.0 Signed-off-by: santong <weipeng.swp@alibaba-inc.com> * docs:modify user guide Signed-off-by: santong <weipeng.swp@alibaba-inc.com> * docs:modify user guide Signed-off-by: santong <weipeng.swp@alibaba-inc.com> * docs: add readme Signed-off-by: santong <weipeng.swp@alibaba-inc.com> * docs: add user guide Signed-off-by: santong <weipeng.swp@alibaba-inc.com> * docs: add user guide Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
dragonflyoss · Jun 30, 2021 · 15f249a · 15f249a
1 parent 6b0ff5d
commit 15f249a
Show file tree

Hide file tree

Showing 25 changed files with 685 additions and 253 deletions.
diff --git a/FAQ.md b/FAQ.md
@@ -0,0 +1 @@
+FAQ
diff --git a/README.md b/README.md
@@ -15,6 +15,17 @@
 
 Provide efficient, stable, secure, low-cost file and image distribution services to be the best practice and standard solution in the related Cloud-Native area.
 
+## Introduction
+
+Dragonfly is an open source intelligent P2P based image and file distribution system. Its goal is to tackle all distribution problems in cloud native scenarios. Currently Dragonfly focuses on being:
+
+- Simple: well-defined user-facing API (HTTP), non-invasive to all container engines;
+- Efficient: CDN support, P2P based file distribution to save enterprise bandwidth;
+- Intelligent: host level speed limit, intelligent flow control due to host detection;
+- Secure: block transmission encryption, HTTPS connection support.
+
+Dragonfly is now hosted by the Cloud Native Computing Foundation (CNCF) as an Incubating Level Project. Originally it was born to solve all kinds of distribution at very large scales, such as application distribution, cache distribution, log distribution, image distribution, and so on.
+
 ## Features
 
 - Implement P2P files distribution with various storage types (HDFS, storage services from various cloud vendors, Maven, Yum, etc.) through a unified back-to-source adapter layer.
@@ -45,9 +56,8 @@ Provide efficient, stable, secure, low-cost file and image distribution services
 
 ## Getting Started
 
-- [Introduction][introduction]
-- [Installation][installation]
 - [Quick start][quickstart]
+- [Installation][installation]
 
 ## Documentation
 You can find the Dragonfly documentation [on the website][website].
@@ -64,12 +74,11 @@ You should check out our [CONTRIBUTING][contributing] and develop the project to
 ## Code of Conduct
 Please refer to our [Code of Conduct][codeconduct].
 
-[contributing]: CONTRIBUTING.md
-[codeconduct]: CODE_OF_CONDUCT.md
-[introduction]: https://github.com/dragonflyoss/Dragonfly2
-[installation]: https://github.com/dragonflyoss/Dragonfly2
-[quickstart]: https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/en/user-guide/quick-start.md
+[arch]: docs/en/images/arch.png
+[logo-linear]: docs/en/images/logo/dragonfly-linear.svg
+[quickstart]: docs/en/user-guide/quick-start.md
+[installation]: docs/en/user-guide/install/README.md
 [website]: https://d7y.io
 [discussion]: https://github.com/dragonflyoss/Dragonfly2/discussions
-[logo-linear]: docs/en/images/logo/dragonfly-linear.svg
-[arch]: docs/en/images/arch.png
+[contributing]: CONTRIBUTING.md
+[codeconduct]: CODE_OF_CONDUCT.md
diff --git a/ROADMAP.md b/ROADMAP.md
@@ -0,0 +1 @@
+ROADMAP
diff --git a/api/README.md b/api/README.md
@@ -3,6 +3,8 @@
 We encourage users to experience Dragonfly in different ways. When doing this, there are three ways users could choose to interact with dragonfly mostly:
 
 * For end-users, command line tool `dfget`, `dfdaemon` is mostly used.
-* For developers, Dragonfly manager's raw API is the original thing they would make use of. For more details about API docs, please refer to [api.md](../docs/api_reference/api.md). We should also keep it in mind that doc [apis.md](../docs/api_reference/api.md) is automatically generated by [swagger2markup](https://github.com/Swagger2Markup/swagger2markup). Please **DO NOT** edit [api.md](../docs/api_reference/api.md) directly.
+* For developers, Dragonfly manager's raw API is the original thing they would make use of. For more details about API docs, please refer to 
+  [api.md](/docs/en/api-reference/api.md). We should also keep it in mind that doc [apis.md](../docs/en/api-reference/api.md) is automatically generated by 
+  [swagger2markup](https://github.com/Swagger2Markup/swagger2markup). Please **DO NOT** edit [api.md](../docs/en/api-reference/api.md) directly.
 
 Directory `/api` mainly describes the second part **Dragonfly Manager's Raw API**.
diff --git a/docs/README.md b/docs/README.md
@@ -1,6 +1,6 @@
 # Dragonfly Document
 
-[English](./en/README.md)
+[English](en/README.md)
 
 [TODO 简体中文](./zh-CN/README.md)
 
diff --git a/docs/en/README.md b/docs/en/README.md
@@ -7,14 +7,14 @@ Organization of document is as following:
 * [Quick Start](#Quick-Start)
 * [User Guide](#User-Guide)
 * [CLI Reference](#CLI-Reference)
-    * [WIP dfget](./cli-reference/dfget.md)
-    * [WIP cdnsystem](./cli-reference/cdn.md)
-    * [WIP scheduler](./cli-reference/scheduler.md)
-    * [TODO manager](./cli-reference/manager.md)
+    * [WIP dfget](cli-reference/dfget.md)
+    * [WIP cdnsystem](cli-reference/cdn.md)
+    * [WIP scheduler](cli-reference/scheduler.md)
+    * [TODO manager](cli-reference/manager.md)
 * [TODO API Reference](#API-Reference)
 * [Ecosystem](#Ecosystem)
-    * [Kubernetes Integration](./ecosystem/Kubernetes-with-Dragonfly.md)
-    * [WIP Harbor Integration](./ecosystem/Harbor-with-Dragonfly.md)
+    * [Kubernetes Integration](ecosystem/Kubernetes-with-Dragonfly.md)
+    * [WIP Harbor Integration](ecosystem/Harbor-with-Dragonfly.md)
 * [Developer Guide](#Developer-Guide)
     * [Design Doc](#Design-Doc)
     * [Test Guide](#Test-Guide)
@@ -25,27 +25,33 @@ Find `WIP` or `TODO` in this page and follow [CONTRIBUTING](../../CONTRIBUTING.m
 
 ## Quick Start
 
-[Quick Started](./user-guide/quick-start.md) is exactly what you need if you would give Dragonfly a try. This document includes what are the prerequisites, how to install Dragonfly and how to experience Dragonfly's usage.
+[Quick Started](user-guide/quick-start.md) is exactly what you need if you would give Dragonfly a try. This document includes what are the 
+prerequisites, 
+how to install Dragonfly and how to experience Dragonfly's usage.
 
-## [WIP] User Guide
+## User Guide
 
-[User Guide](./user-guide) helps all kinds of guidance end users need to experience Dragonfly. Not only the very brief [Quick Start](./quick-start), but the detailed binary installation and configure illustration. In addition, any concept and function which help users understand Dragonfly better would be included as well.
+[User Guide](user-guide/README.md) helps all kinds of guidance end users need to experience Dragonfly. Not only the very brief [Quick Start]
+(user-guide/quick-start.md), but the detailed binary installation and configure illustration. In addition, any concept and function which help users 
+understand Dragonfly better would be included as well.
 
-## [WIP] CLI Reference
+## CLI Reference
 
-For almost all users, commandline is the first reference you may need. Document in directory [CLI Reference](./cli-reference) is about command detailed usage of Dragonfly CLI including `dfget`, `cdnsystem`, `scheduler` and `manager`. You can get introductions, synopsis, examples, options about command. Last but not least, Dragonfly can guarantee commandline docs is strongly consistent with Dragonfly CLI's source code. What's more, all commandline docs are auto generated via source code.
+For almost all users, commandline is the first reference you may need. Document in directory [CLI Reference](cli-reference) is about command detailed usage of Dragonfly CLI including `dfget`, `cdnsystem`, `scheduler` and `manager`. You can get introductions, synopsis, examples, options about command. Last but not least, Dragonfly can guarantee commandline docs is strongly consistent with Dragonfly CLI's source code. What's more, all commandline docs are auto generated via source code.
 
 ## [TODO] API Reference
 
 Commandline is the easiest way to experience Dragonfly's ability. API extension will bring more further experience of Dragonfly. Commandline is just one kind of combination usage of API, if you wish to hack or take more advantages of Dragonfly, please see [API Reference](./api-reference). Like command line document, all API docs are auto generated via source code.
 
-## [WIP] Ecosystem
+## Ecosystem
 
-Ecosystem documents show connections between Dragonfly and popular tool or system in cloud native ecosystem. They guide end users how to experience cloud native systems with Dragonfly, such as other CNCF project Kubernetes and Harbor.
+Ecosystem documents show connections between Dragonfly and popular tool or system in cloud native ecosystem. They guide end users how to experience cloud 
+native systems with Dragonfly, such as other CNCF project [Kubernetes](ecosystem/Kubernetes-with-Dragonfly.md) and [Harbor](ecosystem/Harbor-with-Dragonfly.md).
 
-## [WIP] Developer Guide
+## Developer Guide
 
-Develop Guide helps (potential) developers/contributors to understand the theory inside Dragonfly rather than the interface it exposes. With better understanding of how Dragonfly is designed, developer could learn source code of Dragonfly much easier and know how to debug, test and hack.
+[Develop Guide](development/local.md) helps (potential) developers/contributors to understand the theory inside Dragonfly rather than the interface it exposes. With 
+better understanding of how Dragonfly is designed, developer could learn source code of Dragonfly much easier and know how to debug, test and hack.
 
 ### [TODO] Design Doc
 
@@ -66,7 +72,7 @@ For more details, please refer to [test](./test-guide).
 
 The folder `/docs` does not contain all the document about Dragonfly. There are still other really helpful documents in other path of this repo, like:
 
-* [TODO FAQ.md](../FAQ.md)
+* [TODO FAQ.md](../../FAQ.md)
 * [CHANGELOG.md](../../CHANGELOG.md)
 * [TODO ROADMAP.md](../../ROADMAP.md)
 * others.

diff --git a/docs/en/cli-reference/dfget.1 → docs/en/api-reference/api.md b/docs/en/cli-reference/dfget.1 → docs/en/api-reference/api.md
diff --git a/docs/en/cli-reference/cdn.md b/docs/en/cli-reference/cdn.md
@@ -7,7 +7,7 @@ CDN is a long-running process which caches downloaded data from source to avoid
 go run cmd/cdnsystem/main.go [Option]
 ```
 ## Log configuration
-set environment variable DF_ACTIVE_PROFILE=local if you want to print logs to Terminal
+set environment variable console=true if you want to print logs to Terminal
 
 ## Runtime metrics monitoring 
 ```
@@ -16,20 +16,11 @@ go run cmd/cdnsystem/main.go --profiler
 ### Options
 
 ```
-        --advertise-ip string             the cdn node ip is the ip we advertise to other peers in the p2p-network
-        --clean-ratio int                 CleanRatio is the ratio to clean the disk and it is based on 10. the value of CleanRatio should be [1-10] (default 1)
-        --config string                   the path of cdn configuration file (default "/etc/dragonfly/cdn.yaml")
-        --download-port int               downloadPort is the port for download files from cdnNode (default 8001)
-        --fail-access-interval duration   fail access interval is the interval time after failed to access the URL (default 3m0s)
-        --gc-disk-interval duration       gc disk interval is the interval time to execute GC disk. (default 15s)
-        --gc-initial-delay duration       gc initial delay is the delay time from the start to the first GC execution (default 6s)
-        --gc-meta-interval duration       gc meta interval is the interval time to execute the GC meta (default 2m0s)
-    -h, --help                            help for cdn
-        --home-dir string                 homeDir is the working directory of cdnNode (default "/Users/su*__*nweipeng1/cdn-system")
-        --max-bandwidth rate              network rate that cdnNode can use (default 200MB)
-        --port int                        listenPort is the port that cdn server listens on (default 8003)
-        --profiler                        profiler sets whether cdnNode HTTP server setups profiler
-        --system-bandwidth rate           network rate reserved for system (default 20MB)
-        --task-expire-time duration       task expire time is the time that a task is treated expired if the task is not accessed within the time (default 3m0s)
-        --young-gc-threshold file-size    gc disk interval is the interval time to execute GC disk. (default 100GB)
+      --config string    the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/cdn.yaml, it can also be 
+      set by env var:CDN_CONFIG，The settings and uses of each configuration item can refer to cdn.yaml in config directory
+      --console          whether logger output records to the stdout
+  -h, --help             help for cdn
+      --jaeger string    jaeger endpoint url, like: http://localhost:14250/api/traces
+      --pprof-port int   listen port for pprof, 0 represents random port (default -1)
+      --verbose          whether logger use debug level
 ```
diff --git a/docs/en/cli-reference/dfget.md b/docs/en/cli-reference/dfget.md
@@ -21,7 +21,7 @@ dfget --schedulers 127.0.0.1:8002 -o /path/to/output -u "http://example.com/obje
 
 ## Log configuration
 
-set environment variable DF_ACTIVE_PROFILE=local if you want to print logs to Terminal
+set environment variable console=true if you want to print logs to Terminal
 
 ### Options
 
@@ -64,7 +64,7 @@ set environment variable DF_ACTIVE_PROFILE=local if you want to print logs to Te
 
 ## Log configuration
 
-set environment variable DF_ACTIVE_PROFILE=local if you want to print logs to Terminal
+set environment variable console=true if you want to print logs to Terminal
 
 ### Options
 

diff --git a/docs/en/cli-reference/manager.md b/docs/en/cli-reference/manager.md
@@ -0,0 +1,28 @@
+## Manager
+
+Manager is a process that runs in the background and plays the role of the brain of each subsystem cluster in Dragonfly. It is used to manage the dynamic 
+configuration of each system module and provide functions such as heartbeat keeping alive, monitoring the market, and product functions.
+
+## Try it
+```
+go run cmd/manager/main.go [Option]
+```
+
+## Log configuration
+set environment variable console=local if you want to print logs to Terminal
+
+## Runtime metrics monitoring
+```
+go run cmd/scheduler/main.go --profiler
+```
+### Options
+
+```
+      --config string    the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/scheduler.yaml, it can 
+      also be set by env var:SCHEDULER_CONFIG，The settings and uses of each configuration item can refer to scheduler.yaml in config directory
+      --console          whether logger output records to the stdout
+  -h, --help             help for cdn
+      --jaeger string    jaeger endpoint url, like: http://localhost:14250/api/traces
+      --pprof-port int   listen port for pprof, 0 represents random port (default -1)
+      --verbose          whether logger use debug level
+```
diff --git a/docs/en/cli-reference/scheduler.md b/docs/en/cli-reference/scheduler.md
@@ -1,51 +1,26 @@
-## 调度器 
+## Scheduler
 
-调度器生成并维护下载过程中的P2P网络
-
-### 说明
-
-调度器是一个常驻后台运行的进程，用于接收和管理客户端的下载任务，通知CDN进行回源， 在下载过程中生成维护P2P网络，给客户端推送适合的下载节点
-
-### 用法
+Scheduler is a long-running process which receives and manages download tasks from the client, notify the CDN to return to the source, 
+generate and maintain a P2P network during the download process, and push suitable download nodes to the client
+## Try it
 ```
-scheduler [flags]
+go run cmd/scheduler/main.go [Option]
 ```
+## Log configuration
+set environment variable console=true if you want to print logs to Terminal
 
-### 可选参数
-
+## Runtime metrics monitoring
 ```
-      --config string              the path of scheduler's configuration file (default "conf/scheduler.yaml")
-  -h, --help                       help for scheduler
-      --port int                   port is the port that scheduler server listens on (default 8002)
-      --sender-job-pool-size int   sender-job-pool-size is used for scheduler and do not change it (default 10000)
-      --sender-num int             sender-num is used for scheduler and do not change it (default 50)
-      --worker-job-pool-size int   worker-job-pool-size is used for scheduler and do not change it (default 10000)
-      --worker-num int             worker-num is used for scheduler and do not change it (default 12)
+go run cmd/scheduler/main.go --profiler
 ```
-
-### 使用示例
-
-scheduler --config your-config-path/scheduler.yaml
-
-### 配置文件说明
+### Options
 
 ```
-server:
-  port: 8001                        rpc 端口
-
-scheduler:
-
-worker:
-  worker-num: 5                     工作线程数
-  worker-job-pool-size: 10000       工作队列长度
-  sender-num: 10                    发送消息线程数
-  sender-job-pool-size: 10000       发送消息队列长度
-
-cdn:
-  list:                             CDN列表
-    -
-      - cdn-name : "cdn"            CDN服务器的HostName
-        ip: "127.0.0.1"             CDN服务器的IP地址
-        rpcPort: 8003               CDN的RPC端口
-        download-port: 8002         CDN的下载端口
+      --config string    the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/scheduler.yaml, it can 
+      also be set by env var:SCHEDULER_CONFIG，The settings and uses of each configuration item can refer to scheduler.yaml in config directory
+      --console          whether logger output records to the stdout
+  -h, --help             help for cdn
+      --jaeger string    jaeger endpoint url, like: http://localhost:14250/api/traces
+      --pprof-port int   listen port for pprof, 0 represents random port (default -1)
+      --verbose          whether logger use debug level
 ```
diff --git a/docs/en/config/cdn-nginx.conf b/docs/en/config/cdn-nginx.conf
@@ -45,7 +45,7 @@ http {
     server {
         listen              8001;
         location / {
-            root /home/admin/cdn-system/repo;
+            root /home/admin/ftp;
         }
     }
 }
diff --git a/docs/en/config/scheduler.yaml b/docs/en/config/scheduler.yaml
@@ -4,7 +4,7 @@
 debug: false
 
 server:
-  # ListenPort is the ip and port supernode server listens on.
+  # ListenPort is the ip and port scheduler server listens on.
   # default: 8002
   port: 8002
 

diff --git a/docs/en/user-guide/README.md b/docs/en/user-guide/README.md
diff --git a/docs/en/user-guide/install/README.md b/docs/en/user-guide/install/README.md
@@ -0,0 +1,5 @@
+[install-client](./install-client.md) is the installation instructions of client
+[install-manager](./install-manager.md) is the installation instructions of manager
+[install-scheduler](./install-scheduler.md) is the installation instructions of scheduler
+[install-cdn](./install-cdn.md) is the installation instructions of CDN
+