feat: readme

Signed-off-by: Gaius <gaius.qi@gmail.com>
dragonflyoss · Nov 3, 2021 · 036f126 · 036f126
1 parent 6378f18
commit 036f126
Show file tree

Hide file tree

Showing 54 changed files with 718 additions and 422 deletions.
diff --git a/docs/en/FAQ.md b/docs/en/FAQ.md
diff --git a/docs/en/README.md b/docs/en/README.md
@@ -1,6 +1,11 @@
 # Dragonfly Document
 
-Dragonfly Document is written, drawn, memorialized representation of all things about Dragonfly. For those who are generally interested in Dragonfly, README.md of this repo is sufficient. While for end users, all details contained in `/docs` is the best guide all should have. And for developers, contents in [Developer Guide](developer-guide/developer-guide.md) part is that all need.
+Dragonfly Document is written, drawn, memorialized representation of
+all things about Dragonfly. For those who are generally interested in Dragonfly,
+README.md of this repo is sufficient. While for end users,
+all details contained in `/docs` is the best guide all should have.
+And for developers, contents in [Developer Guide](developer-guide/developer-guide.md)
+part is that all need.
 
 Organization of document is as following:
 

diff --git a/docs/en/ROADMAP.md b/docs/en/ROADMAP.md
diff --git a/docs/en/deployment/installation/kubernetes/helm.md b/docs/en/deployment/installation/kubernetes/helm.md
@@ -308,5 +308,13 @@ kubectl -n dragonfly-system exec -it ${pod_name} -- grep "peer task done" /var/l
 
 Example output:
 ```
-{"level":"info","ts":"2021-06-28 06:02:30.924","caller":"peer/peertask_stream_callback.go:77","msg":"stream peer task done, cost: 2838ms","peer":"172.17.0.9-1-ed7a32ae-3f18-4095-9f54-6ccfc248b16e","task":"3c658c488fd0868847fab30976c2a079d8fd63df148fb3b53fd1a418015723d7","component":"streamPeerTask"}
-```
+{
+    "level": "info",
+    "ts": "2021-06-28 06:02:30.924",
+    "caller": "peer/peertask_stream_callback.go:77",
+    "msg": "stream peer task done, cost: 2838ms",
+    "peer": "172.17.0.9-1-ed7a32ae-3f18-4095-9f54-6ccfc248b16e",
+    "task": "3c658c488fd0868847fab30976c2a079d8fd63df148fb3b53fd1a418015723d7",
+    "component": "streamPeerTask"
+}
+```
diff --git a/docs/en/deployment/installation/kubernetes/kustomize.md b/docs/en/deployment/installation/kubernetes/kustomize.md
@@ -81,5 +81,13 @@ kubectl -n dragonfly-system exec -it ${pod_name} -- grep "peer task done" /var/l
 
 Example output:
 ```
-{"level":"info","ts":"2021-06-28 06:02:30.924","caller":"peer/peertask_stream_callback.go:77","msg":"stream peer task done, cost: 2838ms","peer":"172.17.0.9-1-ed7a32ae-3f18-4095-9f54-6ccfc248b16e","task":"3c658c488fd0868847fab30976c2a079d8fd63df148fb3b53fd1a418015723d7","component":"streamPeerTask"}
+{
+    "level": "info",
+    "ts": "2021-06-28 06:02:30.924",
+    "caller": "peer/peertask_stream_callback.go:77",
+    "msg": "stream peer task done, cost: 2838ms",
+    "peer": "172.17.0.9-1-ed7a32ae-3f18-4095-9f54-6ccfc248b16e",
+    "task": "3c658c488fd0868847fab30976c2a079d8fd63df148fb3b53fd1a418015723d7",
+    "component": "streamPeerTask"
+}
 ```
diff --git a/docs/en/design/README.md b/docs/en/design/README.md
@@ -1,6 +1,10 @@
 # Design
 
-Design is content all about design of Dragonfly. It includes all things taken into consideration at the very beginning, the architecture designed for all components in Dragonfly, the interactive workflow between components, all APIs in Dragonfly and some technical things else.
+Design is content all about design of Dragonfly.
+It includes all things taken into consideration at the very beginning,
+the architecture designed for all components in Dragonfly,
+the interactive workflow between components,
+all APIs in Dragonfly and some technical things else.
 
 Table of contents:
 

diff --git a/docs/en/design/architecture.md b/docs/en/design/architecture.md
@@ -2,37 +2,62 @@
 
 ## Positioning
 
-Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented) file distribution and management services, and become the cloud-native best practice and standard solution in this realm.
+Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented)
+file distribution and management services,
+and become the cloud-native best practice and standard solution in this realm.
 
 ## What problems are solved
 
-**Architecture design flaws:** The existing architecture is difficult to meet the growing business needs of file distribution, and has gradually exposed deficiencies in stability, efficiency, and security, and is facing more and more challenges
+**Architecture design flaws:** The existing architecture is difficult to
+meet the growing business needs of file distribution,
+and has gradually exposed deficiencies in stability,
+efficiency, and security, and is facing more and more challenges
 
-**Insufficient value penetration:** Currently only supports HTTP back-to-source protocol, and lacks adaptation to other types of storage (HDFS, storage services of various cloud vendors, Maven, YUM, etc.)
+**Insufficient value penetration:** Currently only supports HTTP back-to-source protocol,
+and lacks adaptation to other types of
+storage (HDFS, storage services of various cloud vendors, Maven, YUM, etc.)
 
-**Single distribution mode:** Currently only supports active pull mode, lack of active push and active synchronization capabilities
+**Single distribution mode:** Currently only supports active pull mode,
+lack of active push and active synchronization capabilities
 
-**Lack of product capabilities:** No well-experienced console functions are capabilities, such as distribution task management and control, data market, multi-tenancy, and permission control, etc.
+**Lack of product capabilities:** No well-experienced console
+functions are capabilities, such as distribution task management and control,
+data market, multi-tenancy, and permission control, etc.
 
 ## Core Competence
 
-- Through a unified back-to-source adaptation layer and realization of P2P file distribution capabilities that support various types of storage (HDFS, 
+- Through a unified back-to-source adaptation layer and
+  realization of P2P file distribution capabilities that
+  support various types of storage (HDFS,
   storage services of various cloud vendors, Maven, YUM, etc.)
-- Support more distribution modes: active pull, active push, real-time synchronization, remote copy, automatic warm-up, cross-cloud transmission, etc.
-- Separation and decoupling between systems, scheduling and CDN plug-in, and support on-demand deployment, light or heavy, internal or external, flexible to 
+- Support more distribution modes: active pull, active push,
+  real-time synchronization, remote copy, automatic warm-up,
+  cross-cloud transmission, etc.
+- Separation and decoupling between systems, scheduling and CDN plug-in,
+  and support on-demand deployment, light or heavy,
+  internal or external, flexible to
   meet the actual needs of different scenarios
-- Based on the newly designed P2P protocol framework of grpc, with better efficiency and stability
-- Support encrypted transmission, account-based transmission authentication and current limiting, and multi-tenant isolation mechanism
+- Based on the newly designed P2P protocol framework of grpc,
+  with better efficiency and stability
+- Support encrypted transmission, account-based transmission
+  authentication and current limiting, and multi-tenant isolation mechanism
 - Support more efficient IO methods: multi-threaded IO, memory mapping, DMA, etc.
-- Support dynamic compression, memory file system and more efficient scheduling algorithms to improve distribution efficiency
-- The client supports third-party software's native integration of Dragonfly's P2P capabilities through the C/S mode
-- Production capabilities: support file upload, task management of various distribution modes, data views, and global management and control functions
-- One set of internal and external, core features are shared with each other, and non-common features are extended separately
-- Further integration with the ecosystem: Harbor, Nydus (image downloading solution on demand), warehouse services of various cloud vendors, etc.
+- Support dynamic compression, memory file system and
+  more efficient scheduling algorithms to improve distribution efficiency
+- The client supports third-party software's native integration
+  of Dragonfly's P2P capabilities through the C/S mode
+- Production capabilities: support file upload, task management of
+  various distribution modes, data views, and global management and control functions
+- One set of internal and external, core features are
+  shared with each other, and non-common features are extended separately
+- Further integration with the ecosystem: Harbor, Nydus
+  (image downloading solution on demand),
+  warehouse services of various cloud vendors, etc.
 
 ## Architecture Design
 
 ### Overall structure
+
 ![alt][arch]
 
 ### Entity relationship
@@ -51,7 +76,8 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented)
 
 ### Sub-system function
 
-**Manager**
+#### Manager
+
 - Dynamic configuration management
 - Data Big Disk & Bandwidth Measurement
 - File upload & distribution management and control
@@ -61,15 +87,17 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented)
 - Multi-tenant management
 - Command channel service (channel integration)
 
-**Scheduler**
+#### Scheduler
+
 - Multi-dimensional P2P scheduling strategy
 - Intelligent scheduling algorithm (advanced)
 - The scheduling results can be quantitatively compared and support A/B Testing
 - Scheduling strategy plug-in and CDN subsystem plug-in
 - Meta-information distribution pipeline
 - Client download results and back-to-source results statistics and docking with monitoring services
 
-**CDN**
+#### CDN
+
 - Multi-source adaptation capabilities, including catalog download and range download functions
 - DMA read and write
 - Transfer back to the source
@@ -80,7 +108,8 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented)
 - Secure transmission (symmetric encryption, integrity check, etc.)
 - Storage media plug-in
 
-**Dfget daemon**
+#### Dfget daemon
+
 - Multi-source adaptation capabilities, including catalog download and range download functions
 - Efficient IO mechanism
 - IO scheduling of multiple download tasks (high IO throughput)
@@ -94,12 +123,12 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented)
 - Client elegant upgrade and deployment plan
 - Command execution
 
-**Framework**
+#### Framework
+
 - High availability, high performance, easy integration
 - Consistent Hash algorithm selection
 - Client connection management
 
-
 [arch]: ../images/arch.png
 [association]: ../images/association.png
-[download-process]: ../images/download-process.png
+[download-process]: ../images/download-process.png
diff --git a/docs/en/design/cdn.md b/docs/en/design/cdn.md
@@ -1 +1,3 @@
-# WIP CDN
+# WIP CDN
+
+TODO
diff --git a/docs/en/design/dfdaemon.md b/docs/en/design/dfdaemon.md
@@ -1 +1,3 @@
-# WIP Dfdaemon
+# WIP Dfdaemon
+
+TODO
diff --git a/docs/en/design/manager.md b/docs/en/design/manager.md
@@ -1,20 +1,27 @@
 # Manager
 
-Used to manage the dynamic configuration that each module depends on, and provide keepalive and metrics functions.
+Used to manage the dynamic configuration that each module depends on,
+and provide keepalive and metrics functions.
 
 ## Relationship
 
 - CDN cluster and Scheduler cluster have a `1:N` relationship
 - CDN cluster and CDN instance have a `1:N` relationship
 - Scheduler cluster and Scheduler instance have a `1:N` relationship
 
+<!-- markdownlint-disable -->
 <div align="center">
   <img src="../images/manager-console/relationship.jpg" width="500" title="manager-relationship">
 </div>
+<!-- markdownlint-restore -->
 
-When the Scheduler instance starts, it reports to the manager the Scheduler Cluster ID. Refer to [document](../deployment/configuration/scheduler.yaml) to configure `schedulerClusterID`.
+When the Scheduler instance starts,
+it reports to the manager the Scheduler Cluster ID.
+Refer to [document](../deployment/configuration/scheduler.yaml) to configure `schedulerClusterID`.
 
-When the CDN instance starts, it reports to the manager the CDN Cluster ID. Refer to [document](../deployment/configuration/cdn.yaml) to configure `cdnClusterID`.
+When the CDN instance starts,
+it reports to the manager the CDN Cluster ID.
+Refer to [document](../deployment/configuration/cdn.yaml) to configure `cdnClusterID`.
 
 ## User account
 
@@ -46,15 +53,19 @@ Default root username: `root` password: `dragonfly`.
 
 ##### Configure Scheduler Cluster
 
+<!-- markdownlint-disable -->
 <p align="center">
   <img width="500" height="500" src="../images/manager-console/configure-scheduler-cluster.jpg">
 </p>
+<!-- markdownlint-restore -->
 
 ##### Configure Scheduler Cluster's Client
 
+<!-- markdownlint-disable -->
 <p align="center">
   <img width="500" height="500" src="../images/manager-console/configure-scheduler-cluster-client.jpg">
 </p>
+<!-- markdownlint-restore -->
 
 - `load_limit`: client host can provide the maximum upload load.
 
@@ -70,13 +81,14 @@ Default root username: `root` password: `dragonfly`.
 
 ##### Configure CDN Cluster
 
+<!-- markdownlint-disable -->
 <p align="center">
   <img width="500" height="500" src="../images/manager-console/configure-cdn-cluster.jpg">
 </p>
+<!-- markdownlint-restore -->
 
 - `load_limit`: CDN host can provide the maximum upload load.
 
-
 [signin]: ../images/manager-console/signin.jpg
 [signup]: ../images/manager-console/signup.jpg
 [scheduler-cluster]: ../images/manager-console/scheduler-cluster.jpg

diff --git a/docs/en/design/scheduler.md b/docs/en/design/scheduler.md
@@ -1 +1,3 @@
-# WIP Scheduler
+# WIP Scheduler
+
+TODO
diff --git a/docs/en/developer-guide/developer-guide.md b/docs/en/developer-guide/developer-guide.md
@@ -2,11 +2,11 @@
 
 Easily set up a local development environment.
 
-### Step 1: Install docker and docker compose
+## Step 1: Install docker and docker compose
 
 See documentation on [docs.docker.com]
 
-### Step 2: Start dragonfly
+## Step 2: Start dragonfly
 
 Enter dragonfly documentation and start docker-compose.
 
@@ -19,10 +19,11 @@ Creating dfdaemon  ... done
 Attaching to cdn, scheduler, dfdaemon
 ```
 
-### Step 3: Log analysis
+## Step 3: Log analysis
 
 Show dragonfly logs.
 
+<!-- markdownlint-disable -->
 ```bash
 $ tail -f log/**/*.log
 ==> log/dragonfly/cdn/core.log <==
@@ -48,8 +49,9 @@ $ tail -f log/**/*.log
 {"level":"info","ts":"2021-02-26 05:43:37.342","caller":"worker/sender.go:49","msg":"start sender worker : 50"}
 {"level":"info","ts":"2021-02-26 05:43:37.343","caller":"worker/worker_group.go:64","msg":"start scheduler worker number:6"}
 ```
+<!-- markdownlint-restore -->
 
-### Step 4: Stop dragonfly
+## Step 4: Stop dragonfly
 
 ```bash
 $ docker-compose down

diff --git a/docs/en/preheat/README.md b/docs/en/preheat/README.md
@@ -1,6 +1,7 @@
 # Preheat
 
-P2P provides two types of preheating: `image` and `file`. Users can be preheat in the `console` or directly call `preheat api` for preheating
+P2P provides two types of preheating: `image` and `file`.
+Users can be preheat in the `console` or directly call `preheat api` for preheating
 
 Table of contents:
 

diff --git a/docs/en/preheat/api.md b/docs/en/preheat/api.md
@@ -1,8 +1,10 @@
 # API
 
-Use preheat apis for preheating. First create a POST request for preheating, you can refer to [create job api document](../api-reference/api-reference.md#create-job)
+Use preheat apis for preheating. First create a POST request for preheating,
+you can refer to [create job api document](../api-reference/api-reference.md#create-job)
 
-If the `scheduler_cluster_ids` does not exist, it means to preheat all scheduler clusters.
+If the `scheduler_cluster_ids` does not exist,
+it means to preheat all scheduler clusters.
 
 ```bash
 curl --location --request POST 'http://dragonfly-manager:8080/api/v1/jobs' \
@@ -19,10 +21,23 @@ curl --location --request POST 'http://dragonfly-manager:8080/api/v1/jobs' \
 If the output of command above has content like
 
 ```bash
-{ "id": 1 "task_id": "group_4d1ea00e-740f-4dbf-a47e-dbdc08eb33e1", "type": "preheat", "status": "PENDING", "args": { "filter": "", "headers": null, "type": "image", "url": "https://registry-1.docker.io/v2/library/redis/manifests/latest" }}
+{
+    "id": 1,
+    "task_id": "group_4d1ea00e-740f-4dbf-a47e-dbdc08eb33e1",
+    "type": "preheat",
+    "status": "PENDING",
+    "args": {
+        "filter": "",
+        "headers": null,
+        "type": "image",
+        "url": "https://registry-1.docker.io/v2/library/redis/manifests/latest"
+    }
+}
 ```
 
-Polling the preheating status with id. if status is `SUCCESS`, preheating is successful, you can refer to [get job api document](../api-reference/api-reference.md#get-job)
+Polling the preheating status with id. if status is `SUCCESS`,
+preheating is successful, you can refer to
+[get job api document](../api-reference/api-reference.md#get-job)
 
 ```bash
 curl --request GET 'http://dragonfly-manager:8080/api/v1/jobs/1'
@@ -31,5 +46,16 @@ curl --request GET 'http://dragonfly-manager:8080/api/v1/jobs/1'
 If the status is `SUCCESS`, the preheating is successful.
 
 ```bash
-{ "id": 1 "task_id": "group_4d1ea00e-740f-4dbf-a47e-dbdc08eb33e1", "type": "preheat", "status": "SUCCESS", "args": { "filter": "", "headers": null, "type": "image", "url": "https://registry-1.docker.io/v2/library/redis/manifests/latest" }}
+{
+    "id": 1,
+    "task_id": "group_4d1ea00e-740f-4dbf-a47e-dbdc08eb33e1",
+    "type": "preheat",
+    "status": "SUCCESS",
+    "args": {
+        "filter": "",
+        "headers": null,
+        "type": "image",
+        "url": "https://registry-1.docker.io/v2/library/redis/manifests/latest"
+    }
+}
 ```
diff --git a/docs/en/preheat/console.md b/docs/en/preheat/console.md
@@ -1 +1,3 @@
-# WIP console
+# WIP console
+
+TODO