Skip to content

Commit

Permalink
Merge pull request #5187 from FederatedAI/feature-2.0.0-quick_guide_u…
Browse files Browse the repository at this point in the history
…pdate

Feature 2.0.0 quick guide update
  • Loading branch information
talkingwallace authored Sep 14, 2023
2 parents f2a2969 + c3b3310 commit 754be54
Show file tree
Hide file tree
Showing 10 changed files with 163 additions and 427 deletions.
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,18 @@ FATE is an open source project hosted by Linux Foundation. The [Technical Charte
### Version < 2.0
Releases history can be found in [releases](https://github.com/FederatedAI/FATE/releases), deployment resources can be found on [wiki](https://github.com/FederatedAI/FATE/wiki/Download)

### Version == 2.0.0-beta
#### Standalone deployment
- Deploying FATE on a single node via PyPI, pre-built docker images or installers. It is for simple testing purposes. Refer to this [guide](./deploy/standalone-deploy/).

### Cluster deployment
Deploying FATE to multiple nodes to achieve scalability, reliability and manageability.

- [Cluster deployment by CLI](./deploy/cluster-deploy): Using CLI to deploy a FATE cluster.

### Quick Start
- [Training Demo With Installing From Pypi](doc/2.0/quick_start.md)
- [Training Demo With Installing FATE AND FATE-Flow From Pypi](doc/2.0/quick_start.md)
- [Training Demo With Installing FATE Only From Pypi](doc/2.0/fate/ml)

## Related Repositories (Projects)
- [KubeFATE](https://github.com/FederatedAI/KubeFATE): An operational tool for the FATE platform using cloud native technologies such as containers and Kubernetes.
Expand Down
10 changes: 9 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,16 @@ FATE于2019年2月首次对外开源,并成立
### 2.0以前的版本
FATE 2.0以前的版本在[发布页](https://github.com/FederatedAI/FATE/releases), 下载资源汇总页在[wiki](https://github.com/FederatedAI/FATE/wiki/Download)

### 2.0.0-beta 版本
#### 单机版部署
在单节点上部署FATE单机版,支持从 PyPI 直接安装,docker,主机安装包三种方式。
- [单机版部署教程](./deploy/standalone-deploy)
#### 集群
- [原生集群安装](./deploy/cluster-deploy): Using CLI to deploy a FATE cluster.

### 快速开始
- [从Pypi下载安装并启动训练任务示例](doc/2.0/quick_start.md)
- [从 PyPI 下载安装 FATE 和 FATE-Flow 并启动训练任务示例](doc/2.0/quick_start.md)
- [从 PyPI 下载安装 FATE,并启动训练任务示例](doc/2.0/fate/ml)

## 关联仓库
- [KubeFATE](https://github.com/FederatedAI/KubeFATE)
Expand Down
85 changes: 68 additions & 17 deletions deploy/standalone-deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,67 @@

The single-node version provides 3 deployment methods, which can be selected based on your needs:

- Install FATE from PyPI
- Install FATE using Docker Images
- Install FATE on the host machine (using pre-compiled installation packages)

## 2. Install FATE using Docker Images (Recommended)
## 2. Install FATE from PyPI (Recommended)

It is recommended to use Docker images to greatly reduce the likelihood of encountering problems.
### 2.1 Installing Python Environment
- Prepare and install [conda](https://docs.conda.io/projects/miniconda/en/latest/) environment.
- Create a virtual environment:
```shell
# FATE requires Python >= 3.8
conda create -n fate_env python=3.8
conda activate fate_env
```

### 2.2 Installing FATE
This section introduces two ways to installing FATE from pypi, with and without FATE-Flow service.

#### 2.2.1 Installing FATE With FATE-FLow Service
FATE-Flow provides federated job life cycle management, includes scheduling, data management, model and metric management, etc.

##### 2.2.1.1 Installing FATE, FATE-Flow, FATE-Client
```shell
pip install fate_client[fate,fate_flow]==2.0.0.b0
```
#### 2.2.1.2 Service Initialization
```shell
fate_flow init --ip 127.0.0.1 --port 9380 --home $HOME_DIR
pipeline init --ip 127.0.0.1 --port 9380
```
- `ip`: The IP address where the service runs.
- `port`: The HTTP port the service runs on.
- `home`: The data storage directory, including data, models, logs, job configurations, and SQLite databases.

#### 2.2.1.3 Start Fate-Flow Service

```shell
fate_flow start
```

#### 2.2.1.4 Testing

- [Test Items](#5-Test-Items)

### 2.2.2 Installing FATE Directly
FATE provides multiple federated algorithm and secure protocols,
users can directly import fate and use built-in algorithms and secure protocols directly.

#### 2.2.2.1 Installing FATE
```shell
pip install pyfate==2.0.0.b0
```
#### 2.2.2.2 Using Guides
Refer to [examples](../../doc/2.0/fate/ml)


## 3. Install FATE using Docker Images

**Note:** Replace `${version}` in the examples below with the actual version number.

### 2.1 Pre-deployment Environment Check
### 3.1 Pre-deployment Environment Check

- The host machine should have access to the external network to pull installation packages and Docker images from public networks.
- Dependency on [Docker](https://download.docker.com/linux/). Docker version 18.09 is recommended. You can verify the Docker environment using the following command: `docker --version`. For Docker start/stop and other operations, refer to `docker --help`.
Expand All @@ -40,9 +91,9 @@ Example:
export version=2.0.0-beta
```

### 2.2 Pull Docker Images
### 3.2 Pull Docker Images

#### 2.2.1 Via Public Image Services
#### 3.2.1 Via Public Image Services

```bash
# Docker Hub
Expand All @@ -53,7 +104,7 @@ docker pull ccr.ccs.tencentyun.com/federatedai/standalone_fate:${version}
docker tag ccr.ccs.tencentyun.com/federatedai/standalone_fate:${version} federatedai/standalone_fate:${version}
```

#### 2.2.2 Via Image Package
#### 3.2.2 Via Image Package

```bash
wget https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate/${version}/release/standalone_fate_docker_image_${version}_release.tar.gz
Expand All @@ -63,25 +114,25 @@ docker images | grep federatedai/standalone_fate

If you see an image corresponding to `${version}`, it means the image download was successful.

### 2.3 Start
### 3.3 Start

```bash
docker run -it --name standalone_fate -p 8080:8080 federatedai/standalone_fate:${version}
```

### 2.4 Testing
### 3.4 Testing

```bash
source /data/projects/fate/fate_flow/bin/init_env.sh
```

- [Test Items](#4-Test-Items)
- [Test Items](#5-Test-Items)

## 3. Install FATE on the Host Machine (Using Pre-Compiled Installation Packages)
## 4. Install FATE on the Host Machine (Using Pre-Compiled Installation Packages)

**Note:** Replace `${version}` in the examples below with the actual version number.

### 3.1 Pre-deployment Environment Check
### 4.1 Pre-deployment Environment Check

Check if local ports 8080, 9360, and 9380 are already occupied.

Expand All @@ -97,7 +148,7 @@ Because operating system dependencies need to be installed, root privileges are
echo "{username to be used} ALL=(ALL) NOPASSWD:ALL" | tee /etc/sudoers.d/{username to be used}
```

### 3.2 Get Installation Package
### 4.2 Get Installation Package

Download the installation package and unpack it.

Expand All @@ -106,7 +157,7 @@ wget https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate/${version}/
tar -xzvf standalone_fate_install_${version}_release.tar.gz
```

### 3.3 Installation
### 4.3 Installation

Navigate to the unpacked directory and use `bin/init.sh` for installation.

Expand All @@ -126,14 +177,14 @@ cd standalone_fate_install_${version}_release;
bash bin/init.sh init
```

### 3.4 Start
### 4.4 Start

```bash
bash bin/init.sh status
bash bin/init.sh start
```

### 3.5 Testing
### 4.5 Testing

- Load environment variables

Expand All @@ -143,9 +194,9 @@ source bin/init_env.sh

- [Test Items](#4-Test-Items)

## 4. Test Items
## 5. Test Items

### 4.1 Toy Test
### 5.1 Toy Test

```bash
flow test toy -gid 10000 -hid 10000
Expand Down
84 changes: 68 additions & 16 deletions deploy/standalone-deploy/README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,62 @@

单机版提供 3 种部署方式,可以根据实际情况选择:

- 从 PyPI 安装 FATE
- 使用 Docker 镜像安装 FATE
- 在主机中安装 FATE (使用已编译的安装包)

## 2. 使用 Docker 镜像安装 FATE(推荐)
## 2. 从 PyPI 安装 FATE
### 2.1 安装Python环境
- [conda](https://docs.conda.io/projects/miniconda/en/latest/) 环境准备及安装
- 创建虚拟环境
```shell
# fate的运行环境为python>=3.8
conda create -n fate_env python=3.8
conda activate fate_env
```

### 2.2 安装 FATE
本节介绍从 PyPI 安装携带FATE-Flow服务,以及无服务直接使用FATE包的两种 FATE 安装方法

#### 2.2.1 安装 FATE,同时携带 FATE-Flow 服务
FATE-Flow提供了联邦作业生命周期管理,包括调度、数据管理、模型和指标管理等。

##### 2.2.1.1 安装FATE、FATE-FLOW、FATE-Client
```shell
pip安装fate_client[fate,fate_flow]==2.0.0.b0
```

#### 2.2.1.2 服务初始化
```shell
fate_flow init --ip 127.0.0.1 --port 9380 --home $HOME_DIR
pipeline --ip 127.0.0.1 --端口 9380
```
- `ip`:服务运行的ip
- `port`:服务运行的 http 端口
- `home`:数据存储目录。主要包括:数据/模型/日志/作业配置/sqlite.db等内容

#### 2.2.1.3 服务启动
```shell
fate_flow start
```

#### 2.2.1.4 测试

- [测试项](#5-测试项)

### 2.2.2 直接安装FATE
FATE提供多种联邦算法和安全协议, 用户可以在安装 FATE 后直接使用内置算法和安全协议。

#### 2.2.1.1 安装 FATE
```shell
pip pyfate==2.0.0.b0
```

#### 2.2.2.2 使用指引
参考 [examples](../../doc/2.0/fate/ml)


## 3. 使用 Docker 镜像安装 FATE(推荐)

建议使用 Docker 镜像,这样可以大大降低遇到问题的可能性。

Expand All @@ -40,9 +92,9 @@ export version={本次部署的 FATE 版本号, 如 2.0.0-beta}
export version=2.0.0-beta
```

### 2.2 拉取镜像
### 3.2 拉取镜像

#### 2.2.1 通过公共镜像服务
#### 3.2.1 通过公共镜像服务

```bash
# Docker Hub
Expand All @@ -53,7 +105,7 @@ docker pull ccr.ccs.tencentyun.com/federatedai/standalone_fate:${version}
docker tag ccr.ccs.tencentyun.com/federatedai/standalone_fate:${version} federatedai/standalone_fate:${version}
```

#### 2.2.2 通过镜像包
#### 3.2.2 通过镜像包

```bash
wget https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate/${version}/release/standalone_fate_docker_image_${version}_release.tar.gz
Expand All @@ -63,25 +115,25 @@ docker images | grep federatedai/standalone_fate

如果您能看到对应 `${version}` 的镜像,则表示镜像下载成功。

### 2.3 启动
### 3.3 启动

```bash
docker run -it --name standalone_fate -p 8080:8080 federatedai/standalone_fate:${version}
```

### 2.4 测试
### 3.4 测试

```bash
source /data/projects/fate/fate_flow/bin/init_env.sh
```

- [测试项](#4-测试项)
- [测试项](#5-测试项)

## 3. 在主机中安装 FATE(使用已编译的安装包)
## 4. 在主机中安装 FATE(使用已编译的安装包)

**注意:** 请使用实际的版本号替换示例中的 `${version}`

### 3.1 部署前环境检查
### 4.1 部署前环境检查

检查本地端口 8080、9360 和 9380 是否被占用。

Expand All @@ -97,7 +149,7 @@ netstat -apln|grep 9380
echo "{要使用的用户名} ALL=(ALL) NOPASSWD:ALL" | tee /etc/sudoers.d/{要使用的用户名}
```

### 3.2 获取安装包
### 4.2 获取安装包

下载安装包并解压缩。

Expand All @@ -106,7 +158,7 @@ wget https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate/${version}/
tar -xzvf standalone_fate_install_${version}_release.tar.gz
```

### 3.3 安装
### 4.3 安装

进入解压后的目录并使用 `bin/init.sh` 进行安装。

Expand All @@ -126,26 +178,26 @@ cd standalone_fate_install_${version}_release;
bash bin/init.sh init
```

### 3.4 启动
### 4.4 启动

```bash
bash bin/init.sh status
bash bin/init.sh start
```

### 3.5 测试
### 4.5 测试

- 加载环境变量

```bash
source bin/init_env.sh
```

- [测试项](#4-测试项)
5 [测试项](#5-测试项)

## 4. 测试项
## 5. 测试项

### 4.1 Toy 测试
### 5.1 Toy 测试

```bash
flow test toy -gid 10000 -hid 10000
Expand Down
Loading

0 comments on commit 754be54

Please sign in to comment.