Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Hackathon + No.151】PaddleRS运行环境打包,并制作端到端遥感建筑物提取教程 #147

Merged
merged 25 commits into from
May 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
51245ae
[Fix] Fix setup error about .version
geoyee Mar 30, 2023
ef3cf11
Merge branch 'PaddlePaddle:develop' into develop
geoyee Mar 31, 2023
3677927
Merge branch 'PaddlePaddle:develop' into develop
geoyee Mar 31, 2023
b0f202a
Merge branch 'PaddlePaddle:develop' into develop
geoyee Mar 31, 2023
14b2465
Merge branch 'PaddlePaddle:develop' into develop
geoyee Apr 2, 2023
1cef8c7
Merge branch 'PaddlePaddle:develop' into develop
geoyee Apr 5, 2023
2e9ed29
Merge branch 'PaddlePaddle:develop' into develop
geoyee Apr 6, 2023
bdb31f0
Merge branch 'PaddlePaddle:develop' into develop
geoyee Apr 8, 2023
4405141
Merge branch 'PaddlePaddle:develop' into develop
geoyee Apr 18, 2023
4a4694d
Merge branch 'PaddlePaddle:develop' into develop
geoyee Apr 21, 2023
267d20f
Merge branch 'PaddlePaddle:develop' into develop
geoyee Apr 29, 2023
45e54c8
Merge branch 'PaddlePaddle:develop' into develop
geoyee May 9, 2023
47724ef
Merge branch 'PaddlePaddle:develop' into develop
geoyee May 9, 2023
293f64a
[New] Init docker
geoyee May 9, 2023
054fa4f
[Docker] Fix and update docs
geoyee May 9, 2023
ae2908a
[Docker] Update docs
geoyee May 9, 2023
2b0523d
Delete Dockerfile
geoyee May 9, 2023
7e27389
[docker] Update with GPU
geoyee May 9, 2023
829ab9b
[Docker] Improve the training process in docker
geoyee May 10, 2023
59065ab
[Docker] Update
geoyee May 10, 2023
0004955
[Docker] Update file organization and add english
geoyee May 10, 2023
7b2c705
Update README.md
geoyee May 10, 2023
b48a96c
[Docker] Update
geoyee May 10, 2023
766ee3d
Merge branch 'docker' of github.com:geoyee/PaddleRS into docker
geoyee May 10, 2023
daaccde
Update README.md
geoyee May 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -139,4 +139,4 @@ dmypy.json
/tutorials/train/**/output/
/log

/playground/
/playground/
23 changes: 21 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,24 @@ WORKDIR /usr/src
RUN pip install git+https://github.com/lucasb-eyer/pydensecrf.git \
&& rm -rf /usr/src/pydensecrf

# 6. finish
WORKDIR /opt/PaddleRS
# 6. (optional) install eiseg
ARG EISEG
RUN if [ "$EISEG" = "ON" ] ; then \
pip install --upgrade pip \
&& pip install eiseg rasterio -i https://mirror.baidu.com/pypi/simple \
&& pip uninstall -y opencv-python-headless \
&& pip install opencv-python==4.2.0.34 -i https://mirror.baidu.com/pypi/simple \
&& apt-get update \
&& apt-get install -y \
libgl1-mesa-glx libxcb-xinerama0 libxkbcommon-x11-0 \
libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 \
libxcb-render-util0 libxcb-shape0 libxcb-xfixes0 \
x11-xserver-utils x11-apps locales \
&& locale-gen zh_CN \
&& locale-gen zh_CN.utf8 \
&& apt-get install -y ttf-wqy-microhei ttf-wqy-zenhei xfonts-wqy ; \
fi
ENV DISPLAY host.docker.internal:0

# 7. finish
WORKDIR /opt/PaddleRS
47 changes: 47 additions & 0 deletions docs/docker_cn.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# PaddleRS镜像构建与使用

## 1. 镜像构建

首先需要拉取仓库:

```shell
git clone https://github.com/PaddlePaddle/PaddleRS
```

- 安装CPU版本,默认为2.4.1:

```shell
docker build -t paddlers:latest -f Dockerfile .
```

- (可选)安装GPU版本,若要使用PaddleRS进行训练,最好使用GPU版本,请确保Docker版本大于19,其他环境的`PPTAG`可以参考[https://hub.docker.com/r/paddlepaddle/paddle/tags](https://hub.docker.com/r/paddlepaddle/paddle/tags):

```shell
docker build -t paddlers:latest -f Dockerfile . --build-arg PPTAG=2.4.1-gpu-cuda11.7-cudnn8.4-trt8.4
```

- (可选)若需要使用[EISeg](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/EISeg)提供的交互式分割标注功能,可设置`EISEG="ON"`,默认只安装了支持遥感标注的扩展:

```shell
docker build -t paddlers:latest -f Dockerfile . --build-arg EISEG="ON"
```

## 2. 镜像使用

- 查看当前构建好的镜像,记住需要启动的镜像的`<imageID>`:

```shell
docker images
```

- 仅使用PaddleRS(包括EISeg),可直接启动镜像,将本机存放模型参数的文件夹挂载到docker中,若要使用GPU,在docker 19之后,可以添加中括号内的参数启用GPU:

```shell
docker run -it -v <本机文件夹绝对路径:容器文件夹绝对路径> [--gpus all -e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all] <imageID>
```

- (可选)若需要使用EISeg,则需要在本机安装和开启X11,用于接收Qt的GUI界面。Windows可使用[VcXsrv](https://sourceforge.net/projects/vcxsrv/),Linux可使用[Xserver](https://blog.csdn.net/a806689294/article/details/111462627)。在相关工具启动之后,再启动EISeg:

```shell
eiseg
```
47 changes: 47 additions & 0 deletions docs/docker_en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# PaddleRS Image Build and Use

## 1. Image Build

First, you need to clone the repository:

```shell
git clone https://github.com/PaddlePaddle/PaddleRS
```

- Install the CPU version, which is 2.4.1 by default:

```shell
docker build -t paddlers:latest -f Dockerfile .
```

- (Optional) Install the GPU version. If you want to use PaddleRS for training, it is recommended to use the GPU version. Make sure that the Docker version is greater than 19. For other environments, the `PPTAG` can refer to [https://hub.docker.com/r/paddlepaddle/paddle/tags](https://hub.docker.com/r/paddlepaddle/paddle/tags):

```shell
docker build -t paddlers:latest -f Dockerfile . --build-arg PPTAG=2.4.1-gpu-cuda11.7-cudnn8.4-trt8.4
```

- (Optional) If you need to use the interactive segmentation annotation function provided by [EISeg](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/EISeg), you can set `EISEG="ON"`. By default, only the extensions that support remote sensing annotation are installed:

```shell
docker build -t paddlers:latest -f Dockerfile . --build-arg EISEG="ON"
```

## 2. Image Use

- View the currently built images and remember the `<imageID>` of the image to be started:

```shell
docker images
```

- To use only PaddleRS (including EISeg), you can directly start the image, and mount the folder where the model parameters are stored on the local machine to Docker. If you want to use the GPU, after docker 19, you can add the parameter in square brackets to enable the GPU:

```shell
docker run -it -v <local_folder_absolute_path:container_folder_absolute_path> [--gpus all -e NVIDIA_DRIVER_CAPABILITIES=compute,utility -e NVIDIA_VISIBLE_DEVICES=all] <imageID>
```

- (Optional) If you need to use EISeg, you need to install and enable X11 on the local machine to receive the GUI interface of Qt. Windows can use [VcXsrv](https://sourceforge.net/projects/vcxsrv/), and Linux can use [Xserver](https://blog.csdn.net/a806689294/article/details/111462627). After the relevant tools are started, start EISeg:

```shell
eiseg
```
25 changes: 1 addition & 24 deletions docs/quick_start_cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,30 +62,7 @@ python setup.py install
```


除了采用上述安装步骤以外,PaddleRS也提供Docker安装方式。具体步骤如下:

1. 从dockerhub拉取镜像:

```shell
docker pull paddlepaddle/paddlers:1.0.0 # 暂无
```

或者,可以选择从头开始构建。通过修改`Dockerfile`文件中的`PPTAG`,可选择PaddlePaddle的多种基础镜像。

```shell
git clone https://github.com/PaddlePaddle/PaddleRS
cd PaddleRS
docker build -t <imageName> . # 默认使用PaddlePaddle 2.4.1的CPU版本
# docker build -t <imageName> . --build-arg PPTAG=2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0 # 构建使用GPU版本PaddlePaddle的环境
# 其余tag可以参考:https://hub.docker.com/r/paddlepaddle/paddle/tags
```

2. 启动容器

```shell
docker images # 查看镜像的ID
docker run -it <imageID>
```
除了采用上述安装步骤以外,PaddleRS也提供Docker安装方式,具体请参考[文档](./docker_cn.md)。

## 模型训练

Expand Down
25 changes: 1 addition & 24 deletions docs/quick_start_en.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,30 +55,7 @@ cd paddlers/models/ppdet/ext_op
python setup.py install
```

We also provide a docker image for installation:

1. Pull from dockerhub:

```shell
docker pull paddlepaddle/paddlers:1.0.0
```

Optionally, you can build the image from scratch. You can change the base images for different PaddlePaddle versions by setting `PPTAG` in `Dockerfile`.

```shell
git clone https://github.com/PaddlePaddle/PaddleRS
cd PaddleRS
docker build -t <imageName> . # Default is PaddlePaddle-2.4.1-CPU
# docker build -t <imageName> . --build-arg PPTAG=2.4.1-gpu-cuda10.2-cudnn7.6-trt7.0 # Use a GPU version of PaddlePaddle
# For more tags, please refer to: https://hub.docker.com/r/paddlepaddle/paddle/tags
```

2. Start a container

```shell
docker images # View the ID of the image
docker run -it <imageID>
```
We also provide a docker image for installation, see [here](./docker_en.md).

## Model Training

Expand Down
27 changes: 27 additions & 0 deletions examples/building_extraction/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# 0. from base paddlers image
FROM paddlers:latest

# 1. install mysql and nodejs
RUN apt-get update \
&& apt-get install -y mysql-server mysql-client libmysqlclient-dev \
git curl \
&& curl -sL https://deb.nodesource.com/setup_16.x | bash - \
&& apt-get install -y nodejs

# 2. clone geoview
WORKDIR /opt
RUN git clone https://github.com/PaddleCV-SIG/GeoView.git \
&& mv PaddleRS GeoView
ENV PYTHONPATH /opt/GeoView/PaddleRS

# 3. install backend requirements
WORKDIR /opt/GeoView/backend
RUN pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple \
&& mv .flaskenv_template .flaskenv

# 4. install frontend requirements
WORKDIR /opt/GeoView/frontend
RUN npm install

# 5. finish
WORKDIR /opt/GeoView
Loading