diff --git a/README.md b/README.md
index 62f6dca7c6..e6ecfcd8bb 100644
--- a/README.md
+++ b/README.md
@@ -88,6 +88,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
Auto-gbdt
Cifar10-pytorch
Scikit-learn
+ EfficientNet
More...
@@ -166,7 +167,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
-
+
@@ -192,18 +193,18 @@ Within the following table, we summarized the current NNI capabilities, we are g
Support TrainingService
Implement TrainingService
-
-
+
+
-## **Install & Verify**
+## **Installation**
-**Install through pip**
+### **Install**
-* We support Linux, MacOS and Windows (local, remote and pai mode) in current stage, Ubuntu 16.04 or higher, MacOS 10.14.1 along with Windows 10.1809 are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`.
+NNI supports and is tested on Ubuntu >= 16.04, macOS >= 10.14.1, and Windows 10 >= 1809. Simply run the following `pip install` in an environment that has `python 64-bit >= 3.5`.
-Linux and MacOS
+Linux or macOS
```bash
python3 -m pip install --upgrade nni
@@ -215,65 +216,39 @@ Windows
python -m pip install --upgrade nni
```
-Note:
-
-* `--user` can be added if you want to install NNI in your home directory, which does not require any special privileges.
-* Currently NNI on Windows support local, remote and pai mode. Anaconda or Miniconda is highly recommended to install NNI on Windows.
-* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.md)
-
-**Install through source code**
-
-* We support Linux (Ubuntu 16.04 or higher), MacOS (10.14.1) and Windows (10.1809) in our current stage.
-
-Linux and MacOS
-
-* Run the following commands in an environment that has `python >= 3.5`, `git` and `wget`.
-
-```bash
- git clone -b v1.3 https://github.com/Microsoft/nni.git
- cd nni
- source install.sh
-```
-
-Windows
-
-* Run the following commands in an environment that has `python >=3.5`, `git` and `PowerShell`
+If you want to try latest code, please [install NNI](docs/en_US/Tutorial/Installation.md) from source code.
-```bash
- git clone -b v1.3 https://github.com/Microsoft/nni.git
- cd nni
- powershell -ExecutionPolicy Bypass -file install.ps1
-```
+For detail system requirements of NNI, please refer to [here](docs/en_US/Tutorial/Installation.md#system-requirements).
-For the system requirements of NNI, please refer to [Install NNI](docs/en_US/Tutorial/Installation.md)
+Note:
-For NNI on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/NniOnWindows.md)
+* If there is any privilege issue, add `--user` to install NNI in the user directory.
+* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install NNI on Windows.
+* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.md). For FAQ on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/NniOnWindows.md).
-**Verify install**
+### **Verify installation**
-The following example is an experiment built on TensorFlow. Make sure you have **TensorFlow 1.x installed** before running it. Note that **currently Tensorflow 2.0 is NOT supported**.
+The following example is built on TensorFlow 1.x. Make sure **TensorFlow 1.x is used** when running it.
* Download the examples via clone the source code.
-```bash
- git clone -b v1.3 https://github.com/Microsoft/nni.git
-```
-
-Linux and MacOS
+ ```bash
+ git clone -b v1.3 https://github.com/Microsoft/nni.git
+ ```
* Run the MNIST example.
-```bash
- nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
-```
+ Linux or macOS
-Windows
+ ```bash
+ nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
+ ```
-* Run the MNIST example.
+ Windows
-```bash
- nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
-```
+ ```bash
+ nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
+ ```
* Wait for the message `INFO: Successfully started experiment!` in the command line. This message indicates that your experiment has been successfully started. You can explore the experiment using the `Web UI url`.
@@ -323,9 +298,10 @@ When you submit a pull request, a CLA-bot will automatically determine whether y
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the Code of [Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact opencode@microsoft.com with any additional questions or comments.
After getting familiar with contribution agreements, you are ready to create your first PR =), follow the NNI developer tutorials to get start:
-* We recommend new contributors to start with ['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22), these issues are simple and easy to start.
+* We recommend new contributors to start with simple issues: ['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22).
* [NNI developer environment installation tutorial](docs/en_US/Tutorial/SetupNniDeveloperEnvironment.md)
* [How to debug](docs/en_US/Tutorial/HowToDebug.md)
+* If you have any questions on usage, review [FAQ](https://github.com/microsoft/nni/blob/master/docs/en_US/Tutorial/FAQ.md) first, if there are no relevant issues and answers to your question, try contact NNI dev team and users in [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) or [File an issue](https://github.com/microsoft/nni/issues/new/choose) on GitHub.
* [Customize your own Tuner](docs/en_US/Tuner/CustomizeTuner.md)
* [Implement customized TrainingService](docs/en_US/TrainingService/HowToImplementTrainingService.md)
* [Implement a new NAS trainer on NNI](https://github.com/microsoft/nni/blob/master/docs/en_US/NAS/NasInterface.md#implement-a-new-nas-trainer-on-nni)
@@ -369,4 +345,3 @@ We encourage researchers and students leverage these projects to accelerate the
## **License**
The entire codebase is under [MIT license](LICENSE)
-
diff --git a/README_zh_CN.md b/README_zh_CN.md
index ec77fcbd50..9aca68dde8 100644
--- a/README_zh_CN.md
+++ b/README_zh_CN.md
@@ -4,7 +4,7 @@
* * *
-[](LICENSE) [](https://msrasrg.visualstudio.com/NNIOpenSource/_build/latest?definitionId=6) [](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen) [](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3Abug) [](https://github.com/Microsoft/nni/pulls?q=is%3Apr+is%3Aopen) [](https://github.com/Microsoft/nni/releases) [](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [](https://nni.readthedocs.io/zh/latest/?badge=latest)
+[](LICENSE) [](https://msrasrg.visualstudio.com/NNIOpenSource/_build/latest?definitionId=17&branchName=master) [](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen) [](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3Abug) [](https://github.com/Microsoft/nni/pulls?q=is%3Apr+is%3Aopen) [](https://github.com/Microsoft/nni/releases) [](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [](https://nni.readthedocs.io/zh/latest/?badge=latest)
[English](README.md)
@@ -83,6 +83,7 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
Auto-gbdt
Cifar10-pytorch
Scikit-learn
+ EfficientNet
更多...
@@ -121,6 +122,7 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
ENAS
DARTS
P-DARTS
+ CDARTS
Network Morphism
@@ -160,7 +162,7 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
-
+
@@ -186,18 +188,18 @@ NNI 提供命令行工具以及友好的 WebUI 来管理训练的 Experiment。
支持训练平台
实现训练平台
-
-
+
+
-## **安装和验证**
+## **安装**
-**通过 pip 命令安装**
+### **安装**
-* 当前支持 Linux，MacOS 和 Windows（本机，远程，OpenPAI 模式），在 Ubuntu 16.04 或更高版本，MacOS 10.14.1 以及 Windows 10.1809 上进行了测试。 在 `python >= 3.5` 的环境中，只需要运行 `pip install` 即可完成安装。
+NNI 支持并在 Ubuntu >= 16.04, macOS >= 10.14.1, 和 Windows 10 >= 1809 通过了测试。 在 `python 64-bit >= 3.5` 的环境中，只需要运行 `pip install` 即可完成安装。
-Linux 和 macOS
+Linux 或 macOS
```bash
python3 -m pip install --upgrade nni
@@ -209,65 +211,39 @@ Windows
python -m pip install --upgrade nni
```
-注意：
-
-* 如果需要将 NNI 安装到自己的 home 目录中，可使用 `--user`，这样也不需要任何特殊权限。
-* 目前，Windows 上的 NNI 支持本机，远程和 OpenPAI 模式。 强烈推荐使用 Anaconda 或 Miniconda 在 Windows 上安装 NNI。
-* 如果遇到如`Segmentation fault` 这样的任何错误请参考[常见问题](docs/zh_CN/Tutorial/FAQ.md)。
-
-**通过源代码安装**
-
-* 当前支持 Linux（Ubuntu 16.04 或更高版本），MacOS（10.14.1）以及 Windows 10（1809 版）。
-
-Linux 和 MacOS
-
-* 在 `python >= 3.5` 的环境中运行命令： `git` 和 `wget`，确保安装了这两个组件。
-
-```bash
- git clone -b v1.3 https://github.com/Microsoft/nni.git
- cd nni
- source install.sh
-```
-
-Windows
-
-* 在 `python >=3.5` 的环境中运行命令： `git` 和 `PowerShell`，确保安装了这两个组件。
+如果想要尝试最新代码，可通过源代码[安装 NNI](docs/zh_CN/Tutorial/Installation.md)。
-```bash
- git clone -b v1.3 https://github.com/Microsoft/nni.git
- cd nni
- powershell -ExecutionPolicy Bypass -file install.ps1
-```
+有关 NNI 的详细系统要求，参考[这里](docs/zh_CN/Tutorial/Installation.md#system-requirements)。
-参考[安装 NNI](docs/zh_CN/Tutorial/Installation.md) 了解系统需求。
+注意：
-Windows 上参考 [Windows 上使用 NNI](docs/zh_CN/Tutorial/NniOnWindows.md)。
+* 如果遇到任何权限问题，可添加 `--user` 在用户目录中安装 NNI。
+* 目前，Windows 上的 NNI 支持本机，远程和 OpenPAI 模式。 强烈推荐使用 Anaconda 或 Miniconda 在 Windows 上安装 NNI。
+* 如果遇到如 `Segmentation fault` 等错误参考[常见问题](docs/zh_CN/Tutorial/FAQ.md)。 Windows 上的 FAQ 参考[在 Windows 上使用 NNI](docs/zh_CN/Tutorial/NniOnWindows.md)。
-**验证安装**
+### **验证安装**
-以下示例 Experiment 依赖于 TensorFlow 。 在运行前确保安装了 **TensorFlow 1.x**。 注意，**目前不支持 TensorFlow 2.0**。
+以下示例基于 TensorFlow 1.x 。确保运行环境中使用的的是 ** TensorFlow 1.x**。
* 通过克隆源代码下载示例。
-
-```bash
- git clone -b v1.3 https://github.com/Microsoft/nni.git
-```
-
-Linux 和 MacOS
-
-* 运行 MNIST 示例。
-
-```bash
- nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
-```
-
-Windows
+
+ ```bash
+ git clone -b v1.3 https://github.com/Microsoft/nni.git
+ ```
* 运行 MNIST 示例。
-
-```bash
- nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
-```
+
+ Linux 或 macOS
+
+ ```bash
+ nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
+ ```
+
+ Windows
+
+ ```bash
+ nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
+ ```
* 在命令行中等待输出 `INFO: Successfully started experiment!`。 此消息表明 Experiment 已成功启动。 通过命令行输出的 `Web UI url` 来访问 Experiment 的界面。
@@ -319,11 +295,12 @@ You can use these commands to get more information about the experiment
该项目采用了 [ Microsoft 开源行为准则 ](https://opensource.microsoft.com/codeofconduct/)。 有关详细信息，请参阅[常见问题解答](https://opensource.microsoft.com/codeofconduct/faq/)，如有任何疑问或意见可联系 opencode@microsoft.com。
-熟悉贡献协议后，即可按照 NNI 开发人员教程，创建第一个 PR =)：
+熟悉贡献协议后，即可按照 NNI 开发人员教程，创建第一个 PR：
-* 推荐新贡献者先找到标有 ['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) 或 ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22) 标签的 Issue。这些都比较简单，可以从这些问题开始。
+* 推荐新贡献者先从简单的问题开始：['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) 或 ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22)。
* [NNI 开发环境安装教程](docs/zh_CN/Tutorial/SetupNniDeveloperEnvironment.md)
* [如何调试](docs/zh_CN/Tutorial/HowToDebug.md)
+* 如果有使用上的问题，可先查看[常见问题解答](https://github.com/microsoft/nni/blob/master/docs/zh_CN/Tutorial/FAQ.md)。如果没能解决问题，可通过 [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) 联系 NNI 开发团队或在 GitHub 上 [报告问题](https://github.com/microsoft/nni/issues/new/choose)。
* [自定义 Tuner](docs/zh_CN/Tuner/CustomizeTuner.md)
* [实现定制的训练平台](docs/zh_CN/TrainingService/HowToImplementTrainingService.md)
* [在 NNI 上实现新的 NAS Trainer](https://github.com/microsoft/nni/blob/master/docs/zh_CN/NAS/NasInterface.md#implement-a-new-nas-trainer-on-nni)
@@ -349,7 +326,7 @@ You can use these commands to get more information about the experiment
* [使用 NNI 为 SPTAG 自动调参](docs/zh_CN/CommunitySharings/SptagAutoTune.md)
* [使用 NNI 为 scikit-learn 查找超参](https://towardsdatascience.com/find-thy-hyper-parameters-for-scikit-learn-pipelines-using-microsoft-nni-f1015b1224c1)
* **博客** - [AutoML 工具（Advisor，NNI 与 Google Vizier）的对比](http://gaocegege.com/Blog/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/katib-new#%E6%80%BB%E7%BB%93%E4%B8%8E%E5%88%86%E6%9E%90) 作者：[@gaocegege](https://github.com/gaocegege) - kubeflow/katib 的设计与实现的总结与分析章节
- * **Blog (中文)** - [NNI 2019 新功能汇总](https://mp.weixin.qq.com/s/7_KRT-rRojQbNuJzkjFMuA) by @squirrelsc
+ * **博客** - [NNI 2019 新功能汇总](https://mp.weixin.qq.com/s/7_KRT-rRojQbNuJzkjFMuA) by @squirrelsc
## **反馈**
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 3f4238e413..45dc10a976 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -26,8 +26,8 @@ jobs:
yarn eslint
displayName: 'Run eslint'
- script: |
- python3 -m pip install torch==0.4.1 --user
- python3 -m pip install torchvision==0.2.1 --user
+ python3 -m pip install torch==1.2.0 --user
+ python3 -m pip install torchvision==0.4.0 --user
python3 -m pip install tensorflow==1.13.1 --user
python3 -m pip install keras==2.1.6 --user
python3 -m pip install gym onnx --user
@@ -91,8 +91,8 @@ jobs:
echo "##vso[task.setvariable variable=PATH]${HOME}/Library/Python/3.7/bin:${PATH}"
displayName: 'Install nni toolkit via source code'
- script: |
- python3 -m pip install torch==0.4.1 --user
- python3 -m pip install torchvision==0.2.1 --user
+ python3 -m pip install torch==1.2.0 --user
+ python3 -m pip install torchvision==0.4.0 --user
python3 -m pip install tensorflow==1.13.1 --user
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null
brew install swig@3
@@ -131,7 +131,7 @@ jobs:
- script: |
python -m pip install scikit-learn==0.20.0 --user
python -m pip install keras==2.1.6 --user
- python -m pip install https://download.pytorch.org/whl/cu90/torch-0.4.1-cp36-cp36m-win_amd64.whl --user
+ python -m pip install torch===1.2.0 torchvision===0.4.1 -f https://download.pytorch.org/whl/torch_stable.html --user
python -m pip install torchvision --user
python -m pip install tensorflow==1.13.1 --user
displayName: 'Install dependencies'
diff --git a/docs/en_US/CommunitySharings/NNI_AutoFeatureEng.md b/docs/en_US/CommunitySharings/NNI_AutoFeatureEng.md
new file mode 100644
index 0000000000..40a1e2f8c1
--- /dev/null
+++ b/docs/en_US/CommunitySharings/NNI_AutoFeatureEng.md
@@ -0,0 +1,99 @@
+# NNI review article from Zhihu: - By Garvin Li
+
+The article is by a NNI user on Zhihu forum. In the article, Garvin had shared his experience on using NNI for Automatic Feature Engineering. We think this article is very useful for users who are interested in using NNI for feature engineering. With author's permission, we translated the original article into English.
+
+**原文(source)**: [如何看待微软最新发布的AutoML平台NNI？By Garvin Li](https://www.zhihu.com/question/297982959/answer/964961829?utm_source=wechat_session&utm_medium=social&utm_oi=28812108627968&from=singlemessage&isappinstalled=0)
+
+## 01 Overview of AutoML
+
+In author's opinion, AutoML is not only about hyperparameter optimization, but
+also a process that can target various stages of the machine learning process,
+including feature engineering, NAS, HPO, etc.
+
+## 02 Overview of NNI
+
+NNI (Neural Network Intelligence) is an open source AutoML toolkit from
+Microsoft, to help users design and tune machine learning models, neural network
+architectures, or a complex system’s parameters in an efficient and automatic
+way.
+
+Link:[ https://github.com/Microsoft/nni](https://github.com/Microsoft/nni)
+
+In general, most of Microsoft tools have one prominent characteristic: the
+design is highly reasonable (regardless of the technology innovation degree).
+NNI's AutoFeatureENG basically meets all user requirements of AutoFeatureENG
+with a very reasonable underlying framework design.
+
+## 03 Details of NNI-AutoFeatureENG
+
+>The article is following the github project: [https://github.com/SpongebBob/tabular_automl_NNI](https://github.com/SpongebBob/tabular_automl_NNI).
+
+Each new user could do AutoFeatureENG with NNI easily and efficiently. To exploring the AutoFeatureENG capability, downloads following required files, and then run NNI install through pip.
+
+
+NNI treats AutoFeatureENG as a two-steps-task, feature generation exploration and feature selection. Feature generation exploration is mainly about feature derivation and high-order feature combination.
+
+## 04 Feature Exploration
+
+For feature derivation, NNI offers many operations which could automatically generate new features, which list [as following](https://github.com/SpongebBob/tabular_automl_NNI/blob/master/AutoFEOp.md) :
+
+**count**: Count encoding is based on replacing categories with their counts computed on the train set, also named frequency encoding.
+
+**target**: Target encoding is based on encoding categorical variable values with the mean of target variable per value.
+
+**embedding**: Regard features as sentences, generate vectors using *Word2Vec.*
+
+**crosscout**: Count encoding on more than one-dimension, alike CTR (Click Through Rate).
+
+**aggregete**: Decide the aggregation functions of the features, including min/max/mean/var.
+
+**nunique**: Statistics of the number of unique features.
+
+**histsta**: Statistics of feature buckets, like histogram statistics.
+
+Search space could be defined in a **JSON file**: to define how specific features intersect, which two columns intersect and how features generate from corresponding columns.
+
+
+
+The picture shows us the procedure of defining search space. NNI provides count encoding for 1-order-op, as well as cross count encoding, aggerate statistics (min max var mean median nunique) for 2-order-op.
+
+For example, we want to search the features which are a frequency encoding (valuecount) features on columns name {“C1”, ...,” C26”}, in the following way:
+
+
+
+we can define a cross frequency encoding (value count on cross dims) method on columns {"C1",...,"C26"} x {"C1",...,"C26"} in the following way:
+
+
+
+The purpose of Exploration is to generate new features. You can use **get_next_parameter** function to get received feature candidates of one trial.
+
+>RECEIVED_PARAMS = nni.get_next_parameter()
+
+## 05 Feature selection
+
+To avoid feature explosion and overfitting, feature selection is necessary. In the feature selection of NNI-AutoFeatureENG, LightGBM (Light Gradient Boosting Machine), a gradient boosting framework developed by Microsoft, is mainly promoted.
+
+
+
+If you have used **XGBoost** or **GBDT**, you would know the algorithm based on tree structure can easily calculate the importance of each feature on results. LightGBM is able to make feature selection naturally.
+
+The issue is that selected features might be applicable to *GBDT* (Gradient Boosting Decision Tree), but not to the linear algorithm like *LR* (Logistic Regression).
+
+
+
+## 06 Summary
+
+NNI's AutoFeatureEng sets a well-established standard, showing us the operation procedure, available modules, which is highly convenient to use. However, a simple model is probably not enough for good results.
+
+## Suggestions to NNI
+
+About Exploration: If consider using DNN (like xDeepFM) to extract high-order feature would be better.
+
+About Selection: There could be more intelligent options, such as automatic selection system based on downstream models.
+
+Conclusion: NNI could offer users some inspirations of design and it is a good open source project. I suggest researchers leverage it to accelerate the AI research.
+
+Tips: Because the scripts of open source projects are compiled based on gcc7, Mac system may encounter problems of gcc (GNU Compiler Collection). The solution is as follows:
+
+#brew install libomp
+
diff --git a/docs/en_US/CommunitySharings/community_sharings.rst b/docs/en_US/CommunitySharings/community_sharings.rst
index 6938000a9b..23431301c1 100644
--- a/docs/en_US/CommunitySharings/community_sharings.rst
+++ b/docs/en_US/CommunitySharings/community_sharings.rst
@@ -13,3 +13,4 @@ In addtion to the official tutorilas and examples, we encourage community contri
Hyper-parameter Tuning Algorithm Comparsion
Parallelizing Optimization for TPE
Automatically tune systems with NNI
+ NNI review article from Zhihu: - By Garvin Li
diff --git a/docs/en_US/TrainingService/PaiMode.md b/docs/en_US/TrainingService/PaiMode.md
index 3174e1079b..6f5068d320 100644
--- a/docs/en_US/TrainingService/PaiMode.md
+++ b/docs/en_US/TrainingService/PaiMode.md
@@ -3,7 +3,36 @@
NNI supports running an experiment on [OpenPAI](https://github.com/Microsoft/pai) (aka pai), called pai mode. Before starting to use NNI pai mode, you should have an account to access an [OpenPAI](https://github.com/Microsoft/pai) cluster. See [here](https://github.com/Microsoft/pai#how-to-deploy) if you don't have any OpenPAI account and want to deploy an OpenPAI cluster. In pai mode, your trial program will run in pai's container created by Docker.
## Setup environment
-Install NNI, follow the install guide [here](../Tutorial/QuickStart.md).
+Step 1. Install NNI, follow the install guide [here](../Tutorial/QuickStart.md).
+
+Step 2. Get PAI token.
+Click `My profile` button in the top-right side of PAI's webprotal.
+
+Find the token management region, copy one of the token as your account token.
+
+
+Step 3. Mount NFS storage to local machine.
+ Click `Submit job` button in PAI's webportal.
+
+ Find the data management region in job submission page.
+
+The `DEFAULT_STORAGE`field is the path to be mounted in PAI's container when a job is started. The `Preview container paths` is the NFS host and path that PAI provided, you need to mount the corresponding host and path to your local machine first, then NNI could use the PAI's NFS storage.
+For example, use the following command:
+```
+sudo mount nfs://gcr-openpai-infra02:/pai/data /local/mnt
+```
+Then the `/data` folder in container will be mounted to `/local/mnt` folder in your local machine.
+You could use the following configuration in your NNI's config file:
+```
+nniManagerNFSMountPath: /local/mnt
+containerNFSMountPath: /data
+```
+
+Step 4. Get PAI's storage plugin name.
+Contact PAI's admin, and get the PAI's storage plugin name for NFS storage. The default storage name is `teamwise_storage`, the configuration in NNI's config file is in following value:
+```
+paiStoragePlugin: teamwise_storage
+```
## Run an experiment
Use `examples/trials/mnist-annotation` as an example. The NNI config YAML file's content is like:
@@ -37,6 +66,7 @@ trial:
virtualCluster: default
nniManagerNFSMountPath: /home/user/mnt
containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
# Configuration to access OpenPAI Cluster
paiConfig:
userName: your_pai_nni_user
@@ -48,12 +78,12 @@ Note: You should set `trainingServicePlatform: pai` in NNI config YAML file if y
Compared with [LocalMode](LocalMode.md) and [RemoteMachineMode](RemoteMachineMode.md), trial configuration in pai mode have these additional keys:
* cpuNum
- * Required key. Should be positive number based on your trial program's CPU requirement
+ * Optional key. Should be positive number based on your trial program's CPU requirement. If it is not set in trial configuration, it should be set in the config file specified in `paiConfigPath` field.
* memoryMB
- * Required key. Should be positive number based on your trial program's memory requirement
+ * Optional key. Should be positive number based on your trial program's memory requirement. If it is not set in trial configuration, it should be set in the config file specified in `paiConfigPath` field.
* image
- * Required key. In pai mode, your trial program will be scheduled by OpenPAI to run in [Docker container](https://www.docker.com/). This key is used to specify the Docker image used to create the container in which your trial will run.
- * We already build a docker image [nnimsra/nni](https://hub.docker.com/r/msranni/nni/) on [Docker Hub](https://hub.docker.com/). It contains NNI python packages, Node modules and javascript artifact files required to start experiment, and all of NNI dependencies. The docker file used to build this image can be found at [here](https://github.com/Microsoft/nni/tree/master/deployment/docker/Dockerfile). You can either use this image directly in your config file, or build your own image based on it.
+ * Optional key. In pai mode, your trial program will be scheduled by OpenPAI to run in [Docker container](https://www.docker.com/). This key is used to specify the Docker image used to create the container in which your trial will run.
+ * We already build a docker image [nnimsra/nni](https://hub.docker.com/r/msranni/nni/) on [Docker Hub](https://hub.docker.com/). It contains NNI python packages, Node modules and javascript artifact files required to start experiment, and all of NNI dependencies. The docker file used to build this image can be found at [here](https://github.com/Microsoft/nni/tree/master/deployment/docker/Dockerfile). You can either use this image directly in your config file, or build your own image based on it. If it is not set in trial configuration, it should be set in the config file specified in `paiConfigPath` field.
* virtualCluster
* Optional key. Set the virtualCluster of OpenPAI. If omitted, the job will run on default virtual cluster.
* nniManagerNFSMountPath
@@ -61,7 +91,9 @@ Compared with [LocalMode](LocalMode.md) and [RemoteMachineMode](RemoteMachineMod
* containerNFSMountPath
* Required key. Set the mount path in your container used in PAI.
* paiStoragePlugin
- * Required key. Set the storage plugin name used in PAI.
+ * Optional key. Set the storage plugin name used in PAI. If it is not set in trial configuration, it should be set in the config file specified in `paiConfigPath` field.
+* paiConfigPath
+ * Optional key. Set the file path of pai job configuration, the file is in yaml format.
Once complete to fill NNI experiment config file and save (for example, save as exp_pai.yml), then run the following command
diff --git a/docs/en_US/TrainingService/PaiYarnMode.md b/docs/en_US/TrainingService/PaiYarnMode.md
index eb2864f94c..64dda5e465 100644
--- a/docs/en_US/TrainingService/PaiYarnMode.md
+++ b/docs/en_US/TrainingService/PaiYarnMode.md
@@ -6,7 +6,7 @@ The original `pai` mode is modificated to `paiYarn` mode, which is a distributed
Install NNI, follow the install guide [here](../Tutorial/QuickStart.md).
## Run an experiment
-Use `examples/trials/mnist-annotation` as an example. The NNI config YAML file's content is like:
+Use `examples/trials/mnist-tfv1` as an example. The NNI config YAML file's content is like:
```yaml
authorName: your_name
@@ -22,14 +22,14 @@ trainingServicePlatform: paiYarn
# search space file
searchSpacePath: search_space.json
# choice: true, false
-useAnnotation: true
+useAnnotation: false
tuner:
builtinTunerName: TPE
classArgs:
optimize_mode: maximize
trial:
command: python3 mnist.py
- codeDir: ~/nni/examples/trials/mnist-annotation
+ codeDir: ~/nni/examples/trials/mnist-tfv1
gpuNum: 0
cpuNum: 1
memoryMB: 8196
diff --git a/docs/en_US/TrainingService/RemoteMachineMode.md b/docs/en_US/TrainingService/RemoteMachineMode.md
index 7e1df06ccc..54a4e45159 100644
--- a/docs/en_US/TrainingService/RemoteMachineMode.md
+++ b/docs/en_US/TrainingService/RemoteMachineMode.md
@@ -1,24 +1,32 @@
-# Run an Experiment on Multiple Machines
+# Run an Experiment on Remote Machines
-NNI supports running an experiment on multiple machines through SSH channel, called `remote` mode. NNI assumes that you have access to those machines, and already setup the environment for running deep learning training code.
+NNI can run one experiment on multiple remote machines through SSH, called `remote` mode. It's like a lightweight training platform. In this mode, NNI can be started from your computer, and dispatch trials to remote machines in parallel.
-e.g. Three machines and you login in with account `bob` (Note: the account is not necessarily the same on different machine):
+## Remote machine requirements
-| IP | Username| Password |
-| -------- |---------|-------|
-| 10.1.1.1 | bob | bob123 |
-| 10.1.1.2 | bob | bob123 |
-| 10.1.1.3 | bob | bob123 |
+* It only supports Linux as remote machines, and [linux part in system specification](../Tutorial/Installation.md) is same as NNI local mode.
-## Setup NNI environment
+* Follow [installation](../Tutorial/Installation.md) to install NNI on each machine.
-Install NNI on each of your machines following the install guide [here](../Tutorial/QuickStart.md).
+* Make sure remote machines meet environment requirements of your trial code. If the default environment does not meet the requirements, the setup script can be added into `command` field of NNI config.
+
+* Make sure remote machines can be accessed through SSH from the machine which runs `nnictl` command. It supports both password and key authentication of SSH. For advanced usages, please refer to [machineList part of configuration](../Tutorial/ExperimentConfig.md).
+
+* Make sure the NNI version on each machine is consistent.
## Run an experiment
-Install NNI on another machine which has network accessibility to those three machines above, or you can just run `nnictl` on any one of the three to launch the experiment.
+e.g. there are three machines, which can be logged in with username and password.
+
+| IP | Username | Password |
+| -------- | -------- | -------- |
+| 10.1.1.1 | bob | bob123 |
+| 10.1.1.2 | bob | bob123 |
+| 10.1.1.3 | bob | bob123 |
+
+Install and run NNI on one of those three machines or another machine, which has network access to them.
-We use `examples/trials/mnist-annotation` as an example here. Shown here is `examples/trials/mnist-annotation/config_remote.yml`:
+Use `examples/trials/mnist-annotation` as the example. Below is content of `examples/trials/mnist-annotation/config_remote.yml`:
```yaml
authorName: default
@@ -58,14 +66,8 @@ machineList:
passwd: bob123
```
-Files in `codeDir` will be automatically uploaded to the remote machine. You can run NNI on different operating systems (Windows, Linux, MacOS) to spawn experiments on the remote machines (only Linux allowed):
+Files in `codeDir` will be uploaded to remote machines automatically. You can run below command on Windows, Linux, or macOS to spawn trials on remote Linux machines:
```bash
nnictl create --config examples/trials/mnist-annotation/config_remote.yml
```
-
-You can also use public/private key pairs instead of username/password for authentication. For advanced usages, please refer to [Experiment Config Reference](../Tutorial/ExperimentConfig.md).
-
-## Version check
-
-NNI support version check feature in since version 0.6, [reference](PaiMode.md).
\ No newline at end of file
diff --git a/docs/en_US/TrainingService/SupportTrainingService.md b/docs/en_US/TrainingService/SupportTrainingService.md
index dfb0df3fe8..ca2b9283fc 100644
--- a/docs/en_US/TrainingService/SupportTrainingService.md
+++ b/docs/en_US/TrainingService/SupportTrainingService.md
@@ -4,10 +4,11 @@ NNI TrainingService provides the training platform for running NNI trial jobs. N
NNI not only provides few built-in training service options, but also provides a method for customers to build their own training service easily.
## Built-in TrainingService
+
|TrainingService|Brief Introduction|
|---|---|
|[__Local__](./LocalMode.md)|NNI supports running an experiment on local machine, called local mode. Local mode means that NNI will run the trial jobs and nniManager process in same machine, and support gpu schedule function for trial jobs.|
-|[__Remote__](./RemoteMachineMode.md)|NNI supports running an experiment on multiple machines through SSH channel, called remote mode. NNI assumes that you have access to those machines, and already setup the environment for running deep learning training code. NNI will submit the trial jobs in remote machine, and schedule suitable machine with enouth gpu resource if specified.|
+|[__Remote__](./RemoteMachineMode.md)|NNI supports running an experiment on multiple machines through SSH channel, called remote mode. NNI assumes that you have access to those machines, and already setup the environment for running deep learning training code. NNI will submit the trial jobs in remote machine, and schedule suitable machine with enough gpu resource if specified.|
|[__Pai__](./PaiMode.md)|NNI supports running an experiment on [OpenPAI](https://github.com/Microsoft/pai) (aka pai), called pai mode. Before starting to use NNI pai mode, you should have an account to access an [OpenPAI](https://github.com/Microsoft/pai) cluster. See [here](https://github.com/Microsoft/pai#how-to-deploy) if you don't have any OpenPAI account and want to deploy an OpenPAI cluster. In pai mode, your trial program will run in pai's container created by Docker.|
|[__Kubeflow__](./KubeflowMode.md)|NNI supports running experiment on [Kubeflow](https://github.com/kubeflow/kubeflow), called kubeflow mode. Before starting to use NNI kubeflow mode, you should have a Kubernetes cluster, either on-premises or [Azure Kubernetes Service(AKS)](https://azure.microsoft.com/en-us/services/kubernetes-service/), a Ubuntu machine on which [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) is setup to connect to your Kubernetes cluster. If you are not familiar with Kubernetes, [here](https://kubernetes.io/docs/tutorials/kubernetes-basics/) is a good start. In kubeflow mode, your trial program will run as Kubeflow job in Kubernetes cluster.|
|[__FrameworkController__](./FrameworkControllerMode.md)|NNI supports running experiment using [FrameworkController](https://github.com/Microsoft/frameworkcontroller), called frameworkcontroller mode. FrameworkController is built to orchestrate all kinds of applications on Kubernetes, you don't need to install Kubeflow for specific deep learning framework like tf-operator or pytorch-operator. Now you can use FrameworkController as the training service to run NNI experiment.|
@@ -16,7 +17,8 @@ NNI not only provides few built-in training service options, but also provides a
TrainingService is designed to be easily implemented, we define an abstract class TrainingService as the parent class of all kinds of TrainingService, users just need to inherit the parent class and complete their own child class if they want to implement customized TrainingService.
The abstract function in TrainingService is shown below:
-```
+
+```javascript
abstract class TrainingService {
public abstract listTrialJobs(): Promise;
public abstract getTrialJob(trialJobId: string): Promise;
@@ -32,5 +34,6 @@ abstract class TrainingService {
public abstract run(): Promise;
}
```
+
The parent class of TrainingService has a few abstract functions, users need to inherit the parent class and implement all of these abstract functions.
For more information about how to write your own TrainingService, please [refer](https://github.com/microsoft/nni/blob/master/docs/en_US/TrainingService/HowToImplementTrainingService.md).
diff --git a/docs/en_US/TrialExample/EfficientNet.md b/docs/en_US/TrialExample/EfficientNet.md
new file mode 100644
index 0000000000..634a1a9593
--- /dev/null
+++ b/docs/en_US/TrialExample/EfficientNet.md
@@ -0,0 +1,21 @@
+# EfficientNet
+
+[EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)
+
+Use Grid search to find the best combination of alpha, beta and gamma for EfficientNet-B1, as discussed in Section 3.3 in paper. Search space, tuner, configuration examples are provided here.
+
+## Instructions
+
+[Example code](https://github.com/microsoft/nni/tree/master/examples/trials/efficientnet)
+
+1. Set your working directory here in the example code directory.
+2. Run `git clone https://github.com/ultmaster/EfficientNet-PyTorch` to clone this modified version of [EfficientNet-PyTorch](https://github.com/lukemelas/EfficientNet-PyTorch). The modifications were done to adhere to the original [Tensorflow version](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet) as close as possible (including EMA, label smoothing and etc.); also added are the part which gets parameters from tuner and reports intermediate/final results. Clone it into `EfficientNet-PyTorch`; the files like `main.py`, `train_imagenet.sh` will appear inside, as specified in the configuration files.
+3. Run `nnictl create --config config_local.yml` (use `config_pai.yml` for OpenPAI) to find the best EfficientNet-B1. Adjust the training service (PAI/local/remote), batch size in the config files according to the environment.
+
+For training on ImageNet, read `EfficientNet-PyTorch/train_imagenet.sh`. Download ImageNet beforehand and extract it adhering to [PyTorch format](https://pytorch.org/docs/stable/torchvision/datasets.html#imagenet) and then replace `/mnt/data/imagenet` in with the location of the ImageNet storage. This file should also be a good example to follow for mounting ImageNet into the container on OpenPAI.
+
+## Results
+
+The follow image is a screenshot, demonstrating the relationship between acc@1 and alpha, beta, gamma.
+
+
diff --git a/docs/en_US/Tuner/HyperbandAdvisor.md b/docs/en_US/Tuner/HyperbandAdvisor.md
index a367b06b13..b7787af199 100644
--- a/docs/en_US/Tuner/HyperbandAdvisor.md
+++ b/docs/en_US/Tuner/HyperbandAdvisor.md
@@ -5,7 +5,7 @@ Hyperband on NNI
[Hyperband][1] is a popular automl algorithm. The basic idea of Hyperband is that it creates several buckets, each bucket has `n` randomly generated hyperparameter configurations, each configuration uses `r` resource (e.g., epoch number, batch number). After the `n` configurations is finished, it chooses top `n/eta` configurations and runs them using increased `r*eta` resource. At last, it chooses the best configuration it has found so far.
## 2. Implementation with fully parallelism
-Frist, this is an example of how to write an automl algorithm based on MsgDispatcherBase, rather than Tuner and Assessor. Hyperband is implemented in this way because it integrates the functions of both Tuner and Assessor, thus, we call it advisor.
+First, this is an example of how to write an automl algorithm based on MsgDispatcherBase, rather than Tuner and Assessor. Hyperband is implemented in this way because it integrates the functions of both Tuner and Assessor, thus, we call it advisor.
Second, this implementation fully leverages Hyperband's internal parallelism. More specifically, the next bucket is not started strictly after the current bucket, instead, it starts when there is available resource.
diff --git a/docs/en_US/Tutorial/FAQ.md b/docs/en_US/Tutorial/FAQ.md
index 6b749f925b..16299e5b20 100644
--- a/docs/en_US/Tutorial/FAQ.md
+++ b/docs/en_US/Tutorial/FAQ.md
@@ -47,5 +47,9 @@ Probably it's a problem with your network config. Here is a checklist.
### NNI on Windows problems
Please refer to [NNI on Windows](NniOnWindows.md)
+
+### More FAQ issues
+[NNI Issues with FAQ labels](https://github.com/microsoft/nni/labels/FAQ)
+
### Help us improve
Please inquiry the problem in https://github.com/Microsoft/nni/issues to see whether there are other people already reported the problem, create a new one if there are no existing issues been created.
diff --git a/docs/en_US/Tutorial/Installation.md b/docs/en_US/Tutorial/Installation.md
index e7711bd2d0..f324366bd8 100644
--- a/docs/en_US/Tutorial/Installation.md
+++ b/docs/en_US/Tutorial/Installation.md
@@ -1,20 +1,22 @@
# Installation of NNI
-Currently we support installation on Linux, Mac and Windows.
+Currently we support installation on Linux, macOS and Windows.
-## **Installation on Linux & Mac**
+## Install on Linux or macOS
-* __Install NNI through pip__
+* Install NNI through pip
- Prerequisite: `python >= 3.5`
+ Prerequisite: `python 64-bit >= 3.5`
```bash
python3 -m pip install --upgrade nni
```
-* __Install NNI through source code__
+* Install NNI through source code
- Prerequisite: `python >=3.5`, `git`, `wget`
+ If you are interested on special or latest code version, you can install NNI through source code.
+
+ Prerequisites: `python 64-bit >=3.5`, `git`, `wget`
```bash
git clone -b v0.8 https://github.com/Microsoft/nni.git
@@ -22,25 +24,27 @@ Currently we support installation on Linux, Mac and Windows.
./install.sh
```
-* __Install NNI in docker image__
+* Use NNI in a docker image
You can also install NNI in a docker image. Please follow the instructions [here](https://github.com/Microsoft/nni/tree/master/deployment/docker/README.md) to build NNI docker image. The NNI docker image can also be retrieved from Docker Hub through the command `docker pull msranni/nni:latest`.
-## **Installation on Windows**
+## Install on Windows
- Anaconda or Miniconda is highly recommended.
+ Anaconda or Miniconda is highly recommended to manage multiple Python environments.
-* __Install NNI through pip__
+* Install NNI through pip
- Prerequisite: `python(64-bit) >= 3.5`
+ Prerequisites: `python 64-bit >= 3.5`
```bash
python -m pip install --upgrade nni
```
-* __Install NNI through source code__
+* Install NNI through source code
+
+ If you are interested on special or latest code version, you can install NNI through source code.
- Prerequisite: `python >=3.5`, `git`, `PowerShell`.
+ Prerequisites: `python 64-bit >=3.5`, `git`, `PowerShell`.
```bash
git clone -b v0.8 https://github.com/Microsoft/nni.git
@@ -48,43 +52,103 @@ Currently we support installation on Linux, Mac and Windows.
powershell -ExecutionPolicy Bypass -file install.ps1
```
-## **System requirements**
-
-Below are the minimum system requirements for NNI on Linux. Due to potential programming changes, the minimum system requirements for NNI may change over time.
-
-||Minimum Requirements|Recommended Specifications|
-|---|---|---|
-|**Operating System**|Ubuntu 16.04 or above|Ubuntu 16.04 or above|
-|**CPU**|Intel® Core™ i3 or AMD Phenom™ X3 8650|Intel® Core™ i5 or AMD Phenom™ II X3 or better|
-|**GPU**|NVIDIA® GeForce® GTX 460|NVIDIA® GeForce® GTX 660 or better|
-|**Memory**|4 GB RAM|6 GB RAM|
-|**Storage**|30 GB available hare drive space|
-|**Internet**|Boardband internet connection|
-|**Resolution**|1024 x 768 minimum display resolution|
-
-Below are the minimum system requirements for NNI on macOS. Due to potential programming changes, the minimum system requirements for NNI may change over time.
-
-||Minimum Requirements|Recommended Specifications|
-|---|---|---|
-|**Operating System**|macOS 10.14.1 (latest version)|macOS 10.14.1 (latest version)|
-|**CPU**|Intel® Core™ i5-760 or better|Intel® Core™ i7-4770 or better|
-|**GPU**|NVIDIA® GeForce® GT 750M or AMD Radeon™ R9 M290 or better|AMD Radeon™ R9 M395X or better|
-|**Memory**|4 GB RAM|8 GB RAM|
-|**Storage**|70GB available space 7200 RPM HDD|70GB available space SSD|
-|**Internet**|Boardband internet connection|
-|**Resolution**|1024 x 768 minimum display resolution|
-
-Below are the minimum system requirements for NNI on Windows, Windows 10.1809 is well tested and recommend. Due to potential programming changes, the minimum system requirements for NNI may change over time.
-
-||Minimum Requirements|Recommended Specifications|
-|---|---|---|
-|**Operating System**|Windows 10|Windows 10|
-|**CPU**|Intel® Core™ i3 or AMD Phenom™ X3 8650|Intel® Core™ i5 or AMD Phenom™ II X3 or better|
-|**GPU**|NVIDIA® GeForce® GTX 460|NVIDIA® GeForce® GTX 660 or better|
-|**Memory**|4 GB RAM|6 GB RAM|
-|**Storage**|30 GB available hare drive space|
-|**Internet**|Boardband internet connection|
-|**Resolution**|1024 x 768 minimum display resolution|
+## Verify installation
+
+The following example is built on TensorFlow 1.x. Make sure **TensorFlow 1.x is used** when running it.
+
+* Download the examples via clone the source code.
+
+ ```bash
+ git clone -b v1.3 https://github.com/Microsoft/nni.git
+ ```
+
+* Run the MNIST example.
+
+ Linux or macOS
+
+ ```bash
+ nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
+ ```
+
+ Windows
+
+ ```bash
+ nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
+ ```
+
+* Wait for the message `INFO: Successfully started experiment!` in the command line. This message indicates that your experiment has been successfully started. You can explore the experiment using the `Web UI url`.
+
+```text
+INFO: Starting restful server...
+INFO: Successfully started Restful server!
+INFO: Setting local config...
+INFO: Successfully set local config!
+INFO: Starting experiment...
+INFO: Successfully started experiment!
+-----------------------------------------------------------------------
+The experiment id is egchD4qy
+The Web UI urls are: http://223.255.255.1:8080 http://127.0.0.1:8080
+-----------------------------------------------------------------------
+
+You can use these commands to get more information about the experiment
+-----------------------------------------------------------------------
+ commands description
+1. nnictl experiment show show the information of experiments
+2. nnictl trial ls list all of trial jobs
+3. nnictl top monitor the status of running experiments
+4. nnictl log stderr show stderr log content
+5. nnictl log stdout show stdout log content
+6. nnictl stop stop an experiment
+7. nnictl trial kill kill a trial job by id
+8. nnictl --help get help information about nnictl
+-----------------------------------------------------------------------
+```
+
+* Open the `Web UI url` in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. [Here](../Tutorial/WebUI.md) are more Web UI pages.
+
+
+
+
+
+## System requirements
+
+Due to potential programming changes, the minimum system requirements of NNI may change over time.
+
+### Linux
+
+| | Recommended | Minimum |
+| -------------------- | ---------------------------------------------- | -------------------------------------- |
+| **Operating System** | Ubuntu 16.04 or above |
+| **CPU** | Intel® Core™ i5 or AMD Phenom™ II X3 or better | Intel® Core™ i3 or AMD Phenom™ X3 8650 |
+| **GPU** | NVIDIA® GeForce® GTX 660 or better | NVIDIA® GeForce® GTX 460 |
+| **Memory** | 6 GB RAM | 4 GB RAM |
+| **Storage** | 30 GB available hare drive space |
+| **Internet** | Boardband internet connection |
+| **Resolution** | 1024 x 768 minimum display resolution |
+
+### macOS
+
+| | Recommended | Minimum |
+| -------------------- | ------------------------------------- | --------------------------------------------------------- |
+| **Operating System** | macOS 10.14.1 or above |
+| **CPU** | Intel® Core™ i7-4770 or better | Intel® Core™ i5-760 or better |
+| **GPU** | AMD Radeon™ R9 M395X or better | NVIDIA® GeForce® GT 750M or AMD Radeon™ R9 M290 or better |
+| **Memory** | 8 GB RAM | 4 GB RAM |
+| **Storage** | 70GB available space SSD | 70GB available space 7200 RPM HDD |
+| **Internet** | Boardband internet connection |
+| **Resolution** | 1024 x 768 minimum display resolution |
+
+### Windows
+
+| | Recommended | Minimum |
+| -------------------- | ---------------------------------------------- | -------------------------------------- |
+| **Operating System** | Windows 10 1809 or above |
+| **CPU** | Intel® Core™ i5 or AMD Phenom™ II X3 or better | Intel® Core™ i3 or AMD Phenom™ X3 8650 |
+| **GPU** | NVIDIA® GeForce® GTX 660 or better | NVIDIA® GeForce® GTX 460 |
+| **Memory** | 6 GB RAM | 4 GB RAM |
+| **Storage** | 30 GB available hare drive space |
+| **Internet** | Boardband internet connection |
+| **Resolution** | 1024 x 768 minimum display resolution |
## Further reading
diff --git a/docs/en_US/Tutorial/Nnictl.md b/docs/en_US/Tutorial/Nnictl.md
index b58d4c4a37..b0a8b33513 100644
--- a/docs/en_US/Tutorial/Nnictl.md
+++ b/docs/en_US/Tutorial/Nnictl.md
@@ -49,7 +49,7 @@ nnictl support commands:
|--config, -c| True| |YAML configure file of the experiment|
|--port, -p|False| |the port of restful server|
|--debug, -d|False||set debug mode|
- |--watch, -w|False||set watch mode|
+ |--foreground, -f|False||set foreground mode, print log content to terminal|
* Examples
@@ -98,7 +98,7 @@ Debug mode will disable version check function in Trialkeeper.
|id| True| |The id of the experiment you want to resume|
|--port, -p| False| |Rest port of the experiment you want to resume|
|--debug, -d|False||set debug mode|
- |--watch, -w|False||set watch mode|
+ |--foreground, -f|False||set foreground mode, print log content to terminal|
* Example
diff --git a/docs/en_US/Tutorial/QuickStart.md b/docs/en_US/Tutorial/QuickStart.md
index f48550cbfc..3e16a46605 100644
--- a/docs/en_US/Tutorial/QuickStart.md
+++ b/docs/en_US/Tutorial/QuickStart.md
@@ -2,14 +2,15 @@
## Installation
-We support Linux MacOS and Windows in current stage, Ubuntu 16.04 or higher, MacOS 10.14.1 and Windows 10.1809 are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`.
-#### Linux and MacOS
+We support Linux macOS and Windows in current stage, Ubuntu 16.04 or higher, macOS 10.14.1 and Windows 10.1809 are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`.
+
+**Linux and macOS**
```bash
python3 -m pip install --upgrade nni
```
-#### Windows
+**Windows**
```bash
python -m pip install --upgrade nni
@@ -17,7 +18,7 @@ We support Linux MacOS and Windows in current stage, Ubuntu 16.04 or higher, Mac
Note:
-* For Linux and MacOS `--user` can be added if you want to install NNI in your home directory, which does not require any special privileges.
+* For Linux and macOS `--user` can be added if you want to install NNI in your home directory, which does not require any special privileges.
* If there is any error like `Segmentation fault`, please refer to [FAQ](FAQ.md)
* For the `system requirements` of NNI, please refer to [Install NNI](Installation.md)
@@ -53,7 +54,7 @@ The above code can only try one set of parameters at a time, if we want to tune
NNI is born for helping user do the tuning jobs, the NNI working process is presented below:
-```
+```text
input: search space, trial code, config file
output: one optimal hyperparameter configuration
@@ -68,7 +69,7 @@ output: one optimal hyperparameter configuration
If you want to use NNI to automatically train your model and find the optimal hyper-parameters, you need to do three changes base on your code:
-**Three things required to do when using NNI**
+**Three steps to start an experiment**
**Step 1**: Give a `Search Space` file in JSON, includes the `name` and the `distribution` (discrete valued or continuous valued) of all the hyperparameters you need to search.
@@ -138,22 +139,25 @@ Note, **for Windows, you need to change trial command `python3` to `python`**
All the codes above are already prepared and stored in [examples/trials/mnist-tfv1/](https://github.com/Microsoft/nni/tree/master/examples/trials/mnist-tfv1).
-#### Linux and MacOS
+**Linux and macOS**
+
Run the **config.yml** file from your command line to start MNIST experiment.
```bash
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
```
-#### Windows
+
+**Windows**
+
Run the **config_windows.yml** file from your command line to start MNIST experiment.
-**Note**, if you're using NNI on Windows, it needs to change `python3` to `python` in the config.yml file, or use the config_windows.yml file to start the experiment.
+Note, if you're using NNI on Windows, it needs to change `python3` to `python` in the config.yml file, or use the config_windows.yml file to start the experiment.
```bash
nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
```
-Note, **nnictl** is a command line tool, which can be used to control experiments, such as start/stop/resume an experiment, start/stop NNIBoard, etc. Click [here](Nnictl.md) for more usage of `nnictl`
+Note, `nnictl` is a command line tool, which can be used to control experiments, such as start/stop/resume an experiment, start/stop NNIBoard, etc. Click [here](Nnictl.md) for more usage of `nnictl`
Wait for the message `INFO: Successfully started experiment!` in the command line. This message indicates that your experiment has been successfully started. And this is what we expected to get:
@@ -195,7 +199,7 @@ The Web UI urls are: [Your IP]:8080
Open the `Web UI url`(In this information is: `[Your IP]:8080`) in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. If you can not open the WebUI link in your terminal, you can refer to [FAQ](FAQ.md).
-#### View summary page
+### View summary page
Click the tab "Overview".
@@ -207,7 +211,7 @@ Top 10 trials will be listed in the Overview page, you can browse all the trials
-#### View trials detail page
+### View trials detail page
Click the tab "Default Metric" to see the point graph of all trials. Hover to see its specific default metric and search space message.
diff --git a/docs/en_US/examples.rst b/docs/en_US/examples.rst
index d260e648ce..77ea9733b4 100644
--- a/docs/en_US/examples.rst
+++ b/docs/en_US/examples.rst
@@ -12,3 +12,4 @@ Examples
GBDT<./TrialExample/GbdtExample>
RocksDB <./TrialExample/RocksdbExamples>
KDExample <./TrialExample/KDExample>
+ EfficientNet <./TrialExample/EfficientNet>
diff --git a/examples/trials/efficientnet/assets/search_result.png b/docs/img/efficientnet_search_result.png
similarity index 100%
rename from examples/trials/efficientnet/assets/search_result.png
rename to docs/img/efficientnet_search_result.png
diff --git a/docs/img/pai_data_management_page.jpg b/docs/img/pai_data_management_page.jpg
new file mode 100644
index 0000000000..852c5fd3d5
Binary files /dev/null and b/docs/img/pai_data_management_page.jpg differ
diff --git a/docs/img/pai_job_submission_page.jpg b/docs/img/pai_job_submission_page.jpg
new file mode 100644
index 0000000000..f49a1c267e
Binary files /dev/null and b/docs/img/pai_job_submission_page.jpg differ
diff --git a/docs/img/pai_token_button.jpg b/docs/img/pai_token_button.jpg
new file mode 100644
index 0000000000..86e911d0e8
Binary files /dev/null and b/docs/img/pai_token_button.jpg differ
diff --git a/docs/img/pai_token_profile.jpg b/docs/img/pai_token_profile.jpg
new file mode 100644
index 0000000000..52d68bb7b5
Binary files /dev/null and b/docs/img/pai_token_profile.jpg differ
diff --git a/docs/requirements.txt b/docs/requirements.txt
index 35082a1136..54e21d307c 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -10,4 +10,4 @@ numpy
scipy
coverage
scikit-learn==0.20
-torch==1.3.1
\ No newline at end of file
+https://download.pytorch.org/whl/cpu/torch-1.3.1%2Bcpu-cp37-cp37m-linux_x86_64.whl
diff --git a/docs/zh_CN/CommunitySharings/NNI_AutoFeatureEng.md b/docs/zh_CN/CommunitySharings/NNI_AutoFeatureEng.md
new file mode 100644
index 0000000000..ec932fc3ec
--- /dev/null
+++ b/docs/zh_CN/CommunitySharings/NNI_AutoFeatureEng.md
@@ -0,0 +1,88 @@
+# 来自知乎的评论： - 作者 Garvin Li
+
+本文由 NNI 用户在知乎论坛上发表。 在这篇文章中，Garvin 分享了在使用 NNI 进行自动特征工程方面的体验。 我们认为本文对于有兴趣使用 NNI 进行特征工程的用户非常有用。 经作者许可，将原始文章摘编如下。
+
+**原文**: [如何看待微软最新发布的AutoML平台NNI？作者 Garvin Li](https://www.zhihu.com/question/297982959/answer/964961829?utm_source=wechat_session&utm_medium=social&utm_oi=28812108627968&from=singlemessage&isappinstalled=0)
+
+## 01 AutoML概述
+
+作者认为 AutoML 不光是调参，应该包含自动特征工程。AutoML 是一个系统化的体系，包括：自动特征工程（AutoFeatureEng）、自动调参（AutoTuning）、自动神经网络探索（NAS）等。
+
+## 02 NNI 概述
+
+NNI（(Neural Network Intelligence）是一个微软的开源 AutoML 工具包，通过自动而有效的方法来帮助用户设计并调优机器学习模型，神经网络架构，或复杂系统的参数。
+
+链接：[ https://github.com/Microsoft/nni](https://github.com/Microsoft/nni)
+
+我目前只学习了自动特征工程这一个模块，总体看微软的工具都有一个比较大的特点，技术可能不一定多新颖，但是设计都非常赞。 NNI 的 AutoFeatureENG 基本包含了用户对于 AutoFeatureENG 的一切幻想。在微软做 PD 应该挺幸福吧，底层的这些个框架的设计都极为合理。
+
+## 03 细说NNI - AutoFeatureENG
+> 本文使用了此项目： [https://github.com/SpongebBob/tabular_automl_NNI](https://github.com/SpongebBob/tabular_automl_NNI)。
+
+新用户可以使用 NNI 轻松高效地进行 AutoFeatureENG。 使用是非常简单的，安装下文件中的 require，然后 pip install NNI。
+
+ NNI把 AutoFeatureENG 拆分成 exploration 和 selection 两个模块。 exploration 主要是特征衍生和交叉，selection 讲的是如何做特征筛选。
+
+## 04 特征 Exploration
+
+对于功能派生，NNI 提供了许多可自动生成新功能的操作，[列表](https://github.com/SpongebBob/tabular_automl_NNI/blob/master/AutoFEOp.md)如下：
+
+**count**：传统的统计，统计一些数据的出现频率
+
+**target**：特征和目标列的一些映射特征
+
+**embedding**：把特征看成句子，用 *word2vector* 的方式制作向量
+
+**crosscount**：特征间除法，有点类似CTR
+
+**aggregete**：特征的 min/max/var/mean
+
+**nunique**：统计唯一特征的数量。
+
+**histsta**：特征存储桶的统计信息，如直方图统计信息。
+
+具体特征怎么交叉，哪一列和哪一列交叉，每一列特征用什么方式衍生呢？可以通过 **search_space. json** 这个文件控制。
+
+
+
+图片展示了定义搜索空间的过程。 NNI 为 1 阶运算提供计数编码，并为 2 阶运算提供聚合的统计（min max var mean median nunique）。
+
+例如，希望以下列方式搜索列名称 {"C1"、"..."，"C26"} 上的频率编码（valuecount）功能的功能：
+
+
+
+可以在列 {"C1",...,"C26"} x {"C1",...,"C26"} 上定义交叉频率编码（交叉维度的值计数）方法：
+
+
+
+Exploration 的目的就是长生出新的特征。 在代码里可以用 **get_next_parameter** 的方式获取 tuning 的参数：
+> RECEIVED_PARAMS = nni.get_next_parameter()
+
+## 05 特征 Selection
+
+为了避免特征泛滥的情况，避免过拟合，一定要有 Selection 的机制挑选特征。 在 NNI-AutoFeatureENG 的 Selection 中，主要使用了微软开发的梯度提升框架 LightGBM（Light Gradient Boosting Machine）。
+
+
+
+了解 xgboost 或者 GBDT 算法同学应该知道，这种树形结构的算法是很容易计算出每个特征对于结果的影响的。 所以使用 lightGBM 可以天然的进行特征筛选。
+
+弊病就是，如果下游是个 *LR*（逻辑回归）这种线性算法，筛选出来的特征是否具备普适性。
+
+
+
+## 06 总结
+
+NNI 的 AutoFeature 模块是给整个行业制定了一个教科书般的标准，告诉大家这个东西要怎么做，有哪些模块，使用起来非常方便。 但是如果只是基于这样简单的模式，不一定能达到很好的效果。
+
+## 对 NNI 的建议
+
+我觉得在Exploration方面可以引用一些 DNN（如：xDeepFM） 的特征组合方式，提取更高维度的特征。
+
+在 Selection 方面可以有更多的智能化方案，比如可以基于下游的算法自动选择 Selection 机制。
+
+总之 NNI 在设计曾给了我一些启发，还是一个挺好的开源项目，推荐给大家~ 建议 AI 研究人员使用它来加速研究。
+
+大家用的时候如果是 Mac 电脑可能会遇到 gcc 的问题，因为开源项目自带的脚本是基于 gcc7 编译的， 可以用下面的方法绕过去：
+
+# brew install libomp
+
diff --git a/docs/zh_CN/CommunitySharings/community_sharings.rst b/docs/zh_CN/CommunitySharings/community_sharings.rst
index 828ff48b4d..e549dba143 100644
--- a/docs/zh_CN/CommunitySharings/community_sharings.rst
+++ b/docs/zh_CN/CommunitySharings/community_sharings.rst
@@ -13,3 +13,4 @@
超参调优算法的对比
TPE 的并行优化
使用 NNI 自动调优系统
+ 来自知乎的评论：作者 Garvin Li
diff --git a/docs/zh_CN/Compressor/Pruner.md b/docs/zh_CN/Compressor/Pruner.md
index 0e7963c9d8..d564109149 100644
--- a/docs/zh_CN/Compressor/Pruner.md
+++ b/docs/zh_CN/Compressor/Pruner.md
@@ -335,5 +335,3 @@ pruner.compress()
- **sparsity:** 卷积过滤器要修剪的百分比。
- **op_types:** 在 ActivationMeanRankFilterPruner 中仅支持 Conv2d。
-
-***
\ No newline at end of file
diff --git a/docs/zh_CN/Compressor/Quantizer.md b/docs/zh_CN/Compressor/Quantizer.md
index d2a571f874..3d63a3b3b9 100644
--- a/docs/zh_CN/Compressor/Quantizer.md
+++ b/docs/zh_CN/Compressor/Quantizer.md
@@ -5,10 +5,9 @@ NNI Compressor 中的 Quantizer
Naive Quantizer 将 Quantizer 权重默认设置为 8 位，可用它来测试量化算法。
### 用法
-tensorflow ```python nni.compression.tensorflow.NaiveQuantizer(model_graph).compress()
-```
pytorch
-```python nni.compression.torch.NaiveQuantizer(model).compress()
+```python
+model = nni.compression.torch.NaiveQuantizer(model).compress()
```
***
@@ -45,7 +44,7 @@ quantizer.compress()
查看示例进一步了解
#### QAT Quantizer 的用户配置
-压缩算法所需的常见配置可在[通用配置](./Overview.md#User-configuration-for-a-compression-algorithm)中找到。
+压缩算法所需的常见配置可在[通用配置](./Overview.md#压缩算法中的用户配置)中找到。
此算法所需的配置：
@@ -78,7 +77,7 @@ quantizer.compress()
查看示例进一步了解
#### DoReFa Quantizer 的用户配置
-压缩算法所需的常见配置可在[通用配置](./Overview.md#User-configuration-for-a-compression-algorithm)中找到。
+压缩算法所需的常见配置可在[通用配置](./Overview.md#压缩算法中的用户配置)中找到。
此算法所需的配置：
@@ -114,7 +113,7 @@ model = quantizer.compress()
可以查看示例 [examples/model_compress/BNN_quantizer_cifar10.py](https://github.com/microsoft/nni/tree/master/examples/model_compress/BNN_quantizer_cifar10.py) 了解更多信息。
#### BNN Quantizer 的用户配置
-压缩算法所需的常见配置可在[通用配置](./Overview.md#User-configuration-for-a-compression-algorithm)中找到。
+压缩算法所需的常见配置可在[通用配置](./Overview.md#压缩算法中的用户配置)中找到。
此算法所需的配置：
diff --git a/docs/zh_CN/NAS/CDARTS.md b/docs/zh_CN/NAS/CDARTS.md
new file mode 100644
index 0000000000..b4347127e7
--- /dev/null
+++ b/docs/zh_CN/NAS/CDARTS.md
@@ -0,0 +1,61 @@
+# CDARTS
+
+## 介绍
+
+CDARTS 在搜索和评估网络之间构建了循环反馈机制。 首先，搜索网络会生成初始结构用于评估，以便优化评估网络的权重。 然后，通过分类中通过的标签，以及评估网络中特征蒸馏的正则化来进一步优化搜索网络中的架构。 重复上述循环来优化搜索和评估网路，从而使结构得到训练，成为最终的评估网络。
+
+在 `CdartsTrainer` 的实现中，首先分别实例化了两个 Model 和 Mutator。 第一个 Model 被称为"搜索网络"，使用 `RegularizedDartsMutator` 来进行变化。它与 `DartsMutator` 稍有差别。 第二个 Model 是“评估网络”，它里用前面搜索网络的 Mutator 来创建了一个离散的 Mutator，来每次采样一条路径。 Trainer 会交替训练 Model 和 Mutator。 如果对 Trainer 和 Mutator 的实现感兴趣，可参考[这里](#reference)。
+
+## 重现结果
+
+这是基于 NNI 平台的 CDARTS，该平台目前支持 CIFAR10 搜索和重新训练。 同时也支持 ImageNet 的搜索和重新训练，并有相应的接口。 在 NNI 上重现的结果略低于论文，但远高于原始 DARTS。 这里展示了在 CIFAR10 上的三个独立实验的结果。
+
+| 运行 | 论文 | NNI |
+| -- |:-----:|:-----:|
+| 1 | 97.52 | 97.44 |
+| 2 | 97.53 | 97.48 |
+| 3 | 97.58 | 97.56 |
+
+
+## 示例
+
+[示例代码](https://github.com/microsoft/nni/tree/master/examples/nas/cdarts)
+
+```bash
+＃如果未克隆 NNI 代码。 如果代码已被克隆，请忽略此行并直接进入代码目录。
+git clone https://github.com/Microsoft/nni.git
+
+# 为分布式训练安装 apex
+git clone https://github.com/NVIDIA/apex
+cd apex
+python setup.py install --cpp_ext --cuda_ext
+
+# 搜索最好的架构
+cd examples/nas/cdarts
+bash run_search_cifar.sh
+
+# 训练最好的架构
+bash run_retrain_cifar.sh
+```
+
+## 参考
+
+### PyTorch
+
+```eval_rst
+.. autoclass:: nni.nas.pytorch.cdarts.CdartsTrainer
+ :members:
+
+ .. automethod:: __init__
+
+.. autoclass:: nni.nas.pytorch.cdarts.RegularizedDartsMutator
+ :members:
+
+.. autoclass:: nni.nas.pytorch.cdarts.DartsDiscreteMutator
+ :members:
+
+ .. automethod:: __init__
+
+.. autoclass:: nni.nas.pytorch.cdarts.RegularizedMutatorParallel
+ :members:
+```
diff --git a/docs/zh_CN/NAS/DARTS.md b/docs/zh_CN/NAS/DARTS.md
index 4f350efa9f..c092070dc4 100644
--- a/docs/zh_CN/NAS/DARTS.md
+++ b/docs/zh_CN/NAS/DARTS.md
@@ -1,4 +1,4 @@
-# NNI 中的 DARTS
+# DARTS
## 介绍
@@ -6,13 +6,45 @@
为了实现，作者在小批量中交替优化网络权重和架构权重。 还进一步探讨了使用二阶优化（unroll）来替代一阶，来提高性能的可能性。
-NNI 的实现基于[官方实现](https://github.com/quark0/darts)以及一个[第三方实现](https://github.com/khanrc/pt.darts)。 目前，在 CIFAR10 上从头训练的一阶和二阶优化均已实现。
+NNI 的实现基于[官方实现](https://github.com/quark0/darts)以及一个[第三方实现](https://github.com/khanrc/pt.darts)。 NNI 上的 DARTS 设计为可用于任何搜索空间。 与原始论文一样，为 CIFAR10 实现了 CNN 的搜索空间，来作为 DARTS 的实际示例。
## 重现结果
-为了重现本文的结果，我们做了一阶和二阶优化的实验。 由于时间限制，我们仅从第二阶段重新训练了*一次**最佳架构*。 我们的结果目前与论文的结果相当。 稍后会增加更多结果
+上述示例旨在重现本文中的结果，我们进行了一阶和二阶优化实验。 由于时间限制，我们仅从第二阶段重新训练了*一次**最佳架构*。 我们的结果目前与论文的结果相当。 稍后会增加更多结果
-| | 论文中 | 重现 |
-| ------------ | ------------- | ---- |
-| 一阶 (CIFAR10) | 3.00 +/- 0.14 | 2.78 |
-| 二阶（CIFAR10） | 2.76 +/- 0.09 | 2.89 |
+| | 论文中 | 重现 |
+| ----------- | ------------- | ---- |
+| 一阶（CIFAR10） | 3.00 +/- 0.14 | 2.78 |
+| 二阶（CIFAR10） | 2.76 +/- 0.09 | 2.89 |
+
+## 示例
+
+### CNN 搜索空间
+
+[示例代码](https://github.com/microsoft/nni/tree/master/examples/nas/darts)
+
+```bash
+＃如果未克隆 NNI 代码。 如果代码已被克隆，请忽略此行并直接进入代码目录。
+git clone https://github.com/Microsoft/nni.git
+
+# 搜索最好的架构
+cd examples/nas/darts
+python3 search.py
+
+# 训练最好的架构
+python3 retrain.py --arc-checkpoint ./checkpoints/epoch_49.json
+```
+
+## 参考
+
+### PyTorch
+
+```eval_rst
+.. autoclass:: nni.nas.pytorch.darts.DartsTrainer
+ :members:
+
+ .. automethod:: __init__
+
+.. autoclass:: nni.nas.pytorch.darts.DartsMutator
+ :members:
+```
diff --git a/docs/zh_CN/NAS/ENAS.md b/docs/zh_CN/NAS/ENAS.md
index c25b27bc9b..dcfa3ec060 100644
--- a/docs/zh_CN/NAS/ENAS.md
+++ b/docs/zh_CN/NAS/ENAS.md
@@ -1,7 +1,46 @@
-# NNI 中的 ENAS
+# ENAS
## 介绍
论文 [Efficient Neural Architecture Search via Parameter Sharing](https://arxiv.org/abs/1802.03268) 通过在子模型之间共享参数来加速 NAS 过程。 在 ENAS 中，Contoller 学习在大的计算图中搜索最有子图的方式来发现神经网络。 Controller 通过梯度策略训练，从而选择出能在验证集上有最大期望奖励的子图。 同时对与所选子图对应的模型进行训练，以最小化规范交叉熵损失。
-NNI 的实现基于 [Tensorflow 的官方实现](https://github.com/melodyguan/enas)，包括了 CIFAR10 上的 Macro/Micro 搜索空间。 NNI 中从头训练的代码还未完成，当前还没有重现结果。
+NNI 基于官方的 [Tensorflow](https://github.com/melodyguan/enas) 实现，包括通用的强化学习的 Controller，以及能交替训练目标网络和 Controller 的 Trainer。 根据论文，也对 CIFAR10 实现了 Macro 和 Micro 搜索空间来展示如何使用 Trainer。 NNI 中从头训练的代码还未完成，当前还没有重现结果。
+
+## 示例
+
+### CIFAR10 Macro/Micro 搜索空间
+
+[示例代码](https://github.com/microsoft/nni/tree/master/examples/nas/enas)
+
+```bash
+＃如果未克隆 NNI 代码。 如果代码已被克隆，请忽略此行并直接进入代码目录。
+git clone https://github.com/Microsoft/nni.git
+
+# 搜索最好的网络架构
+cd examples/nas/enas
+
+# 在 Macro 搜索空间中搜索
+python3 search.py --search-for macro
+
+# 在 Micro 搜索空间中搜索
+python3 search.py --search-for micro
+
+# 查看更多选项
+python3 search.py -h
+```
+
+## 参考
+
+### PyTorch
+
+```eval_rst
+.. autoclass:: nni.nas.pytorch.enas.EnasTrainer
+ :members:
+
+ .. automethod:: __init__
+
+.. autoclass:: nni.nas.pytorch.enas.EnasMutator
+ :members:
+
+ .. automethod:: __init__
+```
diff --git a/docs/zh_CN/NAS/NasInterface.md b/docs/zh_CN/NAS/NasInterface.md
index c7893036d9..dd3f98499f 100644
--- a/docs/zh_CN/NAS/NasInterface.md
+++ b/docs/zh_CN/NAS/NasInterface.md
@@ -98,7 +98,7 @@ trainer.export(file='./chosen_arch')
不同的 Trainer 可能有不同的输入参数，具体取决于其算法。 详细参数可参考具体的 [Trainer 代码](https://github.com/microsoft/nni/tree/master/src/sdk/pynni/nni/nas/pytorch)。 训练完成后，可通过 `trainer.export()` 导出找到的最好的模型。 无需通过 `nnictl` 来启动 NNI Experiment。
-[这里](Overview.md#supported-one-shot-nas-algorithms)是所有支持的 Trainer。 [这里](https://github.com/microsoft/nni/tree/master/examples/nas/simple/train.py)是使用 NNI NAS API 的简单示例。
+[这里](Overview.md#支持的-one-shot-nas-算法)是所有支持的 Trainer。 [这里](https://github.com/microsoft/nni/tree/master/examples/nas/simple/train.py)是使用 NNI NAS API 的简单示例。
### 经典分布式搜索
diff --git a/docs/zh_CN/NAS/Overview.md b/docs/zh_CN/NAS/Overview.md
index 1474a4d788..fc6c734c81 100644
--- a/docs/zh_CN/NAS/Overview.md
+++ b/docs/zh_CN/NAS/Overview.md
@@ -6,93 +6,33 @@
以此为动力，NNI 的目标是提供统一的体系结构，以加速NAS上的创新，并将最新的算法更快地应用于现实世界中的问题上。
-通过[统一的接口](./NasInterface.md)，有两种方式进行架构搜索。 [第一种](#supported-one-shot-nas-algorithms)称为 one-shot NAS，基于搜索空间构建了一个超级网络，并使用 one-shot 训练来生成性能良好的子模型。 [第二种](./NasInterface.md#classic-distributed-search)是传统的搜索方法，搜索空间中每个子模型作为独立的 Trial 运行，将性能结果发给 Tuner，由 Tuner 来生成新的子模型。
+通过[统一的接口](./NasInterface.md)，有两种方式进行架构搜索。 [一种](#supported-one-shot-nas-algorithms)称为 one-shot NAS，基于搜索空间构建了一个超级网络，并使用 one-shot 训练来生成性能良好的子模型。 [第二种](./NasInterface.md#经典分布式搜索)是传统的搜索方法，搜索空间中每个子模型作为独立的 Trial 运行，将性能结果发给 Tuner，由 Tuner 来生成新的子模型。
* [支持的 One-shot NAS 算法](#supported-one-shot-nas-algorithms)
-* [使用 NNI Experiment 的经典分布式 NAS](./NasInterface.md#classic-distributed-search)
+* [使用 NNI Experiment 的经典分布式 NAS](./NasInterface.md#经典分布式搜索)
* [NNI NAS 编程接口](./NasInterface.md)
## 支持的 One-shot NAS 算法
NNI 现在支持以下 NAS 算法，并且正在添加更多算法。 用户可以重现算法或在自己的数据集上使用它。 鼓励用户使用 [NNI API](#use-nni-api) 实现其它算法，以使更多人受益。
-| 名称 | 算法简介 |
-| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
-| [ENAS](#enas) | Efficient Neural Architecture Search via Parameter Sharing [参考论文](https://arxiv.org/abs/1802.03268) |
-| [DARTS](#darts) | DARTS: Differentiable Architecture Search [参考论文](https://arxiv.org/abs/1806.09055) |
-| [P-DARTS](#p-darts) | Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation [参考论文](https://arxiv.org/abs/1904.12760) |
+| 名称 | 算法简介 |
+| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [ENAS](ENAS.md) | [Efficient Neural Architecture Search via Parameter Sharing](https://arxiv.org/abs/1802.03268). 在 ENAS 中，Contoller 学习在大的计算图中搜索最有子图的方式来发现神经网络。 它通过在子模型间共享参数来实现加速和出色的性能指标。 |
+| [DARTS](DARTS.md) | [DARTS: Differentiable Architecture Search](https://arxiv.org/abs/1806.09055) 引入了一种在两级网络优化中使用的可微分算法。 |
+| [P-DARTS](PDARTS.md) | [Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation](https://arxiv.org/abs/1904.12760) 基于DARTS。 它引入了一种有效的算法，可在搜索过程中逐渐增加搜索的深度。 |
+| [SPOS](SPOS.md) | 论文 [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://arxiv.org/abs/1904.00420) 构造了一个采用统一的路径采样方法来训练简化的超网络，并使用进化算法来提高搜索神经网络结构的效率。 |
+| [CDARTS](CDARTS.md) | [Cyclic Differentiable Architecture Search](https://arxiv.org/abs/****) 在搜索和评估的网络见构建了循环反馈的机制。 通过引入的循环的可微分架构搜索框架将两个网络集成为一个架构。 |
-注意，这些算法**不需要 nnictl**，独立运行，仅支持 PyTorch。 将来的版本会支持 Tensorflow 2.0。
+One-shot 算法**不需要 nnictl，可单独运行**。 只实现了 PyTorch 版本。 将来的版本会支持 Tensorflow 2.x。
-### 依赖项
+这是运行示例的一些常见依赖项。 PyTorch 需要高于 1.2 才能使用 `BoolTensor`.
* NNI 1.2+
* tensorboard
* PyTorch 1.2+
* git
-### ENAS
-
-[Efficient Neural Architecture Search via Parameter Sharing](https://arxiv.org/abs/1802.03268). 在 ENAS 中，Contoller 学习在大的计算图中搜索最有子图的方式来发现神经网络。 它通过在子模型间共享参数来实现加速和出色的性能指标。
-
-#### 用法
-
-NNI 中的 ENAS 还在开发中，当前仅支持在 CIFAR10 上 Macro/Micro 搜索空间的搜索阶段。 在 PTB 上从头开始训练及其搜索空间尚未完成。 [详细说明](ENAS.md)。
-
-```bash
-＃如果未克隆 NNI 代码。 如果代码已被克隆，请忽略此行并直接进入代码目录。
-git clone https://github.com/Microsoft/nni.git
-
-# 搜索最好的网络架构
-cd examples/nas/enas
-
-# 在 Macro 搜索空间中搜索
-python3 search.py --search-for macro
-
-# 在 Micro 搜索空间中搜索
-python3 search.py --search-for micro
-
-# 查看更多选项
-python3 search.py -h
-```
-
-### DARTS
-
-[DARTS: Differentiable Architecture Search](https://arxiv.org/abs/1806.09055) 在算法上的主要贡献是，引入了一种在两级网络优化中使用的可微分算法。 [详细说明](DARTS.md)。
-
-#### 用法
-
-```bash
-＃如果未克隆 NNI 代码。 如果代码已被克隆，请忽略此行并直接进入代码目录。
-git clone https://github.com/Microsoft/nni.git
-
-# 搜索最好的架构
-cd examples/nas/darts
-python3 search.py
-
-# 训练最好的架构
-python3 retrain.py --arc-checkpoint ./checkpoints/epoch_49.json
-```
-
-### P-DARTS
-
-[Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation](https://arxiv.org/abs/1904.12760) 基于 [DARTS](#DARTS)。 它在算法上的主要贡献是引入了一种有效的算法，可在搜索过程中逐渐增加搜索的深度。
-
-#### 用法
-
-```bash
-＃如果未克隆 NNI 代码。 如果代码已被克隆，请忽略此行并直接进入代码目录。
-git clone https://github.com/Microsoft/nni.git
-
-# 搜索最好的架构
-cd examples/nas/pdarts
-python3 search.py
-
-# 训练最好的架构，过程与 darts 相同。
-cd ../darts
-python3 retrain.py --arc-checkpoint ../pdarts/checkpoints/epoch_2.json
-```
-
## 使用 NNI API
注意，我们正在尝试通过统一的编程接口来支持各种 NAS 算法，当前处于试验阶段。 这意味着当前编程接口将来会有变化。
@@ -104,7 +44,7 @@ python3 retrain.py --arc-checkpoint ../pdarts/checkpoints/epoch_2.json
1. 在设计神经网络时，可能在层、子模型或连接上有多种选择，并且无法确定是其中一种或某些的组合的结果最好。 因此，需要简单的方法来表达候选的层或子模型。
2. 在神经网络上应用 NAS 时，需要统一的方式来表达架构的搜索空间，这样不必为不同的搜索算法来更改代码。
-NNI 提出的 API 在[这里](https://github.com/microsoft/nni/tree/master/src/sdk/pynni/nni/nas/pytorch)。 [这里](https://github.com/microsoft/nni/tree/master/examples/nas/darts)包含了基于此 API 的 NAS 实现示例。
+NNI 提出的 API 在[这里](https://github.com/microsoft/nni/tree/master/src/sdk/pynni/nni/nas/pytorch)。 [这里](https://github.com/microsoft/nni/tree/master/examples/nas/naive)包含了基于此 API 的 NAS 实现示例。
## **参考和反馈**
* 在 GitHub 中[提交此功能的 Bug](https://github.com/microsoft/nni/issues/new?template=bug-report.md)；
diff --git a/docs/zh_CN/Release.md b/docs/zh_CN/Release.md
index 8a65ee765e..eba498b2b6 100644
--- a/docs/zh_CN/Release.md
+++ b/docs/zh_CN/Release.md
@@ -1,5 +1,44 @@
# 更改日志
+## 发布 1.3 - 12/30/2019
+
+### 主要功能
+
+#### 支持神经网络架构搜索算法
+
+* [单路径一次性](https://github.com/microsoft/nni/tree/v1.3/examples/nas/spos/)算法和示例
+
+#### 模型压缩算法支持
+
+* [知识蒸馏](https://github.com/microsoft/nni/blob/v1.3/docs/zh_CN/TrialExample/KDExample.md)算法和使用示例
+* Pruners
+ * [L2Filter Pruner](https://github.com/microsoft/nni/blob/master/docs/zh_CN/Compressor/Pruner.md#l2filter-pruner)
+ * [ActivationAPoZRankFilterPruner](https://github.com/microsoft/nni/blob/master/docs/zh_CN/Compressor/Pruner.md#activationapozrankfilterpruner)
+ * [ActivationMeanRankFilterPruner](https://github.com/microsoft/nni/blob/master/docs/zh_CN/Compressor/Pruner.md#activationmeanrankfilterpruner)
+* [BNN Quantizer](https://github.com/microsoft/nni/blob/v1.3/docs/zh_CN/Compressor/Quantizer.md#bnn-quantizer)
+
+#### 训练平台
+
+* OpenPAI 的 NFS 支持
+
+ 从 OpenPAI v0.11开始，HDFS 不再用作默认存储，可将 NFS、AzureBlob 或其他存储用作默认存储。 在本次版本中，NNI 扩展了对 OpenPAI 最近改动的支持，可与 OpenPAI v0.11 及后续版本的默认存储集成。
+
+* Kubeflow 更新适配
+
+ 适配 Kubeflow 0.7 对 tf-operator 的新支持。
+
+### 工程（代码和生成自动化）
+
+* 启用 [ESLint](https://eslint.org/) 静态代码分析。
+
+### 小改动和 Bug 修复
+
+* 正确识别内置 Tuner 和定制 Tuner
+* Dispatcher 基类的日志
+* 修复有时 Tuner、Assessor 的失败会终止 Experiment 的 Bug。
+* 修复本机作为远程计算机的[问题](https://github.com/microsoft/nni/issues/1852)
+* SMAC Tuner 中 Trial 配置的去重 [ticket](https://github.com/microsoft/nni/issues/1364)
+
## 发布 1.2 - 12/02/2019
### 主要功能
@@ -30,7 +69,7 @@
- 文档
- 改进了 NNI API 文档，增加了更多的 docstring。
-### 修复的 Bug
+### Bug 修复
- 修复当失败的 Trial 没有指标时，表格的排序问题。 -Issue #1773
- 页面切换时，保留选择的（最大、最小）状态。 -PR#1710
@@ -42,14 +81,14 @@
### 主要功能
* 新 Tuner: [PPO Tuner](https://github.com/microsoft/nni/blob/v1.1/docs/zh_CN/Tuner/PPOTuner.md)
-* [查看已停止的 Experiment](https://github.com/microsoft/nni/blob/v1.1/docs/zh_CN/Tutorial/Nnictl.md#view)
+* [查看已停止的 Experiment](https://github.com/microsoft/nni/blob/master/docs/zh_CN/Tutorial/Nnictl.md#view)
* Tuner 可使用专门的 GPU 资源（参考[教程](https://github.com/microsoft/nni/blob/v1.1/docs/zh_CN/Tutorial/ExperimentConfig.md)中的 `gpuIndices` 了解详情）
* 改进 WEB 界面
- Trial 详情页面可列出每个 Trial 的超参，以及开始结束时间（需要通过 "add column" 添加）
- 优化大型 Experiment 的显示性能
- 更多示例
- [EfficientNet PyTorch 示例](https://github.com/ultmaster/EfficientNet-PyTorch)
- - [Cifar10 NAS 示例](https://github.com/microsoft/nni/blob/v1.1/examples/trials/nas_cifar10/README_zh_CN.md)
+ - [Cifar10 NAS 示例](https://github.com/microsoft/nni/blob/v1.1/examples/trials/nas_cifar10/README.md)
- [模型压缩工具包 - Alpha 发布](https://github.com/microsoft/nni/blob/v1.1/docs/zh_CN/Compressor/Overview.md)：我们很高兴的宣布 NNI 的模型压缩工具包发布了。它还处于试验阶段，会根据使用反馈来改进。 诚挚邀请您使用、反馈，或更多贡献
### 修复的 Bug
@@ -62,26 +101,28 @@
### 主要功能
* Tuners 和 Assessors
-
- - 支持自动特征生成和选择 -Issue#877 -PR #1387 + 提供自动特征接口 + 基于 Beam 搜索的 Tuner + [添加 Pakdd 示例](https://github.com/microsoft/nni/tree/master/examples/trials/auto-feature-engineering)
- - 添加并行算法提高 TPE 在高并发下的性能。 -PR #1052
- - 为 hyperband 支持多阶段 -PR #1257
-- 训练平台
-
- - 支持私有 Docker Registry -PR #755
-
- * 改进
- * 增加 RestFUL API 的 Python 包装，支持通过代码获取指标的值 PR #1318
- * 新的 Python API : get_experiment_id(), get_trial_id() -PR #1353 -Issue #1331 & -Issue#1368
- * 优化 NAS 搜索空间 -PR #1393
+ - 支持自动特征生成和选择 -Issue#877 -PR #1387
+ + 提供自动特征接口
+ + 基于 Beam 搜索的 Tuner
+ + [增加 Pakdd 示例](https://github.com/microsoft/nni/tree/master/examples/trials/auto-feature-engineering)
+ + 添加并行算法提高 TPE 在高并发下的性能。 -PR #1052
+ + 为 hyperband 支持多阶段 -PR #1257
++ 训练平台
+
+ - 支持私有 Docker Registry -PR #755
+
+ * 改进
+ * 增加 RestFUL API 的 Python 包装，支持通过代码获取指标的值 PR #1318
+ * 新的 Python API : get_experiment_id(), get_trial_id() -PR #1353 -Issue #1331 & -Issue#1368
+ * 优化 NAS 搜索空间 -PR #1393
+ 使用 _type 统一 NAS 搜索空间 -- "mutable_type"e
+ 更新随机搜索 Tuner
- + 将 gpuNum 设为可选 -Issue #1365
- + 删除 OpenPAI 模式下的 outputDir 和 dataDir 配置 -Issue #1342
- + 在 Kubeflow 模式下创建 Trial 时，codeDir 不再被拷贝到 logDir -Issue #1224
+ + 将 gpuNum 设为可选 -Issue #1365
+ + 删除 OpenPAI 模式下的 outputDir 和 dataDir 配置 -Issue #1342
+ + 在 Kubeflow 模式下创建 Trial 时，codeDir 不再被拷贝到 logDir -Issue #1224
+ Web 门户和用户体验
-
+
- 在 Web 界面的搜索过程中显示最好指标的曲线 -Issue #1218
- 在多阶段 Experiment 中，显示参数列表的当前值 -Issue1210 -PR #1348
- 在 AddColumn 中增加 "Intermediate count" 选项。 -Issue #1210
@@ -90,12 +131,13 @@
- 在命令行中为 nnictl 命令增加详细文档的连接 -Issue #1260
- 用户体验改进：显示 Error 日志 -Issue #1173
- 文档
-
+
- 更新文档结构 -Issue #1231
- - [多阶段文档的改进](AdvancedFeature/MultiPhase.md) -Issue #1233 -PR #1242 + 增加配置示例
- - [Web 界面描述改进](Tutorial/WebUI.md) -PR #1419
+ - [多阶段文档的改进](AdvancedFeature/MultiPhase.md) -Issue #1233 -PR #1242
+ + 添加配置示例
+ + [Web 界面描述改进](Tutorial/WebUI.md) -PR #1419
-### 修复的 Bug
+### Bug 修复
* (Bug 修复)修复 0.9 版本中的链接 -Issue #1236
* (Bug 修复)自动完成脚本
@@ -116,20 +158,22 @@
### 主要功能
-* 通用 NAS 编程接口
+* 生成 NAS 编程接口
* 为 NAS 接口添加 `enas-mode` 和 `oneshot-mode`：[PR #1201](https://github.com/microsoft/nni/pull/1201#issue-291094510)
* [有 Matern 核的高斯 Tuner](Tuner/GPTuner.md)
* 支持多阶段 Experiment
-
+
* 为多阶段 Experiment 增加新的训练平台：pai 模式从 v0.9 开始支持多阶段 Experiment。
- * 为以下内置 Tuner 增加多阶段的功能：
- * TPE, Random Search, Anneal, Naïve Evolution, SMAC, Network Morphism, Metis Tuner。
-
- 有关详细信息，参考[实现多阶段的 Tuner](AdvancedFeature/MultiPhase.md)。
+ * 为以下内置 Tuner 增加多阶段的功能：
+
+
+ * TPE, Random Search, Anneal, Naïve Evolution, SMAC, Network Morphism, Metis Tuner。
+
+ 有关详细信息，参考[实现多阶段的 Tuner](AdvancedFeature/MultiPhase.md)。
* Web 界面
-
+
* 在 Web 界面中可比较 Trial。 有关详细信息，参考[查看 Trial 状态](Tutorial/WebUI.md)
* 允许用户调节 Web 界面的刷新间隔。 有关详细信息，参考[查看概要页面](Tutorial/WebUI.md)
* 更友好的显示中间结果。 有关详细信息，参考[查看 Trial 状态](Tutorial/WebUI.md)
@@ -158,7 +202,7 @@
* 在已经运行非 NNI 任务的 GPU 上也能运行 Trial
* 支持 Kubeflow v1beta2 操作符
* 支持 Kubeflow TFJob/PyTorchJob v1beta2
-* [通用 NAS 编程接口](AdvancedFeature/GeneralNasInterfaces.md)
+* [通用 NAS 编程接口](https://github.com/microsoft/nni/blob/v0.8/docs/zh_CN/GeneralNasInterfaces.md)
* 实现了 NAS 的编程接口，可通过 NNI Annotation 很容易的表达神经网络架构搜索空间
* 提供新命令 `nnictl trial codegen` 来调试 NAS 代码生成部分
* 提供 NAS 编程接口教程，NAS 在 MNIST 上的示例，用于 NAS 的可定制的随机 Tuner
@@ -274,10 +318,10 @@
#### 支持新的 Tuner 和 Assessor
-* 支持新的 [Metis Tuner](Tuner/MetisTuner.md)。 **在线**超参调优的场景下，Metis 算法已经被证明非常有效。
+* 支持新的 [Metis Tuner](Tuner/MetisTuner.md)。 对于**在线**超参调优的场景，Metis 算法已经被证明非常有效。
* 支持 [ENAS customized tuner](https://github.com/countif/enas_nni)。由 GitHub 社区用户所贡献。它是神经网络的搜索算法，能够通过强化学习来学习神经网络架构，比 NAS 的性能更好。
* 支持 [Curve fitting （曲线拟合）Assessor](Assessor/CurvefittingAssessor.md)，通过曲线拟合的策略来实现提前终止 Trial。
-* 进一步支持 [Weight Sharing（权重共享）](AdvancedFeature/AdvancedNas.md)：为 NAS Tuner 通过 NFS 来提供权重共享。
+* [权重共享的](https://github.com/microsoft/nni/blob/v0.5/docs/AdvancedNAS.md)高级支持：为 NAS Tuner 提供权重共享，当前支持 NFS。
#### 改进训练平台
@@ -361,12 +405,12 @@
### NNICTL 的新功能和更新
* 支持同时运行多个 Experiment。
-
+
在 v0.3 以前，NNI 仅支持一次运行一个 Experiment。 此版本开始，用户可以同时运行多个 Experiment。 每个 Experiment 都需要一个唯一的端口，第一个 Experiment 会像以前版本一样使用默认端口。 需要为其它 Experiment 指定唯一端口：
-
- ```bash
- nnictl create --port 8081 --config
- ```
+
+ ```bash
+ nnictl create --port 8081 --config
+ ```
* 支持更新最大 Trial 的数量。 使用 `nnictl update --help` 了解详情。 或参考 [NNICTL](Tutorial/Nnictl.md) 查看完整帮助。
@@ -375,15 +419,15 @@
* 不兼容的改动：nn.get_parameters() 改为 nni.get_next_parameter。 所有以前版本的示例将无法在 v0.3 上运行，需要重新克隆 NNI 代码库获取新示例。 如果在自己的代码中使用了 NNI，也需要相应的更新。
* 新 API **nni.get_sequence_id()**。 每个 Trial 任务都会被分配一个唯一的序列数字，可通过 nni.get_sequence_id() API 来获取。
-
- ```bash
- git clone -b v0.3 https://github.com/microsoft/nni.git
- ```
+
+ ```bash
+ git clone -b v0.3 https://github.com/microsoft/nni.git
+ ```
* **nni.report_final_result(result)** API 对结果参数支持更多的数据类型。
-
+
可用类型：
-
+
* int
* float
* 包含有 'default' 键值的 dict，'default' 的值必须为 int 或 float。 dict 可以包含任何其它键值对。
@@ -394,11 +438,11 @@
### 新示例
-* 公共的 NNI Docker 映像：
-
- ```bash
- docker pull msranni/nni:latest
- ```
+* 公开的 NNI Docker 映像：
+
+ ```bash
+ docker pull msranni/nni:latest
+ ```
* 新的 Trial 示例：[NNI Sklearn 示例](https://github.com/microsoft/nni/tree/master/examples/trials/sklearn)
diff --git a/docs/zh_CN/TrainingService/PaiYarnMode.md b/docs/zh_CN/TrainingService/PaiYarnMode.md
index c84debfa55..0f930967a2 100644
--- a/docs/zh_CN/TrainingService/PaiYarnMode.md
+++ b/docs/zh_CN/TrainingService/PaiYarnMode.md
@@ -102,7 +102,7 @@ paiYarnConfig:
```
nnictl create --config exp_paiYarn.yml
```
-来在 paiYarn 模式下启动 Experiment。 NNI 会为每个 Trial 创建 OpenPAIYarn 作业，作业名称的格式为 `nni_exp_{experiment_id}_trial_{trial_id}`。 可以在 OpenPAIYarn 集群的网站中看到 NNI 创建的作业，例如： 
+来在 paiYarn 模式下启动 Experiment。 NNI 会为每个 Trial 创建 OpenPAIYarn 作业，作业名称的格式为 `nni_exp_{experiment_id}_trial_{trial_id}`。 可以在 OpenPAIYarn 集群的网站中看到 NNI 创建的作业，例如： 
注意：paiYarn 模式下，NNIManager 会启动 RESTful 服务，监听端口为 NNI 网页服务器的端口加1。 例如，如果网页端口为`8080`，那么 RESTful 服务器会监听在 `8081`端口，来接收运行在 Kubernetes 中的 Trial 作业的指标。 因此，需要在防火墙中启用端口 `8081` 的 TCP 协议，以允许传入流量。
diff --git a/docs/zh_CN/TrainingService/RemoteMachineMode.md b/docs/zh_CN/TrainingService/RemoteMachineMode.md
index eba05921b5..e4b6917f84 100644
--- a/docs/zh_CN/TrainingService/RemoteMachineMode.md
+++ b/docs/zh_CN/TrainingService/RemoteMachineMode.md
@@ -1,8 +1,22 @@
-# 在多机上运行 Experiment
+# 在远程计算机上运行 Experiment
-NNI 支持通过 SSH 通道在多台计算机上运行 Experiment，称为 `remote` 模式。 NNI 需要这些计算机的访问权限，并假定已配置好了深度学习训练环境。
+NNI 可以通过 SSH 在多个远程计算机上运行同一个 Experiment，称为 `remote` 模式。 这就像一个轻量级的训练平台。 在此模式下，可以从计算机启动 NNI，并将 Trial 并行调度到远程计算机。
-例如：有三台服务器，登录账户为 `bob`（注意：账户不必在各台计算机上一致）：
+## 远程计算机的要求
+
+* 仅支持 Linux 作为远程计算机，其[配置需求](../Tutorial/Installation.md)与 NNI 本机模式相同。
+
+* 根据[安装文章](../Tutorial/Installation.md)，在每台计算机上安装 NNI。
+
+* 确保远程计算机满足 Trial 代码的环境要求。 如果默认环境不符合要求，可以将设置脚本添加到 NNI 配置的 `command` 字段。
+
+* 确保远程计算机能被运行 `nnictl` 命令的计算机通过 SSH 访问。 同时支持 SSH 的密码和密钥验证方法。 有关高级用法，参考[配置](../Tutorial/ExperimentConfig.md)的 machineList 部分。
+
+* 确保每台计算机上的 NNI 版本一致。
+
+## 运行 Experiment
+
+例如，有三台机器，可使用用户名和密码登录。
| IP | 用户名 | 密码 |
| -------- | --- | ------ |
@@ -10,15 +24,9 @@ NNI 支持通过 SSH 通道在多台计算机上运行 Experiment，称为 `remo
| 10.1.1.2 | bob | bob123 |
| 10.1.1.3 | bob | bob123 |
-## 设置 NNI 环境
+在这三台计算机或另一台能访问这些计算机的环境中安装并运行 NNI。
-按照[指南](../Tutorial/QuickStart.md)在每台计算机上安装 NNI。
-
-## 运行 Experiment
-
-将 NNI 安装在可以访问上述三台计算机的网络的另一台计算机上，或者仅在三台计算机中的任何一台上运行 `nnictl` 即可启动 Experiment。
-
-以 `examples/trials/mnist-annotation` 为例。 此处示例在 `examples/trials/mnist-annotation/config_remote.yml`：
+以 `examples/trials/mnist-annotation` 为例。 示例文件 `examples/trials/mnist-annotation/config_remote.yml` 的内容如下：
```yaml
authorName: default
@@ -58,14 +66,8 @@ machineList:
passwd: bob123
```
-`codeDir` 中的文件会被自动上传到远程服务器。 可在不同的操作系统上运行 NNI (Windows, Linux, MacOS)，来在远程机器上（仅支持 Linux）运行 Experiment。
+`codeDir` 中的文件会自动上传到远程计算机中。 可在 Windows、Linux 或 macOS 上运行以下命令，在远程 Linux 计算机上启动 Trial：
```bash
nnictl create --config examples/trials/mnist-annotation/config_remote.yml
-```
-
-也可使用公钥/私钥对，而非用户名/密码进行身份验证。 有关高级用法，请参考[实验配置参考](../Tutorial/ExperimentConfig.md)。
-
-## 版本校验
-
-从 0.6 开始，NNI 支持版本校验，详情参考[这里](PaiMode.md)。
\ No newline at end of file
+```
\ No newline at end of file
diff --git a/docs/zh_CN/TrainingService/SupportTrainingService.md b/docs/zh_CN/TrainingService/SupportTrainingService.md
index fbf6f6a2cd..5afcc13020 100644
--- a/docs/zh_CN/TrainingService/SupportTrainingService.md
+++ b/docs/zh_CN/TrainingService/SupportTrainingService.md
@@ -19,21 +19,22 @@ NNI 不仅提供了这些内置的训练平台，还提供了轻松连接自己
TrainingService 在设计上为了便于实现，将平台相关的公共属性抽象成类。用户只需要继承这个抽象类，并根据平台特点实现子类，便能够实现 TrainingService。
TrainingService 的声明如下：
- abstract class TrainingService {
- public abstract listTrialJobs(): Promise;
- public abstract getTrialJob(trialJobId: string): Promise;
- public abstract addTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void;
- public abstract removeTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void;
- public abstract submitTrialJob(form: JobApplicationForm): Promise;
- public abstract updateTrialJob(trialJobId: string, form: JobApplicationForm): Promise;
- public abstract get isMultiPhaseJobSupported(): boolean;
- public abstract cancelTrialJob(trialJobId: string, isEarlyStopped?: boolean): Promise;
- public abstract setClusterMetadata(key: string, value: string): Promise;
- public abstract getClusterMetadata(key: string): Promise;
- public abstract cleanUp(): Promise;
- public abstract run(): Promise;
- }
-
+```javascript
+abstract class TrainingService {
+ public abstract listTrialJobs(): Promise;
+ public abstract getTrialJob(trialJobId: string): Promise;
+ public abstract addTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void;
+ public abstract removeTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void;
+ public abstract submitTrialJob(form: JobApplicationForm): Promise;
+ public abstract updateTrialJob(trialJobId: string, form: JobApplicationForm): Promise;
+ public abstract get isMultiPhaseJobSupported(): boolean;
+ public abstract cancelTrialJob(trialJobId: string, isEarlyStopped?: boolean): Promise;
+ public abstract setClusterMetadata(key: string, value: string): Promise;
+ public abstract getClusterMetadata(key: string): Promise;
+ public abstract cleanUp(): Promise;
+ public abstract run(): Promise;
+}
+```
TrainingService 的父类有一些抽象函数，用户需要继承父类并实现所有这些抽象函数。
有关如何实现 TrainingService 的更多信息，[参考这里](https://github.com/microsoft/nni/blob/master/docs/zh_CN/TrainingService/HowToImplementTrainingService.md)。
\ No newline at end of file
diff --git a/docs/zh_CN/TrialExample/EfficientNet.md b/docs/zh_CN/TrialExample/EfficientNet.md
new file mode 100644
index 0000000000..bf44c695ab
--- /dev/null
+++ b/docs/zh_CN/TrialExample/EfficientNet.md
@@ -0,0 +1,21 @@
+# EfficientNet
+
+[EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)
+
+如论文中 3.3 所述，使用遍历搜索来找到 EfficientNet-B1 的 alpha, beta 和 gamma 的最好组合。 搜索空间，Tuner，配置示例如下。
+
+## 说明
+
+[示例代码](https://github.com/microsoft/nni/tree/master/examples/trials/efficientnet)
+
+1. 将示例代码目录设为当前工作目录。
+2. 运行 `git clone https://github.com/ultmaster/EfficientNet-PyTorch` 来克隆修改过的 [EfficientNet-PyTorch](https://github.com/lukemelas/EfficientNet-PyTorch)。 修改尽可能接近原始的 [TensorFlow 版本](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet) （包括 EMA，标记平滑度等等。）；另外添加了代码从 Tuner 获取参数并回调中间和最终结果。 将其 clone 至 `EfficientNet-PyTorch`；`main.py`，`train_imagenet.sh` 等文件会在配置文件中指定的路径。
+3. 运行 `nnictl create --config config_local.yml` (OpenPAI 可使用 `config_pai.yml`) 来找到最好的 EfficientNet-B1。 根据环境来调整训练平台（OpenPAI、本机、远程），batch size。
+
+在 ImageNet 上的训练，可阅读 `EfficientNet-PyTorch/train_imagenet.sh`。 下载 ImageNet，并参考 [PyTorch 格式](https://pytorch.org/docs/stable/torchvision/datasets.html#imagenet) 来解压，然后将 `/mnt/data/imagenet` 替换为 ImageNet 的路径。 此文件也是如何将 ImageNet 挂载到 OpenPAI 容器的示例。
+
+## 结果
+
+下图展示了 acc@1 和 alpha、beta、gamma 之间的关系。
+
+
diff --git a/docs/zh_CN/TrialExample/KDExample.md b/docs/zh_CN/TrialExample/KDExample.md
index 8f669b3d6d..ef91b9b905 100644
--- a/docs/zh_CN/TrialExample/KDExample.md
+++ b/docs/zh_CN/TrialExample/KDExample.md
@@ -30,4 +30,4 @@ for batch_idx, (data, target) in enumerate(train_loader):
* **kd_teacher_model:** 预训练过的教师模型
* **kd_T:** 用于平滑教师模型输出的温度。
-完整代码可在这里找到
\ No newline at end of file
+完整代码[在这里](https://github.com/microsoft/nni/tree/v1.3/examples/model_compress/knowledge_distill/)。
diff --git a/docs/zh_CN/TrialExample/SklearnExamples.md b/docs/zh_CN/TrialExample/SklearnExamples.md
index 36f9b6fa67..e860358040 100644
--- a/docs/zh_CN/TrialExample/SklearnExamples.md
+++ b/docs/zh_CN/TrialExample/SklearnExamples.md
@@ -20,7 +20,7 @@ nnictl create --config ./config.yml
示例使用了数字数据集，它是由 1797 个 8x8 的图片组成，每个图片都是一个手写数字，目标是将图片分为 10 类。
-在这个示例中，使用 SVC 作为模型，并为此模型选择一些参数，包括 `"C", "keral", "degree", "gamma" 和 "coef0"`。 关于这些参数的更多信息，可参考[这里](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)。
+在这个示例中，使用 SVC 作为模型，并为此模型选择一些参数，包括 `"C", "kernel", "degree", "gamma" 和 "coef0"`。 关于这些参数的更多信息，可参考[这里](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)。
### 2.2 回归
@@ -63,7 +63,7 @@ nnictl create --config ./config.yml
```json
{
"C": {"_type":"uniform","_value":[0.1, 1]},
- "keral": {"_type":"choice","_value":["linear", "rbf", "poly", "sigmoid"]},
+ "kernel": {"_type":"choice","_value":["linear", "rbf", "poly", "sigmoid"]},
"degree": {"_type":"choice","_value":[1, 2, 3, 4]},
"gamma": {"_type":"uniform","_value":[0.01, 0.1]},
"coef0 ": {"_type":"uniform","_value":[0.01, 0.1]}
@@ -75,7 +75,7 @@ nnictl create --config ./config.yml
```python
params = {
'C': 1.0,
- 'keral': 'linear',
+ 'kernel': 'linear',
'degree': 3,
'gamma': 0.01,
'coef0': 0.01
diff --git a/docs/zh_CN/Tutorial/FAQ.md b/docs/zh_CN/Tutorial/FAQ.md
index bcd7fe7a25..7577248612 100644
--- a/docs/zh_CN/Tutorial/FAQ.md
+++ b/docs/zh_CN/Tutorial/FAQ.md
@@ -56,6 +56,10 @@ nnictl 在执行时，使用 tmp 目录作为临时目录来复制 codeDir 下
参考 [Windows 上使用 NNI](NniOnWindows.md)。
+### 更多常见问题解答
+
+[标有常见问题标签的 Issue](https://github.com/microsoft/nni/labels/FAQ)
+
### 帮助改进
在创建新问题前，请在 https://github.com/Microsoft/nni/issues 查看是否有人已经报告了相似的问题。
\ No newline at end of file
diff --git a/docs/zh_CN/Tutorial/HowToDebug.md b/docs/zh_CN/Tutorial/HowToDebug.md
index 580da25dcd..f2c9c72f8c 100644
--- a/docs/zh_CN/Tutorial/HowToDebug.md
+++ b/docs/zh_CN/Tutorial/HowToDebug.md
@@ -81,4 +81,4 @@ NNI 中有不同的错误类型。 根据严重程度，可分为三类。 当 N
如图，每个 Trial 都有日志路径，可以从中找到 Trial 的日志和 stderr。
-除了 Experiment 级调试之外，NNI 还提供调试单个 Trial 的功能，而无需启动整个 Experiment。 有关调试单个 Trial 代码的更多信息，请参考[独立运行模式](../TrialExample/Trials.md#standalone-mode-for-debug)。
\ No newline at end of file
+除了 Experiment 级调试之外，NNI 还提供调试单个 Trial 的功能，而无需启动整个 Experiment。 有关调试单个 Trial 代码的更多信息，请参考[独立运行模式](../TrialExample/Trials.md#用于调试的独立模式)。
\ No newline at end of file
diff --git a/docs/zh_CN/Tutorial/Installation.md b/docs/zh_CN/Tutorial/Installation.md
index 830676ac1b..9a645cdc86 100644
--- a/docs/zh_CN/Tutorial/Installation.md
+++ b/docs/zh_CN/Tutorial/Installation.md
@@ -1,20 +1,22 @@
# 安装 NNI
-当前支持在 Linux，Mac 和 Windows 下安装。
+当前支持在 Linux，macOS 和 Windows 下安装。
-## **在 Linux 和 Mac 下安装**
+## 在 Linux 或 macOS 上安装
-* **通过 pip 命令安装 NNI**
+* 通过 pip 命令安装 NNI
- 先决条件：`python >= 3.5`
+ 先决条件：`python 64-bit >= 3.5`
```bash
python3 -m pip install --upgrade nni
```
-* **通过源代码安装 NNI**
+* 通过源代码安装 NNI
- 先决条件：`python >=3.5`, `git`, `wget`
+ 如果对某个或最新版本的代码感兴趣，可通过源代码安装 NNI。
+
+ 先决条件：`python 64-bit >=3.5`, `git`, `wget`
```bash
git clone -b v0.8 https://github.com/Microsoft/nni.git
@@ -22,25 +24,27 @@
./install.sh
```
-* **在 docker 映像中安装 NNI**
+* 在 Docker 映像中使用 NNI
也可将 NNI 安装到 docker 映像中。 参考[这里](../deployment/docker/README.md)来生成 NNI 的 Docker 映像。 也可通过此命令从 Docker Hub 中直接拉取 NNI 的映像 `docker pull msranni/nni:latest`。
-## **在 Windows 上安装**
+## 在 Windows 上安装
-推荐使用 Anaconda 或 Miniconda。
+强烈建议使用 Anaconda 或 Miniconda 来管理多个 Python 环境。
-* **通过 pip 命令安装 NNI**
+* 通过 pip 命令安装 NNI
- 先决条件：`python(64-bit) >= 3.5`
+ 先决条件：`python 64-bit >= 3.5`
```bash
python -m pip install --upgrade nni
```
-* **通过源代码安装 NNI**
+* 通过源代码安装 NNI
+
+ 如果对某个或最新版本的代码感兴趣，可通过源代码安装 NNI。
- 先决条件：`python >=3.5`, `git`, `PowerShell`
+ 先决条件：`python 64-bit >=3.5`, `git`, `PowerShell`
```bash
git clone -b v0.8 https://github.com/Microsoft/nni.git
@@ -48,43 +52,104 @@
powershell -ExecutionPolicy Bypass -file install.ps1
```
-## **系统需求**
-
-以下是 NNI 在 Linux 下的最低配置。 由于程序变更，NNI 的最低配置会有所更改。
-
-| | 最低配置 | 推荐配置 |
-| -------- | ------------------------------------- | ----------------------------------------- |
-| **操作系统** | Ubuntu 16.04 或以上版本 | Ubuntu 16.04 或以上版本 |
-| **CPU** | Intel® Core™ i3 或 AMD Phenom™ X3 8650 | Intel® Core™ i5 或 AMD Phenom™ II X3 或更高配置 |
-| **GPU** | NVIDIA® GeForce® GTX 460 | NVIDIA® GeForce® GTX 660 或更高配置 |
-| **内存** | 4 GB | 6 GB |
-| **存储** | 30 GB 可用的磁盘空间 | |
-| **网络** | 宽带连接 | |
-| **分辨率** | 1024 x 768 以上 | |
-
-以下是 NNI 在 MacOS 下的最低配置。 由于程序变更，NNI 的最低配置会有所更改。
-
-| | 最低配置 | 推荐配置 |
-| -------- | -------------------------------------------------- | ------------------------ |
-| **操作系统** | macOS 10.14.1 (最新版本) | macOS 10.14.1 (最新版本) |
-| **CPU** | Intel® Core™ i5-760 或更高 | Intel® Core™ i7-4770 或更高 |
-| **GPU** | NVIDIA® GeForce® GT 750M 或 AMD Radeon™ R9 M290 或更高 | AMD Radeon™ R9 M395X 或更高 |
-| **内存** | 4 GB | 8 GB |
-| **存储** | 70GB 可用空间及 7200 RPM 硬盘 | 70GB 可用空间 SSD 硬盘 |
-| **网络** | 宽带连接 | |
-| **分辨率** | 1024 x 768 以上 | |
-
-以下是 NNI 在 Windows 上的最低配置，推荐使用 Windows 10 1809 版。 由于程序变更，NNI 的最低配置会有所更改。
-
-| | 最低配置 | 推荐配置 |
-| -------- | ------------------------------------- | ----------------------------------------- |
-| **操作系统** | Windows 10 | Windows 10 |
-| **CPU** | Intel® Core™ i3 或 AMD Phenom™ X3 8650 | Intel® Core™ i5 或 AMD Phenom™ II X3 或更高配置 |
-| **GPU** | NVIDIA® GeForce® GTX 460 | NVIDIA® GeForce® GTX 660 或更高配置 |
-| **内存** | 4 GB | 6 GB |
-| **存储** | 30 GB 可用的磁盘空间 | |
-| **网络** | 宽带连接 | |
-| **分辨率** | 1024 x 768 以上 | |
+## 验证安装
+
+以下示例基于 TensorFlow 1.x 。确保运行环境中使用的的是 ** TensorFlow 1.x**。
+
+* 通过克隆源代码下载示例。
+
+ ```bash
+ git clone -b v1.3 https://github.com/Microsoft/nni.git
+ ```
+
+* 运行 MNIST 示例。
+
+ Linux 或 macOS
+
+ ```bash
+ nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
+ ```
+
+ Windows
+
+ ```bash
+ nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
+ ```
+
+* 在命令行中等待输出 `INFO: Successfully started experiment!`。 此消息表明 Experiment 已成功启动。 通过命令行输出的 `Web UI url` 来访问 Experiment 的界面。
+
+```text
+INFO: Starting restful server...
+INFO: Successfully started Restful server!
+INFO: Setting local config...
+INFO: Successfully set local config!
+INFO: Starting experiment...
+INFO: Successfully started experiment!
+-----------------------------------------------------------------------
+The experiment id is egchD4qy
+The Web UI urls are: http://223.255.255.1:8080 http://127.0.0.1:8080
+-----------------------------------------------------------------------
+
+You can use these commands to get more information about the experiment
+-----------------------------------------------------------------------
+ commands description
+
+1. nnictl experiment show show the information of experiments
+2. nnictl trial ls list all of trial jobs
+3. nnictl top monitor the status of running experiments
+4. nnictl log stderr show stderr log content
+5. nnictl log stdout show stdout log content
+6. nnictl stop stop an experiment
+7. nnictl trial kill kill a trial job by id
+8. nnictl --help get help information about nnictl
+-----------------------------------------------------------------------
+```
+
+* 在浏览器中打开 `Web UI url`，可看到下图的 Experiment 详细信息，以及所有的 Trial 任务。 查看[这里](../Tutorial/WebUI.md)的更多页面。
+
+
+
+
+
+## 系统需求
+
+由于程序变更，NNI 的最低配置会有所更改。
+
+### Linux
+
+| | 推荐配置 | 最低配置 |
+| -------- | ----------------------------------------- | ------------------------------------- |
+| **操作系统** | Ubuntu 16.04 或以上版本 | |
+| **CPU** | Intel® Core™ i5 或 AMD Phenom™ II X3 或更高配置 | Intel® Core™ i3 或 AMD Phenom™ X3 8650 |
+| **GPU** | NVIDIA® GeForce® GTX 660 或更高配置 | NVIDIA® GeForce® GTX 460 |
+| **内存** | 6 GB | 4 GB |
+| **存储** | 30 GB 可用的磁盘空间 | |
+| **网络** | 宽带连接 | |
+| **分辨率** | 1024 x 768 以上 | |
+
+### macOS
+
+| | 推荐配置 | 最低配置 |
+| -------- | ------------------------ | -------------------------------------------------- |
+| **操作系统** | macOS 10.14.1 或更高版本 | |
+| **CPU** | Intel® Core™ i7-4770 或更高 | Intel® Core™ i5-760 或更高 |
+| **GPU** | AMD Radeon™ R9 M395X 或更高 | NVIDIA® GeForce® GT 750M 或 AMD Radeon™ R9 M290 或更高 |
+| **内存** | 8 GB | 4 GB |
+| **存储** | 70GB 可用空间 SSD 硬盘 | 70GB 可用空间及 7200 RPM 硬盘 |
+| **网络** | 宽带连接 | |
+| **分辨率** | 1024 x 768 以上 | |
+
+### Windows
+
+| | 推荐配置 | 最低配置 |
+| -------- | ----------------------------------------- | ------------------------------------- |
+| **操作系统** | Windows 10 1809 或更高版本 | |
+| **CPU** | Intel® Core™ i5 或 AMD Phenom™ II X3 或更高配置 | Intel® Core™ i3 或 AMD Phenom™ X3 8650 |
+| **GPU** | NVIDIA® GeForce® GTX 660 或更高配置 | NVIDIA® GeForce® GTX 460 |
+| **内存** | 6 GB | 4 GB |
+| **存储** | 30 GB 可用的磁盘空间 | |
+| **网络** | 宽带连接 | |
+| **分辨率** | 1024 x 768 以上 | |
## 更多
diff --git a/docs/zh_CN/Tutorial/Nnictl.md b/docs/zh_CN/Tutorial/Nnictl.md
index acee5d4534..38b66d314b 100644
--- a/docs/zh_CN/Tutorial/Nnictl.md
+++ b/docs/zh_CN/Tutorial/Nnictl.md
@@ -49,6 +49,7 @@ nnictl 支持的命令：
| --config, -c | True | | Experiment 的 YAML 配置文件 |
| --port, -p | False | | RESTful 服务的端口 |
| --debug, -d | False | | 设置为调试模式 |
+ | --watch, -w | False | | 启动为监视模式 |
* 示例
@@ -97,6 +98,7 @@ nnictl 支持的命令：
| id | True | | 要恢复的 Experiment 标识 |
| --port, -p | False | | 要恢复的 Experiment 使用的 RESTful 服务端口 |
| --debug, -d | False | | 设置为调试模式 |
+ | --watch, -w | False | | 启动为监视模式 |
* 示例
diff --git a/docs/zh_CN/Tutorial/QuickStart.md b/docs/zh_CN/Tutorial/QuickStart.md
index 3ed05f3e68..b886debf18 100644
--- a/docs/zh_CN/Tutorial/QuickStart.md
+++ b/docs/zh_CN/Tutorial/QuickStart.md
@@ -2,15 +2,15 @@
## 安装
-当前支持 Linux，MacOS 和 Windows，在 Ubuntu 16.04 或更高版本，MacOS 10.14.1 以及 Windows 10.1809 上进行了测试。 在 `python >= 3.5` 的环境中，只需要运行 `pip install` 即可完成安装。
+当前支持 Linux，macOS 和 Windows，在 Ubuntu 16.04 或更高版本，macOS 10.14.1 以及 Windows 10.1809 上进行了测试。 在 `python >= 3.5` 的环境中，只需要运行 `pip install` 即可完成安装。
-#### Linux 和 MacOS
+**Linux 和 macOS**
```bash
python3 -m pip install --upgrade nni
```
-#### Windows
+**Windows**
```bash
python -m pip install --upgrade nni
@@ -18,7 +18,7 @@
注意：
-* 在 Linux 和 MacOS 上，如果要将 NNI 安装到当前用户的 home 目录中，可使用 `--user`，则不需要特殊权限。
+* 在 Linux 和 macOS 上，如果要将 NNI 安装到当前用户的 home 目录中，可使用 `--user`，则不需要特殊权限。
* 如果遇到如`Segmentation fault` 这样的任何错误请参考[常见问题](FAQ.md)。
* 参考[安装 NNI](Installation.md)，来了解`系统需求`。
@@ -54,21 +54,22 @@ if __name__ == '__main__':
NNI 用来帮助超参调优。它的流程如下：
- 输入: 搜索空间, Trial 代码, 配置文件
- 输出: 一组最佳的超参配置
-
- 1: For t = 0, 1, 2, ..., maxTrialNum,
- 2: hyperparameter = 从搜索空间选择一组参数
- 3: final result = run_trial_and_evaluate(hyperparameter)
- 4: 返回最终结果给 NNI
- 5: If 时间达到上限,
- 6: 停止实验
- 7: return 最好的实验结果
-
+```text
+输入: 搜索空间, Trial 代码, 配置文件
+输出: 一组最佳的超参配置
+
+1: For t = 0, 1, 2, ..., maxTrialNum,
+2: hyperparameter = 从搜索空间选择一组参数
+3: final result = run_trial_and_evaluate(hyperparameter)
+4: 返回最终结果给 NNI
+5: If 时间达到上限,
+6: 停止实验
+7: return 最好的实验结果
+```
如果需要使用 NNI 来自动训练模型，找到最佳超参，需要如下三步：
-**使用 NNI 时的三个步骤**
+**启动 Experiment 的三个步骤**
**第一步**：定义 JSON 格式的`搜索空间`文件，包括所有需要搜索的超参的`名称`和`分布`（离散和连续值均可）。
@@ -140,7 +141,7 @@ trial:
上面的代码都已准备好，并保存在 [examples/trials/mnist-tfv1/](https://github.com/Microsoft/nni/tree/master/examples/trials/mnist-tfv1)。
-#### Linux 和 macOS
+**Linux 和 macOS**
从命令行使用 **config.yml** 文件启动 MNIST Experiment 。
@@ -148,17 +149,17 @@ trial:
nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
```
-#### Windows
+**Windows**
从命令行使用 **config_windows.yml** 文件启动 MNIST Experiment 。
-**注意**：如果使用 Windows，则需要在 config.yml 文件中，将 `python3` 改为 `python`，或者使用 config_windows.yml 来开始 Experiment。
+注意：如果使用 Windows，则需要在 config.yml 文件中，将 `python3` 改为 `python`，或者使用 config_windows.yml 来开始 Experiment。
```bash
nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
```
-注意：**nnictl** 是一个命令行工具，用来控制 NNI Experiment，如启动、停止、继续 Experiment，启动、停止 NNIBoard 等等。 查看[这里](Nnictl.md)，了解 `nnictl` 更多用法。
+注意：`nnictl` 是一个命令行工具，用来控制 NNI Experiment，如启动、停止、继续 Experiment，启动、停止 NNIBoard 等等。 查看[这里](Nnictl.md)，了解 `nnictl` 更多用法。
在命令行中等待输出 `INFO: Successfully started experiment!`。 此消息表明 Experiment 已成功启动。 期望的输出如下：
@@ -201,7 +202,7 @@ Web 地址为：[IP 地址]:8080
在浏览器中打开 `Web 界面地址`(即：`[IP 地址]:8080`)，就可以看到 Experiment 的详细信息，以及所有的 Trial 任务。 如果无法打开终端中的 Web 界面链接，可以参考 [FAQ](FAQ.md)。
-#### 查看概要页面
+### 查看概要页面
点击标签 "Overview"。
@@ -213,7 +214,7 @@ Experiment 相关信息会显示在界面上，配置和搜索空间等。 可
-#### 查看 Trial 详情页面
+### 查看 Trial 详情页面
点击 "Default Metric" 来查看所有 Trial 的点图。 悬停鼠标来查看默认指标和搜索空间信息。
diff --git a/docs/zh_CN/conf.py b/docs/zh_CN/conf.py
index f1336f1c78..d5bec553af 100644
--- a/docs/zh_CN/conf.py
+++ b/docs/zh_CN/conf.py
@@ -47,6 +47,9 @@
'sphinx.ext.napoleon',
]
+# 添加示例模块
+autodoc_mock_imports = ['apex']
+
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -72,7 +75,7 @@
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'Release_v1.0.md']
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = None
diff --git a/docs/zh_CN/examples.rst b/docs/zh_CN/examples.rst
index f76ce9eb3d..95e0f94fee 100644
--- a/docs/zh_CN/examples.rst
+++ b/docs/zh_CN/examples.rst
@@ -11,3 +11,5 @@
EvolutionSQuAD<./TrialExample/SquadEvolutionExamples>
GBDT<./TrialExample/GbdtExample>
RocksDB <./TrialExample/RocksdbExamples>
+ KD 示例 <./TrialExample/KDExample>
+ EfficientNet <./TrialExample/EfficientNet>
diff --git a/docs/zh_CN/model_compression.rst b/docs/zh_CN/model_compression.rst
index 34d05b4844..2e273a79eb 100644
--- a/docs/zh_CN/model_compression.rst
+++ b/docs/zh_CN/model_compression.rst
@@ -18,7 +18,7 @@ NNI 中也内置了一些流程的模型压缩算法。
概述
Level Pruner
AGP Pruner
- L1Filter Pruner
+ L1Filter Pruner
Slim Pruner
Lottery Ticket Pruner
FPGM Pruner
diff --git a/docs/zh_CN/nas.rst b/docs/zh_CN/nas.rst
index a7329dd60d..611c5aefe2 100644
--- a/docs/zh_CN/nas.rst
+++ b/docs/zh_CN/nas.rst
@@ -22,4 +22,6 @@ NAS 算法
NAS 接口
ENAS
DARTS
- P-DARTS
+ P-DARTS
+ SPOS
+ CDARTS
diff --git a/docs/zh_CN/training_services.rst b/docs/zh_CN/training_services.rst
index 4e2969e597..8e75af2ae7 100644
--- a/docs/zh_CN/training_services.rst
+++ b/docs/zh_CN/training_services.rst
@@ -6,5 +6,6 @@ NNI 支持的训练平台介绍
本机<./TrainingService/LocalMode>
远程<./TrainingService/RemoteMachineMode>
OpenPAI<./TrainingService/PaiMode>
+ OpenPAI Yarn 模式<./TrainingService/PaiYarnMode>
Kubeflow<./TrainingService/KubeflowMode>
FrameworkController<./TrainingService/FrameworkControllerMode>
diff --git a/examples/feature_engineering/auto-feature-engineering/README_zh_CN.md b/examples/feature_engineering/auto-feature-engineering/README_zh_CN.md
index 55b50217cd..76cce132ff 100644
--- a/examples/feature_engineering/auto-feature-engineering/README_zh_CN.md
+++ b/examples/feature_engineering/auto-feature-engineering/README_zh_CN.md
@@ -1,8 +1,7 @@
-**NNI 中的自动特征工程**
-===
+ **NNI 中的自动特征工程** ===
-此[示例](https://github.com/SpongebBob/tabular_automl_NNI)在 NNI 中实现了自动特征工程。
+ 此[示例](https://github.com/SpongebBob/tabular_automl_NNI)在 NNI 中实现了自动特征工程。
-代码来自于贡献者。 谢谢可爱的贡献者！
+ 代码来自于贡献者。 谢谢可爱的贡献者！
-欢迎越来越多的人加入我们！
\ No newline at end of file
+ 欢迎越来越多的人加入我们！
diff --git a/examples/trials/auto-gbdt/config_pai.yml b/examples/trials/auto-gbdt/config_pai.yml
index 7393a080a2..e4cd040aec 100644
--- a/examples/trials/auto-gbdt/config_pai.yml
+++ b/examples/trials/auto-gbdt/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/auto-gbdt/config_paiYarn.yml b/examples/trials/auto-gbdt/config_paiYarn.yml
new file mode 100644
index 0000000000..427a6eacd8
--- /dev/null
+++ b/examples/trials/auto-gbdt/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_auto-gbdt
+trialConcurrency: 1
+maxExecDuration: 10h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner, GPTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: minimize
+trial:
+ command: python3 main.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/cifar10_pytorch/config_pai.yml b/examples/trials/cifar10_pytorch/config_pai.yml
index 87d82ff097..97aac1e040 100644
--- a/examples/trials/cifar10_pytorch/config_pai.yml
+++ b/examples/trials/cifar10_pytorch/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
diff --git a/examples/trials/cifar10_pytorch/config_paiYarn.yml b/examples/trials/cifar10_pytorch/config_paiYarn.yml
new file mode 100644
index 0000000000..3ac750f536
--- /dev/null
+++ b/examples/trials/cifar10_pytorch/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_pytorch_cifar10
+trialConcurrency: 1
+maxExecDuration: 100h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 main.py
+ codeDir: .
+ gpuNum: 1
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
diff --git a/examples/trials/efficientnet/README.md b/examples/trials/efficientnet/README.md
index 2d8f436594..0ed4a844e0 100644
--- a/examples/trials/efficientnet/README.md
+++ b/examples/trials/efficientnet/README.md
@@ -1,19 +1 @@
-# EfficientNet
-
-[EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)
-
-Provided here are: Search space and tuners for finding the best tuple (alpha, beta, gamma) for EfficientNet-B1 with grid search, as discussed in Section 3.3 in [paper](https://arxiv.org/abs/1905.11946).
-
-## Instructions
-
-1. Set your working directory here in this directory.
-2. Run `git clone https://github.com/ultmaster/EfficientNet-PyTorch` to clone this modified version of [EfficientNet-PyTorch](https://github.com/lukemelas/EfficientNet-PyTorch). The modifications were done to adhere to the original [Tensorflow version](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet) as close as possible (including EMA, label smoothing and etc.); also added are the part which gets parameters from tuner and reports intermediate/final results. Clone it into `EfficientNet-PyTorch`; the files like `main.py`, `train_imagenet.sh` will appear inside, as specified in the configuration files.
-3. Run `nnictl create --config config_net.yml` to find the best EfficientNet-B1. Adjust the training service (PAI/local/remote), batch size in the config files according to the environment.
-
-For training on ImageNet, read `EfficientNet-PyTorch/train_imagenet.sh`. Download ImageNet beforehand and extract it adhering to [PyTorch format](https://pytorch.org/docs/stable/torchvision/datasets.html#imagenet) and then replace `/mnt/data/imagenet` in with the location of the ImageNet storage. This file should also be a good example to follow for mounting ImageNet into the container on OpenPAI.
-
-## Results
-
-The follow image is a screenshot, demonstrating the relationship between acc@1 and alpha, beta, gamma.
-
-
\ No newline at end of file
+[Documentation](https://nni.readthedocs.io/en/latest/TrialExample/EfficientNet.html)
diff --git a/examples/trials/efficientnet/README_zh_CN.md b/examples/trials/efficientnet/README_zh_CN.md
index 2f4ac5e65f..083689141b 100644
--- a/examples/trials/efficientnet/README_zh_CN.md
+++ b/examples/trials/efficientnet/README_zh_CN.md
@@ -1,19 +1 @@
-# EfficientNet
-
-[EfficientNet: 重新思考卷积神经网络的模型尺度](https://arxiv.org/abs/1905.11946)
-
-这里提供了：使用遍历搜索为 EfficientNet-B1 找到最佳元组（alpha，beta，gamma）的搜索空间和 Tuner。参考[论文](https://arxiv.org/abs/1905.11946) 3.3。
-
-## 说明
-
-1. 设置此目录为当前目录。
-2. 运行 `git clone https://github.com/ultmaster/EfficientNet-PyTorch` 来 clone 修改过的 [EfficientNet-PyTorch](https://github.com/lukemelas/EfficientNet-PyTorch)。 修改尽可能接近原始的 [TensorFlow 版本](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet) （包括 EMA，标记平滑度等等。）；另外添加了代码从 Tuner 获取参数并回调中间和最终结果。 将其 clone 至 `EfficientNet-PyTorch`；`main.py`，`train_imagenet.sh` 等文件会在配置文件中指定的路径。
-3. 运行 `nnictl create --config config_net.yml` 来找到最好的 EfficientNet-B1。 根据环境来调整训练平台（OpenPAI、本机、远程），batch size。
-
-在 ImageNet 上的训练，可阅读 `EfficientNet-PyTorch/train_imagenet.sh`。 下载 ImageNet，并参考 [PyTorch 格式](https://pytorch.org/docs/stable/torchvision/datasets.html#imagenet) 来解压，然后将 `/mnt/data/imagenet` 替换为 ImageNet 的路径。 此文件也是如何将 ImageNet 挂载到 OpenPAI 容器的示例。
-
-## 结果
-
-下图展示了 acc@1 和 alpha、beta、gamma 之间的关系。
-
-
\ No newline at end of file
+[文档](https://nni.readthedocs.io/en/latest/TrialExample/EfficientNet.html)
\ No newline at end of file
diff --git a/examples/trials/efficientnet/config_local.yml b/examples/trials/efficientnet/config_local.yml
new file mode 100644
index 0000000000..bbb0978be2
--- /dev/null
+++ b/examples/trials/efficientnet/config_local.yml
@@ -0,0 +1,18 @@
+authorName: unknown
+experimentName: example_efficient_net
+trialConcurrency: 4
+maxExecDuration: 99999d
+maxTrialNum: 100
+trainingServicePlatform: local
+searchSpacePath: search_net.json
+useAnnotation: false
+tuner:
+ codeDir: .
+ classFileName: tuner.py
+ className: FixedProductTuner
+ classArgs:
+ product: 2
+trial:
+ codeDir: EfficientNet-PyTorch
+ command: python main.py /data/imagenet -j 12 -a efficientnet --batch-size 48 --lr 0.048 --wd 1e-5 --epochs 5 --request-from-nni
+ gpuNum: 1
diff --git a/examples/trials/efficientnet/config_pai.yml b/examples/trials/efficientnet/config_pai.yml
new file mode 100644
index 0000000000..d69634c846
--- /dev/null
+++ b/examples/trials/efficientnet/config_pai.yml
@@ -0,0 +1,31 @@
+authorName: unknown
+experimentName: example_efficient_net
+trialConcurrency: 8
+maxExecDuration: 48h
+maxTrialNum: 100
+trainingServicePlatform: pai
+searchSpacePath: search_net.json
+useAnnotation: false
+tuner:
+ codeDir: .
+ classFileName: tuner.py
+ className: FixedProductTuner
+ classArgs:
+ product: 2
+trial:
+ codeDir: EfficientNet-PyTorch
+ command: sh train_imagenet.sh
+ cpuNum: 4
+ memoryMB: 25000
+ shmMB: 25000
+ gpuNum: 1
+ virtualCluster: nni
+ image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
+nniManagerIp:
+paiConfig:
+ userName:
+ token:
+ host:
diff --git a/examples/trials/efficientnet/config_net.yml b/examples/trials/efficientnet/config_paiYarn.yml
similarity index 91%
rename from examples/trials/efficientnet/config_net.yml
rename to examples/trials/efficientnet/config_paiYarn.yml
index 3ae75ef46c..5c39282211 100644
--- a/examples/trials/efficientnet/config_net.yml
+++ b/examples/trials/efficientnet/config_paiYarn.yml
@@ -3,7 +3,7 @@ experimentName: example_efficient_net
trialConcurrency: 8
maxExecDuration: 48h
maxTrialNum: 100
-trainingServicePlatform: pai
+trainingServicePlatform: paiYarn
searchSpacePath: search_net.json
useAnnotation: false
tuner:
@@ -22,7 +22,7 @@ trial:
virtualCluster: nni
image: msranni/nni:latest
nniManagerIp:
-paiConfig:
+paiYarnConfig:
userName:
passWord:
host:
diff --git a/examples/trials/efficientnet/search_net.json b/examples/trials/efficientnet/search_net.json
index bf45ba918d..360441711f 100644
--- a/examples/trials/efficientnet/search_net.json
+++ b/examples/trials/efficientnet/search_net.json
@@ -1,14 +1,14 @@
{
"alpha": {
"_type": "quniform",
- "_value": [1.0, 2.0, 0.1]
+ "_value": [1.0, 2.0, 0.05]
},
"beta": {
"_type": "quniform",
- "_value": [1.0, 1.5, 0.1]
+ "_value": [1.0, 1.5, 0.05]
},
"gamma": {
"_type": "quniform",
- "_value": [1.0, 1.5, 0.1]
+ "_value": [1.0, 1.5, 0.05]
}
}
diff --git a/examples/trials/efficientnet/tuner.py b/examples/trials/efficientnet/tuner.py
index d091d40ac0..7e5bc8b60c 100644
--- a/examples/trials/efficientnet/tuner.py
+++ b/examples/trials/efficientnet/tuner.py
@@ -14,11 +14,11 @@ def __init__(self, product):
super().__init__()
self.product = product
- def expand_parameters(self, para):
+ def _expand_parameters(self, para):
"""
Filter out all qualified parameters
"""
- para = super().expand_parameters(para)
+ para = super()._expand_parameters(para)
if all([key in para[0] for key in ["alpha", "beta", "gamma"]]): # if this is an interested set
ret_para = []
for p in para:
diff --git a/examples/trials/ga_squad/config_pai.yml b/examples/trials/ga_squad/config_pai.yml
index a2cfb8f381..1921274d32 100644
--- a/examples/trials/ga_squad/config_pai.yml
+++ b/examples/trials/ga_squad/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 32869
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
diff --git a/examples/trials/ga_squad/config_paiYarn.yml b/examples/trials/ga_squad/config_paiYarn.yml
new file mode 100644
index 0000000000..4bded4540e
--- /dev/null
+++ b/examples/trials/ga_squad/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_ga_squad
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+#choice: true, false
+useAnnotation: false
+#Your nni_manager ip
+nniManagerIp: 10.10.10.10
+tuner:
+ codeDir: ../../tuners/ga_customer_tuner
+ classFileName: customer_tuner.py
+ className: CustomerTuner
+ classArgs:
+ optimize_mode: maximize
+trial:
+ command: chmod +x ./download.sh && ./download.sh && python3 trial.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 32869
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
diff --git a/examples/trials/mnist-advisor/config_pai.yml b/examples/trials/mnist-advisor/config_pai.yml
index b26b758f79..c04b15f614 100644
--- a/examples/trials/mnist-advisor/config_pai.yml
+++ b/examples/trials/mnist-advisor/config_pai.yml
@@ -27,10 +27,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
diff --git a/examples/trials/mnist-advisor/config_paiYarn.yml b/examples/trials/mnist-advisor/config_paiYarn.yml
new file mode 100644
index 0000000000..192558a63d
--- /dev/null
+++ b/examples/trials/mnist-advisor/config_paiYarn.yml
@@ -0,0 +1,36 @@
+authorName: default
+experimentName: example_mnist_hyperband
+maxExecDuration: 1h
+maxTrialNum: 10000
+trialConcurrency: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+advisor:
+ #choice: Hyperband, BOHB
+ #(BOHB should be installed through nnictl)
+ builtinAdvisorName: Hyperband
+ classArgs:
+ #R: the maximum trial budget
+ R: 100
+ #eta: proportion of discarded trials
+ eta: 3
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 mnist.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
diff --git a/examples/trials/mnist-annotation/config_pai.yml b/examples/trials/mnist-annotation/config_pai.yml
index f8a825defd..2f8b4d00a8 100644
--- a/examples/trials/mnist-annotation/config_pai.yml
+++ b/examples/trials/mnist-annotation/config_pai.yml
@@ -22,10 +22,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/mnist-annotation/config_paiYarn.yml b/examples/trials/mnist-annotation/config_paiYarn.yml
new file mode 100644
index 0000000000..1a3299d606
--- /dev/null
+++ b/examples/trials/mnist-annotation/config_paiYarn.yml
@@ -0,0 +1,31 @@
+authorName: default
+experimentName: example_mnist
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+#choice: true, false
+useAnnotation: true
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 mnist.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/mnist-batch-tune-keras/config_pai.yml b/examples/trials/mnist-batch-tune-keras/config_pai.yml
index 69c6dd5f61..79bea33f94 100644
--- a/examples/trials/mnist-batch-tune-keras/config_pai.yml
+++ b/examples/trials/mnist-batch-tune-keras/config_pai.yml
@@ -20,10 +20,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
diff --git a/examples/trials/mnist-batch-tune-keras/config_paiYarn.yml b/examples/trials/mnist-batch-tune-keras/config_paiYarn.yml
new file mode 100644
index 0000000000..a81932285f
--- /dev/null
+++ b/examples/trials/mnist-batch-tune-keras/config_paiYarn.yml
@@ -0,0 +1,29 @@
+authorName: default
+experimentName: example_mnist-keras
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: BatchTuner
+trial:
+ command: python3 mnist-keras.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
diff --git a/examples/trials/mnist-keras/config_pai.yml b/examples/trials/mnist-keras/config_pai.yml
index aa08d0ee1c..392c53025a 100644
--- a/examples/trials/mnist-keras/config_pai.yml
+++ b/examples/trials/mnist-keras/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/mnist-keras/config_paiYarn.yml b/examples/trials/mnist-keras/config_paiYarn.yml
new file mode 100644
index 0000000000..4e5279a689
--- /dev/null
+++ b/examples/trials/mnist-keras/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_mnist-keras
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 mnist-keras.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/mnist-pytorch/config_pai.yml b/examples/trials/mnist-pytorch/config_pai.yml
index ac64bb4ce6..233ff8bdb6 100644
--- a/examples/trials/mnist-pytorch/config_pai.yml
+++ b/examples/trials/mnist-pytorch/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/mnist-pytorch/config_paiYarn.yml b/examples/trials/mnist-pytorch/config_paiYarn.yml
new file mode 100644
index 0000000000..d1aae75122
--- /dev/null
+++ b/examples/trials/mnist-pytorch/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_mnist_pytorch
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner, GPTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 mnist.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/mnist-tfv1/config_pai.yml b/examples/trials/mnist-tfv1/config_pai.yml
index c0bb710294..67df714a4b 100644
--- a/examples/trials/mnist-tfv1/config_pai.yml
+++ b/examples/trials/mnist-tfv1/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/mnist-tfv1/config_paiYarn.yml b/examples/trials/mnist-tfv1/config_paiYarn.yml
new file mode 100644
index 0000000000..886ee21c09
--- /dev/null
+++ b/examples/trials/mnist-tfv1/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_mnist
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner, GPTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 mnist.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/nas_cifar10/config_paiYarn_ppo.yml b/examples/trials/nas_cifar10/config_paiYarn_ppo.yml
new file mode 100644
index 0000000000..eb1fb8abc3
--- /dev/null
+++ b/examples/trials/nas_cifar10/config_paiYarn_ppo.yml
@@ -0,0 +1,31 @@
+authorName: Unknown
+experimentName: enas_macro
+trialConcurrency: 20
+maxExecDuration: 2400h
+maxTrialNum: 20000
+#choice: local, remote
+trainingServicePlatform: paiYarn
+#choice: true, false
+useAnnotation: true
+multiPhase: false
+versionCheck: false
+nniManagerIp: 0.0.0.0
+tuner:
+ builtinTunerName: PPOTuner
+ classArgs:
+ optimize_mode: maximize
+ trials_per_update: 60
+ epochs_per_update: 20
+ minibatch_size: 6
+trial:
+ command: sh ./macro_cifar10_pai.sh
+ codeDir: ./
+ gpuNum: 1
+ cpuNum: 1
+ memoryMB: 8196
+ image: msranni/nni:latest
+ virtualCluster: nni
+paiYarnConfig:
+ userName: your_account
+ passWord: your_passwd
+ host: 0.0.0.0
diff --git a/examples/trials/nas_cifar10/config_pai_ppo.yml b/examples/trials/nas_cifar10/config_pai_ppo.yml
index 38156376bd..f5082d87d0 100644
--- a/examples/trials/nas_cifar10/config_pai_ppo.yml
+++ b/examples/trials/nas_cifar10/config_pai_ppo.yml
@@ -25,7 +25,10 @@ trial:
memoryMB: 8196
image: msranni/nni:latest
virtualCluster: nni
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
userName: your_account
- passWord: your_pwd
+ token: your_token
host: 0.0.0.0
diff --git a/examples/trials/network_morphism/FashionMNIST/config_pai.yml b/examples/trials/network_morphism/FashionMNIST/config_pai.yml
index 3562d8dc82..db3d8be6d8 100644
--- a/examples/trials/network_morphism/FashionMNIST/config_pai.yml
+++ b/examples/trials/network_morphism/FashionMNIST/config_pai.yml
@@ -30,10 +30,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/network_morphism/FashionMNIST/config_paiYarn.yml b/examples/trials/network_morphism/FashionMNIST/config_paiYarn.yml
new file mode 100644
index 0000000000..e15ec2ebec
--- /dev/null
+++ b/examples/trials/network_morphism/FashionMNIST/config_paiYarn.yml
@@ -0,0 +1,39 @@
+authorName: default
+experimentName: example_FashionMNIST-network-morphism
+trialConcurrency: 1
+maxExecDuration: 24h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, NetworkMorphism
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: NetworkMorphism
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+ # for now, this tuner only supports cv domain
+ task: cv
+ #input image width
+ input_width: 28
+ #input image channel
+ input_channel: 1
+ #number of classes
+ n_output_node: 10
+trial:
+ command: python3 FashionMNIST_keras.py
+ codeDir: .
+ gpuNum: 1
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/network_morphism/cifar10/config_pai.yml b/examples/trials/network_morphism/cifar10/config_pai.yml
index e14caab934..e2e39c7a4b 100644
--- a/examples/trials/network_morphism/cifar10/config_pai.yml
+++ b/examples/trials/network_morphism/cifar10/config_pai.yml
@@ -30,10 +30,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/network_morphism/cifar10/config_paiYarn.yml b/examples/trials/network_morphism/cifar10/config_paiYarn.yml
new file mode 100644
index 0000000000..3367aa4e36
--- /dev/null
+++ b/examples/trials/network_morphism/cifar10/config_paiYarn.yml
@@ -0,0 +1,39 @@
+authorName: default
+experimentName: example_cifar10-network-morphism
+trialConcurrency: 1
+maxExecDuration: 24h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, NetworkMorphism
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: NetworkMorphism
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+ # for now, this tuner only supports cv domain
+ task: cv
+ #input image width
+ input_width: 32
+ #input image channel
+ input_channel: 3
+ #number of classes
+ n_output_node: 10
+trial:
+ command: python3 cifar10_keras.py
+ codeDir: .
+ gpuNum: 1
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/sklearn/classification/config_pai.yml b/examples/trials/sklearn/classification/config_pai.yml
index d3ffdc8d74..6600894ccb 100644
--- a/examples/trials/sklearn/classification/config_pai.yml
+++ b/examples/trials/sklearn/classification/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/sklearn/classification/config_paiYarn.yml b/examples/trials/sklearn/classification/config_paiYarn.yml
new file mode 100644
index 0000000000..9bec9a4c50
--- /dev/null
+++ b/examples/trials/sklearn/classification/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_sklearn
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 100
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner,MetisTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 main.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/sklearn/regression/config_pai.yml b/examples/trials/sklearn/regression/config_pai.yml
index b6d84f2f6d..d4f7491b6e 100644
--- a/examples/trials/sklearn/regression/config_pai.yml
+++ b/examples/trials/sklearn/regression/config_pai.yml
@@ -23,10 +23,13 @@ trial:
memoryMB: 8196
#The docker image to run nni job on pai
image: msranni/nni:latest
+ nniManagerNFSMountPath: /home/user/mnt
+ containerNFSMountPath: /mnt/data/user
+ paiStoragePlugin: team_wise
paiConfig:
#The username to login pai
userName: username
- #The password to login pai
- passWord: password
+ #The token to login pai
+ token: token
#The host of restful server of pai
host: 10.10.10.10
\ No newline at end of file
diff --git a/examples/trials/sklearn/regression/config_paiYarn.yml b/examples/trials/sklearn/regression/config_paiYarn.yml
new file mode 100644
index 0000000000..0e0e73a0ab
--- /dev/null
+++ b/examples/trials/sklearn/regression/config_paiYarn.yml
@@ -0,0 +1,32 @@
+authorName: default
+experimentName: example_sklearn
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 100
+#choice: local, remote, pai
+trainingServicePlatform: paiYarn
+searchSpacePath: search_space.json
+#choice: true, false
+useAnnotation: false
+tuner:
+ #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner
+ #SMAC (SMAC should be installed through nnictl)
+ builtinTunerName: TPE
+ classArgs:
+ #choice: maximize, minimize
+ optimize_mode: maximize
+trial:
+ command: python3 main.py
+ codeDir: .
+ gpuNum: 0
+ cpuNum: 1
+ memoryMB: 8196
+ #The docker image to run nni job on pai
+ image: msranni/nni:latest
+paiYarnConfig:
+ #The username to login pai
+ userName: username
+ #The password to login pai
+ passWord: password
+ #The host of restful server of pai
+ host: 10.10.10.10
\ No newline at end of file
diff --git a/src/nni_manager/common/log.ts b/src/nni_manager/common/log.ts
index 31a38d6e07..9bf2b92c88 100644
--- a/src/nni_manager/common/log.ts
+++ b/src/nni_manager/common/log.ts
@@ -4,13 +4,11 @@
'use strict';
import * as fs from 'fs';
-import * as path from 'path';
import { Writable } from 'stream';
import { WritableStreamBuffer } from 'stream-buffers';
import { format } from 'util';
import * as component from '../common/component';
import { getExperimentStartupInfo, isReadonly } from './experimentStartupInfo';
-import { getLogDir } from './utils';
const FATAL: number = 1;
const ERROR: number = 2;
@@ -55,23 +53,21 @@ class BufferSerialEmitter {
@component.Singleton
class Logger {
- private DEFAULT_LOGFILE: string = path.join(getLogDir(), 'nnimanager.log');
private level: number = INFO;
- private bufferSerialEmitter: BufferSerialEmitter;
- private writable: Writable;
+ private bufferSerialEmitter?: BufferSerialEmitter;
+ private writable?: Writable;
private readonly: boolean = false;
constructor(fileName?: string) {
- let logFile: string | undefined = fileName;
- if (logFile === undefined) {
- logFile = this.DEFAULT_LOGFILE;
+ const logFile: string | undefined = fileName;
+ if (logFile) {
+ this.writable = fs.createWriteStream(logFile, {
+ flags: 'a+',
+ encoding: 'utf8',
+ autoClose: true
+ });
+ this.bufferSerialEmitter = new BufferSerialEmitter(this.writable);
}
- this.writable = fs.createWriteStream(logFile, {
- flags: 'a+',
- encoding: 'utf8',
- autoClose: true
- });
- this.bufferSerialEmitter = new BufferSerialEmitter(this.writable);
const logLevelName: string = getExperimentStartupInfo()
.getLogLevel();
@@ -84,7 +80,9 @@ class Logger {
}
public close(): void {
- this.writable.destroy();
+ if (this.writable) {
+ this.writable.destroy();
+ }
}
public trace(...param: any[]): void {
@@ -128,12 +126,15 @@ class Logger {
*/
private log(level: string, param: any[]): void {
if (!this.readonly) {
- const buffer: WritableStreamBuffer = new WritableStreamBuffer();
- buffer.write(`[${(new Date()).toLocaleString()}] ${level} `);
- buffer.write(format(param));
- buffer.write('\n');
- buffer.end();
- this.bufferSerialEmitter.feed(buffer.getContents());
+ const logContent = `[${(new Date()).toLocaleString()}] ${level} ${format(param)}\n`;
+ if (this.writable && this.bufferSerialEmitter) {
+ const buffer: WritableStreamBuffer = new WritableStreamBuffer();
+ buffer.write(logContent);
+ buffer.end();
+ this.bufferSerialEmitter.feed(buffer.getContents());
+ } else {
+ console.log(logContent);
+ }
}
}
}
diff --git a/src/nni_manager/main.ts b/src/nni_manager/main.ts
index f707304382..51f964756e 100644
--- a/src/nni_manager/main.ts
+++ b/src/nni_manager/main.ts
@@ -6,6 +6,7 @@
import { Container, Scope } from 'typescript-ioc';
import * as fs from 'fs';
+import * as path from 'path';
import * as component from './common/component';
import { Database, DataStore } from './common/datastore';
import { setExperimentStartupInfo } from './common/experimentStartupInfo';
@@ -34,7 +35,7 @@ function initStartupInfo(
setExperimentStartupInfo(createNew, expId, basePort, logDirectory, experimentLogLevel, readonly);
}
-async function initContainer(platformMode: string, logFileName?: string): Promise {
+async function initContainer(foreground: boolean, platformMode: string, logFileName?: string): Promise {
if (platformMode === 'local') {
Container.bind(TrainingService)
.to(LocalTrainingService)
@@ -71,6 +72,12 @@ async function initContainer(platformMode: string, logFileName?: string): Promis
Container.bind(DataStore)
.to(NNIDataStore)
.scope(Scope.Singleton);
+ const DEFAULT_LOGFILE: string = path.join(getLogDir(), 'nnimanager.log');
+ if (foreground) {
+ logFileName = undefined;
+ } else if (logFileName === undefined) {
+ logFileName = DEFAULT_LOGFILE;
+ }
Container.bind(Logger).provider({
get: (): Logger => new Logger(logFileName)
});
@@ -81,7 +88,7 @@ async function initContainer(platformMode: string, logFileName?: string): Promis
function usage(): void {
console.info('usage: node main.js --port --mode \
- --start_mode --experiment_id ');
+ --start_mode --experiment_id --foreground ');
}
const strPort: string = parseArg(['--port', '-p']);
@@ -90,6 +97,14 @@ if (!strPort || strPort.length === 0) {
process.exit(1);
}
+const foregroundArg: string = parseArg(['--foreground', '-f']);
+if (!('true' || 'false').includes(foregroundArg.toLowerCase())) {
+ console.log(`FATAL: foreground property should only be true or false`);
+ usage();
+ process.exit(1);
+}
+const foreground: boolean = foregroundArg.toLowerCase() === 'true' ? true : false;
+
const port: number = parseInt(strPort, 10);
const mode: string = parseArg(['--mode', '-m']);
@@ -138,7 +153,7 @@ initStartupInfo(startMode, experimentId, port, logDir, logLevel, readonly);
mkDirP(getLogDir())
.then(async () => {
try {
- await initContainer(mode);
+ await initContainer(foreground, mode);
const restServer: NNIRestServer = component.get(NNIRestServer);
await restServer.start();
const log: Logger = getLogger();
@@ -162,6 +177,15 @@ function getStopSignal(): any {
}
}
+function getCtrlCSignal(): any {
+ return 'SIGINT';
+}
+
+process.on(getCtrlCSignal(), async () => {
+ const log: Logger = getLogger();
+ log.info(`Get SIGINT signal!`);
+});
+
process.on(getStopSignal(), async () => {
const log: Logger = getLogger();
let hasError: boolean = false;
diff --git a/src/nni_manager/package.json b/src/nni_manager/package.json
index 93e77cdf48..9b71067467 100644
--- a/src/nni_manager/package.json
+++ b/src/nni_manager/package.json
@@ -13,6 +13,7 @@
"azure-storage": "^2.10.2",
"chai-as-promised": "^7.1.1",
"child-process-promise": "^2.2.1",
+ "deepmerge": "^4.2.2",
"express": "^4.16.3",
"express-joi-validator": "^2.0.0",
"js-base64": "^2.4.9",
diff --git a/src/nni_manager/rest_server/restValidationSchemas.ts b/src/nni_manager/rest_server/restValidationSchemas.ts
index a9ad8cfd9a..c7fa694fb9 100644
--- a/src/nni_manager/rest_server/restValidationSchemas.ts
+++ b/src/nni_manager/rest_server/restValidationSchemas.ts
@@ -38,6 +38,7 @@ export namespace ValidationSchemas {
authFile: joi.string(),
nniManagerNFSMountPath: joi.string().min(1),
containerNFSMountPath: joi.string().min(1),
+ paiConfigPath: joi.string(),
paiStoragePlugin: joi.string().min(1),
nasMode: joi.string().valid('classic_mode', 'enas_mode', 'oneshot_mode', 'darts_mode'),
portList: joi.array().items(joi.object({
diff --git a/src/nni_manager/training_service/pai/paiK8S/paiK8SConfig.ts b/src/nni_manager/training_service/pai/paiK8S/paiK8SConfig.ts
index 70f175683e..26ad2901bd 100644
--- a/src/nni_manager/training_service/pai/paiK8S/paiK8SConfig.ts
+++ b/src/nni_manager/training_service/pai/paiK8S/paiK8SConfig.ts
@@ -31,10 +31,11 @@ export class NNIPAIK8STrialConfig extends TrialConfig {
public readonly nniManagerNFSMountPath: string;
public readonly containerNFSMountPath: string;
public readonly paiStoragePlugin: string;
+ public readonly paiConfigPath?: string;
constructor(command: string, codeDir: string, gpuNum: number, cpuNum: number, memoryMB: number,
image: string, nniManagerNFSMountPath: string, containerNFSMountPath: string,
- paiStoragePlugin: string, virtualCluster?: string) {
+ paiStoragePlugin: string, virtualCluster?: string, paiConfigPath?: string) {
super(command, codeDir, gpuNum);
this.cpuNum = cpuNum;
this.memoryMB = memoryMB;
@@ -43,5 +44,6 @@ export class NNIPAIK8STrialConfig extends TrialConfig {
this.nniManagerNFSMountPath = nniManagerNFSMountPath;
this.containerNFSMountPath = containerNFSMountPath;
this.paiStoragePlugin = paiStoragePlugin;
+ this.paiConfigPath = paiConfigPath;
}
}
diff --git a/src/nni_manager/training_service/pai/paiK8S/paiK8STrainingService.ts b/src/nni_manager/training_service/pai/paiK8S/paiK8STrainingService.ts
index fc64d4dbdc..263009719c 100644
--- a/src/nni_manager/training_service/pai/paiK8S/paiK8STrainingService.ts
+++ b/src/nni_manager/training_service/pai/paiK8S/paiK8STrainingService.ts
@@ -44,6 +44,7 @@ import { PAIClusterConfig, PAITrialJobDetail } from '../paiConfig';
import { PAIJobRestServer } from '../paiJobRestServer';
const yaml = require('js-yaml');
+const deepmerge = require('deepmerge');
/**
* Training Service implementation for OpenPAI (Open Platform for AI)
@@ -59,6 +60,10 @@ class PAIK8STrainingService extends PAITrainingService {
public async setClusterMetadata(key: string, value: string): Promise {
switch (key) {
+ case TrialConfigMetadataKey.NNI_MANAGER_IP:
+ this.nniManagerIpConfig = JSON.parse(value);
+ break;
+
case TrialConfigMetadataKey.PAI_CLUSTER_CONFIG:
this.paiJobRestServer = new PAIJobRestServer(component.get(PAIK8STrainingService));
this.paiClusterConfig = JSON.parse(value);
@@ -185,7 +190,19 @@ class PAIK8STrainingService extends PAITrainingService {
}
}
- return yaml.safeDump(paiJobConfig);
+ if (this.paiTrialConfig.paiConfigPath) {
+ try {
+ const additionalPAIConfig = yaml.safeLoad(fs.readFileSync(this.paiTrialConfig.paiConfigPath, 'utf8'));
+ //deepmerge(x, y), if an element at the same key is present for both x and y, the value from y will appear in the result.
+ //refer: https://github.com/TehShrike/deepmerge
+ const overwriteMerge = (destinationArray: any, sourceArray: any, options: any) => sourceArray;
+ return yaml.safeDump(deepmerge(additionalPAIConfig, paiJobConfig, { arrayMerge: overwriteMerge }));
+ } catch (error) {
+ this.log.error(`Error occurs during loading and merge ${this.paiTrialConfig.paiConfigPath} : ${error}`);
+ }
+ } else {
+ return yaml.safeDump(paiJobConfig);
+ }
}
protected async submitTrialJobToPAI(trialJobId: string): Promise {
@@ -254,7 +271,7 @@ class PAIK8STrainingService extends PAITrainingService {
this.log.info(`nniPAItrial command is ${nniPaiTrialCommand.trim()}`);
const paiJobConfig = this.generateJobConfigInYamlFormat(trialJobId, nniPaiTrialCommand);
-
+ this.log.debug(paiJobConfig);
// Step 3. Submit PAI job via Rest call
// Refer https://github.com/Microsoft/pai/blob/master/docs/rest-server/API.md for more detail about PAI Rest API
const submitJobRequest: request.Options = {
diff --git a/src/nni_manager/yarn.lock b/src/nni_manager/yarn.lock
index 379af7c4b9..ae9c5f6d99 100644
--- a/src/nni_manager/yarn.lock
+++ b/src/nni_manager/yarn.lock
@@ -1112,6 +1112,11 @@ deepmerge@^2.1.1:
version "2.2.1"
resolved "https://registry.yarnpkg.com/deepmerge/-/deepmerge-2.2.1.tgz#5d3ff22a01c00f645405a2fbc17d0778a1801170"
+deepmerge@^4.2.2:
+ version "4.2.2"
+ resolved "https://registry.yarnpkg.com/deepmerge/-/deepmerge-4.2.2.tgz#44d2ea3679b8f4d4ffba33f03d865fc1e7bf4955"
+ integrity sha512-FJ3UgI4gIl+PHZm53knsuSFpE+nESMr7M4v9QcgB7S63Kj/6WqMiFQJpBBYz1Pt+66bZpP3Q7Lye0Oo9MPKEdg==
+
default-require-extensions@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/default-require-extensions/-/default-require-extensions-2.0.0.tgz#f5f8fbb18a7d6d50b21f641f649ebb522cfe24f7"
diff --git a/src/sdk/pynni/nni/compression/torch/pruners.py b/src/sdk/pynni/nni/compression/torch/pruners.py
index 15c6b78262..b49ed77f2f 100644
--- a/src/sdk/pynni/nni/compression/torch/pruners.py
+++ b/src/sdk/pynni/nni/compression/torch/pruners.py
@@ -113,13 +113,13 @@ def calc_mask(self, layer, config, **kwargs):
if not (self.now_epoch >= start_epoch and (self.now_epoch - start_epoch) % freq == 0):
return None
- mask = {'weight': torch.ones(weight.shape).type_as(weight)}
+ mask = {'weight': kwargs['weight_mask'] if 'weight_mask' in kwargs else torch.ones(weight.shape).type_as(weight)}
target_sparsity = self.compute_target_sparsity(config)
k = int(weight.numel() * target_sparsity)
if k == 0 or target_sparsity >= 1 or target_sparsity <= 0:
return mask
# if we want to generate new mask, we should update weigth first
- w_abs = weight.abs()
+ w_abs = weight.abs() * mask['weight']
threshold = torch.topk(w_abs.view(-1), k, largest=False)[0].max()
new_mask = {'weight': torch.gt(w_abs, threshold).type_as(weight)}
if_calculated.copy_(torch.tensor(1)) # pylint: disable=not-callable
diff --git a/src/sdk/pynni/nni/medianstop_assessor/test.py b/src/sdk/pynni/nni/medianstop_assessor/test.py
index bad19911c2..8c7d6d927b 100644
--- a/src/sdk/pynni/nni/medianstop_assessor/test.py
+++ b/src/sdk/pynni/nni/medianstop_assessor/test.py
@@ -31,11 +31,11 @@ def test():
# [1,1,1,1,1,1,1,1,1,1],
# [1,1,1,1,1,1,1,1,1,1]]
- assessor = MedianstopAssessor(FLAGS.start_step, FLAGS.optimize_mode)
- for i in range(4):
+ assessor = MedianstopAssessor(FLAGS.optimize_mode, FLAGS.start_step)
+ for i in range(len(lcs)):
#lc = []
to_complete = True
- for k in range(10):
+ for k in range(len(lcs[0])):
#d = random.randint(i*100+0, i*100+100)
#lc.append(d)
ret = assessor.assess_trial(i, lcs[i][:k+1])
diff --git a/src/sdk/pynni/nni/nas/pytorch/classic_nas/mutator.py b/src/sdk/pynni/nni/nas/pytorch/classic_nas/mutator.py
index e1a0b390f6..3e11ef5b60 100644
--- a/src/sdk/pynni/nni/nas/pytorch/classic_nas/mutator.py
+++ b/src/sdk/pynni/nni/nas/pytorch/classic_nas/mutator.py
@@ -67,6 +67,13 @@ def __init__(self, model):
else:
# get chosen arch from tuner
self._chosen_arch = nni.get_next_parameter()
+ if self._chosen_arch is None:
+ if trial_env_vars.NNI_PLATFORM == "unittest":
+ # happens if NNI_PLATFORM is intentionally set, e.g., in UT
+ logger.warning("`NNI_PLATFORM` is set but `param` is None. Falling back to standalone mode.")
+ self._chosen_arch = self._standalone_generate_chosen()
+ else:
+ raise RuntimeError("Chosen architecture is None. This may be a platform error.")
self.reset()
def _sample_layer_choice(self, mutable, idx, value, search_space_item):
@@ -162,6 +169,8 @@ def _standalone_generate_chosen(self):
elif val["_type"] == INPUT_CHOICE:
choices = val["_value"]["candidates"]
n_chosen = val["_value"]["n_chosen"]
+ if n_chosen is None:
+ n_chosen = len(choices)
chosen_arch[key] = {"_value": choices[:n_chosen], "_idx": list(range(n_chosen))}
else:
raise ValueError("Unknown key '%s' and value '%s'." % (key, val))
diff --git a/src/sdk/pynni/nni/nas/pytorch/darts/mutator.py b/src/sdk/pynni/nni/nas/pytorch/darts/mutator.py
index b3a21f3a31..2aba20dd45 100644
--- a/src/sdk/pynni/nni/nas/pytorch/darts/mutator.py
+++ b/src/sdk/pynni/nni/nas/pytorch/darts/mutator.py
@@ -63,18 +63,23 @@ def sample_final(self):
edges_max[mutable.key] = max_val
result[mutable.key] = F.one_hot(index, num_classes=mutable.length).view(-1).bool()
for mutable in self.mutables:
- if isinstance(mutable, InputChoice) and mutable.n_chosen is not None:
- weights = []
- for src_key in mutable.choose_from:
- if src_key not in edges_max:
- _logger.warning("InputChoice.NO_KEY in '%s' is weighted 0 when selecting inputs.", mutable.key)
- weights.append(edges_max.get(src_key, 0.))
- weights = torch.tensor(weights) # pylint: disable=not-callable
- _, topk_edge_indices = torch.topk(weights, mutable.n_chosen)
- selected_multihot = []
- for i, src_key in enumerate(mutable.choose_from):
- if i not in topk_edge_indices and src_key in result:
- result[src_key] = torch.zeros_like(result[src_key]) # clear this choice to optimize calc graph
- selected_multihot.append(i in topk_edge_indices)
- result[mutable.key] = torch.tensor(selected_multihot, dtype=torch.bool, device=self.device()) # pylint: disable=not-callable
+ if isinstance(mutable, InputChoice):
+ if mutable.n_chosen is not None:
+ weights = []
+ for src_key in mutable.choose_from:
+ if src_key not in edges_max:
+ _logger.warning("InputChoice.NO_KEY in '%s' is weighted 0 when selecting inputs.", mutable.key)
+ weights.append(edges_max.get(src_key, 0.))
+ weights = torch.tensor(weights) # pylint: disable=not-callable
+ _, topk_edge_indices = torch.topk(weights, mutable.n_chosen)
+ selected_multihot = []
+ for i, src_key in enumerate(mutable.choose_from):
+ if i not in topk_edge_indices and src_key in result:
+ # If an edge is never selected, there is no need to calculate any op on this edge.
+ # This is to eliminate redundant calculation.
+ result[src_key] = torch.zeros_like(result[src_key])
+ selected_multihot.append(i in topk_edge_indices)
+ result[mutable.key] = torch.tensor(selected_multihot, dtype=torch.bool, device=self.device()) # pylint: disable=not-callable
+ else:
+ result[mutable.key] = torch.ones(mutable.n_candidates, dtype=torch.bool, device=self.device()) # pylint: disable=not-callable
return result
diff --git a/src/sdk/pynni/nni/nas/pytorch/fixed.py b/src/sdk/pynni/nni/nas/pytorch/fixed.py
index 593a60ae44..9883baa07b 100644
--- a/src/sdk/pynni/nni/nas/pytorch/fixed.py
+++ b/src/sdk/pynni/nni/nas/pytorch/fixed.py
@@ -52,24 +52,24 @@ def _encode_tensor(data):
return data
-def apply_fixed_architecture(model, fixed_arc_path):
+def apply_fixed_architecture(model, fixed_arc):
"""
- Load architecture from `fixed_arc_path` and apply to model.
+ Load architecture from `fixed_arc` and apply to model.
Parameters
----------
model : torch.nn.Module
Model with mutables.
- fixed_arc_path : str
- Path to the JSON that stores the architecture.
+ fixed_arc : str or dict
+ Path to the JSON that stores the architecture, or dict that stores the exported architecture.
Returns
-------
FixedArchitecture
"""
- if isinstance(fixed_arc_path, str):
- with open(fixed_arc_path, "r") as f:
+ if isinstance(fixed_arc, str):
+ with open(fixed_arc) as f:
fixed_arc = json.load(f)
fixed_arc = _encode_tensor(fixed_arc)
architecture = FixedArchitecture(model, fixed_arc)
diff --git a/src/sdk/pynni/nni/nas/pytorch/utils.py b/src/sdk/pynni/nni/nas/pytorch/utils.py
index 007c28a902..44f220b723 100644
--- a/src/sdk/pynni/nni/nas/pytorch/utils.py
+++ b/src/sdk/pynni/nni/nas/pytorch/utils.py
@@ -17,6 +17,14 @@ def global_mutable_counting():
return _counter
+def _reset_global_mutable_counting():
+ """
+ Reset the global mutable counting to count from 1. Useful when defining multiple models with default keys.
+ """
+ global _counter
+ _counter = 0
+
+
def to_device(obj, device):
if torch.is_tensor(obj):
return obj.to(device)
diff --git a/src/sdk/pynni/tests/models/pytorch_models/__init__.py b/src/sdk/pynni/tests/models/pytorch_models/__init__.py
new file mode 100644
index 0000000000..46d4482c86
--- /dev/null
+++ b/src/sdk/pynni/tests/models/pytorch_models/__init__.py
@@ -0,0 +1,6 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT license.
+
+from .mutable_scope import SpaceWithMutableScope
+from .naive import NaiveSearchSpace
+from .nested import NestedSpace
diff --git a/src/sdk/pynni/tests/models/pytorch_models/mutable_scope.py b/src/sdk/pynni/tests/models/pytorch_models/mutable_scope.py
new file mode 100644
index 0000000000..505a14880f
--- /dev/null
+++ b/src/sdk/pynni/tests/models/pytorch_models/mutable_scope.py
@@ -0,0 +1,95 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT license.
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from nni.nas.pytorch.mutables import LayerChoice, InputChoice, MutableScope
+
+
+class Cell(MutableScope):
+ def __init__(self, cell_name, prev_labels, channels):
+ super().__init__(cell_name)
+ self.input_choice = InputChoice(choose_from=prev_labels, n_chosen=1, return_mask=True,
+ key=cell_name + "_input")
+ self.op_choice = LayerChoice([
+ nn.Conv2d(channels, channels, 3, padding=1),
+ nn.Conv2d(channels, channels, 5, padding=2),
+ nn.MaxPool2d(3, stride=1, padding=1),
+ nn.AvgPool2d(3, stride=1, padding=1),
+ nn.Identity()
+ ], key=cell_name + "_op")
+
+ def forward(self, prev_layers):
+ chosen_input, chosen_mask = self.input_choice(prev_layers)
+ cell_out = self.op_choice(chosen_input)
+ return cell_out, chosen_mask
+
+
+class Node(MutableScope):
+ def __init__(self, node_name, prev_node_names, channels):
+ super().__init__(node_name)
+ self.cell_x = Cell(node_name + "_x", prev_node_names, channels)
+ self.cell_y = Cell(node_name + "_y", prev_node_names, channels)
+
+ def forward(self, prev_layers):
+ out_x, mask_x = self.cell_x(prev_layers)
+ out_y, mask_y = self.cell_y(prev_layers)
+ return out_x + out_y, mask_x | mask_y
+
+
+class Layer(nn.Module):
+ def __init__(self, num_nodes, channels):
+ super().__init__()
+ self.num_nodes = num_nodes
+ self.nodes = nn.ModuleList()
+ node_labels = [InputChoice.NO_KEY, InputChoice.NO_KEY]
+ for i in range(num_nodes):
+ node_labels.append("node_{}".format(i))
+ self.nodes.append(Node(node_labels[-1], node_labels[:-1], channels))
+ self.final_conv_w = nn.Parameter(torch.zeros(channels, self.num_nodes + 2, channels, 1, 1),
+ requires_grad=True)
+ self.bn = nn.BatchNorm2d(channels, affine=False)
+
+ def forward(self, pprev, prev):
+ prev_nodes_out = [pprev, prev]
+ nodes_used_mask = torch.zeros(self.num_nodes + 2, dtype=torch.bool, device=prev.device)
+ for i in range(self.num_nodes):
+ node_out, mask = self.nodes[i](prev_nodes_out)
+ nodes_used_mask[:mask.size(0)] |= mask.to(prev.device)
+ # NOTE: which device should we put mask on?
+ prev_nodes_out.append(node_out)
+
+ unused_nodes = torch.cat([out for used, out in zip(nodes_used_mask, prev_nodes_out) if not used], 1)
+ unused_nodes = F.relu(unused_nodes)
+ conv_weight = self.final_conv_w[:, ~nodes_used_mask, :, :, :]
+ conv_weight = conv_weight.view(conv_weight.size(0), -1, 1, 1)
+ out = F.conv2d(unused_nodes, conv_weight)
+ return prev, self.bn(out)
+
+
+class SpaceWithMutableScope(nn.Module):
+ def __init__(self, test_case, num_layers=4, num_nodes=5, channels=16, in_channels=3, num_classes=10):
+ super().__init__()
+ self.test_case = test_case
+ self.num_layers = num_layers
+
+ self.stem = nn.Sequential(
+ nn.Conv2d(in_channels, channels, 3, 1, 1, bias=False),
+ nn.BatchNorm2d(channels)
+ )
+
+ self.layers = nn.ModuleList()
+ for _ in range(self.num_layers + 2):
+ self.layers.append(Layer(num_nodes, channels))
+ self.gap = nn.AdaptiveAvgPool2d(1)
+ self.dense = nn.Linear(channels, num_classes)
+
+ def forward(self, x):
+ prev = cur = self.stem(x)
+ for layer in self.layers:
+ prev, cur = layer(prev, cur)
+
+ cur = self.gap(F.relu(cur)).view(x.size(0), -1)
+ return self.dense(cur)
diff --git a/src/sdk/pynni/tests/models/pytorch_models/naive.py b/src/sdk/pynni/tests/models/pytorch_models/naive.py
new file mode 100644
index 0000000000..0555ec17e4
--- /dev/null
+++ b/src/sdk/pynni/tests/models/pytorch_models/naive.py
@@ -0,0 +1,45 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT license.
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from nni.nas.pytorch.mutables import LayerChoice, InputChoice
+
+
+class NaiveSearchSpace(nn.Module):
+ def __init__(self, test_case):
+ super().__init__()
+ self.test_case = test_case
+ self.conv1 = LayerChoice([nn.Conv2d(3, 6, 3, padding=1), nn.Conv2d(3, 6, 5, padding=2)])
+ self.pool = nn.MaxPool2d(2, 2)
+ self.conv2 = LayerChoice([nn.Conv2d(6, 16, 3, padding=1), nn.Conv2d(6, 16, 5, padding=2)],
+ return_mask=True)
+ self.conv3 = nn.Conv2d(16, 16, 1)
+
+ self.skipconnect = InputChoice(n_candidates=1)
+ self.skipconnect2 = InputChoice(n_candidates=2, return_mask=True)
+ self.bn = nn.BatchNorm2d(16)
+
+ self.gap = nn.AdaptiveAvgPool2d(1)
+ self.fc = nn.Linear(16, 10)
+
+ def forward(self, x):
+ bs = x.size(0)
+
+ x = self.pool(F.relu(self.conv1(x)))
+ x0, mask = self.conv2(x)
+ self.test_case.assertEqual(mask.size(), torch.Size([2]))
+ x1 = F.relu(self.conv3(x0))
+
+ _, mask = self.skipconnect2([x0, x1])
+ x0 = self.skipconnect([x0])
+ if x0 is not None:
+ x1 += x0
+ x = self.pool(self.bn(x1))
+ self.test_case.assertEqual(mask.size(), torch.Size([2]))
+
+ x = self.gap(x).view(bs, -1)
+ x = self.fc(x)
+ return x
diff --git a/src/sdk/pynni/tests/models/pytorch_models/nested.py b/src/sdk/pynni/tests/models/pytorch_models/nested.py
new file mode 100644
index 0000000000..71e1ccf2c3
--- /dev/null
+++ b/src/sdk/pynni/tests/models/pytorch_models/nested.py
@@ -0,0 +1,34 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT license.
+
+import torch.nn as nn
+import torch.nn.functional as F
+
+from nni.nas.pytorch.mutables import LayerChoice, InputChoice
+
+
+class MutableOp(nn.Module):
+ def __init__(self, kernel_size):
+ super().__init__()
+ self.conv = nn.Conv2d(3, 120, kernel_size, padding=kernel_size // 2)
+ self.nested_mutable = InputChoice(n_candidates=10)
+
+ def forward(self, x):
+ return self.conv(x)
+
+
+class NestedSpace(nn.Module):
+ # this doesn't pass tests
+ def __init__(self, test_case):
+ super().__init__()
+ self.test_case = test_case
+ self.conv1 = LayerChoice([MutableOp(3), MutableOp(5)])
+ self.gap = nn.AdaptiveAvgPool2d(1)
+ self.fc1 = nn.Linear(120, 10)
+
+ def forward(self, x):
+ bs = x.size(0)
+ x = F.relu(self.conv1(x))
+ x = self.gap(x).view(bs, -1)
+ x = self.fc(x)
+ return x
diff --git a/src/sdk/pynni/tests/test_nas.py b/src/sdk/pynni/tests/test_nas.py
new file mode 100644
index 0000000000..53b52541ad
--- /dev/null
+++ b/src/sdk/pynni/tests/test_nas.py
@@ -0,0 +1,106 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT license.
+import importlib
+import os
+import sys
+from unittest import TestCase, main
+
+import torch
+import torch.nn as nn
+from nni.nas.pytorch.classic_nas import get_and_apply_next_architecture
+from nni.nas.pytorch.darts import DartsMutator
+from nni.nas.pytorch.enas import EnasMutator
+from nni.nas.pytorch.fixed import apply_fixed_architecture
+from nni.nas.pytorch.random import RandomMutator
+from nni.nas.pytorch.utils import _reset_global_mutable_counting
+
+
+class NasTestCase(TestCase):
+
+ def setUp(self):
+ self.default_input_size = [3, 32, 32]
+ self.model_path = os.path.join(os.path.dirname(__file__), "models")
+ sys.path.append(self.model_path)
+ self.model_module = importlib.import_module("pytorch_models")
+ self.default_cls = [self.model_module.NaiveSearchSpace, self.model_module.SpaceWithMutableScope]
+ self.cuda_test = [0]
+ if torch.cuda.is_available():
+ self.cuda_test.append(1)
+ if torch.cuda.device_count() > 1:
+ self.cuda_test.append(torch.cuda.device_count())
+
+ def tearDown(self):
+ sys.path.remove(self.model_path)
+
+ def iterative_sample_and_forward(self, model, mutator=None, input_size=None, n_iters=20, test_backward=True,
+ use_cuda=False):
+ if input_size is None:
+ input_size = self.default_input_size
+ # support pytorch only
+ input_size = [8 if use_cuda else 2] + input_size # at least 2 samples to enable batch norm
+ for _ in range(n_iters):
+ for param in model.parameters():
+ param.grad = None
+ if mutator is not None:
+ mutator.reset()
+ x = torch.randn(input_size)
+ if use_cuda:
+ x = x.cuda()
+ y = torch.sum(model(x))
+ if test_backward:
+ y.backward()
+
+ def default_mutator_test_pipeline(self, mutator_cls):
+ for model_cls in self.default_cls:
+ for cuda_test in self.cuda_test:
+ _reset_global_mutable_counting()
+ model = model_cls(self)
+ mutator = mutator_cls(model)
+ if cuda_test:
+ model.cuda()
+ mutator.cuda()
+ if cuda_test > 1:
+ model = nn.DataParallel(model)
+ self.iterative_sample_and_forward(model, mutator, use_cuda=cuda_test)
+ _reset_global_mutable_counting()
+ model_fixed = model_cls(self)
+ if cuda_test:
+ model_fixed.cuda()
+ if cuda_test > 1:
+ model_fixed = nn.DataParallel(model_fixed)
+ with torch.no_grad():
+ arc = mutator.export()
+ apply_fixed_architecture(model_fixed, arc)
+ self.iterative_sample_and_forward(model_fixed, n_iters=1, use_cuda=cuda_test)
+
+ def test_random_mutator(self):
+ self.default_mutator_test_pipeline(RandomMutator)
+
+ def test_enas_mutator(self):
+ self.default_mutator_test_pipeline(EnasMutator)
+
+ def test_darts_mutator(self):
+ # DARTS doesn't support DataParallel. To be fixed.
+ self.cuda_test = [t for t in self.cuda_test if t <= 1]
+ self.default_mutator_test_pipeline(DartsMutator)
+
+ def test_apply_twice(self):
+ model = self.model_module.NaiveSearchSpace(self)
+ with self.assertRaises(RuntimeError):
+ for _ in range(2):
+ RandomMutator(model)
+
+ def test_nested_space(self):
+ model = self.model_module.NestedSpace(self)
+ with self.assertRaises(RuntimeError):
+ RandomMutator(model)
+
+ def test_classic_nas(self):
+ for model_cls in self.default_cls:
+ model = model_cls(self)
+ get_and_apply_next_architecture(model)
+ self.iterative_sample_and_forward(model)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/test/config_test.py b/test/config_test.py
index 1db4bf086d..91136a8a95 100644
--- a/test/config_test.py
+++ b/test/config_test.py
@@ -29,6 +29,12 @@ def gen_new_config(config_file, training_service='local'):
config['trial'].pop('command')
if 'gpuNum' in config['trial']:
config['trial'].pop('gpuNum')
+
+ if training_service == 'frameworkcontroller':
+ it_config[training_service]['trial']['taskRoles'][0]['command'] = config['trial']['command']
+ config['trial'].pop('command')
+ if 'gpuNum' in config['trial']:
+ config['trial'].pop('gpuNum')
deep_update(config, it_config['all'])
deep_update(config, it_config[training_service])
@@ -106,7 +112,7 @@ def run(args):
parser = argparse.ArgumentParser()
parser.add_argument("--config", type=str, default=None)
parser.add_argument("--exclude", type=str, default=None)
- parser.add_argument("--ts", type=str, choices=['local', 'remote', 'pai', 'kubeflow'], default='local')
+ parser.add_argument("--ts", type=str, choices=['local', 'remote', 'pai', 'kubeflow', 'frameworkcontroller'], default='local')
parser.add_argument("--local_gpu", action='store_true')
parser.add_argument("--preinstall", action='store_true')
args = parser.parse_args()
diff --git a/test/generate_ts_config.py b/test/generate_ts_config.py
index 53de5d8d0d..fb5784d3b1 100644
--- a/test/generate_ts_config.py
+++ b/test/generate_ts_config.py
@@ -42,6 +42,21 @@ def update_training_service_config(args):
config[args.ts]['kubeflowConfig']['azureStorage']['azureShare'] = args.azs_share
if args.nni_docker_image is not None:
config[args.ts]['trial']['worker']['image'] = args.nni_docker_image
+ elif args.ts == 'frameworkcontroller':
+ if args.nfs_server is not None:
+ config[args.ts]['frameworkcontrollerConfig']['nfs']['server'] = args.nfs_server
+ if args.nfs_path is not None:
+ config[args.ts]['frameworkcontrollerConfig']['nfs']['path'] = args.nfs_path
+ if args.keyvault_vaultname is not None:
+ config[args.ts]['frameworkcontrollerConfig']['keyVault']['vaultName'] = args.keyvault_vaultname
+ if args.keyvault_name is not None:
+ config[args.ts]['frameworkcontrollerConfig']['keyVault']['name'] = args.keyvault_name
+ if args.azs_account is not None:
+ config[args.ts]['frameworkcontrollerConfig']['azureStorage']['accountName'] = args.azs_account
+ if args.azs_share is not None:
+ config[args.ts]['frameworkcontrollerConfig']['azureStorage']['azureShare'] = args.azs_share
+ if args.nni_docker_image is not None:
+ config[args.ts]['trial']['taskRoles'][0]['image'] = args.nni_docker_image
elif args.ts == 'remote':
if args.remote_user is not None:
config[args.ts]['machineList'][0]['username'] = args.remote_user
@@ -69,7 +84,7 @@ def convert_command():
if __name__ == '__main__':
parser = argparse.ArgumentParser()
- parser.add_argument("--ts", type=str, choices=['pai', 'kubeflow', 'remote', 'local'], default='pai')
+ parser.add_argument("--ts", type=str, choices=['pai', 'kubeflow', 'remote', 'local', 'frameworkcontroller'], default='pai')
parser.add_argument("--nni_docker_image", type=str)
parser.add_argument("--nni_manager_ip", type=str)
# args for PAI
@@ -79,7 +94,7 @@ def convert_command():
parser.add_argument("--data_dir", type=str)
parser.add_argument("--output_dir", type=str)
parser.add_argument("--vc", type=str)
- # args for kubeflow
+ # args for kubeflow and frameworkController
parser.add_argument("--nfs_server", type=str)
parser.add_argument("--nfs_path", type=str)
parser.add_argument("--keyvault_vaultname", type=str)
diff --git a/test/pipelines-it-frameworkcontroller.yml b/test/pipelines-it-frameworkcontroller.yml
new file mode 100644
index 0000000000..e29fa3a8b1
--- /dev/null
+++ b/test/pipelines-it-frameworkcontroller.yml
@@ -0,0 +1,55 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT license.
+
+jobs:
+- job: 'integration_test_frameworkController'
+ timeoutInMinutes: 0
+
+ steps:
+ - script: python3 -m pip install --upgrade pip setuptools --user
+ displayName: 'Install python tools'
+
+ - script: |
+ cd deployment/pypi
+ echo 'building prerelease package...'
+ make build
+ ls $(Build.SourcesDirectory)/deployment/pypi/dist/
+ condition: eq( variables['build_docker_img'], 'true' )
+ displayName: 'build nni bdsit_wheel'
+
+ - script: |
+ source install.sh
+ displayName: 'Install nni toolkit via source code'
+
+ - script: |
+ sudo apt-get install swig -y
+ PATH=$HOME/.local/bin:$PATH nnictl package install --name=SMAC
+ PATH=$HOME/.local/bin:$PATH nnictl package install --name=BOHB
+ displayName: 'Install dependencies for integration tests in frameworkcontroller mode'
+
+ - script: |
+ if [ $(build_docker_img) = 'true' ]
+ then
+ cd deployment/pypi
+ docker login -u $(docker_hub_user) -p $(docker_hub_pwd)
+
+ echo 'updating docker file for installing nni from local...'
+ # update Dockerfile to install NNI in docker image from whl file built in last step
+ sed -ie 's/RUN python3 -m pip --no-cache-dir install nni/COPY .\/dist\/* .\nRUN python3 -m pip install nni-*.whl/' ../docker/Dockerfile
+ cat ../docker/Dockerfile
+ export IMG_TAG=`date -u +%y%m%d%H%M`
+ docker build -f ../docker/Dockerfile -t $(test_docker_img_name):$IMG_TAG .
+ docker push $(test_docker_img_name):$IMG_TAG
+ export TEST_IMG=$(test_docker_img_name):$IMG_TAG
+ cd ../../
+ else
+ export TEST_IMG=$(existing_docker_img)
+ fi
+ echo "TEST_IMG:$TEST_IMG"
+ cd test
+ python3 generate_ts_config.py --ts frameworkcontroller --keyvault_vaultname $(keyVault_vaultName) --keyvault_name $(keyVault_name) \
+ --azs_account $(azureStorage_accountName) --azs_share $(azureStorage_azureShare) --nni_docker_image $TEST_IMG --nni_manager_ip $(nni_manager_ip)
+
+ cat training_service.yml
+ PATH=$HOME/.local/bin:$PATH python3 config_test.py --ts frameworkcontroller --exclude multi_phase
+ displayName: 'integration test'
diff --git a/test/pipelines-it-local-windows.yml b/test/pipelines-it-local-windows.yml
index 56a6e99bdc..688b9dcc94 100644
--- a/test/pipelines-it-local-windows.yml
+++ b/test/pipelines-it-local-windows.yml
@@ -8,7 +8,7 @@ jobs:
- script: |
python -m pip install scikit-learn==0.20.0 --user
python -m pip install keras==2.1.6 --user
- python -m pip install https://download.pytorch.org/whl/cu90/torch-0.4.1-cp36-cp36m-win_amd64.whl --user
+ python -m pip install torch===1.2.0 torchvision===0.4.1 -f https://download.pytorch.org/whl/torch_stable.html --user
python -m pip install torchvision --user
python -m pip install tensorflow-gpu==1.11.0 --user
displayName: 'Install dependencies for integration tests'
diff --git a/test/training_service.yml b/test/training_service.yml
index 9fe8a85a0b..2a00acca54 100644
--- a/test/training_service.yml
+++ b/test/training_service.yml
@@ -24,6 +24,32 @@ kubeflow:
image:
trainingServicePlatform: kubeflow
+frameworkcontroller:
+ maxExecDuration: 15m
+ nniManagerIp:
+ frameworkcontrollerConfig:
+ serviceAccountName: frameworkbarrier
+ storage: azureStorage
+ keyVault:
+ vaultName:
+ name:
+ azureStorage:
+ accountName:
+ azureShare:
+ trial:
+ taskRoles:
+ - name: worker
+ taskNum: 1
+ command:
+ gpuNum: 1
+ cpuNum: 1
+ memoryMB: 8192
+ image:
+ frameworkAttemptCompletionPolicy:
+ minFailedTaskCount: 1
+ minSucceededTaskCount: 1
+ trainingServicePlatform: frameworkcontroller
+
local:
trainingServicePlatform: local
pai:
diff --git a/tools/nni_cmd/config_schema.py b/tools/nni_cmd/config_schema.py
index 8017946ce9..4be463c7b6 100644
--- a/tools/nni_cmd/config_schema.py
+++ b/tools/nni_cmd/config_schema.py
@@ -271,16 +271,17 @@ def setPathCheck(key):
pai_trial_schema = {
'trial':{
- 'command': setType('command', str),
'codeDir': setPathCheck('codeDir'),
- 'gpuNum': setNumberRange('gpuNum', int, 0, 99999),
- 'cpuNum': setNumberRange('cpuNum', int, 0, 99999),
- 'memoryMB': setType('memoryMB', int),
- 'image': setType('image', str),
- Optional('virtualCluster'): setType('virtualCluster', str),
'nniManagerNFSMountPath': setPathCheck('nniManagerNFSMountPath'),
'containerNFSMountPath': setType('containerNFSMountPath', str),
- 'paiStoragePlugin': setType('paiStoragePlugin', str)
+ 'command': setType('command', str),
+ Optional('gpuNum'): setNumberRange('gpuNum', int, 0, 99999),
+ Optional('cpuNum'): setNumberRange('cpuNum', int, 0, 99999),
+ Optional('memoryMB'): setType('memoryMB', int),
+ Optional('image'): setType('image', str),
+ Optional('virtualCluster'): setType('virtualCluster', str),
+ Optional('paiStoragePlugin'): setType('paiStoragePlugin', str),
+ Optional('paiConfigPath'): And(os.path.exists, error=SCHEMA_PATH_ERROR % 'paiConfigPath')
}
}
@@ -407,15 +408,8 @@ def setPathCheck(key):
}
machine_list_schema = {
- Optional('machineList'):[Or({
- 'ip': setType('ip', str),
- Optional('port'): setNumberRange('port', int, 1, 65535),
- 'username': setType('username', str),
- 'passwd': setType('passwd', str),
- Optional('gpuIndices'): Or(int, And(str, lambda x: len([int(i) for i in x.split(',')]) > 0), error='gpuIndex format error!'),
- Optional('maxTrialNumPerGpu'): setType('maxTrialNumPerGpu', int),
- Optional('useActiveGpu'): setType('useActiveGpu', bool)
- }, {
+ Optional('machineList'):[Or(
+ {
'ip': setType('ip', str),
Optional('port'): setNumberRange('port', int, 1, 65535),
'username': setType('username', str),
@@ -424,6 +418,15 @@ def setPathCheck(key):
Optional('gpuIndices'): Or(int, And(str, lambda x: len([int(i) for i in x.split(',')]) > 0), error='gpuIndex format error!'),
Optional('maxTrialNumPerGpu'): setType('maxTrialNumPerGpu', int),
Optional('useActiveGpu'): setType('useActiveGpu', bool)
+ },
+ {
+ 'ip': setType('ip', str),
+ Optional('port'): setNumberRange('port', int, 1, 65535),
+ 'username': setType('username', str),
+ 'passwd': setType('passwd', str),
+ Optional('gpuIndices'): Or(int, And(str, lambda x: len([int(i) for i in x.split(',')]) > 0), error='gpuIndex format error!'),
+ Optional('maxTrialNumPerGpu'): setType('maxTrialNumPerGpu', int),
+ Optional('useActiveGpu'): setType('useActiveGpu', bool)
})]
}
diff --git a/tools/nni_cmd/launcher.py b/tools/nni_cmd/launcher.py
index 5d406a0ae3..26332c37b0 100644
--- a/tools/nni_cmd/launcher.py
+++ b/tools/nni_cmd/launcher.py
@@ -9,7 +9,7 @@
import site
import time
import tempfile
-from subprocess import Popen, check_call, CalledProcessError
+from subprocess import Popen, check_call, CalledProcessError, PIPE, STDOUT
from nni_annotation import expand_annotations, generate_search_space
from nni.constants import ModuleName, AdvisorModuleName
from .launcher_utils import validate_all_content
@@ -20,7 +20,7 @@
detect_port, get_user, get_python_dir
from .constants import NNICTL_HOME_DIR, ERROR_INFO, REST_TIME_OUT, EXPERIMENT_SUCCESS_INFO, LOG_HEADER, PACKAGE_REQUIREMENTS
from .command_utils import check_output_command, kill_command
-from .nnictl_utils import update_experiment, set_monitor
+from .nnictl_utils import update_experiment
def get_log_path(config_file_name):
'''generate stdout and stderr log path'''
@@ -78,17 +78,17 @@ def _generate_installation_path(sitepackages_path):
print_error('Fail to find nni under python library')
exit(1)
-def start_rest_server(port, platform, mode, config_file_name, experiment_id=None, log_dir=None, log_level=None):
+def start_rest_server(args, platform, mode, config_file_name, experiment_id=None, log_dir=None, log_level=None):
'''Run nni manager process'''
- if detect_port(port):
+ if detect_port(args.port):
print_error('Port %s is used by another process, please reset the port!\n' \
- 'You could use \'nnictl create --help\' to get help information' % port)
+ 'You could use \'nnictl create --help\' to get help information' % args.port)
exit(1)
- if (platform != 'local') and detect_port(int(port) + 1):
+ if (platform != 'local') and detect_port(int(args.port) + 1):
print_error('PAI mode need an additional adjacent port %d, and the port %d is used by another process!\n' \
'You could set another port to start experiment!\n' \
- 'You could use \'nnictl create --help\' to get help information' % ((int(port) + 1), (int(port) + 1)))
+ 'You could use \'nnictl create --help\' to get help information' % ((int(args.port) + 1), (int(args.port) + 1)))
exit(1)
print_normal('Starting restful server...')
@@ -99,7 +99,7 @@ def start_rest_server(port, platform, mode, config_file_name, experiment_id=None
node_command = 'node'
if sys.platform == 'win32':
node_command = os.path.join(entry_dir[:-3], 'Scripts', 'node.exe')
- cmds = [node_command, entry_file, '--port', str(port), '--mode', platform]
+ cmds = [node_command, entry_file, '--port', str(args.port), '--mode', platform]
if mode == 'view':
cmds += ['--start_mode', 'resume']
cmds += ['--readonly', 'true']
@@ -111,6 +111,8 @@ def start_rest_server(port, platform, mode, config_file_name, experiment_id=None
cmds += ['--log_level', log_level]
if mode in ['resume', 'view']:
cmds += ['--experiment_id', experiment_id]
+ if args.foreground:
+ cmds += ['--foreground', 'true']
stdout_full_path, stderr_full_path = get_log_path(config_file_name)
with open(stdout_full_path, 'a+') as stdout_file, open(stderr_full_path, 'a+') as stderr_file:
time_now = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time()))
@@ -120,9 +122,15 @@ def start_rest_server(port, platform, mode, config_file_name, experiment_id=None
stderr_file.write(log_header)
if sys.platform == 'win32':
from subprocess import CREATE_NEW_PROCESS_GROUP
- process = Popen(cmds, cwd=entry_dir, stdout=stdout_file, stderr=stderr_file, creationflags=CREATE_NEW_PROCESS_GROUP)
+ if args.foreground:
+ process = Popen(cmds, cwd=entry_dir, stdout=PIPE, stderr=STDOUT, creationflags=CREATE_NEW_PROCESS_GROUP)
+ else:
+ process = Popen(cmds, cwd=entry_dir, stdout=stdout_file, stderr=stderr_file, creationflags=CREATE_NEW_PROCESS_GROUP)
else:
- process = Popen(cmds, cwd=entry_dir, stdout=stdout_file, stderr=stderr_file)
+ if args.foreground:
+ process = Popen(cmds, cwd=entry_dir, stdout=PIPE, stderr=PIPE)
+ else:
+ process = Popen(cmds, cwd=entry_dir, stdout=stdout_file, stderr=stderr_file)
return process, str(time_now)
def set_trial_config(experiment_config, port, config_file_name):
@@ -424,7 +432,7 @@ def launch_experiment(args, experiment_config, mode, config_file_name, experimen
if log_level not in ['trace', 'debug'] and (args.debug or experiment_config.get('debug') is True):
log_level = 'debug'
# start rest server
- rest_process, start_time = start_rest_server(args.port, experiment_config['trainingServicePlatform'], \
+ rest_process, start_time = start_rest_server(args, experiment_config['trainingServicePlatform'], \
mode, config_file_name, experiment_id, log_dir, log_level)
nni_config.set_config('restServerPid', rest_process.pid)
# Deal with annotation
@@ -493,8 +501,14 @@ def launch_experiment(args, experiment_config, mode, config_file_name, experimen
experiment_config['experimentName'])
print_normal(EXPERIMENT_SUCCESS_INFO % (experiment_id, ' '.join(web_ui_url_list)))
- if args.watch:
- set_monitor(True, 3, args.port, rest_process.pid)
+ if args.foreground:
+ try:
+ while True:
+ log_content = rest_process.stdout.readline().strip().decode('utf-8')
+ print(log_content)
+ except KeyboardInterrupt:
+ kill_command(rest_process.pid)
+ print_normal('Stopping experiment...')
def create_experiment(args):
'''start a new experiment'''
diff --git a/tools/nni_cmd/launcher_utils.py b/tools/nni_cmd/launcher_utils.py
index f2d2c1d22f..9301e5bf2b 100644
--- a/tools/nni_cmd/launcher_utils.py
+++ b/tools/nni_cmd/launcher_utils.py
@@ -7,7 +7,7 @@
from schema import Schema
from .config_schema import LOCAL_CONFIG_SCHEMA, REMOTE_CONFIG_SCHEMA, PAI_CONFIG_SCHEMA, PAI_YARN_CONFIG_SCHEMA, KUBEFLOW_CONFIG_SCHEMA,\
FRAMEWORKCONTROLLER_CONFIG_SCHEMA, tuner_schema_dict, advisor_schema_dict, assessor_schema_dict
-from .common_utils import print_error, print_warning, print_normal
+from .common_utils import print_error, print_warning, print_normal, get_yml_content
def expand_path(experiment_config, key):
'''Change '~' to user home directory'''
@@ -63,6 +63,8 @@ def parse_path(experiment_config, config_path):
if experiment_config.get('machineList'):
for index in range(len(experiment_config['machineList'])):
expand_path(experiment_config['machineList'][index], 'sshKeyPath')
+ if experiment_config['trial'].get('paiConfigPath'):
+ expand_path(experiment_config['trial'], 'paiConfigPath')
#if users use relative path, convert it to absolute path
root_path = os.path.dirname(config_path)
@@ -94,6 +96,8 @@ def parse_path(experiment_config, config_path):
if experiment_config.get('machineList'):
for index in range(len(experiment_config['machineList'])):
parse_relative_path(root_path, experiment_config['machineList'][index], 'sshKeyPath')
+ if experiment_config['trial'].get('paiConfigPath'):
+ parse_relative_path(root_path, experiment_config['trial'], 'paiConfigPath')
def validate_search_space_content(experiment_config):
'''Validate searchspace content,
@@ -254,6 +258,45 @@ def validate_machine_list(experiment_config):
print_error('Please set machineList!')
exit(1)
+def validate_pai_config_path(experiment_config):
+ '''validate paiConfigPath field'''
+ if experiment_config.get('trainingServicePlatform') == 'pai':
+ if experiment_config.get('trial', {}).get('paiConfigPath'):
+ # validate the file format of paiConfigPath, ensure it is yaml format
+ pai_config = get_yml_content(experiment_config['trial']['paiConfigPath'])
+ if experiment_config['trial'].get('image') is None:
+ if pai_config.get('prerequisites', [{}])[0].get('uri') is None:
+ print_error('Please set image field, or set image uri in your own paiConfig!')
+ exit(1)
+ experiment_config['trial']['image'] = pai_config['prerequisites'][0]['uri']
+ if experiment_config['trial'].get('gpuNum') is None:
+ if pai_config.get('taskRoles', {}).get('taskrole', {}).get('resourcePerInstance', {}).get('gpu') is None:
+ print_error('Please set gpuNum field, or set resourcePerInstance gpu in your own paiConfig!')
+ exit(1)
+ experiment_config['trial']['gpuNum'] = pai_config['taskRoles']['taskrole']['resourcePerInstance']['gpu']
+ if experiment_config['trial'].get('cpuNum') is None:
+ if pai_config.get('taskRoles', {}).get('taskrole', {}).get('resourcePerInstance', {}).get('cpu') is None:
+ print_error('Please set cpuNum field, or set resourcePerInstance cpu in your own paiConfig!')
+ exit(1)
+ experiment_config['trial']['cpuNum'] = pai_config['taskRoles']['taskrole']['resourcePerInstance']['cpu']
+ if experiment_config['trial'].get('memoryMB') is None:
+ if pai_config.get('taskRoles', {}).get('taskrole', {}).get('resourcePerInstance', {}).get('memoryMB', {}) is None:
+ print_error('Please set memoryMB field, or set resourcePerInstance memoryMB in your own paiConfig!')
+ exit(1)
+ experiment_config['trial']['memoryMB'] = pai_config['taskRoles']['taskrole']['resourcePerInstance']['memoryMB']
+ if experiment_config['trial'].get('paiStoragePlugin') is None:
+ if pai_config.get('extras', {}).get('com.microsoft.pai.runtimeplugin', [{}])[0].get('plugin') is None:
+ print_error('Please set paiStoragePlugin field, or set plugin in your own paiConfig!')
+ exit(1)
+ experiment_config['trial']['paiStoragePlugin'] = pai_config['extras']['com.microsoft.pai.runtimeplugin'][0]['plugin']
+ else:
+ pai_trial_fields_required_list = ['image', 'gpuNum', 'cpuNum', 'memoryMB', 'paiStoragePlugin']
+ for trial_field in pai_trial_fields_required_list:
+ if experiment_config['trial'].get(trial_field) is None:
+ print_error('Please set {0} in trial configuration,\
+ or set additional pai configuration file path in paiConfigPath!'.format(trial_field))
+ exit(1)
+
def validate_pai_trial_conifg(experiment_config):
'''validate the trial config in pai platform'''
if experiment_config.get('trainingServicePlatform') in ['pai', 'paiYarn']:
@@ -269,6 +312,7 @@ def validate_pai_trial_conifg(experiment_config):
print_warning(warning_information.format('dataDir'))
if experiment_config.get('trial').get('outputDir'):
print_warning(warning_information.format('outputDir'))
+ validate_pai_config_path(experiment_config)
def validate_all_content(experiment_config, config_path):
'''Validate whether experiment_config is valid'''
diff --git a/tools/nni_cmd/nnictl.py b/tools/nni_cmd/nnictl.py
index 856bd2adc8..d9da570abb 100644
--- a/tools/nni_cmd/nnictl.py
+++ b/tools/nni_cmd/nnictl.py
@@ -51,7 +51,7 @@ def parse_args():
parser_start.add_argument('--config', '-c', required=True, dest='config', help='the path of yaml config file')
parser_start.add_argument('--port', '-p', default=DEFAULT_REST_PORT, dest='port', help='the port of restful server')
parser_start.add_argument('--debug', '-d', action='store_true', help=' set debug mode')
- parser_start.add_argument('--watch', '-w', action='store_true', help=' set watch mode')
+ parser_start.add_argument('--foreground', '-f', action='store_true', help=' set foreground mode, print log content to terminal')
parser_start.set_defaults(func=create_experiment)
# parse resume command
@@ -59,7 +59,7 @@ def parse_args():
parser_resume.add_argument('id', nargs='?', help='The id of the experiment you want to resume')
parser_resume.add_argument('--port', '-p', default=DEFAULT_REST_PORT, dest='port', help='the port of restful server')
parser_resume.add_argument('--debug', '-d', action='store_true', help=' set debug mode')
- parser_resume.add_argument('--watch', '-w', action='store_true', help=' set watch mode')
+ parser_resume.add_argument('--foreground', '-f', action='store_true', help=' set foreground mode, print log content to terminal')
parser_resume.set_defaults(func=resume_experiment)
# parse view command
diff --git a/tools/nni_cmd/nnictl_utils.py b/tools/nni_cmd/nnictl_utils.py
index a66197fac9..4866bcdce4 100644
--- a/tools/nni_cmd/nnictl_utils.py
+++ b/tools/nni_cmd/nnictl_utils.py
@@ -403,11 +403,13 @@ def remote_clean(machine_list, experiment_id=None):
userName = machine.get('username')
host = machine.get('ip')
port = machine.get('port')
+ sshKeyPath = machine.get('sshKeyPath')
+ passphrase = machine.get('passphrase')
if experiment_id:
remote_dir = '/' + '/'.join(['tmp', 'nni', 'experiments', experiment_id])
else:
remote_dir = '/' + '/'.join(['tmp', 'nni', 'experiments'])
- sftp = create_ssh_sftp_client(host, port, userName, passwd)
+ sftp = create_ssh_sftp_client(host, port, userName, passwd, sshKeyPath, passphrase)
print_normal('removing folder {0}'.format(host + ':' + str(port) + remote_dir))
remove_remote_directory(sftp, remote_dir)
diff --git a/tools/nni_cmd/ssh_utils.py b/tools/nni_cmd/ssh_utils.py
index 2e68611206..e3f26a8e24 100644
--- a/tools/nni_cmd/ssh_utils.py
+++ b/tools/nni_cmd/ssh_utils.py
@@ -30,12 +30,16 @@ def copy_remote_directory_to_local(sftp, remote_path, local_path):
except Exception:
pass
-def create_ssh_sftp_client(host_ip, port, username, password):
+def create_ssh_sftp_client(host_ip, port, username, password, ssh_key_path, passphrase):
'''create ssh client'''
try:
paramiko = check_environment()
conn = paramiko.Transport(host_ip, port)
- conn.connect(username=username, password=password)
+ if ssh_key_path is not None:
+ ssh_key = paramiko.RSAKey.from_private_key_file(ssh_key_path, password=passphrase)
+ conn.connect(username=username, pkey=ssh_key)
+ else:
+ conn.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(conn)
return sftp
except Exception as exception:
diff --git a/tools/nni_cmd/tensorboard_utils.py b/tools/nni_cmd/tensorboard_utils.py
index 8cb0bbfc17..60d589083a 100644
--- a/tools/nni_cmd/tensorboard_utils.py
+++ b/tools/nni_cmd/tensorboard_utils.py
@@ -37,12 +37,14 @@ def copy_data_from_remote(args, nni_config, trial_content, path_list, host_list,
machine_dict = {}
local_path_list = []
for machine in machine_list:
- machine_dict[machine['ip']] = {'port': machine['port'], 'passwd': machine['passwd'], 'username': machine['username']}
+ machine_dict[machine['ip']] = {'port': machine['port'], 'passwd': machine['passwd'], 'username': machine['username'],
+ 'sshKeyPath': machine.get('sshKeyPath'), 'passphrase': machine.get('passphrase')}
for index, host in enumerate(host_list):
local_path = os.path.join(temp_nni_path, trial_content[index].get('id'))
local_path_list.append(local_path)
print_normal('Copying log data from %s to %s' % (host + ':' + path_list[index], local_path))
- sftp = create_ssh_sftp_client(host, machine_dict[host]['port'], machine_dict[host]['username'], machine_dict[host]['passwd'])
+ sftp = create_ssh_sftp_client(host, machine_dict[host]['port'], machine_dict[host]['username'], machine_dict[host]['passwd'],
+ machine_dict[host]['sshKeyPath'], machine_dict[host]['passphrase'])
copy_remote_directory_to_local(sftp, path_list[index], local_path)
print_normal('Copy done!')
return local_path_list