Merge pull request #143 from FederatedAI/dev-2.0.0-rc

Update cli doc
FederatedAI · Dec 29, 2023 · 5383f59 · 5383f59
2 parents 998163b + fe515ba
commit 5383f59
Show file tree

Hide file tree

Showing 3 changed files with 181 additions and 43 deletions.
diff --git a/README.md b/README.md
@@ -2,4 +2,8 @@
 Fate-Client contains Pipeline, Flow Cli and Flow SDK:
 * Pipeline is a tool to enable fast federated modeling jobs for FATE. It supports compiling federated modeling job to DSL IR(Intermediate Representation).
 * Flow CLI is a command line interface providing series of commands for user to design, start, and query federated jobs scheduling by fate_flow.
-* Flow SDK provides multiple python api to allow user to interactive with fate_flow, also a based tool used by Flow Cli and Pipeline.
+* Flow SDK provides multiple python api to allow user to interactive with fate_flow, also a based tool used by Flow Cli and Pipeline.
+
+## Tutorials
+[FATE-Flow-Cli Using Guide](./doc/cli_index.md)  
+[Pipeline Using Guide](./doc/pipeline.md)
diff --git a/doc/cli_index.md b/doc/cli_index.md
@@ -0,0 +1,136 @@
+# CLIENT CLI
+
+Description
+
+Introduces how to install and use the FATE Flow Client, which is usually
+included in the FATE Client, which contains several clients of the FATE 
+Project: Pipeline, FATE Flow Client.
+
+Introducing the command line provided by FATE Flow Client, all commands will
+have a common invocation entry, you can type flow in the command line to get all
+the command categories and their subcommands.
+
+    [IN]
+    flow
+
+    [OUT]
+    Usage: flow [OPTIONS] COMMAND [ARGS]...
+
+      Fate Flow Client
+
+    Options.
+      -h, --help Show this message and exit.
+
+    Commands: -h, --help
+      client      -description: Client Operations
+      data        -description: Provides numbers of data operational...
+      init        -description: Flow CLI Init Command.
+      job         -description: Provides numbers of job operational commands,...
+      log         -description: Operations related to job logs.
+      model       -description: Model Operations
+      output      -description: Task output Operations
+      permission  -description: Permission Operations
+      provider    -description: Provider Operations
+      server      -description: Third-party Service Related Operations
+      site        -description: Site Operations
+      table       -description: Data Table Operations, such as Querying and...
+      task        -description: Provides numbers of task operational...
+      test        -description: fate test
+      version     -description: Get fate flow client version -usage: flow...
+
+
+## Install Fate Client
+FATE Client will be distributed to pypi, you can install the corresponding version
+directly using tools such as pip, e.g.
+
+    pip install fate-client
+
+## Init Fate Client
+
+    flow init   -h 
+    Usage: flow init [OPTIONS]
+
+    -description: Flow CLI Init Command. provide ip and port of a valid fate flow server.
+    If the server enables client authentication, you need to configure app-id and app-token
+      
+    -usage: flow init --ip 127.0.0.1 --port 9380
+
+    Options:
+      --ip TEXT         Fate flow server ip address.
+      --port INTEGER    Fate flow server port.
+      --app-id TEXT     APP key for sign requests.
+      --app-token TEXT  Secret key for sign requests.
+      --reset           If specified, initialization settings would be reset to
+                        none. Users should init flow again.
+      -h, --help        Show this message and exit.
+
+
+### Flow Client Cli
+
+After successfully installed FATE Client,and init FATE Client. user needs to configure
+server information for Client. Client cli provides a command line tool for quick setup. 
+Run the following command for more information.
+
+
+## Interface of Flow Client Cli
+
+### Client
+Client operational.
+[examples](../python/fate_client/flow_cli/build/doc/client.md)
+
+
+### data
+Provides numbers of data operational
+[examples](../python/fate_client/flow_cli/build/doc/data.md)
+
+
+## job
+Provides numbers of job operational commands
+[examples](../python/fate_client/flow_cli/build/doc/job.md)
+
+## log
+Operations related to job logs
+[examples](../python/fate_client/flow_cli/build/doc/log.md)
+
+## model
+Model Operations
+[examples](../python/fate_client/flow_cli/build/doc/model.md)
+
+## output
+Task output Operations
+[examples](../python/fate_client/flow_cli/build/doc/output.md)
+
+## permission
+Permission Operations
+[examples](../python/fate_client/flow_cli/build/doc/log.md)
+
+## provider
+Provider Operations
+[examples](../python/fate_client/flow_cli/build/doc/perission.md)
+
+## server
+Third-party Service Related Operations
+[examples](../python/fate_client/flow_cli/build/doc/server.md)
+
+## site
+Site Operations
+[examples](../python/fate_client/flow_cli/build/doc/site.md)
+
+## table
+Data Table Operations
+[examples](../python/fate_client/flow_cli/build/doc/table.md)
+
+## task
+Provides numbers of task operational
+[examples](../python/fate_client/flow_cli/build/doc/task.md)
+
+## test
+fate test
+[examples](../python/fate_client/flow_cli/build/doc/test.md)
+
+
+
+
+
+
+
diff --git a/doc/pipeline.md b/doc/pipeline.md
@@ -9,19 +9,19 @@ pipeline.
 
 ## A FATE Job is A Directed Acyclic Graph
 
-A FATE job is a dag that consists of algorithm component nodes. FATE
+A FATE job is a dag that consists of algorithm task nodes. FATE
 pipeline provides easy-to-use tools to configure order and setting of
 the tasks.
 
 FATE is written in a modular style. Modules are designed to have input
-and output data and model. Therefore, two components are connected when
-a downsteam component takes output from another component as input. By
-tracing how one data set is processed through FATE components, we can see
+and output data and model. Therefore, two tasks are connected when
+a downstream task takes output from another task as input. By
+tracing how one data set is processed through FATE tasks, we can see
 that a FATE job is in fact formed by a sequence of sub-tasks. For
 example, in the [tutorial](https://github.com/FederatedAI/FATE/tree/master/doc/tutorial/pipeline_tutorial_hetero.ipynb),
-guest’s and host's data are read in through `DataWarehouseChannel`.
+guest’s and host's data are read in through `Reader`.
 `PSI` then finds overlapping ids between guest and host. Finally, `CoordinatedLR` is fit
-on the data. Each listed components run a small task with the data, and
+on the data. Each listed tasks run a small task with the data, and
 together they constitute a model training job.
 
 Beyond the given tutorial, a job may include multiple data sets and
@@ -45,21 +45,21 @@ pipeline init --help
 
 ### Component
 
-FATE components are wrapped into `component` in Pipeline API.
-When defining a component, user need to specify components' name,
+FATE tasks are wrapped into `component` in Pipeline API.
+When defining a task, user need to specify task's name,
 input data(may be named as `input_data` or `train_data`), parameters and, possibly, input model(s).
-Each component can take in and output `Data` and `Model`.
+Each task can take in and output `Data` and `Model`.
 Some may take multiple copies of `Data` or `Model`. Parameters of
-components can be set conveniently at the time of initialization.
-Unspecified parameters will take default values. All components have a
-`name`, which can be arbitrarily set. A component’s name is its
+tasks can be set conveniently at the time of initialization.
+Unspecified parameters will take default values. All tasks have a
+`name`, which can be arbitrarily set. A task’s name is its
 identifier, and so it must be unique within a pipeline. We suggest that
-each component name includes a numbering as suffix for easy tracking.
+each task name includes a numbering as suffix for easy tracking.
 
-An example of initializing a component:
+An example of initializing a task:
 
 ```python
-from fate_client.pipeline.components.fate import CoordinatedLR, PSI
+from fate_client.pipeline.components.fate import CoordinatedLR, PSI, Reader
 
 lr_0 = CoordinatedLR("lr_0",
                      epochs=10,
@@ -100,8 +100,9 @@ Below lists data input and output of all components:
 | Coordinated-LR           | CoordinatedLR          | train_data, validate_data, test_data, cv_data | train_output_data, validate_output_data, test_output_data, cv_output_datas |
 | Coordinated-LinR         | CoordinatedLinR        | train_data, validate_data, test_data, cv_data | train_output_data, validate_output_data, test_output_data, cv_output_datas |
 | Homo-LR                  | HomoLR                 | train_data, validate_data, test_data, cv_data | train_output_data, validate_output_data, test_output_data, cv_output_datas |
-| Homo-NN                  | HomoNN                 | train_data, validate_data, test_data, cv_data | train_output_data, validate_output_data, test_output_data, cv_output_datas |
-| Hetero Secure Boosting   | HeteroSecureBoost      | train_data, validate_data, test_data, cv_data | train_output_data, validate_output_data, test_output_data, cv_output_datas |
+| Homo-NN                  | HomoNN                 | train_data, validate_data, test_data | train_data_output, predict_data_output|
+| Hetero-NN   | HeteroNN    | train_data, validate_data, test_data | train_data_output, predict_data_output|
+| Hetero Secure Boosting   | HeteroSecureBoost      | train_data, validate_data, test_data, cv_data | train_data_output, test_output_data, cv_output_datas |
 | Evaluation               | Evaluation             | input_data                                    |                                                                            |
 | Union                    | Union                  | input_data_list                               | output_data                                                                |
 
@@ -125,8 +126,9 @@ Below lists model input and output of all components:
 | Coordinated-LR           | CoordinatedLR          | input_model, warm_start_model | output_model |
 | Coordinated-LinR         | CoordinatedLinR        | input_model, warm_start_model | output_model |
 | Homo-LR                  | HomoLR                 | input_model, warm_start_model | output_model |
-| Homo-NN                  | HomoNN                 | input_model, warm_start_model | output_model |
-| Hetero Secure Boosting   | HeteroSecureBoost      | input_model, warm_start_model | output_model |
+| Homo-NN                  | HomoNN                 | train_model_input, predict_model_input, train_model_output | train_model_output |
+| Hetero-NN|HeteroNN|train_model_input, predict_model_input, train_model_output| train_model_output|
+| Hetero Secure Boosting   | HeteroSecureBoost      | train_model_input, predict_model_input, train_model_output | train_model_output |
 | Evaluation               | Evaluation             |                               |              |
 | Union                    | Union                  |                               |              |
 
@@ -146,28 +148,26 @@ pipeline = FateFlowPipeline().set_roles(guest='9999', host='10000', arbiter='100
 User may also specify runtime configuration:
 
 ```python
-pipeline.conf.set("task_cores", 4)
-pipeline.conf.set("timeout", 3600)
+pipeline.conf.set("cores", 4)
+pipeline.conf.set("task", dict(timeout=3600))
 ```
 
-All pipeline components can be configured individually for different
-roles. For instance, `PSI`
-component can be configured specifically for each party like this:
+All pipeline tasks can be configured individually for different
+roles. For instance, `Reader`
+task can be configured specifically for each party like this:
 
 ```python
-psi_0 = PSI("psi_0")
-psi_0.guest.component_setting(input_data=DataWarehouseChannel(name="breast_hetero_guest",
-                                                              namespace="experiment"))
-psi_0.hosts[0].component_setting(input_data=DataWarehouseChannel(name="breast_hetero_host",
-                                                                 namespace="experiment"))
+reader_0 = Reader("reader_0", runtime_parties=dict(guest="9999", host="10000"))
+reader_0.guest.task_parameters(namespace="experiment", name="breast_hetero_guest")
+reader_0.hosts[0].task_parameters(namespace="experiment", name="breast_hetero_host")
 ```
 
-To include a component in a pipeline, use `add_task`. To add the
-`PSI` component to the previously created pipeline, try
+To include tasks in a pipeline, use `add_tasks`. To add the
+`Reader` component to the previously created pipeline, try
 this:
 
 ```python
-pipeline.add_task(psi_0)
+pipeline.add_tasks([reader_0])
 ```
 
 ## Run A Pipeline
@@ -183,7 +183,7 @@ pipeline.fit()
 
 ## Query on Tasks
 
-FATE Pipeline provides API to query component information, including
+FATE Pipeline provides API to query task information, including
 output data, model, and metrics.
 
 ```python
@@ -210,21 +210,19 @@ prediction.
 predict_pipeline = FateFlowPipeline()
 
 deployed_pipeline = pipeline.get_deployed_pipeline()
-deployed_pipeline.psi_0.guest.component_setting(
-    input_data=DataWarehouseChannel(name="breast_hetero_guest",
-                                    namespace=f"experiment{namespace}"))
-deployed_pipeline.psi_0.hosts[0].component_setting(
-    input_data=DataWarehouseChannel(name="breast_hetero_host",
-                                    namespace=f"experiment{namespace}"))
-
-predict_pipeline.add_task(deployed_pipeline)
+reader_1 = Reader("reader_1", runtime_parties=dict(guest=guest, host=host))
+reader_1.guest.task_parameters(namespace=f"experiment", name="breast_hetero_guest")
+reader_1.hosts[0].task_parameters(namespace=f"experiment", name="breast_hetero_host")
+deployed_pipeline.psi_0.input_data = reader_1.outputs["output_data"]
+
+predict_pipeline.add_tasks([reader_1, deployed_pipeline])
 predict_pipeline.compile()
 ```
 
 New pipeline can then initiate prediction.
 
 ```python
-    predict_pipeline.predict()
+predict_pipeline.predict()
 ```
 
 ## Local File to DataFrame