Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate download and ingest into job manager #1219

Merged
merged 1 commit into from
Apr 7, 2022
Merged

migrate download and ingest into job manager #1219

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions docs-2.0/3.ngql-guide/4.job-statements.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,31 @@ nebula> SUBMIT JOB STATS;
+------------+
```

## SUBMIT JOB DOWNLOAD/INGEST

The `SUBMIT JOB DOWNLOAD HDFS` and `SUBMIT JOB INGEST` commands are used to import the SST file into Nebula Graph. For detail, see [Import data from SST files](../nebula-exchange/use-exchange/ex-ug-import-from-sst.md)。

The `SUBMIT JOB DOWNLOAD HDFS` command will download the SST file on the specified HDFS.

The `SUBMIT JOB INGEST` command will import the downloaded SST file into Nebula Graph.

For example:

```ngql
nebula> SUBMIT JOB DOWNLOAD HDFS "hdfs://192.168.10.100:9000/sst";
+------------+
| New Job Id |
+------------+
| 10 |
+------------+
nebula> SUBMIT JOB INGEST;
+------------+
| New Job Id |
+------------+
| 11 |
+------------+
```

## SHOW JOB

The Meta Service parses a `SUBMIT JOB` request into multiple tasks and assigns them to the nebula-storaged processes. The `SHOW JOB <job_id>` statement shows the information about a specific job and all its tasks in the current graph space.
Expand Down
4 changes: 2 additions & 2 deletions docs-2.0/7.data-security/1.authentication/3.role-list.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ The privileges of roles and the nGQL statements that each role can use are liste
|Read data|Y|Y|Y|Y|Y|`GO`, `SET`, `PIPE`, `MATCH`, `ASSIGNMENT`, `LOOKUP`, `YIELD`, `ORDER BY`, `FETCH VERTICES`, `Find`, `FETCH EDGES`, `FIND PATH`, `LIMIT`, `GROUP BY`, `RETURN`|
|Write data|Y|Y|Y|Y||`INSERT VERTEX`, `UPDATE VERTEX`, `INSERT EDGE`, `UPDATE EDGE`, `DELETE VERTEX`, `DELETE EDGES`, `DELETE TAG`|
|Show operations|Y|Y|Y|Y|Y|`SHOW`, `CHANGE PASSWORD`|
|Job|Y|Y|Y|Y||`SUBMIT JOB COMPACT`, `SUBMIT JOB FLUSH`, `SUBMIT JOB STATS`, `STOP JOB`, `RECOVER JOB`, `BUILD TAG INDEX`, `BUILD EDGE INDEX`|
|Write space|Y|||||`CREATE SPACE`, `DROP SPACE`, `CREATE SNAPSHOT`, `DROP SNAPSHOT`, `BALANCE`, `ADMIN`, `CONFIG`, `INGEST`, `DOWNLOAD`|
|Job|Y|Y|Y|Y||`SUBMIT JOB COMPACT`, `SUBMIT JOB FLUSH`, `SUBMIT JOB STATS`, `STOP JOB`, `RECOVER JOB`, `BUILD TAG INDEX`, `BUILD EDGE INDEX`、`INGEST`, `DOWNLOAD`|
|Write space|Y|||||`CREATE SPACE`, `DROP SPACE`, `CREATE SNAPSHOT`, `DROP SNAPSHOT`, `BALANCE`, `ADMIN`, `CONFIG`|
<!-- balance-3.1
|Write space|Y|||||`CREATE SPACE`, `DROP SPACE`, `CREATE SNAPSHOT`, `DROP SNAPSHOT`, `BALANCE ZONE`, `BALANCE LEADER`, `ADMIN`, `CONFIG`, `INGEST`, `DOWNLOAD`|
-->
Expand Down
8 changes: 4 additions & 4 deletions docs-2.0/nebula-exchange/use-exchange/ex-ug-import-from-sst.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -502,26 +502,26 @@ Connect to the Nebula Graph database using the client tool and import the SST fi
2. Run the following command to download the SST file:

```ngql
nebula> DOWNLOAD HDFS "hdfs://<hadoop_address>:<hadoop_port>/<sst_file_path>";
nebula> SUBMIT JOB DOWNLOAD HDFS "hdfs://<hadoop_address>:<hadoop_port>/<sst_file_path>";
```

For example:

```ngql
nebula> DOWNLOAD HDFS "hdfs://*.*.*.*:9000/sst";
nebula> SUBMIT JOB DOWNLOAD HDFS "hdfs://*.*.*.*:9000/sst";
```

2. Run the following command to import the SST file:

```ngql
nebula> INGEST;
nebula> SUBMIT JOB INGEST;
```

!!! note

- To download the SST file again, delete the `download` folder in the space ID in the `data/storage/nebula` directory in the Nebula Graph installation path, and then download the SST file again. If the space has multiple copies, the `download` folder needs to be deleted on all machines where the copies are saved.

- If there is a problem with the import and re-importing is required, re-execute `INGEST;`.
- If there is a problem with the import and re-importing is required, re-execute `SUBMIT JOB INGEST;`.

### Step 6: (optional) Validate data

Expand Down