Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update full text index #1936

Merged
merged 2 commits into from
Feb 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions docs-2.0/3.ngql-guide/14.native-index-statements/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,22 +38,12 @@ Full-text indexes are used to do prefix, wildcard, regexp, and fuzzy search on a

- Full-text indexes allow indexing just one property.

- Only strings within a specified length (no longer than 256 bytes) are indexed.

- Full-text indexes do not support logical operations such as `AND`, `OR`, and `NOT`.

!!! Note

To do complete string matches, use native indexes.

### Operations on full-text indexes

Before doing any operations on full-text indexes, please make sure that you deploy full-text indexes. Details on full-text indexes deployment, see [Deploy Elasticsearch](../../4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md) and [Deploy Listener](../../4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md).

At this time, full-text indexes are created automatically on the Elasticsearch cluster. And rebuilding or altering full-text indexes are not supported. To drop full-text indexes, you need to drop them on the Elasticsearch cluster manually.

To query full-text indexes, see [Search with full-text indexes](../15.full-text-index-statements/1.search-with-text-based-index.md).

## Null values

Indexes do not support indexing null values.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ Before using the full-text index, make sure that you know the [restrictions](../

## Natural language full-text search

A natural language search interprets the search string as a phrase in natural human language. The search is case-insensitive. By default, each substring (separated by spaces) will be searched separately. For example, there are three vertices with the tag `player`. The tag `player` contains the property `name`. The `name` of these three vertices are `Kevin Durant`, `Tim Duncan`, and `David Beckham`. Now that the full-text index of `player.name` is established, these three vertices will be queried when using the prefix search statement `LOOKUP ON player WHERE PREFIX(player.name,"d");`.
A natural language search interprets the search string as a phrase in natural human language. The search is case-sensitive and by default prefixes the string with a match. For example, there are three vertices with the tag `player`. The tag `player` contains the property `name`. The `name` of these three vertices are `Kevin Durant`, `Tim Duncan`, and `David Beckham`. Now that the full-text index of `player.name` is established, only `David Beckham` will be queried when using the prefix search statement `LOOKUP ON player WHERE PREFIX(player.name,"D");`.

## Syntax

### Create full-text indexes

```ngql
CREATE FULLTEXT {TAG | EDGE} INDEX <index_name> ON {<tag_name> | <edge_name>} ([<prop_name_list>]);
CREATE FULLTEXT {TAG | EDGE} INDEX <index_name> ON {<tag_name> | <edge_name>} ([<prop_name>]);
```

### Show full-text indexes
Expand All @@ -36,6 +36,9 @@ SHOW FULLTEXT INDEXES;
REBUILD FULLTEXT INDEX;
```

!!! caution
When there is a large amount of data, rebuilding full-text index is slow, you can modify `snapshot_send_files=false` in the configuration file of Storage service(`nebula-storaged.conf`).

### Drop full-text indexes

```ngql
Expand Down Expand Up @@ -94,13 +97,7 @@ nebula> SHOW LISTENER;
// This example creates the tag.
nebula> CREATE TAG IF NOT EXISTS player(name string, age int);

// This example creates the native index.
nebula> CREATE TAG INDEX IF NOT EXISTS name ON player(name(20));

// This example rebuilds the native index.
nebula> REBUILD TAG INDEX;

// This example creates the full-text index. The index name starts with "nebula".
// This example creates the full-text index. The index name starts with "nebula_".
nebula> CREATE FULLTEXT TAG INDEX nebula_index_1 ON player(name);

// This example rebuilds the full-text index.
Expand Down
106 changes: 0 additions & 106 deletions docs-2.0/3.ngql-guide/6.functions-and-expressions/17.ES-function.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -8,42 +8,26 @@ For now, full-text search has the following limitations:

- Currently, full-text search supports `LOOKUP` statements only.

- Uppercase letters are not allowed in a full-text index name.

- The maximum indexing string length is 256 bytes. The part of data that exceeds 256 bytes will not be indexed.

- Full-text search is native-index-dependent. You need to first create an index for tags/edge types and then you can use the full-text index feature.
- The full-text index name must starts with `nebula_` and can contain only numbers, lowercase letters, underscores, and strings no longer than 256 bytes.

- If there is a full-text index on the tag/edge type, the tag/edge type cannot be deleted or modified.

- One tag/edge type can only have one full-text index.

- The type of properties must be `string`.
- The type of properties must be `STRING` or `FIXED_STRING`.

- Full-text index can not be applied to search multiple tags/edge types.

- Sorting for the returned results of the full-text search is not supported. Data is returned in the order of data insertion.

- Full-text index can not search properties with value `NULL`.

- Elastic Search creates the corresponding index automatically when writing data. The error `text search not found` will be displayed when users created a full-text index but no corresponding data was written.

- Altering Elasticsearch indexes is not supported at this time.

- The pipe operator is not supported.

- `WHERE` clauses supports full-text search only working on single terms.

- Full-text indexes are not deleted together with the graph space.

- Make sure that you start the Elasticsearch cluster and Nebula Graph at the same time. If not, the data writing on the Elasticsearch cluster can be incomplete.

- Do not contain `'` or `\` in the vertex or edge values. If not, an error will be caused in the Elasticsearch cluster storage.

- It may take a while for Elasticsearch to create indexes. If Nebula Graph warns no index is found, wait for the index to take effect (however, the waiting time is unknown and there is no code to check).

- NebulaGraph clusters deployed with K8s do not support the full-text search feature.

{{ ent.ent_begin }}
- Can not be used together with [calles()](../../3.ngql-guide/6.functions-and-expressions/17.ES-function.md) function.
{{ ent.ent_end }}
Original file line number Diff line number Diff line change
Expand Up @@ -10,65 +10,13 @@ Before you start using the full-text index, please make sure that you know the [

To deploy an Elasticsearch cluster, see [Kubernetes Elasticsearch deployment](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-elasticsearch.html) or [Elasticsearch installation](https://www.elastic.co/guide/en/elasticsearch/reference/7.15/targz.html).

When the Elasticsearch cluster is started, add the template file for the Nebula Graph full-text index.

!!! note

The following template works for ES 7.8. The template content may vary depending on different ES versions. Search [Elasticsearch documents](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html) for the template corresponding to your ES version.

```json
{
"template": "nebula*",
"settings": {
"index": {
"number_of_shards": 3,
"number_of_replicas": 1
}
},
"mappings": {
"properties" : {
"tag_id" : { "type" : "long" },
"column_id" : { "type" : "text" },
"value" :{ "type" : "keyword"}
}
}
}
```

Make sure that you specify the following fields in strict accordance with the preceding template format:
!!! compatibility

```json
"template": "nebula*"
"tag_id" : { "type" : "long" },
"column_id" : { "type" : "text" },
"value" :{ "type" : "keyword"}
```
For NebulaGraph 3.4 and later versions, no additional templates need to be created.

!!! caution

When creating a full-text index, start the index name with `nebula`.

For example:

```bash
curl -H "Content-Type: application/json; charset=utf-8" -XPUT http://127.0.0.1:9200/_template/nebula_index_template -d '
{
"template": "nebula*",
"settings": {
"index": {
"number_of_shards": 3,
"number_of_replicas": 1
}
},
"mappings": {
"properties" : {
"tag_id" : { "type" : "long" },
"column_id" : { "type" : "text" },
"value" :{ "type" : "keyword"}
}
}
}'
```
The full-text index name must starts with `nebula_`.

You can configure the Elasticsearch to meet your business needs. To customize the Elasticsearch, see [Elasticsearch Document](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html).

Expand All @@ -92,6 +40,10 @@ nebula> SIGN IN TEXT SERVICE (127.0.0.1:9200, HTTP);

Elasticsearch does not have a username or password by default. If you configured a username and password, you need to specify them in the `SIGN IN` statement.

!!! caution

The Elasticsearch client can only be logged in once, and if there are changes, you need to `SIGN OUT` and then `SIGN IN` again, and the client takes effect globally, and multiple graph spaces share the same Elasticsearch client.

## Show text search clients

The `SHOW TEXT SEARCH CLIENTS` statement can list the text search clients.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,3 @@ Run the `REMOVE LISTENER ELASTICSEARCH` statement to remove all Listeners in a g
```ngql
nebula> REMOVE LISTENER ELASTICSEARCH;
```

!!! danger

After the Listener is deleted, it cannot be added again. Therefore, the synchronization to the ES cluster cannot be continued and the text index data will be incomplete. If needed, you can only recreate the graph space.

## Next

After deploying the [Elasticsearch cluster](2.deploy-es.md) and the Listener, full-text indexes are created automatically on the Elasticsearch cluster. Users can do full-text search now. For more information, see [Full-Text search](../../3.ngql-guide/15.full-text-index-statements/1.search-with-text-based-index.md).
2 changes: 0 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -332,8 +332,6 @@ nav:
- Conditional expressions: 3.ngql-guide/6.functions-and-expressions/5.conditional-expressions.md
- Predicate functions: 3.ngql-guide/6.functions-and-expressions/8.predicate.md
- Geography functions: 3.ngql-guide/6.functions-and-expressions/14.geo.md
#ent
- Elasticsearch functions: 3.ngql-guide/6.functions-and-expressions/17.ES-function.md
- User-defined functions: 3.ngql-guide/6.functions-and-expressions/9.user-defined-functions.md

- General queries statements:
Expand Down