Skip to content

Commit

Permalink
release 0.1.8, fixed broken API docs and URL
Browse files Browse the repository at this point in the history
  • Loading branch information
wey-gu committed Feb 28, 2023
1 parent 5afd2f2 commit fc721f9
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 29 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ pyspark --driver-class-path /opt/nebulagraph/ngdi/package/nebula-spark-connector
--jars /opt/nebulagraph/ngdi/package/nebula-algo.jar
```

Then we could access Jupyter Notebook with PySpark and refer to [examples/spark_engine.ipynb](examples/spark_engine.ipynb)
Then we could access Jupyter Notebook with PySpark and refer to [examples/spark_engine.ipynb](https://github.com/wey-gu/nebulagraph-di/examples/spark_engine.ipynb)

## Submit Algorithm job to Spark Cluster(Spark Engine)

Expand Down Expand Up @@ -154,13 +154,13 @@ python3 pagerank.py

## Documentation

[API Reference](docs/API.md)
[API Reference](https://github.com/wey-gu/nebulagraph-di/docs/API.md)

## Usage

### Spark Engine Examples

See also: [examples/spark_engine.ipynb](examples/spark_engine.ipynb)
See also: [examples/spark_engine.ipynb](https://github.com/wey-gu/nebulagraph-di/examples/spark_engine.ipynb)

```python
from ngdi import NebulaReader
Expand Down
50 changes: 25 additions & 25 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,63 +2,63 @@

## NebulaGraphConfig

ngdi.NebulaGraphConfig is the configuration for ngdi.NebulaReader, ngdi.NebulaWriter and ngdi.NebulaAlgorithm.
`ngdi.NebulaGraphConfig` is the configuration for `ngdi.NebulaReader`, `ngdi.NebulaWriter` and `ngdi.NebulaAlgorithm`.

See code for details: [ngdi/config.py](../ngdi/config.py)

## NebulaReader

ngdi.NebulaReader reads data from NebulaGraph and constructs NebulaDataFrameObject or NebulaGraphObject.
`ngdi.NebulaReader` reads data from NebulaGraph and constructs `NebulaDataFrameObject` or `NebulaGraphObject`.
It supports different engines, including Spark and NebulaGraph. The default engine is Spark.

Per each engine, ngdi.NebulaReader supports different read modes, including query, scan, and load.
Per each engine, `ngdi.NebulaReader` supports different read modes, including query, scan, and load.
For now, Spark Engine supports query, scan and load modes, while NebulaGraph Engine supports query and scan modes.

In Spark Engine, all modes are underlyingly implemented by NebulaGraph Spark Connector nGQL Reader, while in the future, query mode will be optionally done by opencypher/morpheus to bypass the Nebula-GraphD.
In Spark Engine:
- All modes are underlyingly implemented by NebulaGraph Spark Connector nGQL Reader, while in the future, query mode will be optionally done by opencypher/morpheus to bypass the Nebula-GraphD.
- The `NebulaDataFrameObject` returned by `ngdi.NebulaReader.read()` is a Spark DataFrame, which can be further processed by Spark SQL or Spark MLlib.

The NebulaDataFrameObject returned by ngdi.NebulaReader.read() is a Spark DataFrame, which can be further processed by Spark SQL or Spark MLlib. And the graph returned by ngdi.NebulaReader.to_graph() is a GraphX Graph, which can be further processed by GraphX. In the future, the graph returned by ngdi.NebulaReader.to_graph() will support GraphFrame, too.

In NebulaGraph Engine, query mode is implemented by Python Nebula-GraphD Client, while scan mode the Nebula-StoreageD Client.

The NebulaDataFrameObject returned by ngdi.NebulaReader.read() is a Pandas DataFrame, which can be further processed by Pandas. And the graph returned by ngdi.NebulaReader.to_graph() is a NetworkX Graph, which can be further processed by NetworkX.
In NebulaGraph Engine:
- Query mode is implemented by Python Nebula-GraphD Client, while scan mode the Nebula-StoreageD Client.
- The `NebulaDataFrameObject` returned by `ngdi.NebulaReader.read()` is a Pandas DataFrame, which can be further processed by Pandas. And the graph returned by ngdi.

### Functions

- ngdi.NebulaReader.query() sets the query statement.
- ngdi.NebulaReader.scan() sets the scan statement.
- ngdi.NebulaReader.load() sets the load statement.
- ngdi.NebulaReader.read() executes the read operation and returns a DataFrame or NebulaGraphObject.
- ngdi.NebulaReader.show() shows the DataFrame returned by ngdi.NebulaReader.read().
- `ngdi.NebulaReader.query()` sets the query statement.
- `ngdi.NebulaReader.scan()` sets the scan statement.
- `ngdi.NebulaReader.load()` sets the load statement.
- `ngdi.NebulaReader.read()` executes the read operation and returns a DataFrame or `NebulaGraphObject`.
- `ngdi.NebulaReader.show()` shows the DataFrame returned by `ngdi.NebulaReader.read()`.

## engines

ngdi.engines.SparkEngine is the Spark Engine for ngdi.NebulaReader, ngdi.NebulaWriter and ngdi.NebulaAlgorithm.
- `ngdi.engines.SparkEngine` is the Spark Engine for `ngdi.NebulaReader`, `ngdi.NebulaWriter` and `ngdi.NebulaAlgorithm`.

ngdi.engines.NebulaEngine is the NebulaGraph Engine for ngdi.NebulaReader, ngdi.NebulaWriter.
- `ngdi.engines.NebulaEngine` is the NebulaGraph Engine for `ngdi.NebulaReader`, `ngdi.NebulaWriter`.

ngdi.engines.NetworkXEngine is the NetworkX Engine for ngdi.NebulaAlgorithm.
- `ngdi.engines.NetworkXEngine` is the NetworkX Engine for `ngdi.NebulaAlgorithm`.

## NebulaDataFrameObject
## `NebulaDataFrameObject`

ngdi.NebulaDataFrameObject is a Spark DataFrame or Pandas DataFrame, which can be further processed by Spark SQL or Spark MLlib or Pandas.
ngdi.`NebulaDataFrameObject` is a Spark DataFrame or Pandas DataFrame, which can be further processed by Spark SQL or Spark MLlib or Pandas.

### Functions

- ngdi.NebulaDataFrameObject.algo.pagerank() runs the PageRank algorithm on the Spark DataFrame.
- ngdi.NebulaDataFrameObject.to_graphx() converts the DataFrame to a GraphX Graph. not yet implemented.
- `ngdi.NebulaDataFrameObject.algo.pagerank()` runs the PageRank algorithm on the Spark DataFrame.
- `ngdi.NebulaDataFrameObject.to_graphx()` converts the DataFrame to a GraphX Graph. not yet implemented.

## NebulaGraphObject

ngdi.NebulaGraphObject is a GraphX Graph or NetworkX Graph, which can be further processed by GraphX or NetworkX.
`ngdi.NebulaGraphObject` is a GraphX Graph or NetworkX Graph, which can be further processed by GraphX or NetworkX.

### Functions

- ngdi.NebulaGraphObject.algo.pagerank() runs the PageRank algorithm on the NetworkX Graph. not yet implemented.
- `ngdi.NebulaGraphObject.algo.pagerank()` runs the PageRank algorithm on the NetworkX Graph. not yet implemented.

## NebulaAlgorithm

ngdi.NebulaAlgorithm is a collection of algorithms that can be run on ngdi.NebulaDataFrameObject(spark engine) or ngdi.NebulaGraphObject(networkx engine).
`ngdi.NebulaAlgorithm` is a collection of algorithms that can be run on ngdi.`NebulaDataFrameObject`(spark engine) or `ngdi.NebulaGraphObject`(networkx engine).

## NebulaGNN

ngdi.NebulaGNN is a collection of graph neural network models that can be run on ngdi. not yet implemented.
`ngdi.NebulaGNN` is a collection of graph neural network models that can be run on ngdi. not yet implemented.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[project]
name = "ngdi"
version = "0.1.7"
version = "0.1.8"
description = "NebulaGraph Data Intelligence Suite"
authors = [
{name = "Wey Gu", email = "weyl.gu@gmail.com"},
Expand Down

0 comments on commit fc721f9

Please sign in to comment.