Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model] Move pyg models to examples #299

Merged
merged 1 commit into from
Nov 5, 2021
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Move pyg models
  • Loading branch information
cenyk1230 committed Nov 5, 2021
commit f208cb7d44ee56c211d5551abd448190b65d5a54
1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/bug-report.md
Original file line number Diff line number Diff line change
@@ -28,7 +28,6 @@ Steps to reproduce the behavior:
- OS (e.g., Linux):
- Python version:
- PyTorch version:
- PyG version (if installed):
- CUDA/cuDNN version (if applicable):
- Any other relevant information:

3 changes: 0 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -7,9 +7,6 @@ install:
- pip install https://download.pytorch.org/whl/cpu/torch-1.7.1%2Bcpu-cp37-cp37m-linux_x86_64.whl
- pip install https://pytorch-geometric.com/whl/torch-1.7.0+cpu/torch_scatter-2.0.7-cp37-cp37m-linux_x86_64.whl
- pip install https://pytorch-geometric.com/whl/torch-1.7.0+cpu/torch_sparse-0.6.9-cp37-cp37m-linux_x86_64.whl
- pip install https://pytorch-geometric.com/whl/torch-1.7.0+cpu/torch_cluster-1.5.9-cp37-cp37m-linux_x86_64.whl
- pip install https://pytorch-geometric.com/whl/torch-1.7.0+cpu/torch_spline_conv-1.2.1-cp37-cp37m-linux_x86_64.whl
- pip install torch-geometric
- pip install packaging==20.9
- bash ./scripts/installation/metis.sh
- source ./scripts/installation/gcc.sh
19 changes: 0 additions & 19 deletions Dockerfile

This file was deleted.

33 changes: 3 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -10,7 +10,7 @@

**[Homepage](https://cogdl.ai)** | **[Paper](https://arxiv.org/abs/2103.00959)** | **[100 GNN papers](./gnn_papers.md)** | **[Leaderboards](./results.md)** | **[Documentation](https://cogdl.readthedocs.io)** | **[Datasets](./cogdl/datasets/README.md)** | **[Join our Slack](https://join.slack.com/t/cogdl/shared_invite/zt-b9b4a49j-2aMB035qZKxvjV4vqf0hEg)** | **[中文](./README_CN.md)**

CogDL is a graph representation learning toolkit that allows researchers and developers to easily train and compare baseline or customized models for node classification, graph classification, and other important tasks in the graph domain.
CogDL is a graph deep learning toolkit that allows researchers and developers to easily train and compare baseline or customized models for node classification, graph classification, and other important tasks in the graph domain.

We summarize the contributions of CogDL as follows:

@@ -173,7 +173,7 @@ How to run parallel experiments with GPUs on several models?
If you want to run parallel experiments on your server with multiple GPUs on multiple models, GCN and GAT, on the Cora dataset:

```bash
$ python scripts/parallel_train.py --task node_classification --dataset cora --model gcn gat --device-id 0 1 --seed 0 1 2 3 4
$ python scripts/parallel_train.py --dataset cora --model gcn gat --devices 0 1 --seed 0 1 2 3 4
```

Expected output:
@@ -184,41 +184,14 @@ Expected output:
| ('cora', 'gat') | 0.8262±0.0032 |
</details>

<details>
<summary>
How to use docker container?
</summary>
<br/>
You might also opt to use a Docker container. There is an image available in this repo that you can build with the Torch and CUDA versions available in your system. To build the docker image just run:

```
docker build --build-arg CUDA=YOUR_CUDA_VERSION --build-arg TORCH=YOUR_TORCH_VERSION --tag cogdl .
```

Where `YOUR_CUDA_VERSION` should be cuxxx representing your cuda version (or just cpu) and `YOUR_TORCH_VERSION` should be the version of PyTorch you want to use. For example, to run with CUDA 10.1 and PyTorch 1.7.1 you can run:
```
docker build --build-arg CUDA=cu101 --build-arg TORCH=1.7.1 --tag cogdl .
```

Then you can start the container by running:
```
docker run -it -v cogdl:/cogdl cogdl /bin/bash
```

And then clone your fork or this repository into the cogdl folder:
```
git clone https://github.com/THUDM/cogdl /cogdl
```
</details>

<details>
<summary>
How to use models from other libraries?
</summary>
<br/>
If you are familiar with other popular graph libraries, you can implement your own model in CogDL using modules from PyTorch Geometric (PyG).
For the installation of PyG, you can follow the instructions from PyG (https://github.com/rusty1s/pytorch_geometric/#installation).
For the quick-start usage of how to use layers of PyG, you can find some examples in the [examples/pytorch_geometric](https://github.com/THUDM/cogdl/tree/master/examples/pytorch_geometric/).
For the quick-start usage of how to use layers of PyG, you can find some examples in the [examples/pyg](https://github.com/THUDM/cogdl/tree/master/examples/pyg/).
</details>

<details>
38 changes: 3 additions & 35 deletions README_CN.md
Original file line number Diff line number Diff line change
@@ -10,7 +10,7 @@

**[主页](https://cogdl.ai/zh)** | **[论文](https://arxiv.org/abs/2103.00959)** | **[100篇GNN论文](./gnn_papers.md)** | **[排行榜](./results.md)** | **[文档](https://cogdl.readthedocs.io)** | **[数据集](./cogdl/datasets/README.md)** | **[加入我们的Slack](https://join.slack.com/t/cogdl/shared_invite/zt-b9b4a49j-2aMB035qZKxvjV4vqf0hEg)** | **[English](./README.md)**

CogDL是由清华大学计算机系知识工程实验室(KEG)开发的基于图的深度学习的研究工具,基于Python语言和[PyTorch](https://github.com/pytorch/pytorch)。CogDL允许研究人员和开发人员可以轻松地训练和比较基线算法或自定义模型,以进行结点分类,链接预测,图分类,社区发现等基于图结构的任务。 它提供了许多流行模型的实现,包括:非图神经网络算法例如Deepwalk、LINE、Node2vec、NetMF、ProNE、methpath2vec、PTE、graph2vec、DGK等;图神经网络算法例如GCN、GAT、GraphSAGE、FastGCN、GTN、HAN、GIN、DiffPool等。它也提供了一些下游任务,包括结点分类(分为是否具有节点属性),链接预测(分为同构和异构),图分类(分有监督和⽆监督)以及为这些任务构建各种算法效果的排行榜。
CogDL是一款图深度学习工具包,基于[PyTorch](https://github.com/pytorch/pytorch)框架。CogDL允许研究人员和开发人员可以轻松地训练和比较基线算法或自定义模型,以进行结点分类,链接预测,图分类,社区发现等基于图结构的任务。 它提供了许多流行模型的实现,包括:非图神经网络算法例如Deepwalk、LINE、Node2vec、NetMF、ProNE、methpath2vec、PTE、graph2vec、DGK等;图神经网络算法例如GCN、GAT、GraphSAGE、FastGCN、GTN、HAN、GIN、DiffPool等。它也提供了一些下游任务,包括结点分类(分为是否具有节点属性),链接预测(分为同构和异构),图分类(分有监督和⽆监督)以及为这些任务构建各种算法效果的排行榜。

CogDL的特性包括:

@@ -166,7 +166,7 @@ CogDL提供了一种快速的稀疏矩阵乘的操作([GE-SpMM](https://arxiv.
如果你想使用多个 GPU 同时在 Cora 数据集上运行 GCN 和 GAT 模型,可以使用如下指令:

```bash
$ python scripts/parallel_train.py --task node_classification --dataset cora --model gcn gat --device-id 0 1 --seed 0 1 2 3 4
$ python scripts/parallel_train.py --dataset cora --model gcn gat --devices 0 1 --seed 0 1 2 3 4
```

预计得到的结果如下:
@@ -177,46 +177,14 @@ $ python scripts/parallel_train.py --task node_classification --dataset cora --m
| ('cora', 'gat') | 0.8262±0.0032 |
</details>

<details>
<summary>
如何使用docker容器来配置cogdl的环境?
</summary>
<br/>
您也可以选择使用Docker来配置cogdl所需的环境。要构建Docker镜像,只需运行以下命令。

```bash
docker build --build-arg CUDA=YOUR_CUDA_VERSION --build-arg TORCH=YOUR_TORCH_VERSION --tag cogdl .
```
请根据您的CUDA版本(或CPU)更换 `YOUR_CUDA_VERSION` 以及 更换 `YOUR_TORCH_VERSION` 为您使用的PyTorch版本。


例如,使用 CUDA 10.1 和 PyTorch 1.7.1 一起运行

```bash
docker build --build-arg CUDA=cu101 --build-arg TORCH=1.7.1 --tag cogdl .
```

启动容器

```bash
docker run -it -v cogdl:/cogdl cogdl /bin/bash
```

将cogdl克隆到cogdl目录下:

```bash
git clone https://github.com/THUDM/cogdl /cogdl
```
</details>

<details>
<summary>
如何使用其他图深度学习库中的模型?
</summary>
<br/>
如何你对其他图深度学习库(比如PyTorch Geometric)比较熟悉,你可以基于这些库的模块来在CogDL里实现相关模型。
你可以通过下述的指南来安装相应的库,例如PyTorch Geometric (https://github.com/rusty1s/pytorch_geometric/#installation)。
对于如何使用PyG的模块来实现模型,你可以在示例中找到一些参考:[examples/pytorch_geometric](https://github.com/THUDM/cogdl/tree/master/examples/pytorch_geometric/)。
对于如何使用PyG的模块来实现模型,你可以在示例中找到一些参考:[examples/pyg](https://github.com/THUDM/cogdl/tree/master/examples/pyg/)。
</details>

## CogDL团队
2 changes: 1 addition & 1 deletion cogdl/datasets/planetoid_data.py
Original file line number Diff line number Diff line change
@@ -163,7 +163,7 @@ def get(self, idx):
return self.data

def __repr__(self):
return "{}()".format(self.name)
return "{}".format(self.name)

def __len__(self):
return 1
2 changes: 1 addition & 1 deletion cogdl/layers/mlp_layer.py
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@ class MLP(nn.Module):
hidden_dim : int
Size of hidden layer dimension.
use_bn : bool, optional
Apply batch normalization if True, default: `True).
Apply batch normalization if True, default: `True`.
"""

def __init__(
10 changes: 5 additions & 5 deletions cogdl/layers/rgcn_layer.py
Original file line number Diff line number Diff line change
@@ -7,12 +7,12 @@

class RGCNLayer(nn.Module):
"""
Implementation of Relational-GCN in paper `"Modeling Relational Data with Graph Convolutional Networks"`
<https://arxiv.org/abs/1703.06103>
Implementation of Relational-GCN in paper `"Modeling Relational Data with Graph Convolutional Networks"
<https://arxiv.org/abs/1703.06103>`_

Parameters
----------
in_feats : int
Parameters
----------
in_feats : int
Size of each input embedding.
out_feats : int
Size of each output embedding.
9 changes: 3 additions & 6 deletions cogdl/models/__init__.py
Original file line number Diff line number Diff line change
@@ -73,28 +73,25 @@ def build_model(args):
"dgi": "cogdl.models.nn.dgi.DGIModel",
"mvgrl": "cogdl.models.nn.mvgrl.MVGRL",
"patchy_san": "cogdl.models.nn.patchy_san.PatchySAN",
"chebyshev": "cogdl.models.nn.pyg_cheb.Chebyshev",
"gcn": "cogdl.models.nn.gcn.GCN",
"gdc_gcn": "cogdl.models.nn.gdc_gcn.GDC_GCN",
"graphsage": "cogdl.models.nn.graphsage.Graphsage",
"compgcn": "cogdl.models.nn.compgcn.LinkPredictCompGCN",
"drgcn": "cogdl.models.nn.drgcn.DrGCN",
"unet": "cogdl.models.nn.pyg_graph_unet.GraphUnet",
"unet": "cogdl.models.nn.graph_unet.GraphUnet",
"gcnmix": "cogdl.models.nn.gcnmix.GCNMix",
"diffpool": "cogdl.models.nn.diffpool.DiffPool",
"gcnii": "cogdl.models.nn.gcnii.GCNII",
"sign": "cogdl.models.nn.sign.SIGN",
"pyg_gcn": "cogdl.models.nn.pyg_gcn.GCN",
"mixhop": "cogdl.models.nn.mixhop.MixHop",
"gat": "cogdl.models.nn.gat.GAT",
"han": "cogdl.models.nn.han.HAN",
"ppnp": "cogdl.models.nn.ppnp.PPNP",
"grace": "cogdl.models.nn.grace.GRACE",
"pprgo": "cogdl.models.nn.pprgo.PPRGo",
"gin": "cogdl.models.nn.gin.GIN",
"dgcnn": "cogdl.models.nn.pyg_dgcnn.DGCNN",
"grand": "cogdl.models.nn.grand.Grand",
"gtn": "cogdl.models.nn.pyg_gtn.GTN",
"gtn": "cogdl.models.nn.gtn.GTN",
"rgcn": "cogdl.models.nn.rgcn.LinkPredictRGCN",
"deepergcn": "cogdl.models.nn.deepergcn.DeeperGCN",
"drgat": "cogdl.models.nn.drgat.DrGAT",
@@ -104,7 +101,7 @@ def build_model(args):
"mlp": "cogdl.models.nn.mlp.MLP",
"sgc": "cogdl.models.nn.sgc.sgc",
"sortpool": "cogdl.models.nn.sortpool.SortPool",
"srgcn": "cogdl.models.nn.pyg_srgcn.SRGCN",
"srgcn": "cogdl.models.nn.srgcn.SRGCN",
"gcc": "cogdl.models.nn.gcc_model.GCCModel",
"unsup_graphsage": "cogdl.models.nn.unsup_graphsage.SAGE",
"graphsaint": "cogdl.models.nn.graphsaint.GraphSAINT",
3 changes: 3 additions & 0 deletions cogdl/models/base_model.py
Original file line number Diff line number Diff line change
@@ -25,6 +25,9 @@ def _forward_unimplemented(self, *input: Any) -> None: # abc warning
def forward(self, *args):
raise NotImplementedError

def predict(self, data):
return self.forward(data)

@property
def device(self):
return next(self.parameters()).device
1 change: 0 additions & 1 deletion cogdl/models/nn/gdc_gcn.py
Original file line number Diff line number Diff line change
@@ -124,7 +124,6 @@ def get_diffusion(x, edges):
else:
raise ValueError

# create PyG Data object
edges_i = []
edges_j = []
edge_attr = []
File renamed without changes.
File renamed without changes.
9 changes: 1 addition & 8 deletions cogdl/models/nn/pyg_srgcn.py → cogdl/models/nn/srgcn.py
Original file line number Diff line number Diff line change
@@ -2,17 +2,15 @@
import torch.nn as nn
import torch.nn.functional as F
from cogdl.utils.srgcn_utils import act_attention, act_map, act_normalization
from cogdl.utils import add_remaining_self_loops
from cogdl.utils import spmm, add_remaining_self_loops
from torch_sparse import spspmm

from .. import BaseModel
from cogdl.utils import spmm


class NodeAdaptiveEncoder(nn.Module):
def __init__(self, num_features, dropout=0.5):
super(NodeAdaptiveEncoder, self).__init__()
# self.fc = nn.Linear(num_features, 1, bias=True)
self.fc = nn.Parameter(torch.zeros(size=(num_features, 1)))
nn.init.xavier_normal_(self.fc.data, gain=1.414)
self.bf = nn.Parameter(torch.zeros(size=(1,)))
@@ -73,7 +71,6 @@ def forward(self, graph, x):
edge_index = graph.edge_index
N, dim = x.shape

# nl_adj_mat_ind, nl_adj_mat_val = add_self_loops(edge_index, num_nodes=N)[0], edge_attr.squeeze()
nl_adj_mat_ind = add_remaining_self_loops(edge_index, num_nodes=N)[0]
nl_adj_mat_ind = torch.stack(nl_adj_mat_ind)
nl_adj_mat_val = torch.ones(nl_adj_mat_ind.shape[1]).to(x.device)
@@ -101,9 +98,7 @@ def forward(self, graph, x):
graph.edge_weight = adj_mat_val
for _ in range(i + 1):
val_h = spmm(graph, val_h)
# val_h = spmm(adj_mat_ind, F.dropout(adj_mat_val, p=self.node_dropout, training=self.training), N, N, val_h)

# val_h = val_h / norm
val_h[val_h != val_h] = 0
val_h = val_h + self.bias[i]
val_h = self.adaptive_enc[i](val_h)
@@ -148,7 +143,6 @@ def forward(self, graph, x):
# x = self.dropout(x)

edge_index = graph.edge_index
# adj_mat_ind, adj_mat_val = add_self_loops(edge_index, num_nodes=N)[0], edge_attr.squeeze()
adj_mat_ind = add_remaining_self_loops(edge_index, num_nodes=N)[0]
adj_mat_ind = torch.stack(adj_mat_ind)
adj_mat_val = torch.ones(adj_mat_ind.shape[1]).to(x.device)
@@ -168,7 +162,6 @@ def forward(self, graph, x):
# N, dim = val_h.shape

# MATRIX_MUL
# val_h = spmm(adj_mat_ind, F.dropout(adj_mat_val, p=self.node_dropout, training=self.training), N, N, val_h)
with graph.local_graph():
graph.edge_index = adj_mat_ind
graph.edge_weight = adj_mat_val
Loading