PatRyg99 / PyG-ParallelDataset Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Multi-processing parallel PyTorch Geometric InMemoryDataset.

0 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
README.md		README.md
dataset.py		dataset.py

Repository files navigation

PyG-ParallelDataset

Multi-processing parallel PyTorch Geometric InMemoryDataset.

Features:

Run prefiltering and preprocessing on multiple jobs
Save preprocessed data in chunks of specified length (zipping large files may run into RAM issues even when the dataset itself can fit into RAM)

Dependencies:

torch
pyg
pqdm

About

Multi-processing parallel PyTorch Geometric InMemoryDataset.

Report repository

Releases

No releases published

Packages

No packages published

Languages

Python 100.0%