We introduce GSrec, which aims to design a surface-aligned Gaussian Splatting and benefits the surface reconstruction.
Key idea: Monocular geometry guidance to augment 3DGS with normal attributes, then use neural implicit representation to joint optimize the moving least square field formed by the 3DGS as regularization.
We tested on a workstation configured with Ubuntu 22.04, cuda 11.6 and gcc 9.5.
- Clone this repo:
git clone https://github.com/QianyiWu/gsrec --recursive
cd gsrec
- Install dependencies
conda env create --file environment.yml
conda activate gsrec
We provided our preprocessed Replica dataset (8 scenes, ~1.7GB) for reference. You can download the dataset and put it in the data folder. The data structure is similar to other 3DGS projects and will be organized as follows:
data/
├── dataset_name ("replica" in this case)
│ ├── scan1/
│ │ ├── images
│ │ │ ├── IMG_0.jpg
│ │ │ ├── IMG_1.jpg
│ │ │ ├── ...
│ │ ├── sparse/
│ │ └──0/
│ ├── scan2/
│ │ ├── images
│ │ │ ├── IMG_0.jpg
│ │ │ ├── IMG_1.jpg
│ │ │ ├── ...
│ │ ├── sparse/
│ │ └──0/
...
For custom data, you should process the image sequences with Colmap to obtain the SfM points and camera poses. Then, place the results into data/ folder.
For training a single scene, take scan1 from Replica as an example:
./train_single.sh
It will save the results into outputs/scan1. The training should take about 1 hour till the end.
You can use the following code to extract mesh. Take the outputs from previous steps as an example:
python extract_mesh.py -m outputs/scan1 --mesh_type poisson
If you find our work helpful, please consider citing:
@inproceedings{Wu2024gsrec,
author = {Wu, Qianyi and Zheng, Jianmin and Cai, Jianfei},
title = {Surface Reconstruction from 3D Gaussian Splatting via Local Structural Hints},
booktitle = {European Conference on Computer Vision},
year = {2024}
}Please follow the LICENSE of 3D-GS.
Surface reconstruction for 3DGS is a very important task and we found several concurrent works along this direction. You can check the Related links in our project page. These works are super cool and insightful.
We thank all authors from 3D-GS for presenting such an excellent work. We also thanks all the authors from Scaffold-GS, which we choose as a base model.