GScream: Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal

ECCV 2024

Yuxin Wang¹, Qianyi Wu², Guofeng Zhang³, Dan Xu^1✉️
1HKUST, ²Monash University, ³Zhejiang University

   

Installation

git clone https://github.com/W-Ted/GScream.git

cd GScream
conda env create -f gscream.yaml
conda activate gscream

cd submodules/diff-gaussian-rasterization/ && pip install -e .
cd ../simple-knn && pip install -e .
cd ../..

Since we used RTX 3090, in the setup.py, we hardcoded the gencode=arch with 'compute_86' and 'sm_86' when compiling 'diff-gaussian-rasterization'. For Tesla V100, you may try changing it to 'compute_70' and 'sm_70' before compiling. issue#4

Dataset

We provide the processed SPIN-NeRF dataset with Marigold depths here(~9.7G). You could download it to the ''data'' directory and unzip it.

cd data
pip install gdown && gdown 'https://drive.google.com/uc?id=1EODx3392p1R7CaX5bazhkDrfrDtnqJXv'
unzip spinnerf_dataset_processed.zip && cd ..

Please refer to SPIN-NeRF dataset for the details of this dataset.

Training & Evaluation

python scripts/run.py

All the results will be save in the ''outputs'' directory.

Acknowledgements

This project is built upon Scaffold-GS. The in-painted images are obtained by SD-inpainting and LaMa. The depth maps are estimated by Marigold. The dataset we used is proposed by SPIN-NeRF. Kudos to these researchers.

Citation

@inproceedings{wang2024gscream,
     title={GScream: Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal},
     author={Wang, Yuxin and Wu, Qianyi and Zhang, Guofeng and Xu, Dan},
     booktitle={ECCV},
     year={2024}
     }