Result with Expression -> Artifacts Alleviated (check 0:04)
with out the FLAME expression prior | With the FLAME expression prior |
---|---|
wo_expression.mp4 |
w_expression.mp4 |
※ Use NF-exp branch to implement the 4D gaussian with additional input(expression)
Please follow the 3D-GS to install the relative packages.
git clone https://github.com/whwjdqls/4D-Gaussian-Head
cd 4D-Gaussian-Head
git submodule update --init --recursive
conda create -n 4D-Gaussian-Head python=3.7
conda activate 4D-Gaussian-Head
pip install -r requirements.txt
pip install -e submodules/depth-diff-gaussian-rasterization
pip install -e submodules/simple-knn
In our environment, we use pytorch=1.13.1+cu116.
For synthetic scenes:
The dataset provided in D-NeRF is used. You can download the dataset from dropbox.
For real dynamic scenes:
The dataset provided in HyperNeRF is used. You can download scenes from Hypernerf Dataset and organize them as Nerfies. Meanwhile, Plenoptic Dataset could be downloaded from their official websites. To save the memory, you should extract the frames of each video and then organize your dataset as follows.
├── data
│ | dnerf
│ ├── mutant
│ ├── standup
│ ├── ...
│ | hypernerf
│ ├── interp
│ ├── misc
│ ├── virg
│ | dynerf
│ ├── cook_spinach
│ ├── cam00
│ ├── images
│ ├── 0000.png
│ ├── 0001.png
│ ├── 0002.png
│ ├── ...
│ ├── cam01
│ ├── images
│ ├── 0000.png
│ ├── 0001.png
│ ├── ...
│ ├── cut_roasted_beef
| ├── ...
For training synthetic scenes such as bouncingballs
, run
python train.py -s data/dnerf/bouncingballs --port 6017 --expname "dnerf/bouncingballs" --configs arguments/dnerf/bouncingballs.py
You can customize your training config through the config files.
Run the following script to render the images.
python render.py --model_path "output/dnerf/bouncingballs/" --skip_train --configs arguments/dnerf/bouncingballs.py &
This project face is a fork from 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering project page.
@article{wu20234dgaussians,
title={4D Gaussian Splatting for Real-Time Dynamic Scene Rendering},
author={Wu, Guanjun and Yi, Taoran and Fang, Jiemin and Xie, Lingxi and Zhang, Xiaopeng and Wei Wei and Liu, Wenyu and Tian, Qi and Wang Xinggang},
journal={arXiv preprint arXiv:2310.08528},
year={2023}
}