The following instruction is for reproducing the experiments in Table. 2 in our paper.
Follow the original ARKitScenes instruction to download the data (3dod dataset).
The oversegmentation for ARKitScenes can be download here: train and valid.
After you download the data and our prepared oversegmentations. The Training
and Validation
and oversegmentation folders should be prepared as the following structure for our project:
box2mask/data/ARKitScenes/3dod/
└── Training
├── 44358604 # scene name
├── 44358604_3dod_annotation.json # segmentation label of the scene
├── 44358604_3dod_mesh.ply # mesh file
├── 44358604_frames/ # Containing RGBD camera sequences
├── 45662912
├── 45662912_3dod_annotation.json
├── 45662912_3dod_mesh.ply
├── 45662912_frames/
...
└── Validation/
├── 41069021
├── 41069021_3dod_annotation.json
├── 41069021_3dod_mesh.ply
├── 41069021_frames/
├──
...
└── segmented_train_clean/
├── 47331587_3dod_mesh.0.010000.segs.json
├── 44358604_3dod_mesh.0.010000.segs.json
...
└── segmented_val_clean/
├── 41069021_3dod_mesh.0.010000.segs.json
...
Similar to the main experiment, you can train the model using training.py
from the root folder::
python models/training.py --config configs/arkitscenes.txt
To evaluate with the validation set (producing results like Table 2):
python models/evaluation.py --config configs/arkitscenes.txt
You can also produce visualization by adding option --produce_visualizations
. Producing result for a specific scene can be achived via model/evaluation.py
with --predict_specific_scene
option, see the example below:
python models/evaluation.py --config configs/arkitscenes.txt --predict_specific_scene 42445429 --produce_visualizations
Running the command above will produce the visualization of segmentation result in experiments/arkitscenes/results/[checkpoint]/viz/42445429
where checkpoint
is the loaded checkpoint when running the script.