This repository is built for the official implementation of:
CP3: Unifying Point Cloud Completion by Pretrain-Prompt-Predict Paradigm (T-PAMI) [paper],
Point cloud completion aims to predict complete shape from its partial observation. Current approaches mainly consist of generation and refinement stages in a coarse-to-fine style. However, the generation stage often lacks robustness to tackle different incomplete variations, while the refinement stage blindly recovers point clouds without the semantic awareness.
To tackle these challenges, we unify point cloud Completion by a generic Pretrain-Prompt-Predict paradigm, namely CP3. Inspired by prompting approaches from NLP, we creatively reinterpret point cloud generation and refinement as the prompting and predicting stages, respectively. Then, we introduce a concise self-supervised pretraining stage before prompting. It can effectively increase robustness of point cloud generation, by an Incompletion-Of-Incompletion (IOI) pretext task. Moreover, we develop a novel Semantic Conditional Refinement (SCR) network at the predicting stage. It can discriminatively modulate multi-scale refinement with the guidance of semantics. Finally, extensive experiments demonstrate that our CP3 outperforms the state-of-the-art methods with a large margin.
Download the MVP completion dataset(https://www.dropbox.com/sh/la0kwlqx4n2s5e3/AACjoTzt-_vlX6OF9mfSpFMra?dl=0&lst=)
- pip -r requirements.txt
- source setup.sh (refer to https://github.com/paul007pl/MVP_Benchmark/blob/main/setup.sh )
The pre-trained model path in generation stage: ./log/vrcnet_cd_debug_2022-02-22T17:07:04/pretrain.pth ([download]) You can also retrain the generation stage yourself. During the training process, you need to modify the code to comment out the refinement network.
- To train a model: run
python train.py -c ./cfgs/*.yaml
, e.g.python train.py -c ./cfgs/cp3.yaml
- To test a model: run
python val.py -c ./cfgs/*.yaml
, e.g.python val.py -c ./cfgs/cp3.yaml
The model path in refinement stage: ./log/vrcnet_cd_debug_2022-02-22T17:07:04/best_cd_t_network_refinement_16384.pth (download) ([training log])
If you use this code, please consider citing:
@ARTICLE{10097548,
author={Xu, Mingye and Wang, Yali and Liu, Yihao and He, Tong and Qiao, Yu},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
title={CP3: Unifying Point Cloud Completion by Pretrain-Prompt-Predict Paradigm},
year={2023},
volume={},
number={},
pages={1-15},
doi={10.1109/TPAMI.2023.3257026}}
We include the following libraries and algorithms:
[1] CD
[2] VRCNet
[3] SnowflakeNet