FACE_IMAGE_ANIMATION.md

Face Image Animation

Introduction

We discuss the details about the Face Image Animation here. We use the edge maps as the structure guidance of target frames. Given an input reference image and a guidance video sequence, our model generating a video containing the specific movements.

Left: Input image; Right: Output results.

Dataset Preparation

We use the real video of the FaceForensics dataset. It contains 1000 Videos in total. We use 900 videos for training and 100 videos for testing.

• Crop the videos so that the movement of faces is dominant in the videos.
• Extract video frames and put the images into the forder dataset/FaceForensics/train_data
• Download face key point extractor shape_predictor_68_face_landmarks.dat form here. Put the file under the forder dataset/FaceForensics.
• Run the following code to generate the keypoint files for the extracted frames.

python script/obtain_face_kp.py

Training and Testing

Run the Following code to train our model.

python train.py \
--name=face \
--model=face \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0,1 \
--dataset_mode=face \
--dataroot=./dataset/FaceForensics\

You can download our trained model from here.

Put the obtained weights into ./result/face_checkpoints. Then run the following code to generate the animation videos.

python test.py \
--name=face_checkpoints \
--model=face \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0 \
--dataset_mode=face \
--dataroot=./dataset/FaceForensics \
--results_dir=./eval_results/face \
--nThreads=1

The visdom is required to show the temporary results. You can access these results with:

http://localhost:8096