How to make it process 768 pixel?

[FurkanGozukara]

i am giving 768 px image and 768x768 dense pose video but still getting 512px output

i have edited

inference_config: "configs/inference/inference.yaml"
size: 768
L: 16
S: 1
I: 0
clip: 0
offset: 0
max_length: null
video_type: "condition"
invert_video: false
save_individual_videos: true

save_individual_videos also not working

5_sec_768_dense.mp4

2023-12-05T23-26-23.mp4

[alfredplpl]

You need to rewrite the demo code like this:

demo/gradio_animate.py

    def read_video(video):
        size = (768, 768) #int(size)
        reader = imageio.get_reader(video)
        fps = reader.get_meta_data()['fps']
        assert fps == 25.0, f'Expected video fps: 25, but {fps} fps found'
        return video
    
    def read_image(image, size=512):
        return np.array(Image.fromarray(image).resize((768,768)))

demo/animate.py

            if motion_sequence.endswith('.mp4'):
                control = VideoReader(motion_sequence).read()
                # if control[0].shape[0] != size:
                control = [np.array(Image.fromarray(c).resize((768,768))) for c in control]
                control = np.array(control)

            #if source_image.shape[0] != size:
            source_image = np.array(Image.fromarray(source_image).resize((768,768)))
            H, W, C = source_image.shape

[FurkanGozukara]

@alfredplpl I see it calls animate.py so i have to rewrite it too

let me test

[FurkanGozukara]

I made 768 pixel but quality is even lower haha

2023-12-06T15-02-20.mp4

[zcxu-eric]

That's what I expected, as I mentioned in #18, the quality will decrease because our appearance encoder was trained on 512.