Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

t2m checkpoint model size error #26

Open
yhyu13 opened this issue Apr 10, 2023 · 4 comments
Open

t2m checkpoint model size error #26

yhyu13 opened this issue Apr 10, 2023 · 4 comments

Comments

@yhyu13
Copy link

yhyu13 commented Apr 10, 2023

I've finished setup with py39 and torch 1.12.1 and ran

python demo.py --cfg ./configs/config_mld_humanml3d.yaml --cfg_assets ./configs/assets.yaml --example ./demo/example.txt

But there is an error in loading the pre-trained weight

Length-50: a man kicks with something or someone with his left leg.
Length-100: A person is skipping rope.
Length-100: a person walks backward slowly.
Pointer Pointing at 0
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
/home/hangyu5/anaconda3/envs/t2m_mld/lib/python3.9/site-packages/torch/cuda/__init__.py:146: UserWarning: 
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Traceback (most recent call last):
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/demo.py", line 337, in <module>
    main()
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/demo.py", line 105, in main
    model = get_model(cfg, dataset)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/get_model.py", line 7, in get_model
    return get_module(cfg, datamodule)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/get_model.py", line 17, in get_module
    return Model(cfg=cfg, datamodule=datamodule)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/modeltype/mld.py", line 86, in __init__
    self._get_t2m_evaluator(cfg)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/modeltype/mld.py", line 175, in _get_t2m_evaluator
    self.t2m_moveencoder.load_state_dict(
  File "/home/hangyu5/anaconda3/envs/t2m_mld/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MovementConvEncoder:
	size mismatch for main.0.weight: copying a param with shape torch.Size([512, 259, 4]) from checkpoint, the shape in current model is torch.Size([512, 247, 4]).

Is it an known issue?

@billl-jiang
Copy link
Collaborator

Hi @yhyu13

From the error log, it seems that the input size of evaluators is not correct which indicates the feature size of the dataset is not correct.

self.t2m_moveencoder = t2m_motionenc.MovementConvEncoder(
input_size=cfg.DATASET.NFEATS - 4,
hidden_size=cfg.model.t2m_motionencoder.dim_move_hidden,
output_size=cfg.model.t2m_motionencoder.dim_move_latent,
)

Maybe the problem is caused by the incorrect dataset. The feature size of the HumanML3D data format is 263 and the feature size of the KIT-ML data format is 251. From the log it seems that your HumanML3d dataset feature size is 251 so maybe you are loading the KIT-ML data for the HumanML3D config.

  1. First you can check whether the path of HumanML3D is correct.
    HUMANML3D:
    ROOT: './datasets/humanml3d' # HumanML3D directory
  2. Then you can use NumPy to load the npy files in the HumanML3D folder manually and check whether the shape is [motion_length, 263] which 263 is the feature size of HumanML3D data format.

You can refer to HumanML3D for more details about the HumanML3D dataset.

@ehsanwwe
Copy link

ehsanwwe commented Jul 9, 2024

i have the same error
@yhyu13 how can you solve this ? are you remember ?

@FogSue
Copy link

FogSue commented Oct 17, 2024

Hi @yhyu13

From the error log, it seems that the input size of evaluators is not correct which indicates the feature size of the dataset is not correct.

self.t2m_moveencoder = t2m_motionenc.MovementConvEncoder(
input_size=cfg.DATASET.NFEATS - 4,
hidden_size=cfg.model.t2m_motionencoder.dim_move_hidden,
output_size=cfg.model.t2m_motionencoder.dim_move_latent,
)

Maybe the problem is caused by the incorrect dataset. The feature size of the HumanML3D data format is 263 and the feature size of the KIT-ML data format is 251. From the log it seems that your HumanML3d dataset feature size is 251 so maybe you are loading the KIT-ML data for the HumanML3D config.

  1. First you can check whether the path of HumanML3D is correct.
    HUMANML3D:
    ROOT: './datasets/humanml3d' # HumanML3D directory
  2. Then you can use NumPy to load the npy files in the HumanML3D folder manually and check whether the shape is [motion_length, 263] which 263 is the feature size of HumanML3D data format.

You can refer to HumanML3D for more details about the HumanML3D dataset.

Hello, I believe I have correctly prepared the HumanML3D dataset, as I'm able to obtain the expected shape. However, I'm still encountering persistent shape mismatching errors.
image

@FogSue
Copy link

FogSue commented Oct 17, 2024

Hi @yhyu13
From the error log, it seems that the input size of evaluators is not correct which indicates the feature size of the dataset is not correct.

self.t2m_moveencoder = t2m_motionenc.MovementConvEncoder(
input_size=cfg.DATASET.NFEATS - 4,
hidden_size=cfg.model.t2m_motionencoder.dim_move_hidden,
output_size=cfg.model.t2m_motionencoder.dim_move_latent,
)

Maybe the problem is caused by the incorrect dataset. The feature size of the HumanML3D data format is 263 and the feature size of the KIT-ML data format is 251. From the log it seems that your HumanML3d dataset feature size is 251 so maybe you are loading the KIT-ML data for the HumanML3D config.

  1. First you can check whether the path of HumanML3D is correct.
    HUMANML3D:
    ROOT: './datasets/humanml3d' # HumanML3D directory
  2. Then you can use NumPy to load the npy files in the HumanML3D folder manually and check whether the shape is [motion_length, 263] which 263 is the feature size of HumanML3D data format.

You can refer to HumanML3D for more details about the HumanML3D dataset.

Hello, I believe I have correctly prepared the HumanML3D dataset, as I'm able to obtain the expected shape. However, I'm still encountering persistent shape mismatching errors. image

This is caused by loading kit ckpt for humanml3d!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants