Which part of your open-source code could be used for robotics task directly

[ThomasCai35]

Thanks for your excellent work! Lately I've been trying to put the Theia model into real-life robotics tasks. Now I would love to ensure that which part of your open-sourced code could be used for robotics task directly. Thanks.

[elicassion]

Hi @ThomasCai35

Theia feature is mainly used for robotics tasks, i.e. using Theia as a vision encoder.

import transformers
from transformers import AutoModel
import torch
model = AutoModel.from_pretrained("theaiinstitute/theia-base-patch16-224-cdiv", trust_remote_code=True)
fake_input = torch.zeros((1, 224 ,224, 3), dtype=torch.uint8)

theia_feature = model.forward_feature(fake_input)
# Theia / intermediate feature, mainly used for robot learning.
# To change different feature reduction methods, pass `feature_reduction_method` argument in AutoModel.from_pretrained() method

You can also find the above in our quick start guide.

Good luck and have fun!