Bridge adapter for Vit-Base is different from ResNet

[peterwisu]

Hi, I have a question in the paper it mentioned that in the bridge adapter, after the zoom layer and linear layer. There are the MHSA for each modality before entering the MHCA which are there in the given code for ResNet in class Interactor

ETRIS/model/layers.py

Line 346 in 11cd4db

class Interactor(nn.Module):

but for VIT it used class InteractorT which perform MHCA directly without MHSA.

ETRIS/model/layers.py

Line 326 in 11cd4db

class InteractorT(nn.Module):

Why didn't you do a MHSA for ViT-based as like in ResNet one?