Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get activations of a specific layer of the multifit model #72

Open
francis967 opened this issue May 29, 2020 · 0 comments
Open

Get activations of a specific layer of the multifit model #72

francis967 opened this issue May 29, 2020 · 0 comments

Comments

@francis967
Copy link

Hello everybody,
Anyone knows how to get the activations of an intermediate layer when I make a prediction, despite what I have tried so far, it gives me errors.To give you a little context:

The first thing I do is extract the model layer(for example the first embeddings layer):
self.specific_layer = list(self.classifier.model.modules())[0][0].module.encoder

The second thing I do is put a hook on the layer I want to get the activations from :
def hook_function(module, grad_in, grad_out): self.gradients = grad_out[0]
self.specific_layer.register_backward_hook(hook_function)

Then I vectorize the text input using SentencePiece and give it as input to the model like this, similar to how I've seen it done in other fastai models:
model_output = self.classifier.model(self.inputs)

But when i try to run the above code line it gives me this error:
File "/home/francis/.virtualenvs/my_project/lib/python3.7/site-packages/fastai/text/learner.py", line 261, in forward bs,sl = input.size() AttributeError: 'list' object has no attribute 'size'
Does anyone know what the problem might be?, or if there is a better way to get the activations of an intermediate layer in a multifit model.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant