feat: Add multi-layer, bidirectional RNN with tanh activation #28804
Conversation
func_wrapper.py is a Python module designed to streamline the integration of Hugging Face Transformers into your natural language processing (NLP) projects. It provides a set of input and output conversion wrappers to simplify the process of passing data between your custom functions and Transformers' data structures. Input Conversion Wrappers: inputs_to_transformers_tensors: This wrapper converts input data (text, tensors, etc.) into Transformers-compatible data structures. It is particularly useful when your custom functions expect diverse input types. Output Conversion Wrappers: outputs_to_pytorch_tensors: After your custom function returns data, this wrapper ensures that the output data is converted into PyTorch tensors or other appropriate formats. Usage: Import func_wrapper.py into your project. Initialize a Hugging Face Transformers model and tokenizer. Wrap your custom function with to_transformers_tensors_and_back. This wrapped function can now accept and return Transformers-compatible data. Here's a simple example of how to use func_wrapper.py: import torch from transformers import BertForSequenceClassification, BertTokenizer from ivy.functional.frontends.transformers.func_wrapper import to_transformers_tensors_and_back # Initialize the model and tokenizer model_name = "bert-base-uncased" model = BertForSequenceClassification.from_pretrained(model_name) tokenizer = BertTokenizer.from_pretrained(model_name) # Wrap your custom function using the conversion wrappers wrapped_function = to_transformers_tensors_and_back(your_function, model, tokenizer) # Prepare sample input data sample_input_text = "This is a sample input text." sample_input_tensor = torch.rand((3, 3)) # Call your wrapped function with the sample input data output = wrapped_function(sample_input_text, sample_input_tensor) # The output is automatically converted to PyTorch tensors print(output) Please note that func_wrapper.py is still in development, and further enhancements and refinements are expected. Your feedback and contributions to improve its functionality are welcome.
|
I would like to request your review of this pull request. Your input and insights are greatly valued. Thank you, |
There was a problem hiding this comment.
Hey @muzakkirhussain011 thanks for the contribution!
Please could you add tests for ivy.rnn_tanh and ivy.rnn_tanh_update? The tests will need to go in ivy_tests/test_ivy/test_functional/test_nn/test_layers.py, so you can look at the tests already there to follow the precedent of how to do this.
I think it would also make sense to add the torch frontend _VF.rnn_tanh and _VF.rnn_tanh_cell in this same PR, would you be able to look into that?
Thanks again 😁
ivy/functional/ivy/layers.py
Outdated
| return output[:, -1], output, h_outs | ||
|
|
||
|
|
||
| def _rnn_tanh_cell( |
There was a problem hiding this comment.
We could make this a public rather than private method, so ivy.rnn_tanh_cell can be used in the torch frontend rnn_tanh_cell implementation?
|
Hi @Sam-Armstrong, |
PR Description
This pull request introduces a multi-layer bidirectional RNN with tanh activation.
Changes Made:
rnn_tanhfunction for multi-layer RNNs with optional bidirectional configuration.rnn_tanh_updatefor processing RNN cell updates.Key Features:
Usage:
rnn_tanhwith input data, initial state, weights, and configuration parameters.This update enhances the flexibility and functionality of RNN implementations, providing robust support for various RNN configurations.
Related Issue
#28798
Checklist