You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Post-LN is not supported in MCore transformer at the moment (everything is Pre-LN). In order to convert HF checkpoints, we want to add post-LN style to MCore as well. Post-LN requires an additional LN in the beginning of the network as well.
Describe the solution you'd like
Please refer to NeMo code update here for HF Bert support: #8088
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Post-LN is not supported in MCore transformer at the moment (everything is Pre-LN). In order to convert HF checkpoints, we want to add post-LN style to MCore as well. Post-LN requires an additional LN in the beginning of the network as well.
Describe the solution you'd like
Please refer to NeMo code update here for HF Bert support: #8088
The text was updated successfully, but these errors were encountered: