Fix for old models in change_attention_model #6635
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Vahid <[email protected]>
titu1994
approved these changes
May 11, 2023
ericharper
added a commit
that referenced
this pull request
Aug 11, 2023
* [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <[email protected]> * docs typo fix Signed-off-by: Oleksii Kuchaiev <[email protected]> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Fix ASR notebooks (#4738) Signed-off-by: smajumdar <[email protected]> Signed-off-by: smajumdar <[email protected]> * Multilingual VAD model (#4734) * add ngc link Signed-off-by: fayejf <[email protected]> * add tuned VAD config on ASR data Signed-off-by: fayejf <[email protected]> * yaml note Signed-off-by: fayejf <[email protected]> * update vad asr notebook with mVAD Signed-off-by: fayejf <[email protected]> * update vad infer config comment Signed-off-by: fayejf <[email protected]> * fix Signed-off-by: fayejf <[email protected]> * mvad sd config for ch109 Signed-off-by: fayejf <[email protected]> * update sd readme Signed-off-by: fayejf <[email protected]> * add new mVAD model to doc Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * update sd tutorial with mVAD Signed-off-by: fayejf <[email protected]> * typo fix Signed-off-by: fayejf <[email protected]> Signed-off-by: fayejf <[email protected]> * publish pretrained itn t5 model for English (#4748) Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> * Updated docs and doc paths (#4754) * Updated docs and doc paths Signed-off-by: Virginia Adams <[email protected]> * Update Multitask_Prompt_and_PTuning.ipynb * Update README.rst * Changed branch name to use single quotes Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762) * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <[email protected]> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763) Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> * [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770) Signed-off-by: Xuesong Yang <[email protected]> * Update CMUdict with more recent 0.7b entries (#4768) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * Install pynini in docker container (#4733) Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <[email protected]> * [TTS] deprecated old scripts for ljspeech. (#4780) * deprecated old scripts for ljspeech. * removed relevent function calls in TTS docs. Signed-off-by: Xuesong Yang <[email protected]> * update branch and typos (#4788) Signed-off-by: ericharper <[email protected]> Signed-off-by: ericharper <[email protected]> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <[email protected]> * fixed style. Signed-off-by: Vahid <[email protected]> * dropped seq_range Signed-off-by: Vahid <[email protected]> * fixed indexing in caching methods. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * updated docs. Signed-off-by: Vahid <[email protected]> * addressed comments. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> Signed-off-by: Vahid <[email protected]> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <[email protected]> * Fix style for import Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <[email protected]> * fixed style Signed-off-by: David Mosallanezhad <[email protected]> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <[email protected]> * included comments Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update branch for qa notebook Signed-off-by: ericharper <[email protected]> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <[email protected]> Signed-off-by: sam1373 <[email protected]> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <[email protected]> * Update to reflect new config names Signed-off-by: MaximumEntropy <[email protected]> * Sentencepiece fixes Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Fix finetuning Signed-off-by: MaximumEntropy <[email protected]> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Add finetune eval script Signed-off-by: MaximumEntropy <[email protected]> * Fix name Signed-off-by: MaximumEntropy <[email protected]> * Update Jenkinsfile Signed-off-by: MaximumEntropy <[email protected]> * Update config Signed-off-by: MaximumEntropy <[email protected]> * Fix CI test Signed-off-by: MaximumEntropy <[email protected]> * Update check Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Backward compat Signed-off-by: MaximumEntropy <[email protected]> * Update CI test Signed-off-by: MaximumEntropy <[email protected]> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <[email protected]> * Address comments Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Virginia Adams <[email protected]> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <[email protected]> * fix references Signed-off-by: ekmb <[email protected]> * address review feedback Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <[email protected]> * Fix Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <[email protected]> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <[email protected]> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <[email protected]> * Added padded vocab size Signed-off-by: Virginia Adams <[email protected]> * Fixed val check interval value Signed-off-by: Virginia Adams <[email protected]> * Python format fix Signed-off-by: Virginia Adams <[email protected]> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <[email protected]> * Added back dp=1 check Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> * Update README.rst * Fix uppercasing mismatch for IPA heteronyms (#4860) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <[email protected]> * Revert pin memory Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Sean Naren <[email protected]> * Fix mha (#4866) * fix bug in mha forward function related to cache update return type Signed-off-by: Yang Zhang <[email protected]> * fix lgtm Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> * ipa bug fix (#4871) Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * 1. Applying the same patch to r1.11.0 (#4894) Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Micha Livne <[email protected]> * Update tutorials.rst (#4897) * update readme with apex commit Signed-off-by: ericharper <[email protected]> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <[email protected]> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <[email protected]> * Fix bug with DDP initialization Signed-off-by: Tim Moon <[email protected]> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <[email protected]> * Fix code formatting Signed-off-by: Tim Moon <[email protected]> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Tim Moon <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * latent model support * 1. Debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. * update branch Signed-off-by: ericharper <[email protected]> * fix replace_bos_with_pad not found (#6443) Signed-off-by: Abhinav Khattar <[email protected]> * Support Swiglu in TP PP Conversion (#6437) * Support Swiglu in TP PP Conversion Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) * change bert fork to spawn Signed-off-by: Abhinav Khattar <[email protected]> * num_workers=0 fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) * 1. Meagtron encoder decoder fix for empty validation outputs. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Added/updated new Conformer configs (#6426) * updated conf files. Signed-off-by: Vahid <[email protected]> * added confs. Signed-off-by: Vahid <[email protected]> * moved longconformer confs. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated batch sizes and added fastconformer ctc streaming configs. Signed-off-by: Vahid <[email protected]> * updated batch sizes. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * reduce workers on NMT CI (#6472) Signed-off-by: Abhinav Khattar <[email protected]> * move to nvidia megatron repo (#6465) Signed-off-by: Abhinav Khattar <[email protected]> * Megatron KERPLE positional embeddings (#6478) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. Signed-off-by: Xuesong Yang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xuesong Yang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] FastPitch speaker encoder (#6417) * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * Remove wemb Signed-off-by: hsiehjackson <[email protected]> * Fix import Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss Signed-off-by: hsiehjackson <[email protected]> * Add ConditionalInput Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix import error Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix dataset error Signed-off-by: hsiehjackson <[email protected]> * Remove reference speaker embedding Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove SV encoder Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix length type Signed-off-by: hsiehjackson <[email protected]> * Fix append Signed-off-by: hsiehjackson <[email protected]> * Move error msg Signed-off-by: hsiehjackson <[email protected]> * Add look-up into speaker encoder Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Add valueerror msg Signed-off-by: hsiehjackson <[email protected]> * Move lookup Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove unused Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix error Signed-off-by: hsiehjackson <[email protected]> * Rebase and Fix error Signed-off-by: hsiehjackson <[email protected]> * Fix spk encoder Signed-off-by: hsiehjackson <[email protected]> * Rename n_speakers Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error Signed-off-by: hsiehjackson <[email protected]> --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Sharded manifests for tarred datasets (#6395) * testing sharded manifests Signed-off-by: Dima Rekesh <[email protected]> * compatibility Signed-off-by: Dima Rekesh <[email protected]> * proper fixes Signed-off-by: Dima Rekesh <[email protected]> * adding flag tot convert_to_tarred_audio_dataset Signed-off-by: Dima Rekesh <[email protected]> * shard_manifests conf param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * distributed checks Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py Signed-off-by: Dima Rekesh <[email protected]> * reversing manifest shards flag Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests Signed-off-by: Dima Rekesh <[email protected]> * excluding manifests from webdataset url expansion Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore Signed-off-by: Dima Rekesh <[email protected]> * explicit use of UTF-8 for manifest i/o Signed-off-by: Dima Rekesh <[email protected]> --------- Signed-off-by: Dima Rekesh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) Signed-off-by: Jim O’Regan <[email protected]> * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. Signed-off-by: Mostafa Ghorbandoost <[email protected]> * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. Signed-off-by: Mostafa Ghorbandoost <[email protected]> --------- Signed-off-by: Mostafa Ghorbandoost <[email protected]> * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support Signed-off-by: Dmytro Pykhtar <[email protected]> * added tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * fixed the typos Signed-off-by: Dmytro Pykhtar <[email protected]> * added assertions Signed-off-by: Dmytro Pykhtar <[email protected]> * changed assertion rules Signed-off-by: Dmytro Pykhtar <[email protected]> * deleted unused imports Signed-off-by: Dmytro Pykhtar <[email protected]> * changed tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * updated rampup batch size tests Signed-off-by: Dmytro Pykhtar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling Signed-off-by: Dmytro Pykhtar <[email protected]> * rampup batch size tests changes Signed-off-by: Dmytro Pykhtar <[email protected]> --------- Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default Signed-off-by: Kunal Dhawan <[email protected]> * updated README with the new agg_tokenizer_manifest flag Signed-off-by: Kunal Dhawan <[email protected]> * fixed typo in scripts/speech_recognition/code_switching/README.md Signed-off-by: Kunal Dhawan <[email protected]> * changed agg_tokenizer_manifest to is_lid_manifest Signed-off-by: Kunal Dhawan <[email protected]> --------- Signed-off-by: Kunal Dhawan <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script Signed-off-by: andrusenkoau <[email protected]> * add return encoding embedding option Signed-off-by: andrusenkoau <[email protected]> * update script Signed-off-by: andrusenkoau <[email protected]> * add rnnt and hat ngram decoding script Signed-off-by: andrusenkoau <[email protected]> * add some parameters Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig Signed-off-by: andrusenkoau <[email protected]> * replace return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * generalization of scipt behavior Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * add manual encoder_embeddings calculation Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 Signed-off-by: Andrei Andrusenko <[email protected]> * fix rescoring description Signed-off-by: Andrei Andrusenko <[email protected]> --------- Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * fix replace_bos_with_pad not found (#6443) (#6450) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * reduce workers on NMT CI (#6472) (#6474) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added KERPLE positional embeddings to encoder-decoder. Signed-off-by: Micha Livne <[email protected]> * 1. Added a missing file. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added external index sample. (#6462) Signed-off-by: Micha Livne <[email protected]> * Fix cache aware hybrid bugs (#6466) * Update README to add core installation (#6488) * update README for megatron-core Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Fix typos (#6494) Signed-off-by: smajumdar <[email protected]> * fix broken links r1.18.0 (#6501) * fix broken links Signed-off-by: Evelina <[email protected]> * fix broken links Signed-off-by: Evelina <[email protected]> --------- Signed-off-by: Evelina <[email protected]> * 1. Fixed gaussian hidden transform. Signed-off-by: Micha Livne <[email protected]> * 1. Finished updating hidden loss for MIM. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix custom forward_torch_softmax (#6512) Signed-off-by: Abhinav Khattar <[email protected]> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamp… (#6500) * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator r1.18.0 Signed-off-by: Taejin Park <[email protected]> * ignore keep_logits in FrameBatchASRLogits Signed-off-by: Taejin Park <[email protected]> --------- Signed-off-by: Taejin Park <[email protected]> * [TTS] fixed broken path. (#6514) Signed-off-by: Xuesong Yang <[email protected]> * 1. Added a hiddens module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typos (#6523) (#6539) * Fix typos Signed-off-by: smajumdar <[email protected]> * Fix typos Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> (cherry picked from commit 5468077f5127be1a4c88065de2544f4268b9a6e4) * added back the fast emit section to the configs. (#6540) * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * Fix fp16 (#6543) Signed-off-by: MaximumEntropy <[email protected]> * fix (#6529) Signed-off-by: Abhinav Khattar <[email protected]> * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Change Megatron Enc Dec model to use persistent_workers (#6548) * persistent workers Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by Signed-off-by: KunalDhawan <[email protected]> * updated ASR docs with the fastconformer hybrid checkpoints Signed-off-by: KunalDhawan <[email protected]> * added the fastconformer RNNT and CTC models Signed-off-by: KunalDhawan <[email protected]> --------- Signed-off-by: KunalDhawan <[email protected]> * Add scores for FastConformer models (#6557) Signed-off-by: smajumdar <[email protected]> * Patch transcribe and support offline transcribe for hybrid model (#6550) Signed-off-by: fayejf <[email protected]> * Not doing CastToFloat by default (#6524) * Not doing CastToFloat by default Signed-off-by: Boris Fomitchev <[email protected]> * Added docustring Signed-off-by: Boris Fomitchev <[email protected]> * Dummy commit Signed-off-by: Boris Fomitchev <[email protected]> --------- Signed-off-by: Boris Fomitchev <[email protected]> * temp rtd fix (#6568) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py for speedup (#6565) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths Signed-off-by: He Huang (Steve) <[email protected]> * Update manifest.py Signed-off-by: He Huang (Steve) <[email protected]> --------- Signed-off-by: He Huang (Steve) <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <[email protected]> * address review Signed-off-by: Abhinav Khattar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <[email protected]> * merge Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * More streaming conformer export fixes (#6567) Signed-off-by: Greg Clark <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Fix batch size reconf for T5 FT for multi-validation (#6582) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Updated Megatron LM encoder/decoder to use cfg for hiddens. Signed-off-by: Micha Livne <[email protected]> * 1. Added support to register externalhidden loss / transforms. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make tensor split contiguous (#6580) Signed-off-by: Abhinav Khattar <[email protected]> * Patches from main to r1.18.0 for Virtual Parallel (#6592) * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Switch to megatron core Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 892987169ef277f328e15b71a5a0c9bd961c8ee7) * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 1d813a372ab51688e3af6395d905a4c0366ffd23) * Documentation for ASR-TTS models (#6594) * Add docs about hybrid ASR-TTS models Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about text-only datasets Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS checkpoints Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS configs and training Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * ASR-TTS docs: add to api, fix imports Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * Wrap optional import Signed-off-by: Vladimir Bataev <[email protected]> * Revert general ASR import Signed-off-by: Vladimir Bataev <[email protected]> --------- Signed-off-by: Vladimir Bataev <[email protected]> * Update SDP docs (#6485) * add info about SDP e.g. processor classes in docs Signed-off-by: Elena Rastorgueva <[email protected]> * add link to SDP docs in README Signed-off-by: Elena Rastorgueva <[email protected]> * address code review comments and add SDP overview diagram Signed-off-by: Elena Rastorgueva <[email protected]> * Fix spelling typo Signed-off-by: Elena Rastorgueva <[email protected]> --------- Signed-off-by: Elena Rastorgueva <[email protected]> * Create dummy iters to satisy len checks (#6600) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * Restore GPT support for interleaved pipeline parallelism (#6528) * Restore logic for data-parallel communication with pipeline parallelism in GPT Signed-off-by: Tim Moon <[email protected]> * Support dynamic attention masks in GPT Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos Signed-off-by: Tim Moon <[email protected]> * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit Signed-off-by: Tim Moon <[email protected]> * Distinguish between list of data iterators and data iterator that is a list Signed-off-by: Tim Moon <[email protected]> * Create dummy iters to satisy len checks Signed-off-by: Abhinav Khattar <[email protected]> * Kludge while waiting for Megatron-LM update Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Eric Harper <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * Patch transcribe_util for steaming mode and add wer calculation back to inference scripts (#6601) * fix write Signed-off-by: fayejf <[email protected]> * decoding ctc Signed-off-by: fayejf <[email protected]> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <[email protected]> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * reflect change in asr_evaluator Signed-off-by: fayejf <[email protected]> * reflect som and vahid comment Signed-off-by: fayejf <[email protected]> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <[email protected]> * no text skip Signed-off-by: fayejf <[email protected]> --------- Signed-off-by: fayejf <[email protected]> * 1. Added example conf YAML. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support in tensor_parallel. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hat image to docs (#6619) Signed-off-by: andrusenkoau <[email protected]> * update core commit hash in readme (#6622) Signed-off-by: Abhinav Khattar <[email protected]> * Patch decoding for PC models (#6630) * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * Fix wer.py where 'errors' variable was not set (#6633) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <[email protected]> * fix att_context_size bug for older models. (#6635) Signed-off-by: Vahid <[email protected]> * Add megatron_core to requirements (#6639) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * remove from jenkins Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: ericharper <[email protected]> * remove dup (#6643) Signed-off-by: ericharper <[email protected]> * 1. Fixed config to use names, and added better error messages. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support to pass extra data to hiddens for loss computation. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Working on passing extra data to hiddnes. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed support in loading .nemo without hiddnes module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Improved and fixed logging of validation and testing. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed training logging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed logging of hidden loss. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed logging names. 2. Added logging to hiddens and tokens loss. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed conflicts. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> --------- Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Oleksii Kuchaiev <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: fayejf <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: ericharper <[email protected]> Signed-off-by: Vahid <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: sam1373 <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: ekmb <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Evelina <[email protected]> Signed-off-by: Taejin Park <[email protected]> Signed-off-by: KunalDhawan <[email protected]> Signed-off-by: Boris Fomitchev <[email protected]> Signed-off-by: He Huang (Steve) <[email protected]> Signed-off-by: Greg Clark <[email protected]> Signed-off-by: Elena Rastorgueva <[email protected]> Signed-off-by: Eric Harper <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Oleksii Kuchaiev <[email protected]> Co-authored-by: Jocelyn <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: fayejf <[email protected]> Co-authored-by: bene-ges <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> Co-authored-by: Virginia Adams <[email protected]> Co-authored-by: Zhilin Wang <[email protected]> Co-authored-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> Co-authored-by: anteju <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: David <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Samuel Kriman <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> Co-authored-by: Evelina <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Tim Moon <[email protected]> Co-authored-by: Neha Tadimeti <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Boris Fomitchev <[email protected]> Co-authored-by: He Huang (Steve) <[email protected]> Co-authored-by: Greg Clark <[email protected]> Co-authored-by: Elena Rastorgueva <[email protected]>
guyueh1
pushed a commit
to guyueh1/NeMo
that referenced
this pull request
Aug 14, 2023
* [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <[email protected]> * docs typo fix Signed-off-by: Oleksii Kuchaiev <[email protected]> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Fix ASR notebooks (#4738) Signed-off-by: smajumdar <[email protected]> Signed-off-by: smajumdar <[email protected]> * Multilingual VAD model (#4734) * add ngc link Signed-off-by: fayejf <[email protected]> * add tuned VAD config on ASR data Signed-off-by: fayejf <[email protected]> * yaml note Signed-off-by: fayejf <[email protected]> * update vad asr notebook with mVAD Signed-off-by: fayejf <[email protected]> * update vad infer config comment Signed-off-by: fayejf <[email protected]> * fix Signed-off-by: fayejf <[email protected]> * mvad sd config for ch109 Signed-off-by: fayejf <[email protected]> * update sd readme Signed-off-by: fayejf <[email protected]> * add new mVAD model to doc Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * update sd tutorial with mVAD Signed-off-by: fayejf <[email protected]> * typo fix Signed-off-by: fayejf <[email protected]> Signed-off-by: fayejf <[email protected]> * publish pretrained itn t5 model for English (#4748) Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> * Updated docs and doc paths (#4754) * Updated docs and doc paths Signed-off-by: Virginia Adams <[email protected]> * Update Multitask_Prompt_and_PTuning.ipynb * Update README.rst * Changed branch name to use single quotes Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762) * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <[email protected]> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763) Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> * [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770) Signed-off-by: Xuesong Yang <[email protected]> * Update CMUdict with more recent 0.7b entries (#4768) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * Install pynini in docker container (#4733) Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <[email protected]> * [TTS] deprecated old scripts for ljspeech. (#4780) * deprecated old scripts for ljspeech. * removed relevent function calls in TTS docs. Signed-off-by: Xuesong Yang <[email protected]> * update branch and typos (#4788) Signed-off-by: ericharper <[email protected]> Signed-off-by: ericharper <[email protected]> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <[email protected]> * fixed style. Signed-off-by: Vahid <[email protected]> * dropped seq_range Signed-off-by: Vahid <[email protected]> * fixed indexing in caching methods. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * updated docs. Signed-off-by: Vahid <[email protected]> * addressed comments. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> Signed-off-by: Vahid <[email protected]> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <[email protected]> * Fix style for import Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <[email protected]> * fixed style Signed-off-by: David Mosallanezhad <[email protected]> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <[email protected]> * included comments Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update branch for qa notebook Signed-off-by: ericharper <[email protected]> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <[email protected]> Signed-off-by: sam1373 <[email protected]> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <[email protected]> * Update to reflect new config names Signed-off-by: MaximumEntropy <[email protected]> * Sentencepiece fixes Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Fix finetuning Signed-off-by: MaximumEntropy <[email protected]> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Add finetune eval script Signed-off-by: MaximumEntropy <[email protected]> * Fix name Signed-off-by: MaximumEntropy <[email protected]> * Update Jenkinsfile Signed-off-by: MaximumEntropy <[email protected]> * Update config Signed-off-by: MaximumEntropy <[email protected]> * Fix CI test Signed-off-by: MaximumEntropy <[email protected]> * Update check Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Backward compat Signed-off-by: MaximumEntropy <[email protected]> * Update CI test Signed-off-by: MaximumEntropy <[email protected]> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <[email protected]> * Address comments Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Virginia Adams <[email protected]> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <[email protected]> * fix references Signed-off-by: ekmb <[email protected]> * address review feedback Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <[email protected]> * Fix Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <[email protected]> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <[email protected]> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <[email protected]> * Added padded vocab size Signed-off-by: Virginia Adams <[email protected]> * Fixed val check interval value Signed-off-by: Virginia Adams <[email protected]> * Python format fix Signed-off-by: Virginia Adams <[email protected]> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <[email protected]> * Added back dp=1 check Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> * Update README.rst * Fix uppercasing mismatch for IPA heteronyms (#4860) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <[email protected]> * Revert pin memory Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Sean Naren <[email protected]> * Fix mha (#4866) * fix bug in mha forward function related to cache update return type Signed-off-by: Yang Zhang <[email protected]> * fix lgtm Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> * ipa bug fix (#4871) Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * 1. Applying the same patch to r1.11.0 (#4894) Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Micha Livne <[email protected]> * Update tutorials.rst (#4897) * update readme with apex commit Signed-off-by: ericharper <[email protected]> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <[email protected]> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <[email protected]> * Fix bug with DDP initialization Signed-off-by: Tim Moon <[email protected]> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <[email protected]> * Fix code formatting Signed-off-by: Tim Moon <[email protected]> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Tim Moon <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * latent model support * 1. Debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. * update branch Signed-off-by: ericharper <[email protected]> * fix replace_bos_with_pad not found (#6443) Signed-off-by: Abhinav Khattar <[email protected]> * Support Swiglu in TP PP Conversion (#6437) * Support Swiglu in TP PP Conversion Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) * change bert fork to spawn Signed-off-by: Abhinav Khattar <[email protected]> * num_workers=0 fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) * 1. Meagtron encoder decoder fix for empty validation outputs. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Added/updated new Conformer configs (#6426) * updated conf files. Signed-off-by: Vahid <[email protected]> * added confs. Signed-off-by: Vahid <[email protected]> * moved longconformer confs. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated batch sizes and added fastconformer ctc streaming configs. Signed-off-by: Vahid <[email protected]> * updated batch sizes. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * reduce workers on NMT CI (#6472) Signed-off-by: Abhinav Khattar <[email protected]> * move to nvidia megatron repo (#6465) Signed-off-by: Abhinav Khattar <[email protected]> * Megatron KERPLE positional embeddings (#6478) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. Signed-off-by: Xuesong Yang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xuesong Yang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] FastPitch speaker encoder (#6417) * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * Remove wemb Signed-off-by: hsiehjackson <[email protected]> * Fix import Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss Signed-off-by: hsiehjackson <[email protected]> * Add ConditionalInput Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix import error Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix dataset error Signed-off-by: hsiehjackson <[email protected]> * Remove reference speaker embedding Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove SV encoder Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix length type Signed-off-by: hsiehjackson <[email protected]> * Fix append Signed-off-by: hsiehjackson <[email protected]> * Move error msg Signed-off-by: hsiehjackson <[email protected]> * Add look-up into speaker encoder Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Add valueerror msg Signed-off-by: hsiehjackson <[email protected]> * Move lookup Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove unused Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix error Signed-off-by: hsiehjackson <[email protected]> * Rebase and Fix error Signed-off-by: hsiehjackson <[email protected]> * Fix spk encoder Signed-off-by: hsiehjackson <[email protected]> * Rename n_speakers Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error Signed-off-by: hsiehjackson <[email protected]> --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Sharded manifests for tarred datasets (#6395) * testing sharded manifests Signed-off-by: Dima Rekesh <[email protected]> * compatibility Signed-off-by: Dima Rekesh <[email protected]> * proper fixes Signed-off-by: Dima Rekesh <[email protected]> * adding flag tot convert_to_tarred_audio_dataset Signed-off-by: Dima Rekesh <[email protected]> * shard_manifests conf param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * distributed checks Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py Signed-off-by: Dima Rekesh <[email protected]> * reversing manifest shards flag Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests Signed-off-by: Dima Rekesh <[email protected]> * excluding manifests from webdataset url expansion Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore Signed-off-by: Dima Rekesh <[email protected]> * explicit use of UTF-8 for manifest i/o Signed-off-by: Dima Rekesh <[email protected]> --------- Signed-off-by: Dima Rekesh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) Signed-off-by: Jim O’Regan <[email protected]> * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. Signed-off-by: Mostafa Ghorbandoost <[email protected]> * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. Signed-off-by: Mostafa Ghorbandoost <[email protected]> --------- Signed-off-by: Mostafa Ghorbandoost <[email protected]> * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support Signed-off-by: Dmytro Pykhtar <[email protected]> * added tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * fixed the typos Signed-off-by: Dmytro Pykhtar <[email protected]> * added assertions Signed-off-by: Dmytro Pykhtar <[email protected]> * changed assertion rules Signed-off-by: Dmytro Pykhtar <[email protected]> * deleted unused imports Signed-off-by: Dmytro Pykhtar <[email protected]> * changed tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * updated rampup batch size tests Signed-off-by: Dmytro Pykhtar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling Signed-off-by: Dmytro Pykhtar <[email protected]> * rampup batch size tests changes Signed-off-by: Dmytro Pykhtar <[email protected]> --------- Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default Signed-off-by: Kunal Dhawan <[email protected]> * updated README with the new agg_tokenizer_manifest flag Signed-off-by: Kunal Dhawan <[email protected]> * fixed typo in scripts/speech_recognition/code_switching/README.md Signed-off-by: Kunal Dhawan <[email protected]> * changed agg_tokenizer_manifest to is_lid_manifest Signed-off-by: Kunal Dhawan <[email protected]> --------- Signed-off-by: Kunal Dhawan <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script Signed-off-by: andrusenkoau <[email protected]> * add return encoding embedding option Signed-off-by: andrusenkoau <[email protected]> * update script Signed-off-by: andrusenkoau <[email protected]> * add rnnt and hat ngram decoding script Signed-off-by: andrusenkoau <[email protected]> * add some parameters Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig Signed-off-by: andrusenkoau <[email protected]> * replace return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * generalization of scipt behavior Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * add manual encoder_embeddings calculation Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 Signed-off-by: Andrei Andrusenko <[email protected]> * fix rescoring description Signed-off-by: Andrei Andrusenko <[email protected]> --------- Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * fix replace_bos_with_pad not found (#6443) (#6450) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * reduce workers on NMT CI (#6472) (#6474) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added KERPLE positional embeddings to encoder-decoder. Signed-off-by: Micha Livne <[email protected]> * 1. Added a missing file. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added external index sample. (#6462) Signed-off-by: Micha Livne <[email protected]> * Fix cache aware hybrid bugs (#6466) * Update README to add core installation (#6488) * update README for megatron-core Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Fix typos (#6494) Signed-off-by: smajumdar <[email protected]> * fix broken links r1.18.0 (#6501) * fix broken links Signed-off-by: Evelina <[email protected]> * fix broken links Signed-off-by: Evelina <[email protected]> --------- Signed-off-by: Evelina <[email protected]> * 1. Fixed gaussian hidden transform. Signed-off-by: Micha Livne <[email protected]> * 1. Finished updating hidden loss for MIM. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix custom forward_torch_softmax (#6512) Signed-off-by: Abhinav Khattar <[email protected]> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamp… (#6500) * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator r1.18.0 Signed-off-by: Taejin Park <[email protected]> * ignore keep_logits in FrameBatchASRLogits Signed-off-by: Taejin Park <[email protected]> --------- Signed-off-by: Taejin Park <[email protected]> * [TTS] fixed broken path. (#6514) Signed-off-by: Xuesong Yang <[email protected]> * 1. Added a hiddens module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typos (#6523) (#6539) * Fix typos Signed-off-by: smajumdar <[email protected]> * Fix typos Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> (cherry picked from commit 5468077f5127be1a4c88065de2544f4268b9a6e4) * added back the fast emit section to the configs. (#6540) * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * Fix fp16 (#6543) Signed-off-by: MaximumEntropy <[email protected]> * fix (#6529) Signed-off-by: Abhinav Khattar <[email protected]> * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Change Megatron Enc Dec model to use persistent_workers (#6548) * persistent workers Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by Signed-off-by: KunalDhawan <[email protected]> * updated ASR docs with the fastconformer hybrid checkpoints Signed-off-by: KunalDhawan <[email protected]> * added the fastconformer RNNT and CTC models Signed-off-by: KunalDhawan <[email protected]> --------- Signed-off-by: KunalDhawan <[email protected]> * Add scores for FastConformer models (#6557) Signed-off-by: smajumdar <[email protected]> * Patch transcribe and support offline transcribe for hybrid model (#6550) Signed-off-by: fayejf <[email protected]> * Not doing CastToFloat by default (#6524) * Not doing CastToFloat by default Signed-off-by: Boris Fomitchev <[email protected]> * Added docustring Signed-off-by: Boris Fomitchev <[email protected]> * Dummy commit Signed-off-by: Boris Fomitchev <[email protected]> --------- Signed-off-by: Boris Fomitchev <[email protected]> * temp rtd fix (#6568) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py for speedup (#6565) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths Signed-off-by: He Huang (Steve) <[email protected]> * Update manifest.py Signed-off-by: He Huang (Steve) <[email protected]> --------- Signed-off-by: He Huang (Steve) <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <[email protected]> * address review Signed-off-by: Abhinav Khattar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <[email protected]> * merge Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * More streaming conformer export fixes (#6567) Signed-off-by: Greg Clark <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Fix batch size reconf for T5 FT for multi-validation (#6582) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Updated Megatron LM encoder/decoder to use cfg for hiddens. Signed-off-by: Micha Livne <[email protected]> * 1. Added support to register externalhidden loss / transforms. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make tensor split contiguous (#6580) Signed-off-by: Abhinav Khattar <[email protected]> * Patches from main to r1.18.0 for Virtual Parallel (#6592) * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Switch to megatron core Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 892987169ef277f328e15b71a5a0c9bd961c8ee7) * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 1d813a372ab51688e3af6395d905a4c0366ffd23) * Documentation for ASR-TTS models (#6594) * Add docs about hybrid ASR-TTS models Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about text-only datasets Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS checkpoints Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS configs and training Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * ASR-TTS docs: add to api, fix imports Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * Wrap optional import Signed-off-by: Vladimir Bataev <[email protected]> * Revert general ASR import Signed-off-by: Vladimir Bataev <[email protected]> --------- Signed-off-by: Vladimir Bataev <[email protected]> * Update SDP docs (#6485) * add info about SDP e.g. processor classes in docs Signed-off-by: Elena Rastorgueva <[email protected]> * add link to SDP docs in README Signed-off-by: Elena Rastorgueva <[email protected]> * address code review comments and add SDP overview diagram Signed-off-by: Elena Rastorgueva <[email protected]> * Fix spelling typo Signed-off-by: Elena Rastorgueva <[email protected]> --------- Signed-off-by: Elena Rastorgueva <[email protected]> * Create dummy iters to satisy len checks (#6600) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * Restore GPT support for interleaved pipeline parallelism (#6528) * Restore logic for data-parallel communication with pipeline parallelism in GPT Signed-off-by: Tim Moon <[email protected]> * Support dynamic attention masks in GPT Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos Signed-off-by: Tim Moon <[email protected]> * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit Signed-off-by: Tim Moon <[email protected]> * Distinguish between list of data iterators and data iterator that is a list Signed-off-by: Tim Moon <[email protected]> * Create dummy iters to satisy len checks Signed-off-by: Abhinav Khattar <[email protected]> * Kludge while waiting for Megatron-LM update Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Eric Harper <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * Patch transcribe_util for steaming mode and add wer calculation back to inference scripts (#6601) * fix write Signed-off-by: fayejf <[email protected]> * decoding ctc Signed-off-by: fayejf <[email protected]> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <[email protected]> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * reflect change in asr_evaluator Signed-off-by: fayejf <[email protected]> * reflect som and vahid comment Signed-off-by: fayejf <[email protected]> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <[email protected]> * no text skip Signed-off-by: fayejf <[email protected]> --------- Signed-off-by: fayejf <[email protected]> * 1. Added example conf YAML. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support in tensor_parallel. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hat image to docs (#6619) Signed-off-by: andrusenkoau <[email protected]> * update core commit hash in readme (#6622) Signed-off-by: Abhinav Khattar <[email protected]> * Patch decoding for PC models (#6630) * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * Fix wer.py where 'errors' variable was not set (#6633) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <[email protected]> * fix att_context_size bug for older models. (#6635) Signed-off-by: Vahid <[email protected]> * Add megatron_core to requirements (#6639) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * remove from jenkins Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: ericharper <[email protected]> * remove dup (#6643) Signed-off-by: ericharper <[email protected]> * 1. Fixed config to use names, and added better error messages. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support to pass extra data to hiddens for loss computation. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Working on passing extra data to hiddnes. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed support in loading .nemo without hiddnes module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Improved and fixed logging of validation and testing. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed training logging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed logging of hidden loss. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed logging names. 2. Added logging to hiddens and tokens loss. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed conflicts. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> --------- Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Oleksii Kuchaiev <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: fayejf <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: ericharper <[email protected]> Signed-off-by: Vahid <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: sam1373 <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: ekmb <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Evelina <[email protected]> Signed-off-by: Taejin Park <[email protected]> Signed-off-by: KunalDhawan <[email protected]> Signed-off-by: Boris Fomitchev <[email protected]> Signed-off-by: He Huang (Steve) <[email protected]> Signed-off-by: Greg Clark <[email protected]> Signed-off-by: Elena Rastorgueva <[email protected]> Signed-off-by: Eric Harper <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Oleksii Kuchaiev <[email protected]> Co-authored-by: Jocelyn <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: fayejf <[email protected]> Co-authored-by: bene-ges <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> Co-authored-by: Virginia Adams <[email protected]> Co-authored-by: Zhilin Wang <[email protected]> Co-authored-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> Co-authored-by: anteju <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: David <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Samuel Kriman <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> Co-authored-by: Evelina <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Tim Moon <[email protected]> Co-authored-by: Neha Tadimeti <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Boris Fomitchev <[email protected]> Co-authored-by: He Huang (Steve) <[email protected]> Co-authored-by: Greg Clark <[email protected]> Co-authored-by: Elena Rastorgueva <[email protected]>
dorotat-nv
pushed a commit
to dorotat-nv/NeMo
that referenced
this pull request
Aug 24, 2023
* [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <[email protected]> * docs typo fix Signed-off-by: Oleksii Kuchaiev <[email protected]> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Fix ASR notebooks (#4738) Signed-off-by: smajumdar <[email protected]> Signed-off-by: smajumdar <[email protected]> * Multilingual VAD model (#4734) * add ngc link Signed-off-by: fayejf <[email protected]> * add tuned VAD config on ASR data Signed-off-by: fayejf <[email protected]> * yaml note Signed-off-by: fayejf <[email protected]> * update vad asr notebook with mVAD Signed-off-by: fayejf <[email protected]> * update vad infer config comment Signed-off-by: fayejf <[email protected]> * fix Signed-off-by: fayejf <[email protected]> * mvad sd config for ch109 Signed-off-by: fayejf <[email protected]> * update sd readme Signed-off-by: fayejf <[email protected]> * add new mVAD model to doc Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * update sd tutorial with mVAD Signed-off-by: fayejf <[email protected]> * typo fix Signed-off-by: fayejf <[email protected]> Signed-off-by: fayejf <[email protected]> * publish pretrained itn t5 model for English (#4748) Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> * Updated docs and doc paths (#4754) * Updated docs and doc paths Signed-off-by: Virginia Adams <[email protected]> * Update Multitask_Prompt_and_PTuning.ipynb * Update README.rst * Changed branch name to use single quotes Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762) * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <[email protected]> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763) Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> * [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770) Signed-off-by: Xuesong Yang <[email protected]> * Update CMUdict with more recent 0.7b entries (#4768) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * Install pynini in docker container (#4733) Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <[email protected]> * [TTS] deprecated old scripts for ljspeech. (#4780) * deprecated old scripts for ljspeech. * removed relevent function calls in TTS docs. Signed-off-by: Xuesong Yang <[email protected]> * update branch and typos (#4788) Signed-off-by: ericharper <[email protected]> Signed-off-by: ericharper <[email protected]> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <[email protected]> * fixed style. Signed-off-by: Vahid <[email protected]> * dropped seq_range Signed-off-by: Vahid <[email protected]> * fixed indexing in caching methods. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * updated docs. Signed-off-by: Vahid <[email protected]> * addressed comments. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> Signed-off-by: Vahid <[email protected]> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <[email protected]> * Fix style for import Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <[email protected]> * fixed style Signed-off-by: David Mosallanezhad <[email protected]> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <[email protected]> * included comments Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update branch for qa notebook Signed-off-by: ericharper <[email protected]> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <[email protected]> Signed-off-by: sam1373 <[email protected]> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <[email protected]> * Update to reflect new config names Signed-off-by: MaximumEntropy <[email protected]> * Sentencepiece fixes Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Fix finetuning Signed-off-by: MaximumEntropy <[email protected]> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Add finetune eval script Signed-off-by: MaximumEntropy <[email protected]> * Fix name Signed-off-by: MaximumEntropy <[email protected]> * Update Jenkinsfile Signed-off-by: MaximumEntropy <[email protected]> * Update config Signed-off-by: MaximumEntropy <[email protected]> * Fix CI test Signed-off-by: MaximumEntropy <[email protected]> * Update check Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Backward compat Signed-off-by: MaximumEntropy <[email protected]> * Update CI test Signed-off-by: MaximumEntropy <[email protected]> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <[email protected]> * Address comments Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Virginia Adams <[email protected]> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <[email protected]> * fix references Signed-off-by: ekmb <[email protected]> * address review feedback Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <[email protected]> * Fix Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <[email protected]> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <[email protected]> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <[email protected]> * Added padded vocab size Signed-off-by: Virginia Adams <[email protected]> * Fixed val check interval value Signed-off-by: Virginia Adams <[email protected]> * Python format fix Signed-off-by: Virginia Adams <[email protected]> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <[email protected]> * Added back dp=1 check Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> * Update README.rst * Fix uppercasing mismatch for IPA heteronyms (#4860) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <[email protected]> * Revert pin memory Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Sean Naren <[email protected]> * Fix mha (#4866) * fix bug in mha forward function related to cache update return type Signed-off-by: Yang Zhang <[email protected]> * fix lgtm Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> * ipa bug fix (#4871) Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * 1. Applying the same patch to r1.11.0 (#4894) Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Micha Livne <[email protected]> * Update tutorials.rst (#4897) * update readme with apex commit Signed-off-by: ericharper <[email protected]> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <[email protected]> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <[email protected]> * Fix bug with DDP initialization Signed-off-by: Tim Moon <[email protected]> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <[email protected]> * Fix code formatting Signed-off-by: Tim Moon <[email protected]> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Tim Moon <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * latent model support * 1. Debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. * update branch Signed-off-by: ericharper <[email protected]> * fix replace_bos_with_pad not found (#6443) Signed-off-by: Abhinav Khattar <[email protected]> * Support Swiglu in TP PP Conversion (#6437) * Support Swiglu in TP PP Conversion Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) * change bert fork to spawn Signed-off-by: Abhinav Khattar <[email protected]> * num_workers=0 fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) * 1. Meagtron encoder decoder fix for empty validation outputs. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Added/updated new Conformer configs (#6426) * updated conf files. Signed-off-by: Vahid <[email protected]> * added confs. Signed-off-by: Vahid <[email protected]> * moved longconformer confs. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated batch sizes and added fastconformer ctc streaming configs. Signed-off-by: Vahid <[email protected]> * updated batch sizes. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * reduce workers on NMT CI (#6472) Signed-off-by: Abhinav Khattar <[email protected]> * move to nvidia megatron repo (#6465) Signed-off-by: Abhinav Khattar <[email protected]> * Megatron KERPLE positional embeddings (#6478) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. Signed-off-by: Xuesong Yang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xuesong Yang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] FastPitch speaker encoder (#6417) * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * Remove wemb Signed-off-by: hsiehjackson <[email protected]> * Fix import Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss Signed-off-by: hsiehjackson <[email protected]> * Add ConditionalInput Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix import error Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix dataset error Signed-off-by: hsiehjackson <[email protected]> * Remove reference speaker embedding Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove SV encoder Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix length type Signed-off-by: hsiehjackson <[email protected]> * Fix append Signed-off-by: hsiehjackson <[email protected]> * Move error msg Signed-off-by: hsiehjackson <[email protected]> * Add look-up into speaker encoder Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Add valueerror msg Signed-off-by: hsiehjackson <[email protected]> * Move lookup Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove unused Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix error Signed-off-by: hsiehjackson <[email protected]> * Rebase and Fix error Signed-off-by: hsiehjackson <[email protected]> * Fix spk encoder Signed-off-by: hsiehjackson <[email protected]> * Rename n_speakers Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error Signed-off-by: hsiehjackson <[email protected]> --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Sharded manifests for tarred datasets (#6395) * testing sharded manifests Signed-off-by: Dima Rekesh <[email protected]> * compatibility Signed-off-by: Dima Rekesh <[email protected]> * proper fixes Signed-off-by: Dima Rekesh <[email protected]> * adding flag tot convert_to_tarred_audio_dataset Signed-off-by: Dima Rekesh <[email protected]> * shard_manifests conf param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * distributed checks Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py Signed-off-by: Dima Rekesh <[email protected]> * reversing manifest shards flag Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests Signed-off-by: Dima Rekesh <[email protected]> * excluding manifests from webdataset url expansion Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore Signed-off-by: Dima Rekesh <[email protected]> * explicit use of UTF-8 for manifest i/o Signed-off-by: Dima Rekesh <[email protected]> --------- Signed-off-by: Dima Rekesh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) Signed-off-by: Jim O’Regan <[email protected]> * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. Signed-off-by: Mostafa Ghorbandoost <[email protected]> * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. Signed-off-by: Mostafa Ghorbandoost <[email protected]> --------- Signed-off-by: Mostafa Ghorbandoost <[email protected]> * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support Signed-off-by: Dmytro Pykhtar <[email protected]> * added tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * fixed the typos Signed-off-by: Dmytro Pykhtar <[email protected]> * added assertions Signed-off-by: Dmytro Pykhtar <[email protected]> * changed assertion rules Signed-off-by: Dmytro Pykhtar <[email protected]> * deleted unused imports Signed-off-by: Dmytro Pykhtar <[email protected]> * changed tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * updated rampup batch size tests Signed-off-by: Dmytro Pykhtar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling Signed-off-by: Dmytro Pykhtar <[email protected]> * rampup batch size tests changes Signed-off-by: Dmytro Pykhtar <[email protected]> --------- Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default Signed-off-by: Kunal Dhawan <[email protected]> * updated README with the new agg_tokenizer_manifest flag Signed-off-by: Kunal Dhawan <[email protected]> * fixed typo in scripts/speech_recognition/code_switching/README.md Signed-off-by: Kunal Dhawan <[email protected]> * changed agg_tokenizer_manifest to is_lid_manifest Signed-off-by: Kunal Dhawan <[email protected]> --------- Signed-off-by: Kunal Dhawan <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script Signed-off-by: andrusenkoau <[email protected]> * add return encoding embedding option Signed-off-by: andrusenkoau <[email protected]> * update script Signed-off-by: andrusenkoau <[email protected]> * add rnnt and hat ngram decoding script Signed-off-by: andrusenkoau <[email protected]> * add some parameters Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig Signed-off-by: andrusenkoau <[email protected]> * replace return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * generalization of scipt behavior Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * add manual encoder_embeddings calculation Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 Signed-off-by: Andrei Andrusenko <[email protected]> * fix rescoring description Signed-off-by: Andrei Andrusenko <[email protected]> --------- Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * fix replace_bos_with_pad not found (#6443) (#6450) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * reduce workers on NMT CI (#6472) (#6474) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added KERPLE positional embeddings to encoder-decoder. Signed-off-by: Micha Livne <[email protected]> * 1. Added a missing file. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added external index sample. (#6462) Signed-off-by: Micha Livne <[email protected]> * Fix cache aware hybrid bugs (#6466) * Update README to add core installation (#6488) * update README for megatron-core Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Fix typos (#6494) Signed-off-by: smajumdar <[email protected]> * fix broken links r1.18.0 (#6501) * fix broken links Signed-off-by: Evelina <[email protected]> * fix broken links Signed-off-by: Evelina <[email protected]> --------- Signed-off-by: Evelina <[email protected]> * 1. Fixed gaussian hidden transform. Signed-off-by: Micha Livne <[email protected]> * 1. Finished updating hidden loss for MIM. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix custom forward_torch_softmax (#6512) Signed-off-by: Abhinav Khattar <[email protected]> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamp… (#6500) * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator r1.18.0 Signed-off-by: Taejin Park <[email protected]> * ignore keep_logits in FrameBatchASRLogits Signed-off-by: Taejin Park <[email protected]> --------- Signed-off-by: Taejin Park <[email protected]> * [TTS] fixed broken path. (#6514) Signed-off-by: Xuesong Yang <[email protected]> * 1. Added a hiddens module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typos (#6523) (#6539) * Fix typos Signed-off-by: smajumdar <[email protected]> * Fix typos Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> (cherry picked from commit 5468077f5127be1a4c88065de2544f4268b9a6e4) * added back the fast emit section to the configs. (#6540) * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * Fix fp16 (#6543) Signed-off-by: MaximumEntropy <[email protected]> * fix (#6529) Signed-off-by: Abhinav Khattar <[email protected]> * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Change Megatron Enc Dec model to use persistent_workers (#6548) * persistent workers Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by Signed-off-by: KunalDhawan <[email protected]> * updated ASR docs with the fastconformer hybrid checkpoints Signed-off-by: KunalDhawan <[email protected]> * added the fastconformer RNNT and CTC models Signed-off-by: KunalDhawan <[email protected]> --------- Signed-off-by: KunalDhawan <[email protected]> * Add scores for FastConformer models (#6557) Signed-off-by: smajumdar <[email protected]> * Patch transcribe and support offline transcribe for hybrid model (#6550) Signed-off-by: fayejf <[email protected]> * Not doing CastToFloat by default (#6524) * Not doing CastToFloat by default Signed-off-by: Boris Fomitchev <[email protected]> * Added docustring Signed-off-by: Boris Fomitchev <[email protected]> * Dummy commit Signed-off-by: Boris Fomitchev <[email protected]> --------- Signed-off-by: Boris Fomitchev <[email protected]> * temp rtd fix (#6568) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py for speedup (#6565) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths Signed-off-by: He Huang (Steve) <[email protected]> * Update manifest.py Signed-off-by: He Huang (Steve) <[email protected]> --------- Signed-off-by: He Huang (Steve) <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <[email protected]> * address review Signed-off-by: Abhinav Khattar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <[email protected]> * merge Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * More streaming conformer export fixes (#6567) Signed-off-by: Greg Clark <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Fix batch size reconf for T5 FT for multi-validation (#6582) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Updated Megatron LM encoder/decoder to use cfg for hiddens. Signed-off-by: Micha Livne <[email protected]> * 1. Added support to register externalhidden loss / transforms. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make tensor split contiguous (#6580) Signed-off-by: Abhinav Khattar <[email protected]> * Patches from main to r1.18.0 for Virtual Parallel (#6592) * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Switch to megatron core Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 892987169ef277f328e15b71a5a0c9bd961c8ee7) * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 1d813a372ab51688e3af6395d905a4c0366ffd23) * Documentation for ASR-TTS models (#6594) * Add docs about hybrid ASR-TTS models Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about text-only datasets Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS checkpoints Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS configs and training Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * ASR-TTS docs: add to api, fix imports Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * Wrap optional import Signed-off-by: Vladimir Bataev <[email protected]> * Revert general ASR import Signed-off-by: Vladimir Bataev <[email protected]> --------- Signed-off-by: Vladimir Bataev <[email protected]> * Update SDP docs (#6485) * add info about SDP e.g. processor classes in docs Signed-off-by: Elena Rastorgueva <[email protected]> * add link to SDP docs in README Signed-off-by: Elena Rastorgueva <[email protected]> * address code review comments and add SDP overview diagram Signed-off-by: Elena Rastorgueva <[email protected]> * Fix spelling typo Signed-off-by: Elena Rastorgueva <[email protected]> --------- Signed-off-by: Elena Rastorgueva <[email protected]> * Create dummy iters to satisy len checks (#6600) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * Restore GPT support for interleaved pipeline parallelism (#6528) * Restore logic for data-parallel communication with pipeline parallelism in GPT Signed-off-by: Tim Moon <[email protected]> * Support dynamic attention masks in GPT Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos Signed-off-by: Tim Moon <[email protected]> * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit Signed-off-by: Tim Moon <[email protected]> * Distinguish between list of data iterators and data iterator that is a list Signed-off-by: Tim Moon <[email protected]> * Create dummy iters to satisy len checks Signed-off-by: Abhinav Khattar <[email protected]> * Kludge while waiting for Megatron-LM update Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Eric Harper <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * Patch transcribe_util for steaming mode and add wer calculation back to inference scripts (#6601) * fix write Signed-off-by: fayejf <[email protected]> * decoding ctc Signed-off-by: fayejf <[email protected]> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <[email protected]> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * reflect change in asr_evaluator Signed-off-by: fayejf <[email protected]> * reflect som and vahid comment Signed-off-by: fayejf <[email protected]> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <[email protected]> * no text skip Signed-off-by: fayejf <[email protected]> --------- Signed-off-by: fayejf <[email protected]> * 1. Added example conf YAML. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support in tensor_parallel. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hat image to docs (#6619) Signed-off-by: andrusenkoau <[email protected]> * update core commit hash in readme (#6622) Signed-off-by: Abhinav Khattar <[email protected]> * Patch decoding for PC models (#6630) * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * Fix wer.py where 'errors' variable was not set (#6633) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <[email protected]> * fix att_context_size bug for older models. (#6635) Signed-off-by: Vahid <[email protected]> * Add megatron_core to requirements (#6639) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * remove from jenkins Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: ericharper <[email protected]> * remove dup (#6643) Signed-off-by: ericharper <[email protected]> * 1. Fixed config to use names, and added better error messages. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support to pass extra data to hiddens for loss computation. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Working on passing extra data to hiddnes. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed support in loading .nemo without hiddnes module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Improved and fixed logging of validation and testing. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed training logging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed logging of hidden loss. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed logging names. 2. Added logging to hiddens and tokens loss. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed conflicts. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> --------- Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Oleksii Kuchaiev <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: fayejf <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: ericharper <[email protected]> Signed-off-by: Vahid <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: sam1373 <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: ekmb <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Evelina <[email protected]> Signed-off-by: Taejin Park <[email protected]> Signed-off-by: KunalDhawan <[email protected]> Signed-off-by: Boris Fomitchev <[email protected]> Signed-off-by: He Huang (Steve) <[email protected]> Signed-off-by: Greg Clark <[email protected]> Signed-off-by: Elena Rastorgueva <[email protected]> Signed-off-by: Eric Harper <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Oleksii Kuchaiev <[email protected]> Co-authored-by: Jocelyn <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: fayejf <[email protected]> Co-authored-by: bene-ges <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> Co-authored-by: Virginia Adams <[email protected]> Co-authored-by: Zhilin Wang <[email protected]> Co-authored-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> Co-authored-by: anteju <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: David <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Samuel Kriman <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> Co-authored-by: Evelina <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Tim Moon <[email protected]> Co-authored-by: Neha Tadimeti <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Boris Fomitchev <[email protected]> Co-authored-by: He Huang (Steve) <[email protected]> Co-authored-by: Greg Clark <[email protected]> Co-authored-by: Elena Rastorgueva <[email protected]> Signed-off-by: dorotat <[email protected]>
rohitrango
pushed a commit
to rohitrango/NeMo
that referenced
this pull request
Jun 25, 2024
* [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <[email protected]> * docs typo fix Signed-off-by: Oleksii Kuchaiev <[email protected]> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Fix ASR notebooks (#4738) Signed-off-by: smajumdar <[email protected]> Signed-off-by: smajumdar <[email protected]> * Multilingual VAD model (#4734) * add ngc link Signed-off-by: fayejf <[email protected]> * add tuned VAD config on ASR data Signed-off-by: fayejf <[email protected]> * yaml note Signed-off-by: fayejf <[email protected]> * update vad asr notebook with mVAD Signed-off-by: fayejf <[email protected]> * update vad infer config comment Signed-off-by: fayejf <[email protected]> * fix Signed-off-by: fayejf <[email protected]> * mvad sd config for ch109 Signed-off-by: fayejf <[email protected]> * update sd readme Signed-off-by: fayejf <[email protected]> * add new mVAD model to doc Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * update sd tutorial with mVAD Signed-off-by: fayejf <[email protected]> * typo fix Signed-off-by: fayejf <[email protected]> Signed-off-by: fayejf <[email protected]> * publish pretrained itn t5 model for English (#4748) Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> * Updated docs and doc paths (#4754) * Updated docs and doc paths Signed-off-by: Virginia Adams <[email protected]> * Update Multitask_Prompt_and_PTuning.ipynb * Update README.rst * Changed branch name to use single quotes Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762) * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <[email protected]> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763) Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> * [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770) Signed-off-by: Xuesong Yang <[email protected]> * Update CMUdict with more recent 0.7b entries (#4768) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> * Install pynini in docker container (#4733) Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <[email protected]> * [TTS] deprecated old scripts for ljspeech. (#4780) * deprecated old scripts for ljspeech. * removed relevent function calls in TTS docs. Signed-off-by: Xuesong Yang <[email protected]> * update branch and typos (#4788) Signed-off-by: ericharper <[email protected]> Signed-off-by: ericharper <[email protected]> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <[email protected]> * fixed style. Signed-off-by: Vahid <[email protected]> * dropped seq_range Signed-off-by: Vahid <[email protected]> * fixed indexing in caching methods. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * updated docs. Signed-off-by: Vahid <[email protected]> * addressed comments. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * change frame-wise to cache-aware. Signed-off-by: Vahid <[email protected]> * fixed code style. Signed-off-by: Vahid <[email protected]> Signed-off-by: Vahid <[email protected]> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <[email protected]> * Fix style for import Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <[email protected]> * fixed style Signed-off-by: David Mosallanezhad <[email protected]> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <[email protected]> * included comments Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update branch for qa notebook Signed-off-by: ericharper <[email protected]> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <[email protected]> Signed-off-by: sam1373 <[email protected]> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <[email protected]> * Update to reflect new config names Signed-off-by: MaximumEntropy <[email protected]> * Sentencepiece fixes Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Fix finetuning Signed-off-by: MaximumEntropy <[email protected]> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Add finetune eval script Signed-off-by: MaximumEntropy <[email protected]> * Fix name Signed-off-by: MaximumEntropy <[email protected]> * Update Jenkinsfile Signed-off-by: MaximumEntropy <[email protected]> * Update config Signed-off-by: MaximumEntropy <[email protected]> * Fix CI test Signed-off-by: MaximumEntropy <[email protected]> * Update check Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> * Backward compat Signed-off-by: MaximumEntropy <[email protected]> * Update CI test Signed-off-by: MaximumEntropy <[email protected]> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <[email protected]> * Address comments Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Virginia Adams <[email protected]> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <[email protected]> * fix references Signed-off-by: ekmb <[email protected]> * address review feedback Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <[email protected]> * Fix Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <[email protected]> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <[email protected]> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <[email protected]> * Added padded vocab size Signed-off-by: Virginia Adams <[email protected]> * Fixed val check interval value Signed-off-by: Virginia Adams <[email protected]> * Python format fix Signed-off-by: Virginia Adams <[email protected]> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <[email protected]> * Added back dp=1 check Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> * Update README.rst * Fix uppercasing mismatch for IPA heteronyms (#4860) Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <[email protected]> * Revert pin memory Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Co-authored-by: Sean Naren <[email protected]> * Fix mha (#4866) * fix bug in mha forward function related to cache update return type Signed-off-by: Yang Zhang <[email protected]> * fix lgtm Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> * ipa bug fix (#4871) Signed-off-by: ekmb <[email protected]> Signed-off-by: ekmb <[email protected]> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <[email protected]> * Style Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Virginia Adams <[email protected]> * 1. Applying the same patch to r1.11.0 (#4894) Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Micha Livne <[email protected]> * Update tutorials.rst (#4897) * update readme with apex commit Signed-off-by: ericharper <[email protected]> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <[email protected]> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <[email protected]> * Fix bug with DDP initialization Signed-off-by: Tim Moon <[email protected]> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <[email protected]> * Fix code formatting Signed-off-by: Tim Moon <[email protected]> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Tim Moon <[email protected]> Co-authored-by: Eric Harper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * update readme Signed-off-by: ericharper <[email protected]> * latent model support * 1. Debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. * update branch Signed-off-by: ericharper <[email protected]> * fix replace_bos_with_pad not found (#6443) Signed-off-by: Abhinav Khattar <[email protected]> * Support Swiglu in TP PP Conversion (#6437) * Support Swiglu in TP PP Conversion Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> * Guard activation Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) * change bert fork to spawn Signed-off-by: Abhinav Khattar <[email protected]> * num_workers=0 fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) * 1. Meagtron encoder decoder fix for empty validation outputs. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Added/updated new Conformer configs (#6426) * updated conf files. Signed-off-by: Vahid <[email protected]> * added confs. Signed-off-by: Vahid <[email protected]> * moved longconformer confs. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated readme. Signed-off-by: Vahid <[email protected]> * updated batch sizes and added fastconformer ctc streaming configs. Signed-off-by: Vahid <[email protected]> * updated batch sizes. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> * added hybrid support. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * reduce workers on NMT CI (#6472) Signed-off-by: Abhinav Khattar <[email protected]> * move to nvidia megatron repo (#6465) Signed-off-by: Abhinav Khattar <[email protected]> * Megatron KERPLE positional embeddings (#6478) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. Signed-off-by: Xuesong Yang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xuesong Yang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] FastPitch speaker encoder (#6417) * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * Remove wemb Signed-off-by: hsiehjackson <[email protected]> * Fix import Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss Signed-off-by: hsiehjackson <[email protected]> * Add ConditionalInput Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix import error Signed-off-by: hsiehjackson <[email protected]> * Add initial codes Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix dataset error Signed-off-by: hsiehjackson <[email protected]> * Remove reference speaker embedding Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove SV encoder Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix length type Signed-off-by: hsiehjackson <[email protected]> * Fix append Signed-off-by: hsiehjackson <[email protected]> * Move error msg Signed-off-by: hsiehjackson <[email protected]> * Add look-up into speaker encoder Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Add valueerror msg Signed-off-by: hsiehjackson <[email protected]> * Move lookup Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Remove unused Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <[email protected]> * Fix error Signed-off-by: hsiehjackson <[email protected]> * Rebase and Fix error Signed-off-by: hsiehjackson <[email protected]> * Fix spk encoder Signed-off-by: hsiehjackson <[email protected]> * Rename n_speakers Signed-off-by: hsiehjackson <[email protected]> * Follow comments Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error Signed-off-by: hsiehjackson <[email protected]> --------- Signed-off-by: hsiehjackson <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Sharded manifests for tarred datasets (#6395) * testing sharded manifests Signed-off-by: Dima Rekesh <[email protected]> * compatibility Signed-off-by: Dima Rekesh <[email protected]> * proper fixes Signed-off-by: Dima Rekesh <[email protected]> * adding flag tot convert_to_tarred_audio_dataset Signed-off-by: Dima Rekesh <[email protected]> * shard_manifests conf param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <[email protected]> * distributed checks Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * typo Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * fixes Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py Signed-off-by: Dima Rekesh <[email protected]> * reversing manifest shards flag Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests Signed-off-by: Dima Rekesh <[email protected]> * excluding manifests from webdataset url expansion Signed-off-by: Dima Rekesh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore Signed-off-by: Dima Rekesh <[email protected]> * explicit use of UTF-8 for manifest i/o Signed-off-by: Dima Rekesh <[email protected]> --------- Signed-off-by: Dima Rekesh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) Signed-off-by: Jim O’Regan <[email protected]> * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. Signed-off-by: Mostafa Ghorbandoost <[email protected]> * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. Signed-off-by: Mostafa Ghorbandoost <[email protected]> --------- Signed-off-by: Mostafa Ghorbandoost <[email protected]> * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support Signed-off-by: Dmytro Pykhtar <[email protected]> * added tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * fixed the typos Signed-off-by: Dmytro Pykhtar <[email protected]> * added assertions Signed-off-by: Dmytro Pykhtar <[email protected]> * changed assertion rules Signed-off-by: Dmytro Pykhtar <[email protected]> * deleted unused imports Signed-off-by: Dmytro Pykhtar <[email protected]> * changed tests for rampup batch size Signed-off-by: Dmytro Pykhtar <[email protected]> * updated rampup batch size tests Signed-off-by: Dmytro Pykhtar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling Signed-off-by: Dmytro Pykhtar <[email protected]> * rampup batch size tests changes Signed-off-by: Dmytro Pykhtar <[email protected]> --------- Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- Signed-off-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Micha Livne <[email protected]> * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default Signed-off-by: Kunal Dhawan <[email protected]> * updated README with the new agg_tokenizer_manifest flag Signed-off-by: Kunal Dhawan <[email protected]> * fixed typo in scripts/speech_recognition/code_switching/README.md Signed-off-by: Kunal Dhawan <[email protected]> * changed agg_tokenizer_manifest to is_lid_manifest Signed-off-by: Kunal Dhawan <[email protected]> --------- Signed-off-by: Kunal Dhawan <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script Signed-off-by: andrusenkoau <[email protected]> * add return encoding embedding option Signed-off-by: andrusenkoau <[email protected]> * update script Signed-off-by: andrusenkoau <[email protected]> * add rnnt and hat ngram decoding script Signed-off-by: andrusenkoau <[email protected]> * add some parameters Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig Signed-off-by: andrusenkoau <[email protected]> * replace return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * generalization of scipt behavior Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <[email protected]> * add manual encoder_embeddings calculation Signed-off-by: andrusenkoau <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 Signed-off-by: Andrei Andrusenko <[email protected]> * fix rescoring description Signed-off-by: Andrei Andrusenko <[email protected]> --------- Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * fix replace_bos_with_pad not found (#6443) (#6450) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * reduce workers on NMT CI (#6472) (#6474) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added KERPLE positional embeddings to encoder-decoder. Signed-off-by: Micha Livne <[email protected]> * 1. Added a missing file. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Micha Livne <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * 1. Added external index sample. (#6462) Signed-off-by: Micha Livne <[email protected]> * Fix cache aware hybrid bugs (#6466) * Update README to add core installation (#6488) * update README for megatron-core Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> * Fix typos (#6494) Signed-off-by: smajumdar <[email protected]> * fix broken links r1.18.0 (#6501) * fix broken links Signed-off-by: Evelina <[email protected]> * fix broken links Signed-off-by: Evelina <[email protected]> --------- Signed-off-by: Evelina <[email protected]> * 1. Fixed gaussian hidden transform. Signed-off-by: Micha Livne <[email protected]> * 1. Finished updating hidden loss for MIM. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix custom forward_torch_softmax (#6512) Signed-off-by: Abhinav Khattar <[email protected]> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamp… (#6500) * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator r1.18.0 Signed-off-by: Taejin Park <[email protected]> * ignore keep_logits in FrameBatchASRLogits Signed-off-by: Taejin Park <[email protected]> --------- Signed-off-by: Taejin Park <[email protected]> * [TTS] fixed broken path. (#6514) Signed-off-by: Xuesong Yang <[email protected]> * 1. Added a hiddens module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typos (#6523) (#6539) * Fix typos Signed-off-by: smajumdar <[email protected]> * Fix typos Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> (cherry picked from commit 7fa47bfcd359f1ea80ad056d8bb2d78f97c6206a) * added back the fast emit section to the configs. (#6540) * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> * added back the fast emit section to the configs. Signed-off-by: Vahid <[email protected]> --------- Signed-off-by: Vahid <[email protected]> * Fix fp16 (#6543) Signed-off-by: MaximumEntropy <[email protected]> * fix (#6529) Signed-off-by: Abhinav Khattar <[email protected]> * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Change Megatron Enc Dec model to use persistent_workers (#6548) * persistent workers Signed-off-by: Abhinav Khattar <[email protected]> * fix Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: Eric Harper <[email protected]> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by Signed-off-by: KunalDhawan <[email protected]> * updated ASR docs with the fastconformer hybrid checkpoints Signed-off-by: KunalDhawan <[email protected]> * added the fastconformer RNNT and CTC models Signed-off-by: KunalDhawan <[email protected]> --------- Signed-off-by: KunalDhawan <[email protected]> * Add scores for FastConformer models (#6557) Signed-off-by: smajumdar <[email protected]> * Patch transcribe and support offline transcribe for hybrid model (#6550) Signed-off-by: fayejf <[email protected]> * Not doing CastToFloat by default (#6524) * Not doing CastToFloat by default Signed-off-by: Boris Fomitchev <[email protected]> * Added docustring Signed-off-by: Boris Fomitchev <[email protected]> * Dummy commit Signed-off-by: Boris Fomitchev <[email protected]> --------- Signed-off-by: Boris Fomitchev <[email protected]> * temp rtd fix (#6568) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py for speedup (#6565) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths Signed-off-by: He Huang (Steve) <[email protected]> * Update manifest.py Signed-off-by: He Huang (Steve) <[email protected]> --------- Signed-off-by: He Huang (Steve) <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <[email protected]> * address review Signed-off-by: Abhinav Khattar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <[email protected]> * merge Signed-off-by: Abhinav Khattar <[email protected]> --------- Signed-off-by: Abhinav Khattar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> * More streaming conformer export fixes (#6567) Signed-off-by: Greg Clark <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> * Fix batch size reconf for T5 FT for multi-validation (#6582) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Updated Megatron LM encoder/decoder to use cfg for hiddens. Signed-off-by: Micha Livne <[email protected]> * 1. Added support to register externalhidden loss / transforms. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make tensor split contiguous (#6580) Signed-off-by: Abhinav Khattar <[email protected]> * Patches from main to r1.18.0 for Virtual Parallel (#6592) * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <[email protected]> * Switch to megatron core Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit fa9ab5b41301c9d92c97bf37894f9c94de6b91b5) * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 4d639ef36b10d8536db02bc47ea30f4acbc061bb) * Documentation for ASR-TTS models (#6594) * Add docs about hybrid ASR-TTS models Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about text-only datasets Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS checkpoints Signed-off-by: Vladimir Bataev <[email protected]> * Add docs about ASR-TTS configs and training Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * ASR-TTS docs: add to api, fix imports Signed-off-by: Vladimir Bataev <[email protected]> * Clean up Signed-off-by: Vladimir Bataev <[email protected]> * Wrap optional import Signed-off-by: Vladimir Bataev <[email protected]> * Revert general ASR import Signed-off-by: Vladimir Bataev <[email protected]> --------- Signed-off-by: Vladimir Bataev <[email protected]> * Update SDP docs (#6485) * add info about SDP e.g. processor classes in docs Signed-off-by: Elena Rastorgueva <[email protected]> * add link to SDP docs in README Signed-off-by: Elena Rastorgueva <[email protected]> * address code review comments and add SDP overview diagram Signed-off-by: Elena Rastorgueva <[email protected]> * Fix spelling typo Signed-off-by: Elena Rastorgueva <[email protected]> --------- Signed-off-by: Elena Rastorgueva <[email protected]> * Create dummy iters to satisy len checks (#6600) Signed-off-by: Abhinav Khattar <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * Restore GPT support for interleaved pipeline parallelism (#6528) * Restore logic for data-parallel communication with pipeline parallelism in GPT Signed-off-by: Tim Moon <[email protected]> * Support dynamic attention masks in GPT Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos Signed-off-by: Tim Moon <[email protected]> * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit Signed-off-by: Tim Moon <[email protected]> * Distinguish between list of data iterators and data iterator that is a list Signed-off-by: Tim Moon <[email protected]> * Create dummy iters to satisy len checks Signed-off-by: Abhinav Khattar <[email protected]> * Kludge while waiting for Megatron-LM update Signed-off-by: Tim Moon <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Eric Harper <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Abhinav Khattar <[email protected]> * Patch transcribe_util for steaming mode and add wer calculation back to inference scripts (#6601) * fix write Signed-off-by: fayejf <[email protected]> * decoding ctc Signed-off-by: fayejf <[email protected]> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <[email protected]> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <[email protected]> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <[email protected]> * style fix Signed-off-by: fayejf <[email protected]> * reflect change in asr_evaluator Signed-off-by: fayejf <[email protected]> * reflect som and vahid comment Signed-off-by: fayejf <[email protected]> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <[email protected]> * no text skip Signed-off-by: fayejf <[email protected]> --------- Signed-off-by: fayejf <[email protected]> * 1. Added example conf YAML. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support in tensor_parallel. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hat image to docs (#6619) Signed-off-by: andrusenkoau <[email protected]> * update core commit hash in readme (#6622) Signed-off-by: Abhinav Khattar <[email protected]> * Patch decoding for PC models (#6630) * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> * Patch decoding logic for PC models Signed-off-by: smajumdar <[email protected]> --------- Signed-off-by: smajumdar <[email protected]> * Fix wer.py where 'errors' variable was not set (#6633) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <[email protected]> * fix att_context_size bug for older models. (#6635) Signed-off-by: Vahid <[email protected]> * Add megatron_core to requirements (#6639) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <[email protected]> * remove from jenkins Signed-off-by: ericharper <[email protected]> --------- Signed-off-by: ericharper <[email protected]> * remove dup (#6643) Signed-off-by: ericharper <[email protected]> * 1. Fixed config to use names, and added better error messages. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support to pass extra data to hiddens for loss computation. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Working on passing extra data to hiddnes. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed support in loading .nemo without hiddnes module. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Improved and fixed logging of validation and testing. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed training logging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed logging of hidden loss. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed logging names. 2. Added logging to hiddens and tokens loss. Signed-off-by: Micha Livne <[email protected]> * 1. Fixed conflicts. Signed-off-by: Micha Livne <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> * 1. Debugging. Signed-off-by: Micha Livne <[email protected]> --------- Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: Oleksii Kuchaiev <[email protected]> Signed-off-by: Jocelyn Huang <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: fayejf <[email protected]> Signed-off-by: Alexandra Antonova <[email protected]> Signed-off-by: Virginia Adams <[email protected]> Signed-off-by: Vladimir Bataev <[email protected]> Signed-off-by: ericharper <[email protected]> Signed-off-by: Vahid <[email protected]> Signed-off-by: Ante Jukić <[email protected]> Signed-off-by: David Mosallanezhad <[email protected]> Signed-off-by: sam1373 <[email protected]> Signed-off-by: MaximumEntropy <[email protected]> Signed-off-by: ekmb <[email protected]> Signed-off-by: Yang Zhang <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: Tim Moon <[email protected]> Signed-off-by: Abhinav Khattar <[email protected]> Signed-off-by: smajumdar <[email protected]> Signed-off-by: Micha Livne <[email protected]> Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Dima Rekesh <[email protected]> Signed-off-by: Jim O’Regan <[email protected]> Signed-off-by: Mostafa Ghorbandoost <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Dmytro Pykhtar <[email protected]> Signed-off-by: Kunal Dhawan <[email protected]> Signed-off-by: andrusenkoau <[email protected]> Signed-off-by: Andrei Andrusenko <[email protected]> Signed-off-by: Evelina <[email protected]> Signed-off-by: Taejin Park <[email protected]> Signed-off-by: KunalDhawan <[email protected]> Signed-off-by: Boris Fomitchev <[email protected]> Signed-off-by: He Huang (Steve) <[email protected]> Signed-off-by: Greg Clark <[email protected]> Signed-off-by: Elena Rastorgueva <[email protected]> Signed-off-by: Eric Harper <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: Oleksii Kuchaiev <[email protected]> Co-authored-by: Jocelyn <[email protected]> Co-authored-by: Somshubra Majumdar <[email protected]> Co-authored-by: fayejf <[email protected]> Co-authored-by: bene-ges <[email protected]> Co-authored-by: Alexandra Antonova <[email protected]> Co-authored-by: Virginia Adams <[email protected]> Co-authored-by: Zhilin Wang <[email protected]> Co-authored-by: Vladimir Bataev <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: Eric Harper <[email protected]> Co-authored-by: Vahid Noroozi <[email protected]> Co-authored-by: anteju <[email protected]> Co-authored-by: Ante Jukić <[email protected]> Co-authored-by: David <[email protected]> Co-authored-by: David Mosallanezhad <[email protected]> Co-authored-by: Samuel Kriman <[email protected]> Co-authored-by: Sandeep Subramanian <[email protected]> Co-authored-by: Evelina <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Yang Zhang <[email protected]> Co-authored-by: Sean Naren <[email protected]> Co-authored-by: Tim Moon <[email protected]> Co-authored-by: Neha Tadimeti <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <[email protected]> Co-authored-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: Dima Rekesh <[email protected]> Co-authored-by: Jim O’Regan <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mostafa Ghorbandoost <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Dmytro Pykhtar <[email protected]> Co-authored-by: Kunal Dhawan <[email protected]> Co-authored-by: Andrei Andrusenko <[email protected]> Co-authored-by: Taejin Park <[email protected]> Co-authored-by: Boris Fomitchev <[email protected]> Co-authored-by: He Huang (Steve) <[email protected]> Co-authored-by: Greg Clark <[email protected]> Co-authored-by: Elena Rastorgueva <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Fixes some issues:
-change_attention_model should work for old models which don't have some of the relevant parameters in config
-also it shouldn't run when nothing to change was submitted
Original PR: #6608
Collection: ASR
PR Type: