Remove conversation pipeline tests

* relaxed rope check

* lets also accept rope_type=None, defaulting to the original implementation

* type and rope_type can coexist

* let's not warn when someone is running a foward without cache + self.training

* more models

* fixup

fix resize when deepspeed

* Fix float8_e4m3fn in modeling_utils

* style

* fix

* comment

* support gguf fp16

* support gguf bf16 with pytorch

* add gguf f16 test

* remove bf16

* No more default chat templates

* Add the template to the GPT-SW3 tests since it's not available by default now

* Fix GPT2 test

* Fix Bloom test

* Fix Bloom test

* Remove default templates again

…ingface#32198)

Replaced deprecated unittest method with the correct one.

* [whisper] fix short-form output type

* add test

* make style

* update long-form tests

* fixes

* last fix

* finalise test

….7.0 (huggingface#32210)

remove unnecessary guard code related with pytorch versions 1.4.2 ~
1.7.0

…gingface#32222)

set _supports_param_buffer_assignment to False

fix E721 warnings

* fix

* [test_all] trigger full CI

---------

Co-authored-by: ydshieh <[email protected]>

* translate philosophy.md to chinese

* add the missing link

…tility functions. Default to using the currently active microphone on Mac (huggingface#31846)

* use currently active microphone on mac for ffmpeg_microphone

* Allow ffmpeg_microphone device to be specified

Co-authored-by: amyeroberts <[email protected]>

---------

Co-authored-by: amyeroberts <[email protected]>

Fix code snippet for grounding-dino

* fix

* move changes to prompt lookup

* add test

* set eos in assistant model

* style

* fix flakiness

* changes for new `main`

* Update tests/generation/test_utils.py

Co-authored-by: amyeroberts <[email protected]>

* Update tests/generation/test_utils.py

Co-authored-by: amyeroberts <[email protected]>

* add comment to explain

---------

Co-authored-by: amyeroberts <[email protected]>

* llava w/o images

* tests

* fix resize when deepspeed

* deepsped uses new embeds

* we needed this

…gingface#32143)

* don't log base model architecture in wandb is log model is false

* Update src/transformers/integrations/integration_utils.py

Co-authored-by: amyeroberts <[email protected]>

* convert log model setting into an enum

* fix formatting

---------

Co-authored-by: amyeroberts <[email protected]>

* Refactored to remove un-necessary object base class.

* small fix.

* adds: extra_repr() to RMSNorm layers in multiple models

* adds: extra_repr for deprecated models as well

* formatting as per style guide

…tection` for owlv2 (huggingface#31934)

* Add check for target_sizes is None in post_process_image_guided_detection

* Make sure Owlvit and Owlv2 in sync

* Fix incorrect indentation; add check for correct size of target_sizes

…n_implementation==flash_attention_2` (huggingface#32039)

* add flash attention check

* fix

* fix

…ngface#32241)

* fix

* fix prev test (half of failures)

* [run-slow] llama, gemma2

* [run-slow] llama, gemma2

update

Co-authored-by: ydshieh <[email protected]>

…uggingface#32244)

* replace for loop by tensor ops

* rm assert; readability

* bloom dynamic cache

* bloom follows standard cache format

* no skips for bloom anymore

* use cache position when possible

* clean up

* codestyle

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: amyeroberts <[email protected]>

* pr comments

* isinstance fix

* address comments

* make musicgen test happy

* [run-slow] bloom

---------

Co-authored-by: amyeroberts <[email protected]>

upload

Co-authored-by: ydshieh <[email protected]>

* Optimize t5 tokenize logic to avoid redundant calls

* fix and overwrite copies

…n call (huggingface#32262)

Removed one wrong argument passed to convert_blip_checkpoint function call.

remove exceptions

…uggingface#32076)

* fix

* bug fix

* refine

* fix

Fixed a link in docs.

* mvp

* added test (a few models need fixes)

* fix a few test cases

* test nits

* harder test 😈

* revert changes in stablelm

* test with improved condition

* add todo

* tmp commit

* merged with main

* nits

* add todo

* final corrections

* add docs for generation compilation

* docs nits

* add  tip

* PR suggestions

* add more details to the compilation docs

* fix cache positions

* cache is now init in generate; update docs

* tag test as flaky

* docs

* post rebase make fixup and other nits

* remove unintended changes

* whisper (encoder-decoder) not supported

* move token default updates to ; add tests for token defaults

* push changes

* manual rebase

* chameleon doesn't support this

* fix test_static_cache_mha_mqa_gqa (broken in another PR)

* docs: dynamic is better with end-to-end compilation

* fix _fix_key in PreTrainedModel

* fix _find_longest_common_sequence

* add test

* remove result.json

* nit

* update test

* [pipeline] fix padding for 1-d tensors

* add test

* make style

* Update tests/pipelines/test_pipelines_automatic_speech_recognition.py

Co-authored-by: Kamil Akesbi <[email protected]>

* Update tests/pipelines/test_pipelines_automatic_speech_recognition.py

---------

Co-authored-by: Kamil Akesbi <[email protected]>

)

* Add stream_to_gradio method for running agent in gradio demo

Co-authored-by: ydshieh <[email protected]>

Docs: fix GaLore optimizer example

Fix incorrect usage of GaLore optimizer in Transformers trainer code example.

The GaLore optimizer uses low-rank gradient updates to reduce memory usage. GaLore is quite popular and is implemented by the authors in [https://github.com/jiaweizzhao/GaLore](https://github.com/jiaweizzhao/GaLore). A few months ago GaLore was added to the HuggingFace Transformers library in huggingface#29588.

Documentation of the Trainer module includes a few code examples of how to use GaLore. However, the `optim_targe_modules` argument to the `TrainingArguments` function is incorrect, as discussed in huggingface#29588 (comment). This pull request fixes this issue.

* fix gguf dequantize for gguf==0.9.1

* fix old version

* make style

* fix epochs_trained as int when resuming training

* refactor

---------

Co-authored-by: teddyferdinan <[email protected]>

…ce#31663)

* tentative fix

* do the same for M4T

* doc formatting nits

* ignore non-autodocs

* Apply suggestions from code review

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/esm/modeling_esm.py

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/esm/modeling_esm.py

Co-authored-by: amyeroberts <[email protected]>

* make fixup

---------

Co-authored-by: amyeroberts <[email protected]>

* new agent plan

* plan type assertion

* style corrections

* better prompt naming

* make fixup

Fixed raising of few exceptions.

…ading for prequantized 4bit (huggingface#32276)

…ge of 10% o… (huggingface#32335)

fixes huggingface#32329 : The Torch code is correct - to get an average of 10% of the total, we want to take 50% of the remainder after we've already masked 80% with [MASK] in the previous step.

)

* tmp

* skip files not in the diff

* use git.Repo instead of an external subprocess

* add tiny change to confirm that the diff is working on pushed changes

* add make quality task

* more profesh main commit reference

…ss (huggingface#32191)

* Remove user-defined tokens which can be obtained through merges

* Remove debug line

* formatting

* Refactor spm slow -> fast converter

* revert unnecessary refactor

* set comprehension

* remove test files

* Use `vocab_scores`

* Always replace spiece underline with space in decode

* we no longer need token filtering

* Add save fast load slow unit test

* Remove tokenizers version check

* Remove duplicate code

* Make `<start_of_turn>` and `<end_of_turn>` special tokens

* Bias merge priority with length if score is the same

* Add unit test for merge priority

* CI

fix

* enable flash-attn & static cache

* this works, not the prev

* fix for sliding window layers

* not needed anymore

fix 💩

* Fix FA2 call for Perciever layer

* [run_slow] idefics2

* [run_slow] idefics2

* [run_slow] idefics2

* Fix up

* [run_slow] idefics2

* [run_slow] idefics2

* [run_slow] idefics2

…ggingface#32360)

Fix error when streaming agent run to gradio with non-string tool arguments

…oder models (huggingface#32227)

* draft

* apply changes to all relevant archs

* rerun ci - check_docstrings.py failing?

* fix docstring

* move 2D->4D mask creation to modeling file

* repo consistency

* fix the batch size = 1 case - calling contiguous is not enough

* nit

* style

* propagate to gemma/gemma-2

* prepare inputs for gemma generation

* implement test and tiny fix in gemma2

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: Arthur <[email protected]>

* fix copies

* ci pass

* fix gemma's test_compile_static_cache tests

* flacky

* retrigger ci

---------

Co-authored-by: sanchit-gandhi <[email protected]>
Co-authored-by: Arthur <[email protected]>

* Fixed staticmethods with self as first argument.

* Fixed staticmethods with self as first argument.

* Fixed staticmethods with self as first argument.

* Fixed staticmethods with self as first argument.

cache class flag

…31772)

* [whisper] compile compatibility with long-form decoding

* clarify comment

* fix after rebase

* finalise

* fix bsz

* fix cache split

* remove contiguous

* style

* finish

* update doc

* prevent cuda graph trace

…ngface#32339)

* Remove size check between attn_weights and kv_seq_len

* add unit tests

…uggingface#32359)

Co-authored-by: Guoming Zhang <[email protected]>

…ggingface#31971) (huggingface#32043)

* Remove TPU device map for saving tokenizer config

* Update tokenization_utils_base.py

* Fix error msg when passing non-string device into tokenizer

* Fix error message for non-string tokenizer device

* Print out tokenizer device type in error msg

* Update tokenization_utils_base.py

…sion (huggingface#32342)

empty list in defaults

…gface#31233)

* Fix conflicting key in init kwargs in PreTrainedTokenizerBase

* Update code to check for callable key in save_pretrained

* Apply PR suggestions

* Invoke CI

* Updates based on PR suggestion

* Initial implementation of OffloadedCache

* enable usage via cache_implementation

* Address feedback, add tests, remove legacy methods.

* Remove flash-attn, discover synchronization bugs, fix bugs

* Prevent usage in CPU only mode

* Add a section about offloaded KV cache to the docs

* Fix typos in docs

* Clarifications and better explanation of streams

…#32374)

* fixed hybrid cache init, added test

* Fix Test Typo

---------

Co-authored-by: Aaron Haag <[email protected]>

…e been done (huggingface#32299)

* Test this zach

* Test for improper init w/o zero3

* Move back

* Apply suggestions from code review

Co-authored-by: amyeroberts <[email protected]>

* Get rid of stars in warning

* Make private

* Make clear

---------

Co-authored-by: amyeroberts <[email protected]>

nits

tests! :D

…gingface#32367)

* up

* style

* stopping

…rmers/examples/flax/language-modeling/t5_tokenizer_model.py`. (huggingface#32157)

fix: Exception raised when running .

…ing rotary_seq_len, allowing None position_ids input. (huggingface#31500)

* Mixtral: remove unnecessary plus 1 when calculating rotary_seq_len, allowing position_ids=None (no auto position_ids generation could be unsafe)

* fix typo [:-1] to [:, -1]

* to meet formatting requirement

* to meet formatting requirement

* remove white space

* MixtralFlashAttention2: put "+ 1" inside parentheses when calculating rotary_seq_len, allowing None position_ids input. Fix format/style issue.

* propagate to startcoder2, phi3, mixtral and qwen2

* update qwen2_moe

…on_transformer (huggingface#32393)

Bump keras in /examples/research_projects/decision_transformer

Bumps [keras](https://github.com/keras-team/keras) from 2.8.0 to 2.13.1.
- [Release notes](https://github.com/keras-team/keras/releases)
- [Commits](keras-team/keras@v2.8.0...v2.13.1)

---
updated-dependencies:
- dependency-name: keras
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: SeamlessM4TFeatureExtractor stride remainder

* Added attention mask size test

* Reran ruff for style correction

fix phi

* save total_vocab_size = vocab_size + user added tokens to speed up operation

* updating length when added_tokens_decoder is set

* add test len(tokenizer)

I always forget what typical values are, and I have to look at the paper everytime. This will be a helpful reminder.

…face#32242)

* fix: persist embedding type of MBartConditonalGeneration after resize

* fix: persist embedding type of BartConditonalGeneration after resize

…uggingface#32413)

Fixed tokenizertests for luke, mluke models.

* Respect the config's attn if set

* Update test - can override in from_config

* Fix

…ngface#32434)

* draft

* updates

* works?

* try adding python example in hidden section

* another try

* hwo do i render python

* format as html code?

* Update docs/source/en/kv_cache.md

Co-authored-by: Joao Gante <[email protected]>

* Update docs/source/en/kv_cache.md

Co-authored-by: Joao Gante <[email protected]>

* Update docs/source/en/kv_cache.md

Co-authored-by: Joao Gante <[email protected]>

* Update docs/source/en/kv_cache.md

Co-authored-by: Joao Gante <[email protected]>

* Update docs/source/en/kv_cache.md

Co-authored-by: Joao Gante <[email protected]>

* one more small update

* should render hidden secrtion now

* add outputs

* fix links

* check links

* update all links

* update with offloaded cache

* all cache is importable, so they appear in docs

* fix copies

* docstring...

---------

Co-authored-by: Joao Gante <[email protected]>

fix: add new llava like model bug

* add flash attention check

* fix

* fix

* add the missing marker

* bug fix

* add one more

* remove order

* add one more

…#32024)

* BLIP preprocess

* BIT preprocess

* BRIDGETOWER preprocess

* CHAMELEON preprocess

* CHINESE_CLIP preprocess

* CONVNEXT preprocess

* DEIT preprocess

* DONUT preprocess

* DPT preprocess

* FLAVA preprocess

* EFFICIENTNET preprocess

* FUYU preprocess

* GLPN preprocess

* IMAGEGPT preprocess

* INTRUCTBLIPVIDEO preprocess

* VIVIT preprocess

* ZOEDEPTH preprocess

* VITMATTE preprocess

* VIT preprocess

* VILT preprocess

* VIDEOMAE preprocess

* VIDEOLLAVA

* TVP processing

* TVP fixup

* SWIN2SR preprocess

* SIGLIP preprocess

* SAM preprocess

* RT-DETR preprocess

* PVT preprocess

* POOLFORMER preprocess

* PERCEIVER preprocess

* OWLVIT preprocess

* OWLV2 preprocess

* NOUGAT preprocess

* MOBILEVIT preprocess

* MOBILENETV2 preprocess

* MOBILENETV1 preprocess

* LEVIT preprocess

* LAYOUTLMV2 preprocess

* LAYOUTLMV3 preprocess

* Add test

* Update tests

…huggingface#32438)