Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[docs] LLM Prompting Guide #26272

Closed
wants to merge 27 commits into from
Closed

[docs] LLM Prompting Guide #26272

wants to merge 27 commits into from

Conversation

MKhalusova
Copy link
Contributor

@MKhalusova MKhalusova commented Sep 19, 2023 •
edited
Loading

What does this PR do?

This PR adds an LLM Prompting Guide to the docs that covers the following topics:

  • basics of prompting,
  • encoder-decoder models vs decoder-only models,
  • base vs instruct models,
  • basic prompts to solve common NLP tasks,
  • best practices for prompting,
  • advanced techniques like few-shot learning and chain-of-thought
  • prompting vs fine-tuning

Let me know, if there's anything missing that has to be included.

MKhalusova and others added 27 commits September 18, 2023 15:35
@MKhalusova
WIP prompting guide
e23abcb
@MKhalusova
few-shot prompting
8c67f90
@MKhalusova
first draft complete
f9ebf55
@MKhalusova
minor
baeedc1
@ydshieh @MKhalusova
No doctest for convert_bros_to_pytorch.py (huggingface#26212)
dfed49b 
fix

Co-authored-by: ydshieh <[email protected]>
@ydshieh @MKhalusova
Remove utils/documentation_tests.txt (huggingface#26213)
2b63176 
* update

* update

---------

Co-authored-by: ydshieh <[email protected]>
@julien-c @LysandreJik @MKhalusova
moved ctrl to Salesforce/ctrl (huggingface#26183)
5b0803e 
* moved `ctrl` to `Salesforce/ctrl`

redirects should theoretically work, but still updating those repo references for clarity

* Fixup

* Slow doc tests

* Add modeling file

---------

Co-authored-by: Lysandre <[email protected]>
@Rocketknight1 @MKhalusova
Fix ConversationalPipeline tests (huggingface#26217)
d83ac0d 
Add BlenderbotSmall templates and correct handling for conversation.past_user_inputs
@LysandreJik @MKhalusova
[FSMT] Fix non-shared weights (huggingface#26187)
a746db2 
* Fix non-shared weights

* Add tests

* Edit tied weights keys
@shijie-wu @MKhalusova
refactor decay_parameters production into its own function (huggingfa…
f727ce0 
…ce#26152)
@pphuc25 @MKhalusova
refactor: change default block_size in block size > max position embe…
1387776 
…ddings (huggingface#26069)

* refactor: change default block_size when not initialize

* reformat: add the min of block size
@sanchit-gandhi @MKhalusova
[Wav2Vec2-Conf / LLaMA] Style fix (huggingface#26188)
5f19eaf 
* torch.nn -> nn

* fix llama

* copies
@sanchit-gandhi @MKhalusova
[Permisson] Style fix (huggingface#26228)
b85eecf 
fix copies
@sanchit-gandhi @MKhalusova
[Check] Fix config docstring (huggingface#26222)
cd07c71
@ArthurZucker @sgugger
@amyeroberts @LysandreJik @MKhalusova
🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (huggingface#…
4676774 
…23909)

* fix test for bart. Order is correct now let's skip BPEs

* ouf

* styling

* fix bert....

* slow refactoring

* current updates

* massive refactoring

* update

* NICE!

* update to see where I am at

* updates

* update

* update

* revert

* updates

* updates

* start supporting legacy_save

* styling

* big update

* revert some changes

* nits

* nniiiiiice

* small fixes

* kinda fix t5 with new behaviour

* major update

* fixup

* fix copies

* today's updates

* fix byt5

* upfate

* update

* update

* updates

* update vocab size test

* Barthez does not use not need the fairseq offset ids

* super calll must be after

* calll super

* move all super init

* move other super init

* fixup

* nits

* more fixes

* nits

* more fixes

* nits

* more fix

* remove useless files

* ouch all of them are affected

* and more!

* small imporvements

* no more sanitize token

* more changes around unique no split tokens

* partially fix more things

* keep legacy save but add warning

* so... more fixes

* updates

* guess deberta tokenizer could be nuked

* fixup

* fixup did some bad things

* nuke it if it breaks

* remove prints and pretrain fast from slow with new format.

* fixups

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <[email protected]>

* fiou

* nit

* by default specials should not be normalized?

* update

* remove brakpoint

* updates

* a lot of updates

* fixup

* fixes revert some changes to match fast

* small nits

* that makes it cleaner

* fix camembert accordingly

* update

* some lest breaking changes

* update

* fixup

* fix byt5 and whisper mostly

* some more fixes, canine's byte vocab

* fix gpt2

* fix most of the perceiver tests (4 left)

* fix layout lmv3

* fixup

* fix copies for gpt2 style

* make sure to only warn once

* fix perciever and gpt2 tests

* some more backward compatibility: also read special tokens map because some ppl use it........////.....

* fixup

* add else when reading

* nits

* fresh updates

* fix copies

* will this make everything faster?

* fixes

* more fixes

* update

* more fixes

* fixup

* is the source of truth right?

* sorry camembert for the troubles

* current updates

* fixup

* update led

* update

* fix regression

* fix single word

* more model specific fixes

* fix t5 tests

* fixup

* more comments

* update

* fix nllb

* rstrip removed

* small fixes

* better handle additional_special_tokens and vocab sizes

* fixing

* styling

* fix 4 / 21

* fixup

* fix nlbb's tests

* some fixes

* fix t5

* fixes

* style

* fix canine tests

* damn this is nice

* nits

* m2m100 nit

* fixups

* fixes!

* fixup

* stash

* fix merge

* revert bad change

* fixup

* correct order for code Llama

* fix speecht5 post merge

* styling

* revert source of 11 fails

* small nits

* all changes in one go

* fnet hack

* fix 2 more tests

* update based on main branch of tokenizers

* fixup

* fix VITS issues

* more fixes

* fix mgp test

* fix camembert issues

* oups camembert still has 2 failing tests

* mluke fixes

* decode fixes

* small nits

* nits

* fix llama and vits

* fix camembert

* smal nits

* more fixes when initialising a fast from a slow and etc

* fix one of the last test

* fix CPM tokenizer test

* fixups

* fix pop2piano

* fixup

* ⚠️ Change tokenizers required version ⚠️

* ⚠️ Change tokenizers required version ⚠️

* "tokenizers>=0.14,<0.15", don't forget smaller than

* fix musicgen tests and pretraiendtokenizerfast

* fix owlvit and all

* update t5

* fix 800 red

* fix tests

* fix the fix of the fix of t5

* styling

* documentation nits

* cache _added_tokens_encoder

* fixups

* Nit

* fix red tests

* one last nit!

* make eveything a lot simpler

* Now it's over 😉

* few small nits

* Apply suggestions from code review

Co-authored-by: amyeroberts <[email protected]>

* updates that work for now

* tests that should no be skipped / changed and fixed next

* fixup

* i am ashamed

* pushe the fix

* update

* fixups

* nits

* fix added_tokens_encoder

* fix canine test

* fix pegasus vocab

* fix transfoXL

* fixup

* whisper needs to be fixed for train new

* pegasus nits

* more pegasus fixes

* minor update

* better error message in failed test

* fix whisper failing test

* fix whisper failing test

* fix pegasus

* fixup

* fix **** pegasus

* reset things

* remove another file

* attempts to fix the strange custome encoder and offset

* nits here and there

* update

* fixup

* nit

* fix the whisper test

* nits nits

* Apply suggestions from code review

Co-authored-by: amyeroberts <[email protected]>

* updates based on review

* some small update to potentially remove

* nits

* import rlu cache

* Update src/transformers/tokenization_utils_base.py

Co-authored-by: Lysandre Debut <[email protected]>

* move warning to `from_pretrained`

* update tests results now that the special tokens are always added

---------

Co-authored-by: Sylvain Gugger <[email protected]>
Co-authored-by: amyeroberts <[email protected]>
Co-authored-by: Lysandre Debut <[email protected]>
@nuatmochoi @jungnerd @MKhalusova
🌐 [i18n-KO] Translated whisper.md to Korean (huggingface#26002)
88b5c8a 
* docs: ko-whisper.md

* fix: chatgpt draft

* feat: manual edits

* Feat: manual edits

* fix: resolve suggestions

Co-authored-by: Jungnerd <[email protected]>

---------

Co-authored-by: Jungnerd <[email protected]>
@mksit @MKhalusova
Create the return value on device to avoid unnecessary copying from C…
50e805c 
…PU (huggingface#26151)
@NielsRogge @MKhalusova
[AutoBackbone] Add test (huggingface#26094)
1e657ab 
* Add test

* Add config_class
@NinoRisteski @MKhalusova
Update README.md (huggingface#26198)
3b1512e 
Fixed a few typos
@NinoRisteski @MKhalusova
Update add_new_pipeline.md (huggingface#26197)
dbed182 
fixed a few typos
@Aleksandar1932 @MKhalusova
[docs] Fix model reference in zero shot image classification example (h…
abebc29 
…uggingface#26206)
@muellerz @MKhalusova
Fix the gitlab user mention in issue templates to the correct user (h…
96cac7b 
…uggingface#26237)
@ydshieh @MKhalusova
Fix some docstring in image processors (huggingface#26235)
dd65294 
Fix doc

Co-authored-by: ydshieh <[email protected]>
@Wauplin @MKhalusova
Fix gated repo tests (huggingface#26257)
3aa2c32 
* Fix gated repo tests

* Apply suggestions from code review
@NielsRogge @MKhalusova
Add ViTMatte (huggingface#25843)
31e6a38 
* First draft

* Simplify image processor

* Fix rebase

* Address comments

* Address more comments

* Address more comments

* Address more comments

* Address more comments

* Improve pad_image

* Add tests

* Update integration test

* Fix image processor tests

* Fix model tests

* Convert checkpoints

* Fix doc tests

* Remove file

* Apply suggestions

* Address comments

* Fix typing hint

* Add batch_norm_eps

* Address comments

* Fix style
@ydshieh @MKhalusova
Fix Error not captured in PR doctesting (huggingface#26215)
2fd869b 
* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <[email protected]>
@pacman100 @MKhalusova
DeepSpeed ZeRO-3 handling when resizing embedding layers (huggingface…
8396fa8 
…#26259)

* fix failing deepspeed slow tests

* fixes
@MKhalusova MKhalusova closed this Sep 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.