Add Red-Teaming LLMs

[dapurv5]

https://github.com/dapurv5/awesome-red-teaming-llms

Red-teaming techniques and defenses for Large Language Models (LLMs).

By submitting this pull request I confirm I've read and complied with the below requirements 🖖

Requirements for your pull request

• Don't open a Draft / WIP pull request while you work on the guidelines. A pull request should be 100% ready and should adhere to all the guidelines when you open it.
• You have to review at least 2 other open pull requests.
  I have reviewed the following PRs: #3234 and #3232.
• You have read and understood the instructions for creating a list.
• This pull request has a title in the format Add Name of List.
• Your entry here should include a short description of the project/theme of the list.
• Your entry should be added at the bottom of the appropriate category.
• The title of your entry should be title-cased and the URL to your list should end in #readme.
• No blockchain-related lists.
• The suggested Awesome list complies with the below requirements.

Requirements for your Awesome list

• Has been around for at least 30 days.
• Run awesome-lint on your list and fix the reported issues.
• The default branch should be named main, not master.
• Includes a succinct description of the project/theme at the top of the readme.
• It's the result of hard work and the best I could possibly produce.
• The repo name of your list should be in lowercase slug format: awesome-name-of-list.
• The heading title of your list should be in title case format: # Awesome Name of List.
• Non-generated Markdown file in a GitHub repo.
• The repo should have awesome-list & awesome as GitHub topics.
• Not a duplicate.
• Only has awesome items. Awesome lists are curations of the best, not everything.
• Does not contain items that are unmaintained, has archived repo, deprecated, or missing docs.
• Includes a project logo/illustration whenever possible.
• Entries have a description, unless the title is descriptive enough by itself.
• Includes the Awesome badge.
• Has a Table of Contents section.
• Has an appropriate license.
• Has contribution guidelines.
• All non-important but necessary content is grouped in a Footnotes section at the bottom of the readme.
• Has consistent formatting and proper spelling/grammar.
• Does not use hard-wrapping.
• Does not include a CI (e.g. GitHub Actions) badge.
• Does not include an Inspired by awesome-foo or Inspired by the Awesome project kinda link at the top of the readme.

[dapurv5]

unicorn

[sindresorhus]

Thanks for making an Awesome list! 🙌

It looks like you didn't read the guidelines closely enough. I noticed multiple things that are not followed. Try going through the list point for point to ensure you follow it. I spent a lot of time creating the guidelines so I wouldn't have to comment on common mistakes, and rather spend my time improving Awesome.

[lnxpy]

I think you should use a valid approach for listing the items as the ToC has no depth and lists are depicted as tables!

[dapurv5]

Thank you so much for your comment as I was not able to figure out the issue myself. I had a the list-based version earlier which did not do justice to presenting the information, so I decided to change it to a table. Any easy resolution for this?

[cvyl]

I understand the approach but I think a ToC would be better instead of tables.

[Axorax]

You should add a short description at the top of the README so people can read and understand it quickly

[dapurv5]

• Added a one line summary at the top
• Expanded the table of contents to reflect all the headings in the taxonomy.

[idematos]

The URL to your list should end in #readme.

[idematos]

Hi @dapurv5, thank you for updating the PR! Here are a couple more things I noticed on your list:

• I think the Introduction and Attack Surface sections are too long. I would recommend removing them and instead adding a brief definition of the theme at the top of the list. Also, the list's description would be better presented as a quote (see example).
• The table of contents should then start at Attacks, where the list begins, with a maximum depth of one. You can retain titles with greater depth within the list itself, but they should not appear in the table of contents.

Contents

• Attacks
  • Jailbreak Attack
  • Direct Attack
  • Infusion Attack
  • Inference Attack
  • Training Time Attack
• Defenses

[dapurv5]

Thanks for your suggestions and helpful pointers @idematos . I ran the linter offline and it passes now.