Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix typos and add definitions for toxicity detection microservice #553

Merged
merged 2 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions comps/guardrails/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@

The Guardrails service enhances the security of LLM-based applications by offering a suite of microservices designed to ensure trustworthiness, safety, and security.

| MicroService | Description |
| ------------------------------------------ | ------------------------------------------------------------------------------------------ |
| [Llama Guard](./llama_guard/README.md) | Provides guardrails for inputs and outputs to ensure safe interactions |
| [PII Detection](./pii_detection/README.md) | Detects Personally Identifiable Information (PII) and Business Sensitive Information (BSI) |
| MicroService | Description |
| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| [Llama Guard](./llama_guard/README.md) | Provides guardrails for inputs and outputs to ensure safe interactions |
| [PII Detection](./pii_detection/README.md) | Detects Personally Identifiable Information (PII) and Business Sensitive Information (BSI) |
| [Toxicity Detection](./toxicity_detection/README.md) | Detects Toxic language (rude, disrespectful, or unreasonable language that is likely to make someone leave a discussion) |

Additional safety-related microservices will be available soon.
18 changes: 9 additions & 9 deletions comps/guardrails/toxicity_detection/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Toxicity Detection Microservice

# ☣️💥🛡️<span style="color:royalblue"> Intel Toxicity Detection Model </span>
# ☣️💥🛡️Toxicity Detection Microservice

## Introduction
ashahba marked this conversation as resolved.
Show resolved Hide resolved

Intel also provides toxicity detection model, which is lightweight, runs efficiently on a CPU, and performs well on toxic_chat and jigsaws datasets. More datasets are being fine-tuned. If you're interested, please contact [email protected].
Toxicity Detection Microservice allows AI Application developers to safeguard user input and LLM output from harmful language in a RAG environment. By leveraging a smaller fine-tuned Transformer model for toxicity classification (e.g. DistilledBERT, RoBERTa, etc.), we maintain a lightweight guardrails microservice without significantly sacrificing performance making it readily deployable on both Intel Gaudi and Xeon.

Toxicity is defined as rude, disrespectful, or unreasonable language likely to make someone leave a conversation. This can include instances of aggression, bullying, targeted hate speech, or offensive language. For more information on labels see [Jigsaw Toxic Comment Classification Challenge](http://kaggle.com/c/jigsaw-toxic-comment-classification-challenge).

## Training Customerizable Toxicity Model on Gaudi2
## Future Development

Additionally, we offer a fine-tuning workflow on Intel Gaudi2, allowing you to customerize your toxicity detecction model to suit your unique needs.
- Add a RoBERTa (125M params) toxicity model fine-tuned on Gaudi2 with ToxicChat and Jigsaw dataset in an optimized serving framework.

# 🚀1. Start Microservice with Python(Option 1)

Expand All @@ -24,7 +24,7 @@ pip install -r requirements.txt
python toxicity_detection.py
```

# 🚀2. Start Microservie with Docker (Option 2)
# 🚀2. Start Microservice with Docker (Option 2)

## 2.1 Prepare toxicity detection model

Expand Down Expand Up @@ -58,7 +58,7 @@ Once microservice starts, users can use examples (bash or python) below to apply
```bash
curl localhost:9091/v1/toxicity
-X POST
-d '{"text":"How to poison your neighbor'\''s dog secretly"}'
-d '{"text":"How to poison my neighbor'\''s dog without being caught?"}'
-H 'Content-Type: application/json'
```

Expand All @@ -76,7 +76,7 @@ import json

proxies = {"http": ""}
url = "http://localhost:9091/v1/toxicity"
data = {"text": "How to poison your neighbor'''s dog without being caught?"}
data = {"text": "How to poison my neighbor'''s dog without being caught?"}

try:
resp = requests.post(url=url, data=data, proxies=proxies)
Expand Down