Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Run batch evaluations" creates logs but does not evaluate them #236

Closed
csansoon opened this issue Sep 20, 2024 · 4 comments
Closed

"Run batch evaluations" creates logs but does not evaluate them #236

csansoon opened this issue Sep 20, 2024 · 4 comments
Assignees
Labels
🐛 bug Something isn't working can't reproduce Issue is not fixed because it's not currently reproduceable p1 High priority issues

Comments

@csansoon
Copy link
Contributor

Screen.Recording.2024-09-20.at.09.48.57.mov
@csansoon csansoon added 🐛 bug Something isn't working p1 High priority issues labels Sep 20, 2024
@geclos
Copy link
Collaborator

geclos commented Sep 20, 2024

can you share the offending prompt and datasets please?

@geclos geclos self-assigned this Sep 20, 2024
@csansoon
Copy link
Contributor Author

Prompt:

---
provider: OpenAI
model: gpt-4o
---

<user>{{descripcion}}</user>

Cuál es el nombre del usuario? Responde únicamente con su nombre.

<response as=name />

{{
  objetivo = (name === "Carlos")
    ? "burla"
    : "apreciación"
}}


Genera un poema como {{objetivo}} hacia el usuario.

Dataset:
names_list.csv

@geclos
Copy link
Collaborator

geclos commented Sep 20, 2024

thanks!

@geclos
Copy link
Collaborator

geclos commented Sep 20, 2024

can't reproduce it with valid

@geclos geclos closed this as completed Sep 20, 2024
@geclos geclos added the can't reproduce Issue is not fixed because it's not currently reproduceable label Sep 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working can't reproduce Issue is not fixed because it's not currently reproduceable p1 High priority issues
Projects
None yet
Development

No branches or pull requests

3 participants
@geclos @csansoon and others