Showcase Haystack evaluations on an industry dataset

[mrm1001]

User story

I would like to learn how to apply Haystack core evaluations to improve my RAG pipeline, with an example on how to improve my retriever component, on an industry dataset.

Sub-tasks:

• create an example of how to improve Haystack evaluation metrics by tweaking: chunk size and/or embedding model (e.g. context size). Using: semantic similarity on answers and LLM-based metric context relevance.

More context here: https://www.notion.so/deepsetai/Evaluation-1521712b928d4142828232f2df136856?pvs=4

Tasks

Beta Give feedback

1. find an industry dataset to showcase evaluation metrics #7438
   P1 topic:eval +2
   
2. Update tutorial on model-based evaluation with Haystack #6790
   2.x P2 topic:eval +3