add converter from genai task to model type, fix endpoint results #2518
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
imatiach-msft
requested review from
tongyu-microsoft,
xuke444,
RubyZ10 and
vinuthakaranth
as code owners
imatiach-msft
added a commit
that referenced
this pull request
Feb 2, 2024
* Added info about required packages * Update responsibleaidashboard-question-answering-model-debugging.ipynb * show example prediction * Update responsibleaidashboard-question-answering-model-debugging.ipynb * add genai metrics Signed-off-by: Kartik Choudhary <[email protected]> * Add genai task type Signed-off-by: Kartik Choudhary <[email protected]> * add genai task type (#2494) * Added info about required packages * Update responsibleaidashboard-question-answering-model-debugging.ipynb * show example prediction * Update responsibleaidashboard-question-answering-model-debugging.ipynb * add genai metrics Signed-off-by: Kartik Choudhary <[email protected]> * Add genai task type Signed-off-by: Kartik Choudhary <[email protected]> --------- Signed-off-by: Kartik Choudhary <[email protected]> * remove target_column requirement for genai tasks Signed-off-by: Kartik Choudhary <[email protected]> * genai dev demo notebook Signed-off-by: Kartik Choudhary <[email protected]> * Added info about required packages * Update responsibleaidashboard-question-answering-model-debugging.ipynb * add genai metrics Signed-off-by: Kartik Choudhary <[email protected]> * Add genai task type Signed-off-by: Kartik Choudhary <[email protected]> * remove target_column requirement for genai tasks (#2501) Signed-off-by: Kartik Choudhary <[email protected]> * add demo notebook (#2502) * remove target_column requirement for genai tasks Signed-off-by: Kartik Choudhary <[email protected]> * genai dev demo notebook Signed-off-by: Kartik Choudhary <[email protected]> --------- Signed-off-by: Kartik Choudhary <[email protected]> * added generative text explainer for OpenAI models Signed-off-by: Mohsin Shah <[email protected]> * update UI to support genai task for RAI text dashboard (#2504) * generative ai explanations for all rows of dataset Signed-off-by: Mohsin Shah <[email protected]> * move metrics to utils submodule Signed-off-by: Kartik Choudhary <[email protected]> * error analysis support for genai text Signed-off-by: Kartik Choudhary <[email protected]> * error analysis support (#2510) * move metrics to utils submodule Signed-off-by: Kartik Choudhary <[email protected]> * error analysis support for genai text Signed-off-by: Kartik Choudhary <[email protected]> --------- Signed-off-by: Kartik Choudhary <[email protected]> * fix some linting issues Signed-off-by: Kartik Choudhary <[email protected]> * more linting fixes Signed-off-by: Kartik Choudhary <[email protected]> * Kartikch/add genai metrics (#2513) * update UI to support genai task for RAI text dashboard (#2504) (#2508) * move metrics to utils submodule Signed-off-by: Kartik Choudhary <[email protected]> * error analysis support for genai text Signed-off-by: Kartik Choudhary <[email protected]> * fix some linting issues Signed-off-by: Kartik Choudhary <[email protected]> * more linting fixes Signed-off-by: Kartik Choudhary <[email protected]> --------- Signed-off-by: Kartik Choudhary <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> * add genai metrics endpoint in UI for model overview metrics (#2517) * Add socketio event handler for generative text metrics Signed-off-by: Kartik Choudhary <[email protected]> * add converter from genai task to model type, fix endpoint results (#2518) * Add support for evaluation model in RAITextInsights Signed-off-by: Kartik Choudhary <[email protected]> * fix linting issues Signed-off-by: Kartik Choudhary <[email protected]> * Remove commented out code for generative text tasks Signed-off-by: Kartik Choudhary <[email protected]> * update genai metric compute function for new ml wrapper interface Signed-off-by: Kartik Choudhary <[email protected]> * add output examples in genai metric prompts Signed-off-by: Kartik Choudhary <[email protected]> * fix merge errors Signed-off-by: Kartik Choudhary <[email protected]> * add methods and constants for genai task type Signed-off-by: Kartik Choudhary <[email protected]> * add missing files for genai metrics Signed-off-by: Kartik Choudhary <[email protected]> * update copyright information Signed-off-by: Kartik Choudhary <[email protected]> * Fix target_column assignment in ErrorAnalysisManager Signed-off-by: Kartik Choudhary <[email protected]> * Remove duplicate import and refactor if condition in RAITextInsights class Signed-off-by: Kartik Choudhary <[email protected]> * Refactor rating examples in genai_metrics scripts Signed-off-by: Kartik Choudhary <[email protected]> * Refactor debug_ml method to handle missing text_cols attribute Signed-off-by: Kartik Choudhary <[email protected]> * Remove unnecessary code for generative text models Signed-off-by: Kartik Choudhary <[email protected]> * Rearranged import statements in _compute.py Signed-off-by: Kartik Choudhary <[email protected]> --------- Signed-off-by: Kartik Choudhary <[email protected]> Signed-off-by: Mohsin Shah <[email protected]> Co-authored-by: Mohsin Shah <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Mohsin Shah <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
add converter from genai task to model type, fix endpoint results
Checklist