Add model wrapper for wrapping predictions and test data #1762
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Gaurav Gupta <[email protected]>
gaugup
requested review from
imatiach-msft,
xuke444,
romanlutz,
RubyZ10,
vinuthakaranth and
tongyu-microsoft
as code owners
Codecov Report
@@ Coverage Diff @@
## postga #1762 +/- ##
=======================================
Coverage 88.30% 88.30%
=======================================
Files 89 89
Lines 5250 5250
=======================================
Hits 4636 4636
Misses 614 614
Flags with carried forward coverage won't be shown. Click here to find out more. Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Signed-off-by: Gaurav Gupta <[email protected]>
vinuthakaranth
approved these changes
Oct 5, 2022
gaugup
added a commit
that referenced
this pull request
Nov 18, 2022
* Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change description of cohort selection panel in Aggregate Feature Importance (#1770) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Support cohort filtering of string target in rai_insights (#1771) * Port tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix rai_insights Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Simplify tests in test_cohort_filter.py (#1772) * Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <[email protected]> * Change test name Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort re-labelling logic to Cohort.ts (#1773) * Move cohort re-labelling logic to Cohort.ts Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements.txt (#1779) * Refactor DataAnalysisTab into Chartview/DataAnalysisView and DataBalanceView (#1775) * Refactor DataAnalysisTab into Chartview/DataAnalysisView and DataBalanceView Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * [Highcharts] Enable csv download, xls download and view data table in highcharts (#1783) * enable csv xls download in hightcharts * localization * Add new data APIs for compute_matrix() and compute_error_tree() (#1790) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * update e2e visit function (#1792) * Add rai_test_utils package (#1777) * Add rai_test_utils package Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix and namespace changes Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicated cohorts not retaining the selection in combobox (#1785) * fix and e2e Signed-off-by: vinutha karanth <[email protected]> * update filter Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * release raiwidgets and responsibleai 0.23.0 (#1787) * fix release github action for raiwidgets and responsibleai by making it same as in gated build (#1788) * Add e2e for dependency plot Avg of abs value option (#1768) * fix dep plot Signed-off-by: vinutha karanth <[email protected]> * add e2e Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * update e2e visit function (#1792) * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: vinutha karanth <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: tongy-msft <[email protected]> * sort fix (#1798) Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * [LogarithmicScaling] Enable log scaling for data explorer, feature importance, counterfactual and casual (#1802) * update checkbox to toggle in axis config panel * support log toggle for counterfactual * enable log scaling for feature importance * enable log scaling for casual and data explorer * address comment * Update CODEOWNERS (#1805) * Fix ‘Close button’ and ‘Spin button’ present in new cohort dialog does not have any title or aria-label (#1806) * add aria label Signed-off-by: vinutha karanth <[email protected]> * address comment Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * Update erroranalysis tests to use new rai_test_utils package (#1800) * Update erroranalysis tests to use new rai_test_utils package Signed-off-by: Gaurav Gupta <[email protected]> * Remove shap and xgboost from test requirements Signed-off-by: Gaurav Gupta <[email protected]> * Pin rai_test_utils Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix Heatmap tooltip theme (#1808) * fix tooltip color Signed-off-by: vinutha karanth <[email protected]> * fix theme Signed-off-by: vinutha karanth <[email protected]> * fix Signed-off-by: vinutha karanth <[email protected]> * rev Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * Calculate RAI insights for first 5K samples (#1803) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add API for indicating large data scenario in ModelAssessmentContext (#1804) * [WIP] Add APIs for indicating large data scenario in ModelAssessmentContext Signed-off-by: Gaurav Gupta <[email protected]> * Change API name Signed-off-by: Gaurav Gupta <[email protected]> * Add unit test Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint again Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add RAI version number to oss serialize result (#1812) * Add RAI version number to oss serialize result * update json file namg * add test Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: vinutha karanth <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> Co-authored-by: Ilya Matiach <[email protected]>
gaugup
added a commit
that referenced
this pull request
Nov 29, 2022
…t() and compute_matrix_on_dataset() (#1795) * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change description of cohort selection panel in Aggregate Feature Importance (#1770) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Support cohort filtering of string target in rai_insights (#1771) * Port tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix rai_insights Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Simplify tests in test_cohort_filter.py (#1772) * Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <[email protected]> * Change test name Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort re-labelling logic to Cohort.ts (#1773) * Move cohort re-labelling logic to Cohort.ts Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements.txt (#1779) * Refactor DataAnalysisTab into Chartview/DataAnalysisView and DataBalanceView (#1775) * Refactor DataAnalysisTab into Chartview/DataAnalysisView and DataBalanceView Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * [Highcharts] Enable csv download, xls download and view data table in highcharts (#1783) * enable csv xls download in hightcharts * localization * Add new data APIs for compute_matrix() and compute_error_tree() (#1790) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * update e2e visit function (#1792) * Migrate debug_ml and matrix endpoints to compute_error_tree_on_dataset() and compute_matrix_on_dataset() Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: tongy-msft <[email protected]>
tongyu-microsoft
added a commit
that referenced
this pull request
Dec 7, 2022
…er, counterfactual and causal managers (#1824) * Add initial code for supporting dropped features Signed-off-by: Gaurav Gupta <[email protected]> * Fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix causal path Signed-off-by: Gaurav Gupta <[email protected]> * Validate counerfactual scenario Signed-off-by: Gaurav Gupta <[email protected]> * Fix UI build Signed-off-by: Gaurav Gupta <[email protected]> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change description of cohort selection panel in Aggregate Feature Importance (#1770) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Support cohort filtering of string target in rai_insights (#1771) * Port tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix rai_insights Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Simplify tests in test_cohort_filter.py (#1772) * Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <[email protected]> * Change test name Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Move cohort re-labelling logic to Cohort.ts (#1773) * Move cohort re-labelling logic to Cohort.ts Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements.txt (#1779) * Refactor DataAnalysisTab into Chartview/DataAnalysisView and DataBalanceView (#1775) * Refactor DataAnalysisTab into Chartview/DataAnalysisView and DataBalanceView Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * [Highcharts] Enable csv download, xls download and view data table in highcharts (#1783) * enable csv xls download in hightcharts * localization * Add new data APIs for compute_matrix() and compute_error_tree() (#1790) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * update e2e visit function (#1792) * Add rai_test_utils package (#1777) * Add rai_test_utils package Signed-off-by: Gaurav Gupta <[email protected]> * Lint fix and namespace changes Signed-off-by: Gaurav Gupta <[email protected]> * Fix duplicated cohorts not retaining the selection in combobox (#1785) * fix and e2e Signed-off-by: vinutha karanth <[email protected]> * update filter Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * release raiwidgets and responsibleai 0.23.0 (#1787) * fix release github action for raiwidgets and responsibleai by making it same as in gated build (#1788) * Add e2e for dependency plot Avg of abs value option (#1768) * fix dep plot Signed-off-by: vinutha karanth <[email protected]> * add e2e Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> * lintfix Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * update e2e visit function (#1792) * Fix code review comments Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: vinutha karanth <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: tongy-msft <[email protected]> * sort fix (#1798) Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * [LogarithmicScaling] Enable log scaling for data explorer, feature importance, counterfactual and casual (#1802) * update checkbox to toggle in axis config panel * support log toggle for counterfactual * enable log scaling for feature importance * enable log scaling for casual and data explorer * address comment * Update CODEOWNERS (#1805) * Fix ‘Close button’ and ‘Spin button’ present in new cohort dialog does not have any title or aria-label (#1806) * add aria label Signed-off-by: vinutha karanth <[email protected]> * address comment Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * Update erroranalysis tests to use new rai_test_utils package (#1800) * Update erroranalysis tests to use new rai_test_utils package Signed-off-by: Gaurav Gupta <[email protected]> * Remove shap and xgboost from test requirements Signed-off-by: Gaurav Gupta <[email protected]> * Pin rai_test_utils Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix Heatmap tooltip theme (#1808) * fix tooltip color Signed-off-by: vinutha karanth <[email protected]> * fix theme Signed-off-by: vinutha karanth <[email protected]> * fix Signed-off-by: vinutha karanth <[email protected]> * rev Signed-off-by: vinutha karanth <[email protected]> Signed-off-by: vinutha karanth <[email protected]> * Calculate RAI insights for first 5K samples (#1803) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add API for indicating large data scenario in ModelAssessmentContext (#1804) * [WIP] Add APIs for indicating large data scenario in ModelAssessmentContext Signed-off-by: Gaurav Gupta <[email protected]> * Change API name Signed-off-by: Gaurav Gupta <[email protected]> * Add unit test Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint again Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add RAI version number to oss serialize result (#1812) * Add RAI version number to oss serialize result * update json file namg * add test * enable dropped_features for EA and explainer * enable EA and explainer for classification * remove unused comments * revert notebook change * revert notebook change * revert notebook change * revert causal cf changes * refactor getFeatureNamesAfterDrop to a common method * address comments on modelWrapper * update comments * update index.ts * remove unused variable * fix lint error * fix noneType error * address comments * fix a test error * support cf and causal * hide dropped_features for cf and causal * fix undefined feature_metadata error * fix undefined feature_metadata error * fix wrong cf predict value * fix undefined feature_metadata * fix undefined feature_metadata * address comments * address comments * fix feature_names test error * fix _feature_metadata undefined * fix failing test on undefined model Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: vinutha karanth <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: Vinutha Karanth <[email protected]> Co-authored-by: Ilya Matiach <[email protected]>
gaugup
added a commit
that referenced
this pull request
Dec 10, 2022
) * Add callback for global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Query SDK backend to get global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug in causal serialization Signed-off-by: Gaurav Gupta <[email protected]> * Adding callback for causal ploicy Signed-off-by: Gaurav Gupta <[email protected]> * propagate global and local effects only Signed-off-by: Gaurav Gupta <[email protected]> * Add support for getting global causal effects from SDK backend Signed-off-by: Gaurav Gupta <[email protected]> * lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug Signed-off-by: Gaurav Gupta <[email protected]> * Modify the cohort message Signed-off-by: Gaurav Gupta <[email protected]> * Move to state Signed-off-by: Gaurav Gupta <[email protected]> * Fix imports Signed-off-by: Gaurav Gupta <[email protected]> * Remove console.log Signed-off-by: Gaurav Gupta <[email protected]> * Change the if condition Signed-off-by: Gaurav Gupta <[email protected]> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Remove force update Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]>
gaugup
added a commit
that referenced
this pull request
Jan 6, 2023
) * Add callback for global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Query SDK backend to get global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug in causal serialization Signed-off-by: Gaurav Gupta <[email protected]> * Adding callback for causal ploicy Signed-off-by: Gaurav Gupta <[email protected]> * propagate global and local effects only Signed-off-by: Gaurav Gupta <[email protected]> * Add support for getting global causal effects from SDK backend Signed-off-by: Gaurav Gupta <[email protected]> * lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug Signed-off-by: Gaurav Gupta <[email protected]> * Modify the cohort message Signed-off-by: Gaurav Gupta <[email protected]> * Move to state Signed-off-by: Gaurav Gupta <[email protected]> * Fix imports Signed-off-by: Gaurav Gupta <[email protected]> * Remove console.log Signed-off-by: Gaurav Gupta <[email protected]> * Change the if condition Signed-off-by: Gaurav Gupta <[email protected]> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Remove force update Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]>
gaugup
added a commit
that referenced
this pull request
Jan 6, 2023
* Add missing property 'id' in ICounterfactualData Signed-off-by: Gaurav Gupta <[email protected]> * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * upgrade setup-python github action from v2 to v4 and pin to older ubuntu image for python 3.6 to fix build error (#1838) * Gate call to probability distribution end point (#1839) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * fix cf error on unable to update text (#1842) * Disable partial dependence plots for large data scenarios (#1843) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Remove redundant call to serialize_json_safe() in raiwidgets causal tests (#1840) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fetching global causal effects and global policy from SDK backend (#1753) * Add callback for global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Query SDK backend to get global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug in causal serialization Signed-off-by: Gaurav Gupta <[email protected]> * Adding callback for causal ploicy Signed-off-by: Gaurav Gupta <[email protected]> * propagate global and local effects only Signed-off-by: Gaurav Gupta <[email protected]> * Add support for getting global causal effects from SDK backend Signed-off-by: Gaurav Gupta <[email protected]> * lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug Signed-off-by: Gaurav Gupta <[email protected]> * Modify the cohort message Signed-off-by: Gaurav Gupta <[email protected]> * Move to state Signed-off-by: Gaurav Gupta <[email protected]> * Fix imports Signed-off-by: Gaurav Gupta <[email protected]> * Remove console.log Signed-off-by: Gaurav Gupta <[email protected]> * Change the if condition Signed-off-by: Gaurav Gupta <[email protected]> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Remove force update Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements-linting.txt (#1847) * Remove typo casual and correcting it to causal (#1846) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Disable local causal chart for large data support (#1845) * Disable local causal chart for large data support Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * add multilabel text example and fix correct and incorrect groups in data analysis (#1848) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <[email protected]> * Yarn lintfix Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * add multilabel RAI vision example for fridge multilabel dataset (#1844) * fix install command for speech to text example (#1856) * Send the correct column name to SDK backend for model overview distribution charts (#1835) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Derive fairness metrics from metric stats instead of cohort stats (#1854) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * add upper bounds to greenlet and gevent in rai-core-flask as 5.0 release fails until they are more stable (#1852) * fix loading older versions of RAIInsights due to _FEATURE_METADATA key error (#1857) * Update maintainers (#1860) * Update requirements.txt in `raiwidgets` and `responsibleai` to pin numpy upper bound (#1864) * Update requirements.txt * Update requirements.txt * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <[email protected]> * Add more constants Signed-off-by: Gaurav Gupta <[email protected]> * Add constants.py Signed-off-by: Gaurav Gupta <[email protected]> * Make constants public Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add common APIs for model overview metrics (#1862) * Add metric SDK APIs for Model Overview metrics Signed-off-by: Gaurav Gupta <[email protected]> * remove console.log() Signed-off-by: Gaurav Gupta <[email protected]> * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <[email protected]> * Add more constants Signed-off-by: Gaurav Gupta <[email protected]> * Add constants.py Signed-off-by: Gaurav Gupta <[email protected]> * Make constants public Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements-linting.txt (#1861) * update raiwidgets to rai-core-flask v0.5.0 (#1851) * fix error in erroranalysis heatmap caused by max value over right side of max bin (#1866) * remove dead space in aggregate feature importance (#1869) * display tree branching decisions (#1870) * Remove warning for 'dropped_features are not in use currently.' (#1871) * display tree branching decisions * Remove warning for 'dropped_features are not in use currently.' * turn off fail-fast for python tests (#1878) * add multilabel model overview metrics (#1882) * Show branching decisions on hover over on unselected tree nodes (#1881) * show tree label on hover over * address comments * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * Update requirements-linting.txt (#1847) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <[email protected]> * Yarn lintfix Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements-linting.txt (#1861) * add multilabel model overview metrics (#1882) * Update `raiutils` in `raiwidgets`, `responsibleai`, `erroranalysis` and `nlp_feature_extractors` (#1873) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix test Signed-off-by: Gaurav Gupta <[email protected]> * Undo changes Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]>
romanlutz
added a commit
that referenced
this pull request
Jan 12, 2023
* Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change description of cohort selection panel in Aggregate Feature Importance (#1770) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Support cohort filtering of string target in rai_insights (#1771) * Port tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix rai_insights Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Simplify tests in test_cohort_filter.py (#1772) * Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <[email protected]> * Change test name Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add Confusion Matrix Heatmap * Reconfigure Pivots for Model Overview This is done to solve the cyclomatic complexity es-lint issue. * polish and remove constants, fix axis labels * set fail-fast to false in CI-python * lintfix * simplify and fix logic * lintfix * simplify tests within model overview chart pivot component, add comprehensive tests to ensure the right charts show up * lintfix * remove console.log * fix e2e tests * lintfix * update test structure with flexible features to select in model overview tests * lintfix * fix expected number of cohorts in housing classification * lint * fix e2e tests * lintfix * lintfix * fix build error * fix model overview metric chart assertions * remove wait * allow wait for this one file * lintfix * show confusion matrix only when large data is disabled for now Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: Roman Lutz <[email protected]>
romanlutz
added a commit
that referenced
this pull request
Jan 13, 2023
…comparison (#1874) * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change description of cohort selection panel in Aggregate Feature Importance (#1770) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Support cohort filtering of string target in rai_insights (#1771) * Port tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix rai_insights Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Simplify tests in test_cohort_filter.py (#1772) * Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <[email protected]> * Change test name Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Forecasting Dashboard This commit adds the Forecasting Dashboard capabilities in both UI and SDK. * remove unrelated file * basic set of changes to get it working again * forecasting_grains -> time_series_id_column_names * fix breaking changes in latest main, add quantile prediction, fix date conversion, remove obsolete managers * fix UI components (including table and what-if creation, chart still left tbd) * get dashboard to work with multiple time series and switching back and forth * remove data explorer and model overview, create and edit cohort functionalities from forecasting * lintfix * localizzation * remove console.log * remove question * remove unrelated changes, fix package.json * string fixes and package.json adjustment * localization * localization * more localization and removing of unrelated files * remove unrelated files * remove unrelated file changes * remove is_forecasting_true_y * lintfix * add dropdown label * cache transformation predictions * lintfix * remove isUndefinedOrEmpty * remove isUndefinedOrEmpty * rename isAllDataCohort file as requested * lintfix * remove unused componentDidMount Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: John Wang <[email protected]>
romanlutz
added a commit
that referenced
this pull request
Aug 23, 2023
* Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Change description of cohort selection panel in Aggregate Feature Importance (#1770) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Support cohort filtering of string target in rai_insights (#1771) * Port tests Signed-off-by: Gaurav Gupta <[email protected]> * Fix rai_insights Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Simplify tests in test_cohort_filter.py (#1772) * Simply tests in test_cohort_filter.py Signed-off-by: Gaurav Gupta <[email protected]> * Change test name Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Forecasting Dashboard This commit adds the Forecasting Dashboard capabilities in both UI and SDK. * remove unrelated file * basic set of changes to get it working again * forecasting_grains -> time_series_id_column_names * fix breaking changes in latest main, add quantile prediction, fix date conversion, remove obsolete managers * fix UI components (including table and what-if creation, chart still left tbd) * get dashboard to work with multiple time series and switching back and forth * remove data explorer and model overview, create and edit cohort functionalities from forecasting * lintfix * localizzation * remove console.log * remove question * remove unrelated changes, fix package.json * string fixes and package.json adjustment * localization * localization * more localization and removing of unrelated files * remove unrelated files * remove unrelated file changes * remove is_forecasting_true_y * lintfix * add dropdown label * feature metadata input validation updates * cache transformation predictions * lintfix * remove isUndefinedOrEmpty * remove isUndefinedOrEmpty * rename isAllDataCohort file as requested * lintfix * first draft of notebook * get notebook to run, generate cohorts (not finished) * ntoebook * convert filter args and column names properly * fix datetime handling * fix forecast request handling * working version for bug bash * add wheels * add UI changes * update notebook to fix model download * 15 iterations * lintfix * add example of operation method string * add check for context * add explanatory comment * move description creation logic into getDescription * lintfix * remove datetime_features from snapshot * lintfix and other weirdnesses * undo print removal * remove wheels, lintfix, small cleanup * use META_JSON directly instead of declaring a variable * move constants to common files and reuse * lowercase f * isort * remove datetime_features * sktime notebook (1st half) * isort * debugging the notebook issue * use parentheses instead of backslashes * flake8 * isort * sktime progress * pre-unified RAIInsights commit * pre-unified RAIInsights commit 2 * first pass over unification * unify RAIInsights and RAIForecastingInsights * adjustments to serialization * cleanup, test fixes, consolidation, TODO fixes * fix tests * isort * flake8 * feature metadata and categorical_features consolidation * fix serialization of feature metadata * fix tests * flake8 * tests for feature metadata and cat features * flake8 * remove remnants of RAIForecastingInsights * fix string * move other utils back to rai insights * undo ModelTask import changes * undo ModelTask references to raiutils * undo references to ModelTask in raituils * remove raiutils changes since they're in a separate branch * datetime_features and time_series_id_features replacement * pytest.mark.skip * furthe datetime_features fixes * TS ID col name fix * try out asfreq * notebook try asfreq interpolation * attempt to condense index into a single col * add hidden feature flag, fix tests * remove experimental file, isort * undo string change * restructure wrappers and field to file mapping code * remove unnecessary TODO * get AML notebook to work * get AML notebook fully working * resolve some issues with sktime * notebook * remove duplicated test case * flake8 * isort * flalke8 * wrap zip object with list * use ModelTask from raiutils * remove prints * flake8 * isort * use FORECASTING from raiutils.models * isort * fix sktime issue * get past Python errors * fix predictions array shape * fix UI-Python mismatch with renamed time and time series ID column names (now called "features") * lintfix * undo changes that aren't needed any longer * remove AML notebook * update notebook by removing unnecessary content * notebook tests, basic setup * ids for tests * fix notebook based on latest changes in the repo * add test for adding a what if case * flake8 and cleanup * isort * lintfix * fix test * move notebook * fix nb path * fix kernelspec * add pmdarima dependency * skip forecasting notebook on py 3.6 * remove commented out line * skip explanation * add reason kw arg name * lintfix * fix case for vision, text * fix syntax error * fix python version comparison since it doesn't work with <= --------- Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: Gaurav Gupta <[email protected]> Co-authored-by: John Wang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add model wrapper for wrapping predictions and test data. This only adds the model wrapper in
raiwidgetstests as of now and the purpose of the PR is to demonstrate that some of the RAI components like causal, explanations and error analysis can work without the actual model. The counterfactual component will raise an exception and the prediction model wrappers cannot support the counterfactual scenario. The model wrappers incommon_utils.pywill eventually land in microsoft/ml-wrappers.The model wrapper base class
ModelWrapperPredictionswraps the test data and model prediction and supports thepredictcall. TheModelWrapperPredictionsClassificationalso supports thepredict_probamethod. The raiwidgets tests and fixtures have been modified to use the model and the aforementioned model wrappers.Checklist