[Metadata] Fix counterfactual error on unable to update feature value #1842

tongyu-microsoft · 2022-12-09T18:18:25Z

If dropped features are passed to RAIInsights, the feature index could be wrong, We need to use metadict to query for the feature key.

Description

Before: Unable to update feature values in counterfactual list if dropped features are passed.

After: We are able to update feature values in counterfactual list if dropped features are passed.

Checklist

I have added screenshots above for all UI changes.
I have added e2e tests for all UI changes.
Documentation was updated if it was needed.

codecov-commenter · 2022-12-09T18:36:35Z

Codecov Report

Merging #1842 (07c1625) into main (75e4926) will increase coverage by 13.28%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             main    #1842       +/-   ##
===========================================
+ Coverage   79.00%   92.28%   +13.28%     
===========================================
  Files          19       99       +80     
  Lines        1324     4603     +3279     
===========================================
+ Hits         1046     4248     +3202     
- Misses        278      355       +77

Flag	Coverage Δ
unittests	`92.28% <ø> (+13.28%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
raiwidgets/raiwidgets/dashboard.py
raiwidgets/raiwidgets/fairness_dashboard.py
raiwidgets/raiwidgets/__version__.py
raiwidgets/raiwidgets/model_analysis_dashboard.py
...idgets/raiwidgets/responsibleai_dashboard_input.py
raiwidgets/raiwidgets/interfaces.py
...iwidgets/raiwidgets/explanation_dashboard_input.py
raiwidgets/raiwidgets/error_handling.py
raiwidgets/raiwidgets/responsibleai_dashboard.py
raiwidgets/raiwidgets/constants.py
... and 108 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

github-actions · 2022-12-09T18:50:29Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/tongy/fixMetadataBug/dashboard/index.html

github-actions · 2022-12-09T19:07:58Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/tongy/fixMetadataBug/dashboard/index.html

* Add missing property 'id' in ICounterfactualData Signed-off-by: Gaurav Gupta <[email protected]> * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * upgrade setup-python github action from v2 to v4 and pin to older ubuntu image for python 3.6 to fix build error (#1838) * Gate call to probability distribution end point (#1839) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * fix cf error on unable to update text (#1842) * Disable partial dependence plots for large data scenarios (#1843) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Remove redundant call to serialize_json_safe() in raiwidgets causal tests (#1840) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fetching global causal effects and global policy from SDK backend (#1753) * Add callback for global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Query SDK backend to get global causal effects Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug in causal serialization Signed-off-by: Gaurav Gupta <[email protected]> * Adding callback for causal ploicy Signed-off-by: Gaurav Gupta <[email protected]> * propagate global and local effects only Signed-off-by: Gaurav Gupta <[email protected]> * Add support for getting global causal effects from SDK backend Signed-off-by: Gaurav Gupta <[email protected]> * lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix bug Signed-off-by: Gaurav Gupta <[email protected]> * Modify the cohort message Signed-off-by: Gaurav Gupta <[email protected]> * Move to state Signed-off-by: Gaurav Gupta <[email protected]> * Fix imports Signed-off-by: Gaurav Gupta <[email protected]> * Remove console.log Signed-off-by: Gaurav Gupta <[email protected]> * Change the if condition Signed-off-by: Gaurav Gupta <[email protected]> * Add postga build trigger (#1755) (#1756) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add model wrapper for wrapping predictions and test data (#1762) * Add model wrapper for wrapping predictions and test data Signed-off-by: Gaurav Gupta <[email protected]> * Fix failing tests Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Remove force update Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements-linting.txt (#1847) * Remove typo casual and correcting it to causal (#1846) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Disable local causal chart for large data support (#1845) * Disable local causal chart for large data support Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * add multilabel text example and fix correct and incorrect groups in data analysis (#1848) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <[email protected]> * Yarn lintfix Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * add multilabel RAI vision example for fridge multilabel dataset (#1844) * fix install command for speech to text example (#1856) * Send the correct column name to SDK backend for model overview distribution charts (#1835) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Derive fairness metrics from metric stats instead of cohort stats (#1854) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * add upper bounds to greenlet and gevent in rai-core-flask as 5.0 release fails until they are more stable (#1852) * fix loading older versions of RAIInsights due to _FEATURE_METADATA key error (#1857) * Update maintainers (#1860) * Update requirements.txt in `raiwidgets` and `responsibleai` to pin numpy upper bound (#1864) * Update requirements.txt * Update requirements.txt * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <[email protected]> * Add more constants Signed-off-by: Gaurav Gupta <[email protected]> * Add constants.py Signed-off-by: Gaurav Gupta <[email protected]> * Make constants public Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Add common APIs for model overview metrics (#1862) * Add metric SDK APIs for Model Overview metrics Signed-off-by: Gaurav Gupta <[email protected]> * remove console.log() Signed-off-by: Gaurav Gupta <[email protected]> * Port `cohort.py` from `raiwidgets` to `raiutils` (#1859) * Port cohort.py from raiwidgets to raiutils Signed-off-by: Gaurav Gupta <[email protected]> * Add more constants Signed-off-by: Gaurav Gupta <[email protected]> * Add constants.py Signed-off-by: Gaurav Gupta <[email protected]> * Make constants public Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * lint fix Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements-linting.txt (#1861) * update raiwidgets to rai-core-flask v0.5.0 (#1851) * fix error in erroranalysis heatmap caused by max value over right side of max bin (#1866) * remove dead space in aggregate feature importance (#1869) * display tree branching decisions (#1870) * Remove warning for 'dropped_features are not in use currently.' (#1871) * display tree branching decisions * Remove warning for 'dropped_features are not in use currently.' * turn off fail-fast for python tests (#1878) * add multilabel model overview metrics (#1882) * Show branching decisions on hover over on unselected tree nodes (#1881) * show tree label on hover over * address comments * Update some sample notebooks to include dropped_features (#1836) * update some notebooks to include dropped_features * update some notebooks to include dropped_features * update some notebooks to include dropped_features * remove unused changes * remove unused changes * address comments * Revert "Update some sample notebooks to include dropped_features (#1836)" (#1837) This reverts commit d89b03a. * Update requirements-linting.txt (#1847) * Add number of cohort samples inside generateMetrics() (#1849) * Add number of cohort samples inside generateMetrics() Signed-off-by: Gaurav Gupta <[email protected]> * Yarn lintfix Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Update requirements-linting.txt (#1861) * add multilabel model overview metrics (#1882) * Update `raiutils` in `raiwidgets`, `responsibleai`, `erroranalysis` and `nlp_feature_extractors` (#1873) Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> * Fix test Signed-off-by: Gaurav Gupta <[email protected]> * Undo changes Signed-off-by: Gaurav Gupta <[email protected]> * Fix lint Signed-off-by: Gaurav Gupta <[email protected]> Signed-off-by: Gaurav Gupta <[email protected]> Co-authored-by: tongy-msft <[email protected]> Co-authored-by: Ilya Matiach <[email protected]> Co-authored-by: Roman Lutz <[email protected]>

fix cf error on unable to update text

5ccf25e

tongyu-microsoft requested review from romanlutz, vinuthakaranth, xuke444, RubyZ10 and hawestra as code owners December 9, 2022 18:18

Merge branch 'main' into tongy/fixMetadataBug

07c1625

vinuthakaranth approved these changes Dec 9, 2022

View reviewed changes

tongyu-microsoft merged commit 5fabca0 into main Dec 9, 2022

tongyu-microsoft deleted the tongy/fixMetadataBug branch December 9, 2022 22:52

gaugup pushed a commit that referenced this pull request Jan 6, 2023

fix cf error on unable to update text (#1842)

1d721c4

imatiach-msft mentioned this pull request Jan 6, 2023

release raiwidgets and responsibleai 0.24.0 #1891

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Metadata] Fix counterfactual error on unable to update feature value #1842

[Metadata] Fix counterfactual error on unable to update feature value #1842

tongyu-microsoft commented Dec 9, 2022 •

edited

Loading

codecov-commenter commented Dec 9, 2022 •

edited

Loading

github-actions bot commented Dec 9, 2022

github-actions bot commented Dec 9, 2022

[Metadata] Fix counterfactual error on unable to update feature value #1842

[Metadata] Fix counterfactual error on unable to update feature value #1842

Conversation

tongyu-microsoft commented Dec 9, 2022 • edited Loading

Description

Checklist

codecov-commenter commented Dec 9, 2022 • edited Loading

Codecov Report

github-actions bot commented Dec 9, 2022

github-actions bot commented Dec 9, 2022

tongyu-microsoft commented Dec 9, 2022 •

edited

Loading

codecov-commenter commented Dec 9, 2022 •

edited

Loading