Hotfix to add VEM as soft dependency to `erroranalysis` #2290

Advitya17 · 2023-08-29T13:58:11Z

Hotfix to add VEM as soft dependency to erroranalysis to not require users to have torch for tabular scenarios.

Description

VEM (and hence torch) was becoming a hard dependency for erroranalysis and hence tabular RAI dashboard:

from responsibleai import RAIInsights
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/responsibleai/init.py:8: in
from responsibleai.modelanalysis import ModelAnalysis
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/responsibleai/modelanalysis/init.py:8: in
from responsibleai.modelanalysis.model_analysis import ModelAnalysis
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/responsibleai/modelanalysis/model_analysis.py:15: in
from responsibleai.managers.error_analysis_manager import ErrorAnalysisManager
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/responsibleai/managers/error_analysis_manager.py:15: in
from erroranalysis._internal.error_analyzer import ModelAnalyzer
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/erroranalysis/_internal/error_analyzer/init.py:5: in
from erroranalysis.analyzer import ModelAnalyzer, PredictionsAnalyzer
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/erroranalysis/analyzer/init.py:5: in
from .error_analyzer import ModelAnalyzer, PredictionsAnalyzer
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/erroranalysis/analyzer/error_analyzer.py:12: in
from vision_explanation_methods.error_labeling.error_labeling import
/usr/share/miniconda/envs/test/lib/python3.9/site-packages/vision_explanation_methods/error_labeling/error_labeling.py:10: in
import torchvision
E ModuleNotFoundError: No module named 'torchvision'

Checklist

I have added screenshots above for all UI changes.
I have added e2e tests for all UI changes.
Documentation was updated if it was needed.

erroranalysis/erroranalysis/_internal/matrix_filter.py

codecov-commenter · 2023-08-29T14:15:50Z

Codecov Report

Merging #2290 (7ea9649) into main (225bbf1) will increase coverage by 3.86%.
The diff coverage is 72.72%.

@@            Coverage Diff             @@
##             main    #2290      +/-   ##
==========================================
+ Coverage   88.84%   92.71%   +3.86%     
==========================================
  Files         133      107      -26     
  Lines        7550     5381    -2169     
==========================================
- Hits         6708     4989    -1719     
+ Misses        842      392     -450

Flag	Coverage Δ
unittests	`92.71% <72.72%> (+3.86%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...ranalysis/erroranalysis/_internal/matrix_filter.py	`95.71% <72.72%> (-0.67%)`	⬇️
...is/erroranalysis/_internal/surrogate_error_tree.py	`88.76% <72.72%> (-0.56%)`	⬇️
...ranalysis/erroranalysis/analyzer/error_analyzer.py	`93.00% <72.72%> (-1.22%)`	⬇️

... and 26 files with indirect coverage changes

github-actions · 2023-08-29T16:06:30Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

gaugup

How are you planning to test the vision error analysis flow?

Advitya17 · 2023-08-29T18:06:37Z

How are you planning to test the vision error analysis flow?

Some test cases were already added in https://github.com/microsoft/responsible-ai-toolbox/pull/2225/files#diff-db619f3fdbc5b12b9ebb0d194ced62df8b1de699428f71e9be372f73f419d7e6. Plan to try adding more.

github-actions · 2023-08-29T18:12:45Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

github-actions · 2023-08-29T18:21:19Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

erroranalysis/setup.py

…olbox into agemawat/ea_torch_hotfix

github-actions · 2023-08-29T20:55:25Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

github-actions · 2023-08-29T21:02:12Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

erroranalysis/erroranalysis/_internal/matrix_filter.py

erroranalysis/tests/test_metrics.py

…olbox into agemawat/ea_torch_hotfix

github-actions · 2023-08-30T14:19:43Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

github-actions · 2023-08-30T14:20:17Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

…olbox into agemawat/ea_torch_hotfix

github-actions · 2023-08-30T17:36:27Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

github-actions · 2023-08-30T17:37:16Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

github-actions · 2023-08-30T17:54:21Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

…olbox into agemawat/ea_torch_hotfix

github-actions · 2023-08-31T13:59:16Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

github-actions · 2023-08-31T14:04:43Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

…olbox into agemawat/ea_torch_hotfix

github-actions · 2023-08-31T15:47:10Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

github-actions · 2023-08-31T15:49:37Z

https://responsibleai.blob.core.windows.net/pullrequest/microsoft/responsible-ai-toolbox/agemawat/ea_torch_hotfix/dashboard/index.html

This reverts commit ffa1acb.

vem soft dependency update

0f3df3a

Advitya17 requested review from gaugup, imatiach-msft, RubyZ10, vinuthakaranth, tongyu-microsoft, xuke444 and hawestra as code owners August 29, 2023 13:58

import, flake & isort fixes

421e7c3

romanlutz reviewed Aug 29, 2023

View reviewed changes

erroranalysis/erroranalysis/_internal/matrix_filter.py Outdated Show resolved Hide resolved

gaugup approved these changes Aug 29, 2023

View reviewed changes

Advitya17 added 2 commits August 29, 2023 13:33

comment fixes

2bb7153

lint fixes

8a9f9b4

imatiach-msft reviewed Aug 29, 2023

View reviewed changes

erroranalysis/setup.py Outdated Show resolved Hide resolved

imatiach-msft approved these changes Aug 29, 2023

View reviewed changes

Advitya17 added 2 commits August 29, 2023 16:19

comment fix

cdb0717

Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…

3a523e3

…olbox into agemawat/ea_torch_hotfix

romanlutz reviewed Aug 30, 2023

View reviewed changes

erroranalysis/erroranalysis/_internal/matrix_filter.py Outdated Show resolved Hide resolved

erroranalysis/erroranalysis/_internal/matrix_filter.py Outdated Show resolved Hide resolved

erroranalysis/tests/test_metrics.py Show resolved Hide resolved

Advitya17 added 2 commits August 30, 2023 09:32

comment fixes

1947a04

Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…

ed9cce5

…olbox into agemawat/ea_torch_hotfix

comment update

826e97f

Advitya17 added 2 commits August 30, 2023 13:01

Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…

a0a9919

…olbox into agemawat/ea_torch_hotfix

added extras gate

eeb8433

Advitya17 added 2 commits August 31, 2023 09:27

ci py vision refactor

d2b805a

Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…

f69941c

…olbox into agemawat/ea_torch_hotfix

Advitya17 enabled auto-merge (squash) August 31, 2023 13:45

Advitya17 added 2 commits August 31, 2023 11:11

dependency tweak

dc29e00

Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…

7ea9649

…olbox into agemawat/ea_torch_hotfix

Advitya17 merged commit ffa1acb into main Aug 31, 2023
95 of 97 checks passed

Advitya17 deleted the agemawat/ea_torch_hotfix branch August 31, 2023 17:20

Advitya17 added a commit that referenced this pull request Aug 31, 2023

Revert "Hotfix to add VEM as soft dependency to erroranalysis (#2290)"

38e1d18

This reverts commit ffa1acb.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hotfix to add VEM as soft dependency to `erroranalysis` #2290

Hotfix to add VEM as soft dependency to `erroranalysis` #2290

Advitya17 commented Aug 29, 2023

codecov-commenter commented Aug 29, 2023 •

edited

Loading

github-actions bot commented Aug 29, 2023

gaugup left a comment

Advitya17 commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

Hotfix to add VEM as soft dependency to erroranalysis #2290

Hotfix to add VEM as soft dependency to erroranalysis #2290

Conversation

Advitya17 commented Aug 29, 2023

Description

Checklist

codecov-commenter commented Aug 29, 2023 • edited Loading

Codecov Report

github-actions bot commented Aug 29, 2023

gaugup left a comment

Choose a reason for hiding this comment

Advitya17 commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 30, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

Hotfix to add VEM as soft dependency to `erroranalysis` #2290

Hotfix to add VEM as soft dependency to `erroranalysis` #2290

codecov-commenter commented Aug 29, 2023 •

edited

Loading