Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(QueryContext): validation does not validate query_context metrics #16991

Closed

Conversation

ofekisr
Copy link
Contributor

@ofekisr ofekisr commented Oct 6, 2021

SUMMARY

close
continue in #19753

@villebro villebro self-requested a review October 6, 2021 14:02
@zhaoyongjie zhaoyongjie self-requested a review October 6, 2021 14:46
@ofekisr ofekisr force-pushed the fix/sql_metric_access_validation branch from 3e60a75 to e7b3539 Compare October 7, 2021 10:52
@ofekisr ofekisr marked this pull request as ready for review October 7, 2021 10:52
@codecov
Copy link

codecov bot commented Oct 7, 2021

Codecov Report

Merging #16991 (66ac874) into master (f8a65f8) will increase coverage by 1.33%.
The diff coverage is 86.45%.

❗ Current head 66ac874 differs from pull request most recent head dd48e73. Consider uploading reports for the commit dd48e73 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #16991      +/-   ##
==========================================
+ Coverage   67.10%   68.43%   +1.33%     
==========================================
  Files        1609     1590      -19     
  Lines       64897    65082     +185     
  Branches     6866     6963      +97     
==========================================
+ Hits        43547    44542     +995     
+ Misses      19484    18650     -834     
- Partials     1866     1890      +24     
Flag Coverage Δ
hive ?
mysql 82.04% <86.45%> (-0.15%) ⬇️
postgres 82.05% <86.45%> (-0.20%) ⬇️
presto ?
python 82.13% <86.45%> (-0.60%) ⬇️
sqlite 81.73% <86.45%> (-0.20%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/config.py 91.53% <ø> (-0.11%) ⬇️
superset/common/query_object.py 92.13% <60.00%> (-3.50%) ⬇️
superset/datasets/dao.py 93.24% <66.66%> (-0.55%) ⬇️
superset/common/query_context.py 86.15% <77.27%> (-6.58%) ⬇️
superset/tasks/async_queries.py 90.24% <83.33%> (-0.79%) ⬇️
...set/charts/data/query_context_validator_factory.py 85.71% <85.71%> (ø)
superset/charts/data/commands/get_data_command.py 96.96% <87.50%> (-3.04%) ⬇️
superset/charts/data/query_context_validator.py 89.41% <89.41%> (ø)
superset/charts/data/api.py 88.31% <100.00%> (-0.51%) ⬇️
superset/security/manager.py 91.92% <100.00%> (-0.05%) ⬇️
... and 660 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8a65f8...dd48e73. Read the comment docs.

@ofekisr ofekisr force-pushed the fix/sql_metric_access_validation branch 3 times, most recently from 84282d9 to c92e08e Compare October 7, 2021 14:58
Copy link
Member

@amitmiran137 amitmiran137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extract until you can't

superset/charts/commands/query_context_validator.py Outdated Show resolved Hide resolved
superset/charts/commands/query_context_validator.py Outdated Show resolved Hide resolved
@ofekisr ofekisr force-pushed the fix/sql_metric_access_validation branch 2 times, most recently from e6bd829 to 8038b79 Compare October 10, 2021 10:10
Copy link
Member

@amitmiran137 amitmiran137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !
Feature introduced here will enforce datasource access on custom SQL expression of metrics

@villebro we should add column in a follow up PR

Copy link
Member

@craig-rueda craig-rueda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR - it's bumping up security! I don't see any tests, however. Also, there's a lot of single line methods in the ContextValidator. Can we compress those a bit? Too much delegation makes the code a bit hard to follow along with.

superset/charts/api.py Outdated Show resolved Hide resolved
superset/charts/api.py Outdated Show resolved Hide resolved
@ofekisr ofekisr force-pushed the fix/sql_metric_access_validation branch from 7201bd3 to fb7bfa2 Compare November 22, 2021 19:32
@ofekisr
Copy link
Contributor Author

ofekisr commented Nov 22, 2021

@craig-rueda @hughhhh @jayakrishnankk
After all the recent refactor in ChartData, I recommitted the PR with all the adjustments to refactored code

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass comments. I think this is a great improvement to the security model, but I'm concerned this may cause false positives (I did a similar change some time ago, which caused unpleasant regressions due to triggering false positives in edge cases that hadn't been thought of in the original PR). To make rolling out new security improvements less problematic, I would almost consider introducing (yet another) feature flag "SECURITY_BETA" where new security improvements can be introduced and kept until more comprehensive testing has been done.

superset/security/manager.py Outdated Show resolved Hide resolved
superset/security/manager.py Outdated Show resolved Hide resolved
tests/common/query_context_generator.py Outdated Show resolved Hide resolved
tests/common/query_context_generator.py Outdated Show resolved Hide resolved
superset/charts/data/query_context_validator.py Outdated Show resolved Hide resolved
superset/security/manager.py Show resolved Hide resolved
@jayakrishnankk
Copy link
Contributor

@craig-rueda - requesting review. bumping this.

@ofekisr ofekisr force-pushed the fix/sql_metric_access_validation branch from 04b1537 to 3d204e6 Compare December 14, 2021 10:24
@ofekisr ofekisr force-pushed the fix/sql_metric_access_validation branch from 98e7b63 to 90ac808 Compare December 14, 2021 14:56
@michael-s-molina michael-s-molina removed their request for review March 7, 2022 17:41
@ofekisr ofekisr closed this Apr 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants