Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(QueryContext): separate all processing concerns into QueryContextProcessor #17497

Merged

Conversation

ofekisr
Copy link
Contributor

@ofekisr ofekisr commented Nov 21, 2021

Background

When we have worked on #16991 we wanted to test the new functionalities in concrete and accurate unittest.
All chartData flows and its components are too couple to superset so it is impossible to create unittests.
The flows are not testable and so many components do not meet the very important principle SRP and the code became so dirty

So I've started to refactor it (#17344 ) but many changes were added and it was hard to review so I decided to split those changes into small PRs so will be easier to follow

This is the twelvth PR in a sequence of PRs to meet these

PR description

To meet SRP and preparing QueryContext for decoule it from superset
All the processing task there were moved to QueryContextProcessor

*Note- in the next future PR the creation of QueryContextProcssor will be removed from there to the QueryContext clients
and the clients will call the processor and not the QueryContext itself, then the queryContext methods will be removed

Test plans

There is no logic added so new tests are not required

Previous PRs

  1. refactor(ChartData): move ChartDataResult enums to common #17399
  2. refactor(ChartData): move chart_data_apis from ChartRestApi ChartDataRestApi #17400
  3. refactor(ChartDataCommand): separate loading query_context form cache into different module #17405
  4. refactor(TestChartApi): move chart data api tests into TestChartDataApi #17407
  5. refactor(ChartDataCommand): into two separate commands #17425
  6. refactor(ChartDataCommand): remove create queryContext command's responsibly  #17461
  7. refactor(QueryObject): decouple from queryContext and clean code #17465
  8. refactor(QueryObject): add QueryObjectFactory to meet SRP #17466
  9. refactor(QueryObject): decouple from superset #17479
  10. refactor(QueryContext): add QueryContextFactory to meet SRP #17495
  11. refactor(QueryContext): move df methods utils to utils package #17496

Copy link
Member

@amitmiran137 amitmiran137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simple move of processing logic out of the queryContext model

@ofekisr ofekisr force-pushed the refactor/chart_data_flows_12 branch 2 times, most recently from 73ba8e7 to 86c2c31 Compare November 21, 2021 16:56
@codecov
Copy link

codecov bot commented Nov 21, 2021

Codecov Report

Merging #17497 (69b1aba) into master (4b7d06c) will decrease coverage by 0.22%.
The diff coverage is 91.07%.

❗ Current head 69b1aba differs from pull request most recent head caf9cf2. Consider uploading reports for the commit caf9cf2 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17497      +/-   ##
==========================================
- Coverage   76.97%   76.75%   -0.23%     
==========================================
  Files        1045     1046       +1     
  Lines       56441    56486      +45     
  Branches     7798     7798              
==========================================
- Hits        43448    43357      -91     
- Misses      12737    12873     +136     
  Partials      256      256              
Flag Coverage Δ
hive ?
mysql 81.99% <91.07%> (+0.01%) ⬆️
postgres 82.00% <91.07%> (+0.01%) ⬆️
presto ?
python 82.04% <91.07%> (-0.44%) ⬇️
sqlite ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/common/query_context.py 92.45% <88.88%> (+1.41%) ⬆️
superset/common/query_context_processor.py 91.13% <91.13%> (ø)
superset/db_engine_specs/presto.py 83.50% <100.00%> (-6.87%) ⬇️
superset/db_engines/hive.py 0.00% <0.00%> (-85.19%) ⬇️
superset/db_engine_specs/hive.py 69.49% <0.00%> (-16.99%) ⬇️
superset/db_engine_specs/sqlite.py 91.89% <0.00%> (-5.41%) ⬇️
superset/connectors/sqla/utils.py 88.23% <0.00%> (-3.93%) ⬇️
superset/utils/celery.py 86.20% <0.00%> (-3.45%) ⬇️
superset/views/database/mixins.py 81.03% <0.00%> (-1.73%) ⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4b7d06c...caf9cf2. Read the comment docs.

@amitmiran137 amitmiran137 merged commit 66d7569 into apache:master Nov 22, 2021
AAfghahi pushed a commit that referenced this pull request Jan 10, 2022
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XL 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants