Query formatting and dependency tree/graph visualisation

[maxalbert]

Closes #804.
Related to #786.

I have:

• Formatted any Python files with black
• Brought the branch up to date with master
• Added any relevant Github labels
• Added tests for any new additions
• Added or updated any relevant documentation
• Added an Architectural Decision Record (ADR), if appropriate
• Added an MPLv2 License Header if appropriate
• Updated the Changelog

Description

This PR adds an optional argument show_stored to print_dependency_tree() which allows to display information on whether dependent queries have been stored:

>>> dl = daily_location(date="2016-01-01", method="last")
>>> print_dependency_tree(dl, show_stored=True)
<Query of type: LastLocation, query_id: 'b46b86f42634affb00c510a1778134d4', is_stored: False>
  - <Query of type: JoinToLocation, query_id: '89a463385d85be2e77e5003b3f3b0403', is_stored: False>
     - <Query of type: _SubscriberCells, query_id: '8c568b3b371ecbb943d7a1ad35f1d1b0', is_stored: False>
        - <Query of type: EventsTablesUnion, query_id: '01f9cca502930c3a7322f00121ef9dec', is_stored: False>
           - <Query of type: EventTableSubset, query_id: 'd3b49c25d7b2ec64693e75fdae222fd9', is_stored: False>
              - <Query of type: Table, query_id: '9de507e882f1fb6b0cfcedd324e27839', is_stored: True>
                 - <Query of type: Table, query_id: '7a7f27978925c385bc44a5ec5667d7b3', is_stored: True>
           - <Query of type: EventTableSubset, query_id: 'd5cf1feacd710753f51b630b59c2deee', is_stored: False>
              - <Query of type: Table, query_id: '70a2d392f73aedcc251a6a504f17008a', is_stored: True>
                 - <Query of type: Table, query_id: '057addedac04dbeb1dcbbb6b524b43f0', is_stored: True>
     - <Query of type: CellToAdmin, query_id: 'f73369cb975cfd8c13a50f9ab1754106', is_stored: False>
        - <Query of type: CellToPolygon, query_id: 'e0196a13e5bf8d7c0c62fe5a6c63f87e', is_stored: False>

To implement this I also added a Query.__format__ method which allows more convenient formatting of query objects by optionally specifying attributes that should be included in the resulting string, for example:

>>> dl = daily_location(date="2016-01-01", method="last")
>>> print(f"{dl}")
<Query of type: LastLocation>
>>> print(f"{dl:is_stored,query_state}")
<Query of type: LastLocation, is_stored: False, query_state: <QueryState.KNOWN: 'known'>>

Finally, since I already needed to copy & paste the example code from #786 a few times, I decided to wrap it up in a helper function plot_dependency_tree() which can plot a dependency graph in SVG or PNG format, so that it can be conveniently displayed in a Jupyter notebook.

[codecov]

Codecov Report

Merging #805 into master will decrease coverage by 0.19%.
The diff coverage is 45.71%.

@@            Coverage Diff            @@
##           master     #805     +/-   ##
=========================================
- Coverage   93.34%   93.14%   -0.2%     
=========================================
  Files         130      130             
  Lines        6521     6552     +31     
  Branches      693      695      +2     
=========================================
+ Hits         6087     6103     +16     
- Misses        315      330     +15     
  Partials      119      119

Flag	Coverage Δ
#flowapi_unit_tests	79.84% <ø> (ø)	⬆️
#flowauth_unit_tests	93.97% <ø> (ø)	⬆️
#flowclient_unit_tests	83.57% <ø> (ø)	⬆️
#flowkit_jwt_generator_unit_tests	100% <ø> (ø)	⬆️
#flowmachine_unit_tests	91.11% <100%> (+0.02%)	⬆️
#integration_tests	59.92% <40%> (-0.1%)	⬇️

Impacted Files	Coverage Δ
flowmachine/flowmachine/core/table.py	90.58% <100%> (+0.22%)	⬆️
flowmachine/flowmachine/core/query.py	92.44% <100%> (+0.24%)	⬆️
flowmachine/flowmachine/utils.py	44.31% <13.63%> (-3.69%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b7235b...4661848. Read the comment docs.

[maxalbert]

Hmm, there is some alleged drop in coverage but I think this is due to codecov not picking up test_utils.py (#789)?

[greenape]

Very neat :)