feat(alerts): Select tabs to send backend #17749

m-ajay · 2021-12-14T18:47:03Z

SUMMARY

For a large dashboard that have multiple tabs, users want to get email report on a specific tabs (or nested tabs). So we proposed a feature request to allow user select tab(s) when setup dashboard report. This PR is a backend piece for validating tab selection, store the selected tab with report schedule, and taking a list of selected tabs screenshot for a dashboard report.

TESTING INSTRUCTIONS

added integration tests.

ADDITIONAL INFORMATION

Has associated issue: [alert/report] Solution for dashboards with multiple tabs #14056
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

etr2460

awesome tests! a few minor comments, but everything makes sense to me

etr2460 · 2022-01-06T16:09:31Z

superset/models/reports.py

@@ -16,6 +16,8 @@
 # under the License.
 """A collection of ORM sqlalchemy models for Superset"""
 import enum
+import json
+import typing


i think in most of the code we do from typing import Any, Optional, etc.

etr2460 · 2022-01-06T16:16:44Z

superset/reports/commands/create.py

+            tab_id for tab_id in dashboard_tab_ids if tab_id not in position_data
+        ]
+        if invalid_tab_ids:
+            exceptions.append(ValidationError("Invalid tab IDs selected", "extra"))


should we append the invalid tab ids to the error message? If not, then we don't need to find all of them, and can instead just check if dashboard_tab_ids is a subset of position_data. Like this:

if not set(dashboard_tab_ids).issubset.(set(position_data.keys())): raise Exception

i think let owners know which tab_id is invalid is important. So i prefer to keep invalid_tab_ids and add it to error message.

etr2460 · 2022-01-06T16:18:20Z

superset/reports/commands/execute.py

+            for screenshot in screenshots:
+                image = screenshot.get_screenshot(user=user)
+                if image is not None:
+                    image_data.append(image)
        except SoftTimeLimitExceeded as ex:


This is a bit concerning, because now we're asking you to complete all the screenshots in the same time originally alotted to a single one. I guess we could either increase the time limit, or try to do them in parallel

can we try add time limit on each of screenshot?

codecov · 2022-01-07T00:28:24Z

Codecov Report

Merging #17749 (ae4699e) into master (07998fe) will decrease coverage by 0.02%.
The diff coverage is 98.46%.

@@            Coverage Diff             @@
##           master   #17749      +/-   ##
==========================================
- Coverage   67.07%   67.05%   -0.03%     
==========================================
  Files        1609     1610       +1     
  Lines       64899    65073     +174     
  Branches     6866     6866              
==========================================
+ Hits        43533    43634     +101     
- Misses      19500    19573      +73     
  Partials     1866     1866

Flag	Coverage Δ
hive	`53.23% <18.46%> (-0.06%)`	⬇️
mysql	`?`
postgres	`82.14% <98.46%> (-0.11%)`	⬇️
presto	`53.07% <18.46%> (-0.06%)`	⬇️
python	`82.54% <98.46%> (-0.15%)`	⬇️
sqlite	`81.82% <98.46%> (-0.10%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/reports/commands/create.py	`89.18% <92.30%> (+0.66%)`	⬆️
superset/models/reports.py	`100.00% <100.00%> (ø)`
superset/reports/commands/execute.py	`91.51% <100.00%> (+0.22%)`	⬆️
superset/reports/notifications/base.py	`95.83% <100.00%> (ø)`
superset/reports/notifications/email.py	`100.00% <100.00%> (ø)`
superset/reports/notifications/slack.py	`87.32% <100.00%> (+0.18%)`	⬆️
superset/reports/schemas.py	`98.83% <100.00%> (+0.02%)`	⬆️
superset/utils/encrypt.py	`41.46% <0.00%> (-46.54%)`	⬇️
superset/db_engine_specs/mysql.py	`94.04% <0.00%> (-3.58%)`	⬇️
superset/models/core.py	`88.75% <0.00%> (-0.76%)`	⬇️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07998fe...ae4699e. Read the comment docs.

graceguo-supercat · 2022-01-07T19:24:45Z

@dpgaspar @betodealmeida we are trying to implements an improvement for dashboard email report: allow user to select one or more dashboard tabs, and send them in a single email report. see description in this issue. This PR is backend part and we have another one for front-end tab selector. Ajay started the project but left, so I will take over the rest and finish it.

i saw you had some commits for the Alert and Report feature. please let me know if you have any concerns on our solution. Thanks.

graceguo-supercat · 2022-01-07T22:59:46Z

@m-ajay can you update the summary with more context on what we're trying to achieve here?

Ajay will not work for Superset anymore...i will update description.

graceguo-supercat · 2022-01-10T17:52:28Z

ping @etr2460 @betodealmeida @dpgaspar

betodealmeida

This looks great! I left a few minor nits, but this looks solid!

betodealmeida · 2022-01-10T23:27:27Z

superset/reports/notifications/email.py

-            else ""
-        )
+        img_tag = ""
+        for msgid, _ in images.items():


Nit:

Suggested change

for msgid, _ in images.items():

for msgid in images.keys():

(Or just images instead of images.keys(), though I think explicit is better.)

betodealmeida · 2022-01-10T23:27:44Z

superset/reports/notifications/base.py

@@ -27,7 +27,7 @@
 class NotificationContent:
    name: str
    csv: Optional[bytes] = None  # bytes for csv file
-    screenshot: Optional[bytes] = None  # bytes for the screenshot
+    screenshots: Optional[List[bytes]] = None  # bytes for the screenshot


Nit:

Suggested change

screenshots: Optional[List[bytes]] = None # bytes for the screenshot

screenshots: Optional[List[bytes]] = None # bytes for the screenshots

betodealmeida · 2022-01-10T23:30:43Z

superset/reports/notifications/email.py

-        )
+        img_tag = ""
+        for msgid, _ in images.items():
+            img_tag = img_tag + (


Nit, it's more idiomatic to do this by joining a List[str]:

img_tags = [] for msgid in images.keys(): img_tags.append('<div ...>') img_tag = ''.join(img_tags)

graceguo-supercat · 2022-01-11T18:25:52Z

Thanks @betodealmeida! i added extra fixes per comments, will merge after CI passed.

* Adding the extra config and validation * wip * reports working * Tests working * fix type * Fix lint errors * Fixing type issues * add licence header * fix the fixture deleting problem * scope to session * fix integration test * fix review comments * fix review comments patch 2 Co-authored-by: Grace Guo <[email protected]>

m-ajay added 2 commits December 8, 2021 13:39

Adding the extra config and validation

d749ae1

wip

861ddd3

pull-request-size bot added the size/L label Dec 14, 2021

m-ajay added 3 commits December 17, 2021 12:30

reports working

00077fb

Tests working

f2e20c4

fix type

91f7b93

m-ajay marked this pull request as ready for review December 21, 2021 23:18

m-ajay added 7 commits December 21, 2021 16:48

fix conflict

31c0fe6

Fix lint errors

2fd555d

Fixing type issues

e3fc9f8

conflict resolution

ba0a669

add licence header

823a8d4

fix the fixture deleting problem

996fef9

scope to session

51647a2

graceguo-supercat requested review from graceguo-supercat, etr2460 and dpgaspar December 28, 2021 04:20

Merge branch 'master' into feat/alerts-select-tabs-to-send-backend

4c70f49

etr2460 reviewed Jan 6, 2022

View reviewed changes

fix integration test

4683f5c

graceguo-supercat force-pushed the feat/alerts-select-tabs-to-send-backend branch from 04a154f to 4683f5c Compare January 7, 2022 00:13

graceguo-supercat force-pushed the feat/alerts-select-tabs-to-send-backend branch 8 times, most recently from 7163167 to ed53f56 Compare January 7, 2022 18:36

graceguo-supercat requested a review from etr2460 January 7, 2022 18:42

fix review comments

d954835

graceguo-supercat force-pushed the feat/alerts-select-tabs-to-send-backend branch from ed53f56 to d954835 Compare January 7, 2022 19:16

betodealmeida approved these changes Jan 10, 2022

View reviewed changes

fix review comments patch 2

ae4699e

graceguo-supercat merged commit bdc35a2 into apache:master Jan 11, 2022

graceguo-supercat mentioned this pull request Jan 29, 2022

fix(alert): remove extra < character in email report #18224

Merged

1 task

graceguo-supercat mentioned this pull request Mar 25, 2022

feat(report): add tab selector for dashboard reports #19354

Closed

9 tasks

ktmud mentioned this pull request Jun 29, 2022

feat(report): allow capturing dashboard reports in specific state #20552

Merged

11 tasks

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(alerts): Select tabs to send backend #17749

feat(alerts): Select tabs to send backend #17749

m-ajay commented Dec 14, 2021 •

edited by graceguo-supercat

Loading

etr2460 left a comment

etr2460 Jan 6, 2022

etr2460 Jan 6, 2022

graceguo-supercat Jan 7, 2022 •

edited

Loading

etr2460 Jan 6, 2022

graceguo-supercat Jan 7, 2022

codecov bot commented Jan 7, 2022 •

edited

Loading

graceguo-supercat commented Jan 7, 2022

graceguo-supercat commented Jan 7, 2022

graceguo-supercat commented Jan 10, 2022

betodealmeida left a comment

betodealmeida Jan 10, 2022

betodealmeida Jan 10, 2022

betodealmeida Jan 10, 2022

graceguo-supercat commented Jan 11, 2022

	screenshots: Optional[List[bytes]] = None # bytes for the screenshot
	screenshots: Optional[List[bytes]] = None # bytes for the screenshots

feat(alerts): Select tabs to send backend #17749

feat(alerts): Select tabs to send backend #17749

Conversation

m-ajay commented Dec 14, 2021 • edited by graceguo-supercat Loading

SUMMARY

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

etr2460 left a comment

Choose a reason for hiding this comment

etr2460 Jan 6, 2022

Choose a reason for hiding this comment

etr2460 Jan 6, 2022

Choose a reason for hiding this comment

graceguo-supercat Jan 7, 2022 • edited Loading

Choose a reason for hiding this comment

etr2460 Jan 6, 2022

Choose a reason for hiding this comment

graceguo-supercat Jan 7, 2022

Choose a reason for hiding this comment

codecov bot commented Jan 7, 2022 • edited Loading

Codecov Report

graceguo-supercat commented Jan 7, 2022

graceguo-supercat commented Jan 7, 2022

graceguo-supercat commented Jan 10, 2022

betodealmeida left a comment

Choose a reason for hiding this comment

betodealmeida Jan 10, 2022

Choose a reason for hiding this comment

betodealmeida Jan 10, 2022

Choose a reason for hiding this comment

betodealmeida Jan 10, 2022

Choose a reason for hiding this comment

graceguo-supercat commented Jan 11, 2022

m-ajay commented Dec 14, 2021 •

edited by graceguo-supercat

Loading

graceguo-supercat Jan 7, 2022 •

edited

Loading

codecov bot commented Jan 7, 2022 •

edited

Loading