Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add new report, and alert type: webhook #30044

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

kistoth90
Copy link

SUMMARY

I add webhook as a new selectable report type.

Képernyőkép ekkor: 2024-08-13 11-24-05

By the use of webhook the user can post csv/png/pdf to any endpoint, with some informative parameters as:

  • name of the report
  • id of the report
  • report type (chart, or dashboard)
  • content id
  • content format

For security reasons I made a new optional parameter: WEBHOOK_SECRET. It's a secret string between the Superset and the webhook endpoint. In the case where WEBHOOK_SECRET is filled, Superset adds an "X-Webhook-Signature" parameter to the header of the post call, which hashes the json data to be sent. The receiving party can verify that the party sending the webhook is the real sender by hashing the received json data and comparing it with the sending party's "X-Webhook-Signature" parameter. If the two parameters do not match, the receiving party may reject the call because the sender is not the supposed host.

Képernyőkép ekkor: 2024-08-13 12-04-57

On the report log, we can get feedback about the successful, and not successful requests as well.

Képernyőkép ekkor: 2024-08-13 12-27-31

I'm happy to share my code with you guys!

TESTING INSTRUCTIONS

  1. (optional) Fill the WEBHOOK_SECRET input parameter in the config file
  2. On the Alerts & reports page, press the "+ Report" or "+ Alert" button.
  3. On the notification panel, you can select Webhook as Notification method
  4. Fill the webhook endpoint url
  5. Fill all the mandatory input

By the result you will get the selectend chart/dashboard file on your webhook endpoint with some informative parameters as:

  • name of the report
  • id of the report
  • report type (chart, or dashboard)
  • content id
  • content format

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

- new report type
- new notification logic
- update NotificationMethod.tsx with webhook
- update config with WEBHOOK_SECRET
@dosubot dosubot bot added alert-reports Namespace | Anything related to the Alert & Reports feature api Related to the REST API change:frontend Requires changing the frontend labels Aug 29, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congrats on making your first PR and thank you for contributing to Superset! 🎉 ❤️

We hope to see you in our Slack community too! Not signed up? Use our Slack App to self-register.

@michael-s-molina michael-s-molina changed the title Add new report, and alert type: webhook feat: Add new report, and alert type: webhook Aug 29, 2024
Copy link

codecov bot commented Aug 29, 2024

Codecov Report

Attention: Patch coverage is 54.05405% with 34 lines in your changes missing coverage. Please review.

Project coverage is 70.84%. Comparing base (76d897e) to head (a4aca41).
Report is 909 commits behind head on master.

Files with missing lines Patch % Lines
superset/reports/notifications/webhook.py 47.91% 25 Missing ⚠️
.../features/alerts/components/NotificationMethod.tsx 62.50% 3 Missing and 3 partials ⚠️
...d/src/features/alerts/components/RecipientIcon.tsx 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master   #30044       +/-   ##
===========================================
+ Coverage   60.48%   70.84%   +10.35%     
===========================================
  Files        1931     1990       +59     
  Lines       76236    80378     +4142     
  Branches     8568     9154      +586     
===========================================
+ Hits        46114    56940    +10826     
+ Misses      28017    21216     -6801     
- Partials     2105     2222      +117     
Flag Coverage Δ
hive 48.90% <52.72%> (-0.26%) ⬇️
javascript 58.60% <52.63%> (+0.89%) ⬆️
mysql 76.76% <54.54%> (?)
postgres 76.86% <54.54%> (?)
presto 53.38% <52.72%> (-0.42%) ⬇️
python 83.88% <54.54%> (+20.39%) ⬆️
sqlite 76.32% <54.54%> (?)
unit 60.85% <52.72%> (+3.23%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@villebro
Copy link
Member

Really cool idea @kistoth90!

- add license template to webhook.py
- revert label rename
@github-actions github-actions bot removed the api Related to the REST API label Aug 30, 2024
@rusackas
Copy link
Member

rusackas commented Sep 4, 2024

I love the idea here. Approving the next CI run... ping me if it needs to be kicked again.

@kistoth90
Copy link
Author

At this point I don't know what, or how should I fix the errors:

  • I tried to run the cypress on my desk (cypress-run-chrome) but I got constant "400 Bad Request: The CSRF token is missing." during the login call. I was able to log in manually by the test credentials.
  • the pre-commit works locally fine... i dont know why it want to refresh the source code during the CI run.

I would appreciate some assistance about, becauseI'm stuck.

"""
Retrieves the secret token for HMAC signature from the application configuration.
"""
with app.app_context():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to create a new app_context here?

return hmac.new(
secret.encode("utf-8"),
json.dumps(data).encode("utf-8"),
digestmod="sha256",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

digestmod could be a new configuration key

return json.loads(self._recipient.recipient_config_json)["target"]

def _generate_signature(
self, data: Dict[str, Optional[str | None | int]]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still support python 3.9, in order to use the new union type annotation we need to import:

from __future__ import annotations

return None

def _get_header(
self, data: Dict[str, Optional[str | None | int]]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if using from __future__ import annotations let's change this to:

def _get_header(
    self, data: dict[str, Optional[str | None | int]]
...

),
"content_format": self._content.header_data.get("notification_format"),
"metadata": json.loads(self._recipient.recipient_config_json).get(
"JSONData"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about creating a TypeDict to make this payload structure more explicit

timeout=10,
)
response.raise_for_status()
logger.info("Report webhook has been successfully sent to {host}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use:

logger.info("Report webhook has been successfully sent to %s", host)

instead

@fisjac
Copy link
Contributor

fisjac commented Sep 19, 2024

For the Pre-commit issue, have you tried running ruff and black locally to remove the formatting blocks?

kistoth and others added 5 commits September 19, 2024 11:49
- add __future__ package
- remove app_context from _get_secre_token()
- move digestmod to config as constant
- remove metadata from payload
- fix logger.info message
@kistoth90
Copy link
Author

Hi @fisjac,

Yepp, the ruff, and Blacklist passed as well!

I made some changes by the guidance of dpgaspar. Let's hope the best! 🤞

@sadpandajoe
Copy link
Member

Closing and reopening to try to get tests to kick off.

@sadpandajoe sadpandajoe reopened this Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert-reports Namespace | Anything related to the Alert & Reports feature change:frontend Requires changing the frontend size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants