Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enhanced support for dmt with source config changes as well as retrying with backoff #351

Merged
merged 10 commits into from
Aug 2, 2023

Conversation

desusai7
Copy link
Contributor

@desusai7 desusai7 commented Aug 1, 2023

Enhanced support for DMT

  1. Source Config Flag change from areTransformationsConnected to shouldApplyDeviceModeTransformation to understand if a device mode destination had transformations connected to it.
  2. Parsing a New Flag propagateEventsUntransformedOnError from, which says should original events be sent back to a destination on transformation failure
  3. Retrying with exponential backoff for the requests made to the transformation service with a delay starting at 500ms and a max limit of 1000ms with almost 2 retries
  4. When network requests to DMT service fails even after backing off and retries getting exhausted, then we would dump back the original events to the destination factories based on the new flag in propagateEventsUntransformedOnError.
  1. When we received success response from the DMT service then we will check the status at each event level.
    -> If 200 we will dump the transformed event to factories,
    -> If the status is 410|4xx we will dump the original events based on the new flag propagateEventsUntransformedOnError and detailed logs accordingly
  1. Include destination Ids at each event level based on the user consent, event filtering, and integrations object in the request being sent to the DMT service
  2. If we are dumping original events, utilize the destination Ids from the request object and dump to them
  3. Disable Gzip for requests made to the DMT service
  4. Handle network un-availability by breaking out of do-while immediately and stop the data loss because even retries would fail

@desusai7
Copy link
Contributor Author

desusai7 commented Aug 1, 2023

same as #342, but opened it again because of github not adding the merge commit to the PR and still showing the conflicts

@sonarcloud
Copy link

sonarcloud bot commented Aug 1, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

3.3% 3.3% Coverage
0.0% 0.0% Duplication

@desusai7 desusai7 merged commit 1ee4807 into develop Aug 2, 2023
10 checks passed
@pallabmaiti pallabmaiti deleted the fix/dmt branch August 23, 2023 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants