Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sampling rate functionality added #122

Conversation

eliasyishak
Copy link
Contributor

@eliasyishak eliasyishak commented Jul 11, 2023

This addresses part of the umbrella issue:

This feature adds a utility function that can take two different strings and return a decimal value that we can compare against the sampling rate found in the remotely hosted metadata survey file (sample below)

[
    {
        "uniqueId": "test_channel",
        "url": "https://google.qualtrics.com/jfe/form/SV_5gsB2EuG5Et5Yy2",
        "startDate": "2023-06-01T09:00:00-07:00",
        "endDate": "2023-06-30T09:00:00-07:00",
	"description": "Help improve Flutter's release builds with this 3-question survey!",
	"dismissForDays": "10",
	"moreInfoURL": "https://docs.flutter.dev/reference/crash-reporting",
	"samplingRate": "0.1",    <---- the newly generated decimal value will be compared against value
	"conditions": [
	    {
	        "field": "logFileStats.recordCount",
	        "operator": ">=",
	        "value": 1000
            }
	]
    }
]

Sample usage of the new utility function

/// Will use two strings to produce a double for applying a sampling
/// rate for [Survey] to be returned to the user
double sampleRate(String string1, String string2) {
  final int1 = string1.codeUnits.reduce((a, b) => a + b);
  final int2 = string2.codeUnits.reduce((a, b) => a + b);

  return ((int1 * int2) % 100) / 100;
}

// Very simplified example to show usage
void main() {
  final sampleRate = 0.30;

  final producedValue = sampleRate('string1', 'string2');
  // Suppose the above output was 0.25

  if (sampleRate >= producedValue) passSurveyToUser();  // sudo code
}

  • I’ve reviewed the contributor guide and applied the relevant portions to this PR.
Contribution guidelines:

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

@eliasyishak
Copy link
Contributor Author

@bwilkerson any thoughts on how I implemented the algorithm to produce a decimal value to compare against a given survey's sampling rate?

@bwilkerson
Copy link
Member

Sorry for the slow response.

I'm not sure that I have the context to understand what you're trying to accomplish (beyond the obvious intent of deciding when to display a survey). In particular, I don't understand what the 'sample rate' is suppose to represent.

@eliasyishak
Copy link
Contributor Author

@bwilkerson this feature is part of the larger issue #81 which is to enable unified analytics to provide the user surveys based on the data found in the log file (which contains all the events sent from a client). This PR is for surveys that use a sampling rate for delivering surveys since we may not want to send it all users.

Examples of a survey for this is if we wanted to ask 15% of users about which flutter channel (master, stable, beta) they should experiment with based on their historical events in the log file. Does that help?

Copy link
Member

@bwilkerson bwilkerson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how this produces the linear distribution we want, but it appears to do so, so lgtm.

@eliasyishak eliasyishak marked this pull request as ready for review July 19, 2023 13:23
@eliasyishak eliasyishak merged commit 9e090e7 into dart-lang:survey-handler-feature Jul 19, 2023
@eliasyishak eliasyishak deleted the sampling-rate-implementation branch July 19, 2023 13:25
eliasyishak added a commit that referenced this pull request Aug 3, 2023
* Survey handler functionality to fetch available surveys (#91)

* Add constant for endpoint that stores metadata json file

* Development began on survey handler class with fetch

* Update survey_handler.dart

* Parsing functionality added in `survey_handler`

* Condition class `operator` relabeled to `operatorString`

* `Analytics` test and default constructors to use `SurveyHandler`

* Refactor + cleanup + error handling

* `dart format` fix

* Evaluating functionality added to `Analytics`

* Format fix

* `!=` operator added to `Condition` class

* Refactor for fake survey handler to use list of surveys or string

* Initial test cases added

* Tests added to use json in `FakeSurveyHandler`

* Fix nit

* Early exit if on null `logFileStats`

* Test to check each field in `Survey` and `Condition`

* Documentation update

* No surveys returned for opted out users

* Revert "No surveys returned for opted out users"

This reverts commit f6d9f8e.

* No surveys for opted out users (#99)

* Check `okToSend` before fetching surveys

* Added test

* dart format fix

* Update CHANGELOG.md

* Mark as dev

* Change version suffix

* `dart fix --apply --code=combinators_ordering`

* Fix `survey_handler.dart` with new lints

* Add'l fixes to survey_handler

* Remove left hand types from `analytics.dart`

* Fix `survey_handler_test.dart` with new lints

* Fix tests with survey_handler class from lint fixes

* `dart format` fix

* Sampling rate functionality added (#122)

* Sampling rate functionality added

* Update tests to have 100% sampling rate

* Tests added to test sampling rate

* Update survey_handler_test.dart

* Fix type for `jsonDecode`

* New utility function to convert string into integer

* Fix tests with new outputs for sample rate

* Use uniqueId for survey instead of description

* Add hyphen to lookup

* Fix documentation

* Fix survey handler tests to use new .send method

* Fix tests to use new maps for `LogFileStats`

* Dismissing and persisting surveys (#127)

* Add constant for new file name + clean up session handler

Removing NoOp session instance since that was only being used before `2.0.0`

* Updating survey handler to create file to persist ids

* Revert changes to session handler

* Update constant to be a json file

* Initialize dismiss surveys file with empty json

* Initializer for dismissed file made static

* Functionality added to check if survey snoozed or dismissed

* Dismiss functionality added

* `dismissForDays` -> `dismissForMinutes`

* Update survey_handler_test.dart

* Clean up external functions to be class methods

* Tests added for snoozing and dismissing permanently

* Test added for malformed json

* Check sample rate before using LogFileStats

* Add `surveyShown` API to snooze surveys

* Use new URL for survey metadata

* Error handling for missing json file

* Sample rate example added (#130)

* Added example file

* Including example's output in code

* Update sample_rate.dart

* Fix nits

* Send event for surveys shown and surveys dismissed (#133)

* Added enum and event constructor survey actions

* Fix format errors

* Using two events for survey shown and survey action

* Created mock class to confirm events are sent

* Clean up constructors

* Fix nits

* Refactor for buttons array with `SurveyButton` class (#134)

* Added newe `SurveyButton` class

* Fix tests

* Add documentation for enums

* Update sample_rate.dart

* Update tests to check for `SurveyButton` classes

* Remove enum for status of action

* Use `snoozeForMinutes` instead of dismiss

* Expose `SurveyButton`

* Fixing documentation for event class

* Order members in survey handler

* Refactor to pass button to `surveyInteracted(..)`

* `surveyButtonList` --> `buttonList` renaming

* Adding example file for how to use survey handler feature

* Adding conditional check for url to display

* Format fix

* Allow surveys with no conditions to be passed

Only checking if `logFileStats` is null if there is a condition in the condition array in the json

* Update version

* Simplify utility functions for sample rate + check date

* `const` constructor for `Survey` unnamed constructor

* Fix test to unit test sampling function

* Fix dartdocs + check for null outside loop + breaks removed

* Add documentation to example files

* `dart format`

* Catch `TypeError` when parsing json survey

* Adding tests for the sampling rate with margin of error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants