Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: record Success-Fail telemetry event on job attachments upload and download. #393

Merged
merged 1 commit into from
Jul 10, 2024
Merged

feat: record Success-Fail telemetry event on job attachments upload and download. #393

merged 1 commit into from
Jul 10, 2024

Conversation

leongdl
Copy link
Contributor

@leongdl leongdl commented Jul 5, 2024

What was the problem/requirement? (What/Why)

  • deadline cloud had a regression when 0.48.2 release was pushed to Pypi.
  • The regression was root caused to Job Attachments download code paths after enabling boto3 with CRT.
  • The client side lacked telemetry of different upload / download use cases that can be good signals of a regression.

What was the solution? (How)

  • Add a try-caught wrapper decorator that can be used to emit simple success or fail telemetries.
  • Add the decorator to both Job Submission (CLI / GUI) code paths and to Job output download code paths.

What is the impact of this change?

  • Capture more data about how the component is behaving after release.

How was this change tested?

  • hatch run build
  • hatch run fmt
  • hatch run test
------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                         4808    672   2110    204    84%
Coverage HTML written to dir build/coverage
Coverage XML written to file build/coverage/coverage.xml

Required test coverage of 80.0% reached. Total coverage: 84.42%
  • hatch run integ:test
scheduling tests via LoadScheduling

test/integ/deadline_job_attachments/test_job_attachments.py::test_upload_input_files_all_assets_in_cas[2023-03-03] 
[gw0] [  7%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_upload_input_files_all_assets_in_cas[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_no_job_attachment_settings_in_job[2023-03-03] 
[gw0] [ 14%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_no_job_attachment_settings_in_job[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_no_job_attachment_s3_settings[2023-03-03] 
[gw0] [ 21%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_no_job_attachment_s3_settings[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_with_symlink[2023-03-03] 
[gw0] [ 28%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_with_symlink[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_inputs_with_step_dependencies[2023-03-03] 
[gw0] [ 35%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_inputs_with_step_dependencies[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_with_job_id_step_id_task_id_and_download_directory[2023-03-03] 
[gw0] [ 42%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_with_job_id_step_id_task_id_and_download_directory[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_with_job_id_step_id_and_download_directory[2023-03-03] 
[gw0] [ 50%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_with_job_id_step_id_and_download_directory[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_with_job_id_and_download_directory[2023-03-03] 
[gw0] [ 57%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_with_job_id_and_download_directory[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_upload_input_files_no_download_paths[2023-03-03] 
[gw0] [ 64%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_upload_input_files_no_download_paths[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_inputs_no_inputs[2023-03-03] 
[gw0] [ 71%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_inputs_no_inputs[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_upload_bucket_wrong_account[2023-03-03] 
[gw0] [ 78%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_upload_bucket_wrong_account[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_inputs_bucket_wrong_account[2023-03-03] 
[gw0] [ 85%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_inputs_bucket_wrong_account[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_bucket_wrong_account[2023-03-03] 
[gw0] [ 92%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_bucket_wrong_account[2023-03-03] 
test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_bucket_wrong_account[2023-03-03] 
[gw0] [100%] PASSED test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_bucket_wrong_account[2023-03-03] 

================================================================ slowest 5 durations =================================================================
3.04s setup    test/integ/deadline_job_attachments/test_job_attachments.py::test_upload_input_files_all_assets_in_cas[2023-03-03]
1.56s setup    test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_inputs_with_step_dependencies[2023-03-03]
1.36s teardown test/integ/deadline_job_attachments/test_job_attachments.py::test_download_outputs_bucket_wrong_account[2023-03-03]
1.25s setup    test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_no_job_attachment_settings_in_job[2023-03-03]
1.18s call     test/integ/deadline_job_attachments/test_job_attachments.py::test_sync_outputs_no_job_attachment_settings_in_job[2023-03-03]
================================================================ 14 passed in 17.45s =================================================================

Manual testing:

  • Performed a Job submission using a blender bundle I have. Assets were hashed and uploaded.
  • Downloaded the output of the blender job I submitted. The job outputs were downloaded correctly.

Was this change documented?

  • Not applicable, this is not changing any contracts.

Is this a breaking change?

  • No, this is internal.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@leongdl leongdl requested a review from a team as a code owner July 5, 2024 19:48
@leongdl leongdl changed the title Record Success-Fail telemetry event on job attachments upload and download. feat: record Success-Fail telemetry event on job attachments upload and download. Jul 5, 2024
@leongdl leongdl force-pushed the telemetry branch 2 times, most recently from 466b70c to d731262 Compare July 8, 2024 19:19
leongdl
leongdl commented Jul 9, 2024
View reviewed changes
src/deadline/client/api/_telemetry.py
@@ -349,3 +353,35 @@ def get_deadline_cloud_library_telemetry_client(
:return: Telemetry client to make requests with.
"""
return get_telemetry_client("deadline-cloud-library", version, config=config)


def record_success_fail_telemetry_event(**decorator_kwargs: Dict[str, Any]) -> Callable[..., F]:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made this a decorator so we can tag any code path to emit success / fail from now on.

crowecawcaw
crowecawcaw previously approved these changes Jul 9, 2024
View reviewed changes
YutongLi291
YutongLi291 previously approved these changes Jul 10, 2024
View reviewed changes
@leongdl leongdl dismissed stale reviews from YutongLi291 and crowecawcaw via 6a89ab4 July 10, 2024 22:04
@leongdl leongdl enabled auto-merge (squash) July 10, 2024 22:49
@leongdl leongdl merged commit 9bc590b into aws-deadline:mainline Jul 10, 2024
15 checks passed
github-actions bot pushed a commit to Ahuge/deadline-cloud that referenced this pull request Jul 11, 2024
@leongdl @github-actions
feat: record Success-Fail telemetry event on job attachments upload. (a…
1d99924 
…ws-deadline#393)

Signed-off-by: David Leong <[email protected]>
rena-cheng pushed a commit to rena-cheng/deadline-cloud that referenced this pull request Jul 17, 2024
@leongdl
feat: record Success-Fail telemetry event on job attachments upload. (a…
f63b987 
…ws-deadline#393)

Signed-off-by: David Leong <[email protected]>
Signed-off-by: Cloud User <[email protected]>
@client-software-ci client-software-ci mentioned this pull request Jul 24, 2024
Merged
rena-cheng pushed a commit to rena-cheng/deadline-cloud that referenced this pull request Aug 26, 2024
@leongdl
feat: record Success-Fail telemetry event on job attachments upload. (a…
9cbf992 
…ws-deadline#393)

Signed-off-by: David Leong <[email protected]>
Signed-off-by: Cloud User <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@crowecawcaw crowecawcaw crowecawcaw approved these changes

@benl-2023 benl-2023 benl-2023 approved these changes

@YutongLi291 YutongLi291 YutongLi291 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@leongdl @crowecawcaw @YutongLi291 @benl-2023