Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix data race #492

Merged
merged 3 commits into from
Feb 28, 2023
Merged

Fix data race #492

merged 3 commits into from
Feb 28, 2023

Conversation

mjh1
Copy link
Contributor

@mjh1 mjh1 commented Feb 27, 2023

We were seeing intermittent data race failures in our unit tests. This backoff implementation is advertised as not thread safe so we need a new instance each time. https://github.com/cenkalti/backoff/blob/v4.2.0/tries.go#L10-L12

We were seeing intermittent data race failures in our unit tests. This backoff implementation is advertised as not thread safe so we need a new instance each time. https://github.com/cenkalti/backoff/blob/v4.2.0/tries.go#L10-L12
@mjh1
Copy link
Contributor Author

mjh1 commented Feb 27, 2023

Damn, just noticed there are a few more occurrences that need fixing too.

@codecov
Copy link

codecov bot commented Feb 27, 2023

Codecov Report

Merging #492 (2015667) into main (58f3f44) will decrease coverage by 0.36470%.
The diff coverage is 43.39623%.

Impacted file tree graph

@@                 Coverage Diff                 @@
##                main        #492         +/-   ##
===================================================
- Coverage   55.41085%   55.04615%   -0.36470%     
===================================================
  Files             40          40                 
  Lines           3225        3250         +25     
===================================================
+ Hits            1787        1789          +2     
- Misses          1293        1324         +31     
+ Partials         145         137          -8     
Impacted Files Coverage Δ
clients/input_copy.go 17.47573% <0.00000%> (-44.59324%) ⬇️
clients/s3.go 0.00000% <0.00000%> (ø)
pipeline/external.go 0.00000% <0.00000%> (ø)
pipeline/mist.go 10.79812% <ø> (+1.19812%) ⬆️
pipeline/coordinator.go 79.35484% <74.46809%> (+1.91123%) ⬆️
clients/arweave_ipfs_s3.go 75.00000% <100.00000%> (+0.58140%) ⬆️
clients/mediaconvert.go 70.88949% <100.00000%> (-0.12115%) ⬇️
clients/object_store_client.go 70.52632% <100.00000%> (-2.59196%) ⬇️
transcode/manifest.go 72.58065% <100.00000%> (ø)
transcode/transcode.go 62.94118% <100.00000%> (+0.44118%) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3877db5...2015667. Read the comment docs.

Impacted Files Coverage Δ
clients/input_copy.go 17.47573% <0.00000%> (-44.59324%) ⬇️
clients/s3.go 0.00000% <0.00000%> (ø)
pipeline/external.go 0.00000% <0.00000%> (ø)
pipeline/mist.go 10.79812% <ø> (+1.19812%) ⬆️
pipeline/coordinator.go 79.35484% <74.46809%> (+1.91123%) ⬆️
clients/arweave_ipfs_s3.go 75.00000% <100.00000%> (+0.58140%) ⬆️
clients/mediaconvert.go 70.88949% <100.00000%> (-0.12115%) ⬇️
clients/object_store_client.go 70.52632% <100.00000%> (-2.59196%) ⬇️
transcode/manifest.go 72.58065% <100.00000%> (ø)
transcode/transcode.go 62.94118% <100.00000%> (+0.44118%) ⬆️
... and 4 more

@mjh1
Copy link
Contributor Author

mjh1 commented Feb 27, 2023

Ok fixed all of them now 👍

@@ -155,3 +153,7 @@ type StubInputCopy struct{}
func (s *StubInputCopy) CopyInputToS3(requestID string, inputFile, osTransferURL *url.URL) (inputVideoProbe video.InputVideo, signedURL string, err error) {
return video.InputVideo{}, "", nil
}

func RetryBackoff() backoff.BackOff {
return backoff.WithMaxRetries(newExponentialBackOffExecutor(), 5)
Copy link
Collaborator

@emranemran emranemran Feb 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to create two functions for better readability?
func RetryExponentialBackoff() and func RetryConstantBackoff()

@mjh1 mjh1 merged commit d1a38db into main Feb 28, 2023
@mjh1 mjh1 deleted the mh/fix-data-race branch February 28, 2023 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants