Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix:(controller) leader election bug #2280

Closed
wants to merge 17 commits into from

Conversation

zachaller
Copy link
Collaborator

@zachaller zachaller commented Sep 30, 2022

Fixes #2117

This change has a behavior change in that we now always spin up metrics server even on children they just have zero values because they are not processing anything.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 30, 2022

Go Published Test Results

0 tests   0 ✔️  0s ⏱️
0 suites  0 💤
0 files    0

Results for commit c369d1c.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 30, 2022

E2E Tests Published Test Results

0 tests   0 ✔️  0s ⏱️
0 suites  0 💤
0 files    0

Results for commit c369d1c.

♻️ This comment has been updated with latest results.

Signed-off-by: zachaller <[email protected]>
Signed-off-by: zachaller <[email protected]>
Signed-off-by: zachaller <[email protected]>
@zachaller
Copy link
Collaborator Author

Signed-off-by: zachaller <[email protected]>
Signed-off-by: zachaller <[email protected]>
@codecov
Copy link

codecov bot commented Oct 3, 2022

Codecov Report

Base: 82.38% // Head: 82.37% // Decreases project coverage by -0.01% ⚠️

Coverage data is based on head (c369d1c) compared to base (33ddaf5).
Patch coverage: 86.30% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2280      +/-   ##
==========================================
- Coverage   82.38%   82.37%   -0.02%     
==========================================
  Files         121      121              
  Lines       18476    18471       -5     
==========================================
- Hits        15221    15215       -6     
- Misses       2468     2471       +3     
+ Partials      787      785       -2     
Impacted Files Coverage Δ
rollout/controller.go 77.99% <0.00%> (ø)
service/service.go 68.37% <0.00%> (ø)
controller/controller.go 89.35% <88.57%> (-0.10%) ⬇️
controller/metrics/metrics.go 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Signed-off-by: zachaller <[email protected]>
@zachaller zachaller marked this pull request as ready for review October 4, 2022 00:29
@zachaller zachaller added the ready-for-review Ready for final review label Oct 4, 2022
@zachaller zachaller changed the title Fix leader election bug fix:(controller) leader election bug Oct 4, 2022
@zachaller zachaller requested a review from leoluz October 4, 2022 17:46
Copy link
Contributor

@leoluz leoluz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation looks good but I think PR should improve the test coverage preferably in form of unit tests and/or integration tests if possible.

@zachaller
Copy link
Collaborator Author

Implementation looks good but I think PR should improve the test coverage preferably in form of unit tests and/or integration tests if possible.

Added one more test to just close/exit on a single instance setup to cover the log lines. It's currently from my understanding impossible to cover the actual shutdown of a controller because go test framework has no way to capture exit codes from the test process.

I like the idea of switching the e2e tests to using HA, in order to do that though will require a bit of work including creating a HA manifest that we can use to deploy for e2e then have a test that somehow kills one of the controllers (not sure how) and looks for a switch but not sure how all that would work it would be quite complicated because I am not sure how we would be able to tell that the standby controller took over and we didn't just spin up a new instance.

@zachaller
Copy link
Collaborator Author

I will also point out that all the other lines except the OnStopLeading callback where also never tested before this they just got flagged due to rename not that that is good but you can see that here https://github.com/argoproj/argo-rollouts/pull/2282/files where I just undid the rename.

@zachaller zachaller marked this pull request as draft October 5, 2022 21:16
@zachaller zachaller marked this pull request as ready for review October 5, 2022 21:28
Signed-off-by: zachaller <[email protected]>
@sonarcloud
Copy link

sonarcloud bot commented Oct 5, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
2.9% 2.9% Duplication

@zachaller zachaller marked this pull request as draft October 5, 2022 22:18
@zachaller
Copy link
Collaborator Author

closing in favor of #2291

@zachaller zachaller closed this Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-review Ready for final review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

argo-rollouts controller leader resumes working after it stops leading, running alongside the new leader
2 participants