Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MANOPD-70719] Verify plugins restart after template applying #74

Merged
merged 12 commits into from
Feb 9, 2022

Conversation

iLeonidze
Copy link
Contributor

@iLeonidze iLeonidze commented Dec 10, 2021

Description

  • When applying pod template on upgrading, old plugin pods can stuck and kubetools will detect them as already restarted new pods

Fixes MANOPD-70719

Solution

  • Added DaemonSet expectation API in cluster.yaml
  • Added Deployment expectation API in cluster.yaml
  • Added deployment and daemonset expectations to default plugins
  • Added time synchronisation validation between nodes after NTP configuring
  • Added time synchronisation PAAS check. If time is not synced, test will warn
  • Updated docs about time synchronisation between nodes

How to apply

  • Nothing to apply

Test Cases

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

TestCase 1

Test Configuration:

  • Hardware: Any
  • OS: Any
  • Inventory: Any

Steps:

  1. Start upgrade and make Calico old pods stuck

Results:

Before After
Installation failed with Calico backend/frontend versions misleading Installation finished without errors

Checklist

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • Integration CI passed
  • Unit tests. If Yes list of new/changed tests with brief description
  • There is no merge conflicts

Unit tests

No new unit tests

Reviewers

@koryaga @iLeonidze @Zaborin @alexarefev @Yaroslav-Lahtachev @dmyar21

@iLeonidze iLeonidze removed the draft label Dec 15, 2021
@koryaga koryaga requested review from Yaroslav-Lahtachev and removed request for theboringstuff December 20, 2021 12:17
@iLeonidze
Copy link
Contributor Author

Please review this PR again after major improvements

kubetool/system.py Outdated Show resolved Hide resolved
@n549
Copy link
Collaborator

n549 commented Jan 12, 2022

@iLeonidze what if pods have always the same names (like they do for statefulsets)?

@iLeonidze iLeonidze changed the title [MANOPD-70719] Verify plugins restart after template applying [DRAFT][MANOPD-70719] Verify plugins restart after template applying Jan 14, 2022
@iLeonidze
Copy link
Contributor Author

After some discussion, we decided to rework this PR

@iLeonidze iLeonidze force-pushed the manopd70719 branch 2 times, most recently from c5882a5 to bbaacd3 Compare January 25, 2022 12:43
@iLeonidze iLeonidze changed the title [DRAFT][MANOPD-70719] Verify plugins restart after template applying [MANOPD-70719] Verify plugins restart after template applying Jan 27, 2022
@iLeonidze iLeonidze removed the draft label Jan 27, 2022
# Conflicts:
#	kubemarine/resources/configurations/globals.yaml
@alku89
Copy link
Collaborator

alku89 commented Feb 3, 2022

tested on local qa env. Upgrade procedures finish successfully. issue with the calico plugin not reproduced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants