Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(controller): don't timeout rollout when still waiting for scale down delay #3417

Merged
merged 2 commits into from
Mar 6, 2024

Merge branch 'master' into fix_progress_deadline_exceeded_with_scale_…

6460c18
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
Merged

fix(controller): don't timeout rollout when still waiting for scale down delay #3417

Merge branch 'master' into fix_progress_deadline_exceeded_with_scale_…
6460c18
Select commit
Loading
Failed to load commit list.
GitHub Actions / E2E Tests Published Test Results succeeded Mar 5, 2024 in 0s

4 fail, 6 skipped, 98 pass in 3h 43m 58s

  4 files    4 suites   3h 43m 58s ⏱️
108 tests  98 ✅  6 💤  4 ❌
448 runs  408 ✅ 24 💤 16 ❌

Results for commit 6460c18.

Annotations

Check warning on line 0 in github.com/argoproj/argo-rollouts/test/e2e

See this annotation in the file changed.

@github-actions github-actions / E2E Tests Published Test Results

4 out of 8 runs failed: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (github.com/argoproj/argo-rollouts/test/e2e) 

artifacts/E2E Test Results (k8s 1.23)/junit.xml [took 13s]
artifacts/E2E Test Results (k8s 1.24)/junit.xml [took 13s]
Raw output 
Failed
=== RUN   TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress
time="2024-03-05T17:11:39Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress"
W0305 17:11:39.721919   21807 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0305 17:11:39.726030   21807 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
time="2024-03-05T17:11:39Z" level=info msg="service/alb-rollout-root created\n" rollout=alb-rollout
time="2024-03-05T17:11:40Z" level=info msg="service/alb-rollout-canary created\n" rollout=alb-rollout
time="2024-03-05T17:11:40Z" level=info msg="service/alb-rollout-stable created\n" rollout=alb-rollout
time="2024-03-05T17:11:40Z" level=info msg="ingress.networking.k8s.io/alb-rollout-multi-ingress-1 created\n" rollout=alb-rollout
time="2024-03-05T17:11:40Z" level=info msg="ingress.networking.k8s.io/alb-rollout-multi-ingress-2 created\n" rollout=alb-rollout
time="2024-03-05T17:11:41Z" level=info msg="rollout.argoproj.io/alb-rollout created\n" rollout=alb-rollout
time="2024-03-05T17:11:41Z" level=info msg="Waiting for condition: status=Healthy" rollout=alb-rollout
time="2024-03-05T17:11:42Z" level=info msg="Condition 'status=Healthy' met after 1s" rollout=alb-rollout
time="2024-03-05T17:11:42Z" level=info msg="Experiment expectation 'experiment count == 0' met" rollout=alb-rollout
time="2024-03-05T17:11:42Z" level=info msg="Updated rollout pod spec: 2024-03-05T17:11:42.867066894Z" rollout=alb-rollout
time="2024-03-05T17:11:42Z" level=info msg="Sleeping 10s" rollout=alb-rollout
    require.go:72: 
        	Error Trace:	/home/runner/go/pkg/mod/github.com/tj/[email protected]/require.go:72
        	            				/home/runner/work/argo-rollouts/argo-rollouts/test/e2e/aws_test.go:298
        	            				/home/runner/work/argo-rollouts/argo-rollouts/test/fixtures/then.go:29
        	            				/home/runner/work/argo-rollouts/argo-rollouts/test/e2e/aws_test.go:287
        	Error:      	Not equal: 
        	            	expected: "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"\",\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}"
        	            	actual  : "{\"Type\":\"forward\",\"ForwardConfig\":{\"TargetGroups\":[{\"ServiceName\":\"alb-rollout-canary\",\"ServicePort\":\"80\",\"Weight\":0},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServicePort\":\"80\",\"Weight\":20},{\"ServiceName\":\"alb-rollout-stable\",\"ServicePort\":\"80\",\"Weight\":60}]}}"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"","ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}}
        	            	+{"Type":"forward","ForwardConfig":{"TargetGroups":[{"ServiceName":"alb-rollout-canary","ServicePort":"80","Weight":0},{"ServicePort":"80","Weight":20},{"ServicePort":"80","Weight":20},{"ServiceName":"alb-rollout-stable","ServicePort":"80","Weight":60}]}}
        	Test:       	TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress
Name:            alb-rollout
Namespace:       default
Status:          �[94m◌�[0m Progressing
Message:         more replicas need to be updated
Strategy:        Canary
  Step:          0/1
  SetWeight:     0
  ActualWeight:  0
Images:          nginx:1.19-alpine (�[32mstable�[0m, �[0mΣ:experiment-alb-canary�[0m, �[0mΣ:experiment-alb-stable�[0m)
Replicas:
  Desired:       1
  Current:       1
  Updated:       0
  Ready:         1
  Available:     1

NAME                                                                  KIND        STATUS         AGE  INFO
⟳ alb-rollout                                                         Rollout     �[94m◌�[0m Progressing  12s  
├──# revision:2                                                                                       
│  ├──⧉ �[33malb-rollout-754f5c885d�[0m                                        ReplicaSet  �[0m•�[0m ScaledDown   10s  �[33mcanary�[0m
│  └──Σ �[94malb-rollout-754f5c885d-2-0�[0m                                    Experiment  �[94m◌�[0m Running      10s  
│     ├──⧉ alb-rollout-754f5c885d-2-0-experiment-alb-canary           ReplicaSet  �[32m✔�[0m Healthy      10s  
│     │  └──□ alb-rollout-754f5c885d-2-0-experiment-alb-canary-f22sw  Pod         �[32m✔�[0m Running      10s  ready:1/1
│     └──⧉ alb-rollout-754f5c885d-2-0-experiment-alb-stable           ReplicaSet  �[32m✔�[0m Healthy      10s  
│        └──□ alb-rollout-754f5c885d-2-0-experiment-alb-stable-4thbt  Pod         �[32m✔�[0m Running      10s  ready:1/1
└──# revision:1                                                                                       
   └──⧉ �[32malb-rollout-d6f4fdcdc�[0m                                         ReplicaSet  �[32m✔�[0m Healthy      11s  �[32mstable�[0m
      └──□ alb-rollout-d6f4fdcdc-hc542                                Pod         �[32m✔�[0m Running      11s  ready:1/1

---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  annotations:
    rollout.argoproj.io/revision: "2"
  creationTimestamp: "2024-03-05T17:11:40Z"
  generation: 3
  labels:
    argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e
    e2e-test-name: TestALBExperimentStepNoSetWeightMultiIngress
  name: alb-rollout
  namespace: default
  resourceVersion: "4218"
  uid: b8fa61a7-8ba3-46d5-83b6-001adbf90f6c
spec:
  replicas: 1
  selector:
    matchLabels:
      app: alb-rollout
  strategy:
    canary:
      canaryService: alb-rollout-canary
      stableService: alb-rollout-stable
      steps:
      - experiment:
          analysisRunMetadata: {}
          duration: 15s
          templates:
          - metadata: {}
            name: experiment-alb-canary
            specRef: canary
            weight: 20
          - metadata: {}
            name: experiment-alb-stable
            specRef: stable
            weight: 20
      trafficRouting:
        alb:
          ingresses:
          - alb-rollout-multi-ingress-1
          - alb-rollout-multi-ingress-2
          rootService: alb-rollout-root
          servicePort: 80
  template:
    metadata:
      annotations:
        update: "2024-03-05T17:11:42.867066894Z"
      creationTimestamp: null
      labels:
        app: alb-rollout
    spec:
      containers:
      - image: nginx:1.19-alpine
        name: alb-rollout
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        resources:
          requests:
            cpu: 5m
            memory: 16Mi
status:
  HPAReplicas: 1
  availableReplicas: 1
  blueGreen: {}
  canary:
    currentExperiment: alb-rollout-754f5c885d-2-0
    weights:
      additional:
      - weight: 20
      - weight: 20
      canary:
        podTemplateHash: 754f5c885d
        serviceName: alb-rollout-canary
        weight: 0
      stable:
        podTemplateHash: d6f4fdcdc
        serviceName: alb-rollout-stable
        weight: 60
  conditions:
  - lastTransitionTime: "2024-03-05T17:11:42Z"
    lastUpdateTime: "2024-03-05T17:11:42Z"
    message: Rollout has minimum availability
    reason: AvailableReason
    status: "True"
    type: Available
  - lastTransitionTime: "2024-03-05T17:11:42Z"
    lastUpdateTime: "2024-03-05T17:11:42Z"
    message: Rollout is not healthy
    reason: RolloutHealthy
    status: "False"
    type: Healthy
  - lastTransitionTime: "2024-03-05T17:11:42Z"
    lastUpdateTime: "2024-03-05T17:11:42Z"
    message: RolloutCompleted
    reason: RolloutCompleted
    status: "False"
    type: Completed
  - lastTransitionTime: "2024-03-05T17:11:41Z"
    lastUpdateTime: "2024-03-05T17:11:42Z"
    message: Created new replica set "alb-rollout-754f5c885d"
    reason: NewReplicaSetCreated
    status: "True"
    type: Progressing
  currentPodHash: 754f5c885d
  currentStepHash: 8b59978
  currentStepIndex: 0
  message: more replicas need to be updated
  observedGeneration: "3"
  phase: Progressing
  readyReplicas: 1
  replicas: 1
  selector: app=alb-rollout
  stableRS: d6f4fdcdc

2024-03-05T17:11:41Z         Normal    RolloutAddedToInformer    Rollout resource added to informer: default/alb-rollout
2024-03-05T17:11:41Z         Normal    RolloutNotCompleted       Rollout not completed, started update to revision 1 (d6f4fdcdc)
2024-03-05T17:11:41Z         Normal    RolloutUpdated            Rollout updated to revision 1
2024-03-05T17:11:41Z         Normal    NewReplicaSetCreated      Created ReplicaSet alb-rollout-d6f4fdcdc (revision 1)
2024-03-05T17:11:41Z (x2)    Normal    TrafficWeightUpdated      Traffic weight updated to 0
2024-03-05T17:11:41Z         Normal    ScalingReplicaSet         Scaled up ReplicaSet alb-rollout-d6f4fdcdc (revision 1) from 0 to 1
2024-03-05T17:11:41Z         Normal    RolloutCompleted          Rollout completed update to revision 1 (d6f4fdcdc): Initial deploy
2024-03-05T17:11:42Z         Normal    SwitchService             Switched selector for service 'alb-rollout-stable' from '' to 'd6f4fdcdc'
2024-03-05T17:11:42Z         Normal    SwitchService             Switched selector for service 'alb-rollout-canary' from '' to 'd6f4fdcdc'
2024-03-05T17:11:42Z         Normal    RolloutNotCompleted       Rollout not completed, started update to revision 2 (754f5c885d)
2024-03-05T17:11:42Z         Normal    RolloutUpdated            Rollout updated to revision 2
2024-03-05T17:11:42Z         Normal    NewReplicaSetCreated      Created ReplicaSet alb-rollout-754f5c885d (revision 2)
2024-03-05T17:11:42Z (x2)    Normal    TrafficWeightUpdated      Traffic weight updated 
2024-03-05T17:11:42Z         Normal    ExperimentCreated         Created Experiment 'alb-rollout-754f5c885d-2-0'
2024-03-05T17:11:44Z (x2)    Normal    PatchingALBIngress        Updating Ingress `alb-rollout-multi-ingress-1` to desiredWeight '0'
2024-03-05T17:11:44Z (x2)    Normal    PatchingALBIngress        Updating Ingress `alb-rollout-multi-ingress-2` to desiredWeight '0'
2024-03-05T17:11:44Z         Normal    TrafficWeightUpdated      Traffic weight updated additional: [{20  } {20  }]

time="2024-03-05T17:11:53Z" level=info msg="Deleting e2e-test-name=TestALBExperimentStepNoSetWeightMultiIngress"
W0305 17:11:53.291461   21807 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0305 17:11:53.294530   21807 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
    --- FAIL: TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress (13.66s)

Check warning on line 0 in github.com/argoproj/argo-rollouts/test/e2e

See this annotation in the file changed.

@github-actions github-actions / E2E Tests Published Test Results

4 out of 8 runs failed: TestAWSSuite (github.com/argoproj/argo-rollouts/test/e2e) 

artifacts/E2E Test Results (k8s 1.23)/junit.xml [took 1m 59s]
artifacts/E2E Test Results (k8s 1.24)/junit.xml [took 13s]
Raw output 
Failed
=== RUN   TestAWSSuite
--- FAIL: TestAWSSuite (118.58s)

Check warning on line 0 in github.com/argoproj/argo-rollouts/test/e2e

See this annotation in the file changed.

@github-actions github-actions / E2E Tests Published Test Results

4 out of 8 runs failed: TestCanarySuite/TestCanaryDynamicStableScale (github.com/argoproj/argo-rollouts/test/e2e) 

artifacts/E2E Test Results (k8s 1.24)/junit.xml [took 18s]
artifacts/E2E Test Results (k8s 1.26)/junit.xml [took 19s]
Raw output 
Failed
=== RUN   TestCanarySuite/TestCanaryDynamicStableScale
time="2024-03-05T17:15:00Z" level=info msg="Deleting e2e-test-name=TestCanaryDynamicStableScale"
time="2024-03-05T17:15:00Z" level=info msg="service/dynamic-stable-scale-root created\n" rollout=dynamic-stable-scale
time="2024-03-05T17:15:00Z" level=info msg="service/dynamic-stable-scale-canary created\n" rollout=dynamic-stable-scale
time="2024-03-05T17:15:00Z" level=info msg="service/dynamic-stable-scale-stable created\n" rollout=dynamic-stable-scale
time="2024-03-05T17:15:00Z" level=info msg="ingress.networking.k8s.io/dynamic-stable-scale-ingress created\n" rollout=dynamic-stable-scale
time="2024-03-05T17:15:00Z" level=info msg="rollout.argoproj.io/dynamic-stable-scale created\n" rollout=dynamic-stable-scale
time="2024-03-05T17:15:01Z" level=info msg="Marked 4 revision 1 pods as ready" rollout=dynamic-stable-scale
time="2024-03-05T17:15:01Z" level=info msg="Waiting for condition: status=Healthy" rollout=dynamic-stable-scale
time="2024-03-05T17:15:05Z" level=info msg="Condition 'status=Healthy' met after 3s" rollout=dynamic-stable-scale
time="2024-03-05T17:15:05Z" level=info msg="Updated rollout pod spec: 2024-03-05T17:15:05.292269593Z" rollout=dynamic-stable-scale
time="2024-03-05T17:15:06Z" level=info msg="Marked 1 revision 2 pods as ready" rollout=dynamic-stable-scale
time="2024-03-05T17:15:06Z" level=info msg="Waiting for condition: status=Paused" rollout=dynamic-stable-scale
time="2024-03-05T17:15:07Z" level=info msg="Condition 'status=Paused' met after 0s" rollout=dynamic-stable-scale
time="2024-03-05T17:15:07Z" level=info msg="Sleeping 2s" rollout=dynamic-stable-scale
time="2024-03-05T17:15:09Z" level=info msg="Pod expectation 'revision:1 pod count == 3' met" rollout=dynamic-stable-scale
time="2024-03-05T17:15:09Z" level=info msg="Pod expectation 'revision:2 pod count == 1' met" rollout=dynamic-stable-scale
time="2024-03-05T17:15:09Z" level=info msg="Promoted rollout" rollout=dynamic-stable-scale
time="2024-03-05T17:15:10Z" level=info msg="Marked 2 revision 2 pods as ready" rollout=dynamic-stable-scale
time="2024-03-05T17:15:10Z" level=info msg="Waiting for condition: status.currentStepIndex=3" rollout=dynamic-stable-scale
time="2024-03-05T17:15:12Z" level=info msg="Condition 'status.currentStepIndex=3' met after 1s" rollout=dynamic-stable-scale
time="2024-03-05T17:15:12Z" level=info msg="Sleeping 2s" rollout=dynamic-stable-scale
time="2024-03-05T17:15:14Z" level=info msg="Pod expectation 'revision:1 pod count == 1' met" rollout=dynamic-stable-scale
time="2024-03-05T17:15:14Z" level=info msg="Pod expectation 'revision:2 pod count == 3' met" rollout=dynamic-stable-scale
time="2024-03-05T17:15:14Z" level=info msg="Aborted rollout" rollout=dynamic-stable-scale
time="2024-03-05T17:15:15Z" level=info msg="Marked 2 revision 1 pods as ready" rollout=dynamic-stable-scale
time="2024-03-05T17:15:15Z" level=info msg="Waiting for condition: rev=2 podcount=1" rollout=dynamic-stable-scale
time="2024-03-05T17:15:17Z" level=info msg="Condition 'rev=2 podcount=1' met after 2s" rollout=dynamic-stable-scale
time="2024-03-05T17:15:17Z" level=info msg="Pod expectation 'revision:1 pod count == 4' met" rollout=dynamic-stable-scale
time="2024-03-05T17:15:18Z" level=info msg="Marked 1 revision 1 pods as ready" rollout=dynamic-stable-scale
time="2024-03-05T17:15:18Z" level=info msg="Waiting for condition: rev=2 podcount=0" rollout=dynamic-stable-scale
time="2024-03-05T17:15:19Z" level=info msg="Condition 'rev=2 podcount=0' met after 0s" rollout=dynamic-stable-scale
    canary_test.go:663: Expected dynamic-stable-scale-canary selector: map[app:dynamic-stable-scale rollouts-pod-template-hash:868d98995b]. Actual: map[app:dynamic-stable-scale rollouts-pod-template-hash:74844784d5]
Name:            dynamic-stable-scale
Namespace:       default
Status:          �[31m✖�[0m Degraded
Message:         RolloutAborted: Rollout aborted update to revision 2
Strategy:        Canary
  Step:          0/4
  SetWeight:     0
  ActualWeight:  0
Images:          nginx:1.19-alpine (�[32mstable�[0m)
Replicas:
  Desired:       4
  Current:       4
  Updated:       0
  Ready:         4
  Available:     4

NAME                                              KIND        STATUS         AGE  INFO
⟳ dynamic-stable-scale                            Rollout     �[31m✖�[0m Degraded     19s  
├──# revision:2                                                                   
│  └──⧉ �[33mdynamic-stable-scale-74844784d5�[0m           ReplicaSet  �[0m•�[0m ScaledDown   14s  �[33mcanary�[0m
│     └──□ dynamic-stable-scale-74844784d5-vct62  Pod         �[94m◌�[0m Terminating  14s  ready:1/1
└──# revision:1                                                                   
   └──⧉ �[32mdynamic-stable-scale-868d98995b�[0m           ReplicaSet  �[32m✔�[0m Healthy      19s  �[32mstable�[0m
      ├──□ dynamic-stable-scale-868d98995b-sdlqb  Pod         �[32m✔�[0m Running      19s  ready:1/1
      ├──□ dynamic-stable-scale-868d98995b-2lpjv  Pod         �[32m✔�[0m Running      5s   ready:1/1
      ├──□ dynamic-stable-scale-868d98995b-6r5pb  Pod         �[32m✔�[0m Running      5s   ready:1/1
      └──□ dynamic-stable-scale-868d98995b-lkfs2  Pod         �[32m✔�[0m Running      5s   ready:1/1

---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  annotations:
    rollout.argoproj.io/revision: "2"
  creationTimestamp: "2024-03-05T17:15:00Z"
  generation: 2
  labels:
    argo-rollouts.argoproj.io/controller-instance-id: argo-rollouts-e2e
    e2e-test-name: TestCanaryDynamicStableScale
  name: dynamic-stable-scale
  namespace: default
  resourceVersion: "6554"
  uid: 184692eb-2a30-47c0-b9c8-5fb75d876927
spec:
  replicas: 4
  selector:
    matchLabels:
      app: dynamic-stable-scale
  strategy:
    canary:
      canaryService: dynamic-stable-scale-canary
      dynamicStableScale: true
      stableService: dynamic-stable-scale-stable
      steps:
      - setWeight: 25
      - pause: {}
      - setWeight: 75
      - pause: {}
      trafficRouting:
        alb:
          ingress: dynamic-stable-scale-ingress
          rootService: dynamic-stable-scale-root
          servicePort: 80
  template:
    metadata:
      annotations:
        update: "2024-03-05T17:15:05.292269593Z"
      creationTimestamp: null
      labels:
        app: dynamic-stable-scale
    spec:
      containers:
      - image: nginx:1.19-alpine
        name: dynamic-stable-scale
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        resources:
          requests:
            cpu: 5m
            memory: 16Mi
      readinessGates:
      - conditionType: argoproj.io/e2e-readiness
status:
  HPAReplicas: 5
  abort: true
  abortedAt: "2024-03-05T17:15:14Z"
  availableReplicas: 5
  blueGreen: {}
  canary:
    weights:
      canary:
        podTemplateHash: 74844784d5
        serviceName: dynamic-stable-scale-canary
        weight: 0
      stable:
        podTemplateHash: 868d98995b
        serviceName: dynamic-stable-scale-stable
        weight: 100
  conditions:
  - lastTransitionTime: "2024-03-05T17:15:05Z"
    lastUpdateTime: "2024-03-05T17:15:05Z"
    message: Rollout has minimum availability
    reason: AvailableReason
    status: "True"
    type: Available
  - lastTransitionTime: "2024-03-05T17:15:05Z"
    lastUpdateTime: "2024-03-05T17:15:05Z"
    message: Rollout is not healthy
    reason: RolloutHealthy
    status: "False"
    type: Healthy
  - lastTransitionTime: "2024-03-05T17:15:05Z"
    lastUpdateTime: "2024-03-05T17:15:05Z"
    message: RolloutCompleted
    reason: RolloutCompleted
    status: "False"
    type: Completed
  - lastTransitionTime: "2024-03-05T17:15:12Z"
    lastUpdateTime: "2024-03-05T17:15:12Z"
    message: Rollout is paused
    reason: RolloutPaused
    status: "True"
    type: Paused
  - lastTransitionTime: "2024-03-05T17:15:14Z"
    lastUpdateTime: "2024-03-05T17:15:14Z"
    message: Rollout aborted update to revision 2
    reason: RolloutAborted
    status: "False"
    type: Progressing
  currentPodHash: 74844784d5
  currentStepHash: 7c5b668566
  currentStepIndex: 0
  message: 'RolloutAborted: Rollout aborted update to revision 2'
  observedGeneration: "2"
  phase: Degraded
  readyReplicas: 5
  replicas: 5
  selector: app=dynamic-stable-scale
  stableRS: 868d98995b
  updatedReplicas: 1

2024-03-05T17:15:00Z         Normal    RolloutAddedToInformer    Rollout resource added to informer: default/dynamic-stable-scale
2024-03-05T17:15:00Z         Normal    RolloutNotCompleted       Rollout not completed, started update to revision 1 (868d98995b)
2024-03-05T17:15:00Z         Normal    RolloutUpdated            Rollout updated to revision 1
2024-03-05T17:15:00Z         Normal    NewReplicaSetCreated      Created ReplicaSet dynamic-stable-scale-868d98995b (revision 1)
2024-03-05T17:15:00Z         Normal    PatchingALBIngress        Updating Ingress `dynamic-stable-scale-ingress` to desiredWeight '0'
2024-03-05T17:15:00Z (x2)    Normal    TrafficWeightUpdated      Traffic weight updated to 0
2024-03-05T17:15:00Z         Normal    ScalingReplicaSet         Scaled up ReplicaSet dynamic-stable-scale-868d98995b (revision 1) from 0 to 4
2024-03-05T17:15:00Z         Normal    RolloutCompleted          Rollout completed update to revision 1 (868d98995b): Initial deploy
2024-03-05T17:15:05Z         Normal    SwitchService             Switched selector for service 'dynamic-stable-scale-stable' from '' to '868d98995b'
2024-03-05T17:15:05Z         Normal    SwitchService             Switched selector for service 'dynamic-stable-scale-canary' from '' to '868d98995b'
2024-03-05T17:15:05Z         Normal    RolloutNotCompleted       Rollout not completed, started update to revision 2 (74844784d5)
2024-03-05T17:15:05Z         Normal    RolloutUpdated            Rollout updated to revision 2
2024-03-05T17:15:05Z         Normal    NewReplicaSetCreated      Created ReplicaSet dynamic-stable-scale-74844784d5 (revision 2)
2024-03-05T17:15:05Z (x3)    Normal    TrafficWeightUpdated      Traffic weight updated 
2024-03-05T17:15:05Z         Normal    ScalingReplicaSet         Scaled up ReplicaSet dynamic-stable-scale-74844784d5 (revision 2) from 0 to 1
2024-03-05T17:15:07Z         Normal    SwitchService             Switched selector for service 'dynamic-stable-scale-canary' from '868d98995b' to '74844784d5'
2024-03-05T17:15:07Z         Normal    PatchingALBIngress        Updating Ingress `dynamic-stable-scale-ingress` to desiredWeight '25'
2024-03-05T17:15:07Z         Normal    TrafficWeightUpdated      Traffic weight updated from 0 to 25
2024-03-05T17:15:07Z         Normal    RolloutStepCompleted      Rollout step 1/4 completed (setWeight: 25)
2024-03-05T17:15:07Z         Normal    ScalingReplicaSet         Scaled down ReplicaSet dynamic-stable-scale-868d98995b (revision 1) from 4 to 3
2024-03-05T17:15:07Z         Normal    RolloutPaused             Rollout is paused (CanaryPauseStep)
2024-03-05T17:15:09Z         Normal    RolloutStepCompleted      Rollout step 2/4 completed (pause)

time="2024-03-05T17:15:19Z" level=info msg="Deleting e2e-test-name=TestCanaryDynamicStableScale"
    --- FAIL: TestCanarySuite/TestCanaryDynamicStableScale (19.25s)

Check warning on line 0 in github.com/argoproj/argo-rollouts/test/e2e

See this annotation in the file changed.

@github-actions github-actions / E2E Tests Published Test Results

4 out of 8 runs failed: TestCanarySuite (github.com/argoproj/argo-rollouts/test/e2e) 

artifacts/E2E Test Results (k8s 1.24)/junit.xml [took 18s]
artifacts/E2E Test Results (k8s 1.26)/junit.xml [took 6m 4s]
Raw output 
Failed
=== RUN   TestCanarySuite
time="2024-03-05T17:15:00Z" level=info msg="analysistemplate.argoproj.io/sleep-job created\n"
--- FAIL: TestCanarySuite (364.13s)

Check notice on line 0 in .github

See this annotation in the file changed.

@github-actions github-actions / E2E Tests Published Test Results

6 skipped tests found 

There are 6 skipped tests, see "Raw output" for the full list of skipped tests.
Raw output 
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBBlueGreenUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBCanaryUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBCanaryUpdateMultiIngress
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAppMeshSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestSMIIngressSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestSMISuite

Check notice on line 0 in .github

See this annotation in the file changed.

@github-actions github-actions / E2E Tests Published Test Results

108 tests found 

There are 108 tests, see "Raw output" for the full list of tests.
Raw output 
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAPISIXSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAPISIXSuite/TestAPISIXCanarySetHeaderStep
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAPISIXSuite/TestAPISIXCanaryStep
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBBlueGreenUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBCanaryUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBCanaryUpdateMultiIngress
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBExperimentStep
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBExperimentStepMultiIngress
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBExperimentStepNoSetWeight
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBExperimentStepNoSetWeightMultiIngress
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBPingPongUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestALBPingPongUpdateMultiIngress
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestAlbHeaderRoute
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAWSSuite/TestAlbHeaderRouteMultiIngress
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestAnalysisWithArgs
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestAnalysisWithSecret
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestBackgroundAnalysisWithArgs
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestBlueGreenAbortAndUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestBlueGreenAnalysis
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestBlueGreenKitchenSink
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestBlueGreenPostPromotionFail
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestBlueGreenPrePromotionFail
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestCanaryBackgroundAnalysis
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestCanaryFailInlineMultipleAnalysis
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestCanaryInlineAnalysis
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestCanaryInlineMultipleAnalysis
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAnalysisSuite/TestMultipleAnalysis
github.com/argoproj/argo-rollouts/test/e2e ‑ TestAppMeshSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestBlueGreenSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestBlueGreenSuite/TestBlueGreenPreviewReplicaCount
github.com/argoproj/argo-rollouts/test/e2e ‑ TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithPause
github.com/argoproj/argo-rollouts/test/e2e ‑ TestBlueGreenSuite/TestBlueGreenProgressDeadlineExceededWithoutPause
github.com/argoproj/argo-rollouts/test/e2e ‑ TestBlueGreenSuite/TestEphemeralMetadata
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryDynamicStableScale
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryDynamicStableScaleRollbackToStable
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryProgressDeadlineExceededWithPause
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryScaleDownDelay
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryScaleDownDelayWithProgressDeadline
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryScaleDownOnAbort
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryScaleDownOnAbortNoTrafficRouting
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanarySetCanaryScale
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryUnScaleDownOnAbort
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestCanaryWithPausedRollout
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestEphemeralMetadata
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestReduceWeightAndHonorMaxUnavailable
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestRolloutScalingDuringUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestRolloutScalingWhenPaused
github.com/argoproj/argo-rollouts/test/e2e ‑ TestCanarySuite/TestRolloutWithMaxSurgeScalingDuringUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestExperimentSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestExperimentSuite/TestExperimentWithDryRunMetrics
github.com/argoproj/argo-rollouts/test/e2e ‑ TestExperimentSuite/TestExperimentWithMeasurementRetentionMetrics
github.com/argoproj/argo-rollouts/test/e2e ‑ TestExperimentSuite/TestExperimentWithMultiportServiceAndScaleDownDelay
github.com/argoproj/argo-rollouts/test/e2e ‑ TestExperimentSuite/TestExperimentWithServiceAndScaleDownDelay
github.com/argoproj/argo-rollouts/test/e2e ‑ TestExperimentSuite/TestExperimentWithServiceNameAndScaleDownDelay
github.com/argoproj/argo-rollouts/test/e2e ‑ TestExperimentSuite/TestRolloutWithExperimentAndAnalysis
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestBlueGreenExceedProgressDeadlineAbort
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestBlueGreenPreviewReplicaCountPromoteFull
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestBlueGreenPromoteFull
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestBlueGreenScaleDownDelay
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestBlueGreenScaleDownOnAbort
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestBlueGreenToCanary
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestBlueGreenUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestCanaryPromoteFull
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestCompleteRolloutRestart
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestContainerResourceFormats
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestControllerMetrics
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestFixInvalidSpec
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestKubectlWaitForCompleted
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestKubectlWaitForPaused
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestMalformedRollout
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestMalformedRolloutEphemeralCtr
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestNeverScaleDown
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutAbortRetryPromote
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutBadToBadUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutBadToGoodSetWeightUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutBadToGoodUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutGoodToBadUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutPDBRestart
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutPauseDurationGreaterThanProgressDeadlineSeconds
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutPodVolumesItemsSpec
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestRolloutRestart
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestScaleDownOnSuccess
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestScaleDownProgressively
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestWorkloadRef
github.com/argoproj/argo-rollouts/test/e2e ‑ TestFunctionalSuite/TestWorkloadRefTemplate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestHeaderRoutingSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestHeaderRoutingSuite/TestIstioHostHeaderRoute
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioAbortUpdate
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioAbortUpdateDeleteAllCanaryPods
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioHostSplit
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioHostSplitExperimentStep
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioHostSplitOnlyTls
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioSubsetSplit
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioSubsetSplitExperimentStep
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioSubsetSplitSingleRoute
github.com/argoproj/argo-rollouts/test/e2e ‑ TestIstioSuite/TestIstioUpdateInMiddleZeroCanaryReplicas
github.com/argoproj/argo-rollouts/test/e2e ‑ TestMirrorRouteSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestMirrorRouteSuite/TestIstioHostMirrorRoute
github.com/argoproj/argo-rollouts/test/e2e ‑ TestRollbackSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestRollbackSuite/TestRollbackAnalysisOutsideWindow
github.com/argoproj/argo-rollouts/test/e2e ‑ TestRollbackSuite/TestRollbackAnalysisWithinWindow
github.com/argoproj/argo-rollouts/test/e2e ‑ TestSMIIngressSuite
github.com/argoproj/argo-rollouts/test/e2e ‑ TestSMISuite
View more details on GitHub Actions