Backport of helm: add configuration for proxy lifecycle management into release/1.0.x #2465

hc-github-team-consul-core · 2023-06-27T17:55:28Z

Backport

This PR is auto-generated from #2233 to be assessed for backporting due to the inclusion of the label backport/1.0.x.

🚨

Warning automatic cherry-pick of commits failed. If the first commit failed,
you will see a blank no-op commit below. If at least one commit succeeded, you
will see the cherry-picked commits up to, not including, the commit where
the merge conflict occurred.

The person who merged in the original PR is:
@curtbushko
This person should manually cherry-pick the original PR into a new backport PR,
and close this one when the manual backport PR is merged in.

merge conflict error: POST https://api.github.com/repos/hashicorp/consul-k8s/merges: 409 Merge conflict []

The below text is copied from the body of the original PR.

Changes proposed in this PR:

Adds Helm chart and pod annotation support for configuring proxy lifecycle management graceful shutdown functionality.

Fixes #536, #650, closes #2119

How I've tested this PR:

Acceptance test authored in acceptance: add connect proxy lifecycle shutdown test #2119 has been updated to reflect the fixed expected behavior with proxy lifecycle management graceful shutdown functionality enabled.

bats ./charts/consul/test/unit/connect-inject-deployment.bats --filter "sidecar proxy lifecycle management"

kind create cluster

pushd ../consul-dataplane
make dev
kind load docker-image consul-dataplane/release-default:1.2.0-dev
popd

DEV_IMAGE=consul-k8s-control-plane-dev:dev make control-plane-dev-docker
kind load docker-image consul-k8s-control-plane-dev:dev

cd acceptance/tests/
go test ./connect/... -p 1 -run TestConnectInject_ProxyLifecycle -consul-image=hashicorppreview/consul:1.16-dev -consul-k8s-image=consul-k8s-control-plane-dev:dev -consul-dataplane-image=consul-dataplane/release-default:1.2.0-dev -consul-dataplane-version=1.2.0
go test ./connect/... -p 1 -run TestConnectInject_ProxyLifecycle -consul-image=hashicorppreview/consul:1.16-dev -consul-k8s-image=consul-k8s-control-plane-dev:dev -consul-dataplane-image=consul-dataplane/release-default:1.2.0-dev -consul-dataplane-version=1.2.0 -enable-transparent-proxy

Logs

consul-dataplane logs showing graceful shutdown behavior as configured through HelmValues in this acceptance test:

2023-06-03T00:02:26.149Z [INFO]  consul-dataplane.lifecycle: initiating shutdown
2023-06-03T00:02:26.149Z [INFO]  consul-dataplane.lifecycle: waiting 30 seconds before terminating dataplane proxy
2023-06-03T00:02:26.149Z [DEBUG] consul-dataplane: draining inbound connections to proxy
2023-06-03T00:02:26.152Z+00:00 [debug] envoy.conn_handler(18) [C7] new connection from 127.0.0.1:44296
2023-06-03T00:02:26.152Z+00:00 [debug] envoy.http(18) [C7] new stream
2023-06-03T00:02:26.153Z+00:00 [debug] envoy.http(18) [C7][S7864296776147560620] request headers complete (end_stream=true):
':authority', '127.0.0.1:19000'
':path', '/drain_listeners?inboundonly&graceful'
':method', 'POST'
'user-agent', 'Go-http-client/1.1'
'content-length', '0'
'content-type', 'text/plain'
'accept-encoding', 'gzip'

2023-06-03T00:02:26.153Z+00:00 [debug] envoy.http(18) [C7][S7864296776147560620] request end stream
2023-06-03T00:02:26.153Z+00:00 [debug] envoy.connection(18) [C7] current connecting state: false
2023-06-03T00:02:26.154Z+00:00 [debug] envoy.admin(18) [C7][S7864296776147560620] request complete: path: /drain_listeners?inboundonly&graceful
2023-06-03T00:02:26.154Z+00:00 [debug] envoy.http(18) [C7][S7864296776147560620] drain closing connection
2023-06-03T00:02:26.154Z+00:00 [debug] envoy.http(18) [C7][S7864296776147560620] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'text/plain; charset=UTF-8'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Sat, 03 Jun 2023 00:02:26 GMT'
'server', 'envoy'
'connection', 'close'

2023-06-03T00:02:26.154Z+00:00 [debug] envoy.misc(18) nextChunk: response.length=3 more_data=false end_stream=true
2023-06-03T00:02:26.154Z+00:00 [debug] envoy.http(18) [C7][S7864296776147560620] Codec completed encoding stream.
2023-06-03T00:02:26.253Z+00:00 [debug] envoy.connection(18) [C7] closing data_to_write=0 type=2
2023-06-03T00:02:26.253Z+00:00 [debug] envoy.connection(18) [C7] closing socket: 1
2023-06-03T00:02:26.253Z+00:00 [debug] envoy.conn_handler(18) [C7] adding to cleanup list
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.filter(28) [C8] new tcp proxy session
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.filter(28) [C8] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.connection(28) [C9] connecting to 10.244.0.9:20000
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.connection(28) [C9] connection in progress
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.conn_handler(28) [C8] new connection from 127.0.0.1:49052
2023-06-03T00:02:26.733Z+00:00 [debug] envoy.connection(28) [C9] connected
2023-06-03T00:02:26.734Z+00:00 [debug] envoy.connection(28) [C9] Async cert validation completed
2023-06-03T00:02:26.734Z+00:00 [debug] envoy.pool(28) [C9] attaching to next stream
2023-06-03T00:02:26.734Z+00:00 [debug] envoy.pool(28) [C9] creating stream
2023-06-03T00:02:26.734Z+00:00 [debug] envoy.filter(28) [C8] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.connection(28) [C9] SSL shutdown: rc=0
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.connection(28) [C9] remote close
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.connection(28) [C9] closing socket: 0
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.pool(28) [C9] client disconnected, failure reason:
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.pool(28) [C9] destroying stream: 0 remaining
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.connection(28) [C8] closing data_to_write=0 type=0
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.connection(28) [C8] closing socket: 1
2023-06-03T00:02:26.736Z+00:00 [debug] envoy.conn_handler(28) [C8] adding to cleanup list
2023-06-03T00:02:28.123Z+00:00 [debug] envoy.main(18) flushing stats
2023-06-03T00:02:28.835Z+00:00 [debug] envoy.filter(28) [C10] new tcp proxy session
2023-06-03T00:02:28.835Z+00:00 [debug] envoy.filter(28) [C10] Creating connection to cluster local_app
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.connection(28) [C11] connecting to 127.0.0.1:0
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.connection(28) [C11] connection in progress
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.conn_handler(28) [C10] new connection from 10.244.0.1:48360
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.connection(28) [C10] closing socket: 0
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.pool(28) cancelling pending stream
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.connection(28) [C11] closing data_to_write=0 type=1
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.connection(28) [C11] closing socket: 1
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.pool(28) [C11] client disconnected, failure reason:
2023-06-03T00:02:28.836Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:28.837Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:28.837Z+00:00 [debug] envoy.conn_handler(28) [C10] adding to cleanup list
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.filter(29) [C12] new tcp proxy session
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.filter(29) [C12] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.misc(29) Allocating TCP conn pool
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.pool(29) trying to create new connection
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.pool(29) creating a new connection (connecting=0)
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.connection(29) [C13] connecting to 10.244.0.9:20000
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.connection(29) [C13] connection in progress
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.conn_handler(29) [C12] new connection from 127.0.0.1:49062
2023-06-03T00:02:28.918Z+00:00 [debug] envoy.connection(29) [C13] connected
2023-06-03T00:02:28.919Z+00:00 [debug] envoy.connection(29) [C13] Async cert validation completed
2023-06-03T00:02:28.920Z+00:00 [debug] envoy.pool(29) [C13] attaching to next stream
2023-06-03T00:02:28.920Z+00:00 [debug] envoy.pool(29) [C13] creating stream
2023-06-03T00:02:28.920Z+00:00 [debug] envoy.filter(29) [C12] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:28.922Z+00:00 [debug] envoy.connection(29) [C13] SSL shutdown: rc=0
2023-06-03T00:02:28.922Z+00:00 [debug] envoy.connection(29) [C13] remote close
2023-06-03T00:02:28.922Z+00:00 [debug] envoy.connection(29) [C13] closing socket: 0
2023-06-03T00:02:28.922Z+00:00 [debug] envoy.pool(29) [C13] client disconnected, failure reason:
2023-06-03T00:02:28.923Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:28.923Z+00:00 [debug] envoy.pool(29) [C13] destroying stream: 0 remaining
2023-06-03T00:02:28.923Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:28.923Z+00:00 [debug] envoy.connection(29) [C12] closing data_to_write=0 type=0
2023-06-03T00:02:28.923Z+00:00 [debug] envoy.connection(29) [C12] closing socket: 1
2023-06-03T00:02:28.923Z+00:00 [debug] envoy.conn_handler(29) [C12] adding to cleanup list
2023-06-03T00:02:31.101Z+00:00 [debug] envoy.filter(29) [C14] new tcp proxy session
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.filter(29) [C14] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.misc(29) Allocating TCP conn pool
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.pool(29) trying to create new connection
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.pool(29) creating a new connection (connecting=0)
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.connection(29) [C15] connecting to 10.244.0.9:20000
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.connection(29) [C15] connection in progress
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.conn_handler(29) [C14] new connection from 127.0.0.1:49344
2023-06-03T00:02:31.102Z+00:00 [debug] envoy.connection(29) [C15] connected
2023-06-03T00:02:31.103Z+00:00 [debug] envoy.connection(29) [C15] Async cert validation completed
2023-06-03T00:02:31.103Z+00:00 [debug] envoy.pool(29) [C15] attaching to next stream
2023-06-03T00:02:31.103Z+00:00 [debug] envoy.pool(29) [C15] creating stream
2023-06-03T00:02:31.103Z+00:00 [debug] envoy.filter(29) [C14] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:31.105Z+00:00 [debug] envoy.connection(29) [C15] SSL shutdown: rc=0
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.connection(29) [C15] remote close
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.connection(29) [C15] closing socket: 0
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.pool(29) [C15] client disconnected, failure reason:
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.pool(29) [C15] destroying stream: 0 remaining
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.connection(29) [C14] closing data_to_write=0 type=0
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.connection(29) [C14] closing socket: 1
2023-06-03T00:02:31.106Z+00:00 [debug] envoy.conn_handler(29) [C14] adding to cleanup list
2023-06-03T00:02:33.124Z+00:00 [debug] envoy.main(18) flushing stats
2023-06-03T00:02:33.268Z+00:00 [debug] envoy.filter(29) [C16] new tcp proxy session
2023-06-03T00:02:33.268Z+00:00 [debug] envoy.filter(29) [C16] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:33.268Z+00:00 [debug] envoy.misc(29) Allocating TCP conn pool
2023-06-03T00:02:33.268Z+00:00 [debug] envoy.pool(29) trying to create new connection
2023-06-03T00:02:33.268Z+00:00 [debug] envoy.pool(29) creating a new connection (connecting=0)
2023-06-03T00:02:33.268Z+00:00 [debug] envoy.connection(29) [C17] connecting to 10.244.0.9:20000
2023-06-03T00:02:33.269Z+00:00 [debug] envoy.connection(29) [C17] connection in progress
2023-06-03T00:02:33.269Z+00:00 [debug] envoy.conn_handler(29) [C16] new connection from 127.0.0.1:49360
2023-06-03T00:02:33.269Z+00:00 [debug] envoy.connection(29) [C17] connected
2023-06-03T00:02:33.270Z+00:00 [debug] envoy.connection(29) [C17] Async cert validation completed
2023-06-03T00:02:33.270Z+00:00 [debug] envoy.pool(29) [C17] attaching to next stream
2023-06-03T00:02:33.270Z+00:00 [debug] envoy.pool(29) [C17] creating stream
2023-06-03T00:02:33.270Z+00:00 [debug] envoy.filter(29) [C16] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.connection(29) [C17] SSL shutdown: rc=0
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.connection(29) [C17] remote close
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.connection(29) [C17] closing socket: 0
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.pool(29) [C17] client disconnected, failure reason:
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.pool(29) [C17] destroying stream: 0 remaining
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.connection(29) [C16] closing data_to_write=0 type=0
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.connection(29) [C16] closing socket: 1
2023-06-03T00:02:33.271Z+00:00 [debug] envoy.conn_handler(29) [C16] adding to cleanup list
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.filter(28) [C18] new tcp proxy session
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.filter(28) [C18] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.connection(28) [C19] connecting to 10.244.0.9:20000
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.connection(28) [C19] connection in progress
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.conn_handler(28) [C18] new connection from 127.0.0.1:49376
2023-06-03T00:02:35.449Z+00:00 [debug] envoy.connection(28) [C19] connected
2023-06-03T00:02:35.450Z+00:00 [debug] envoy.connection(28) [C19] Async cert validation completed
2023-06-03T00:02:35.451Z+00:00 [debug] envoy.pool(28) [C19] attaching to next stream
2023-06-03T00:02:35.451Z+00:00 [debug] envoy.pool(28) [C19] creating stream
2023-06-03T00:02:35.451Z+00:00 [debug] envoy.filter(28) [C18] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:35.452Z+00:00 [debug] envoy.connection(28) [C19] SSL shutdown: rc=0
2023-06-03T00:02:35.452Z+00:00 [debug] envoy.connection(28) [C19] remote close
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.connection(28) [C19] closing socket: 0
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.pool(28) [C19] client disconnected, failure reason:
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.pool(28) [C19] destroying stream: 0 remaining
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.connection(28) [C18] closing data_to_write=0 type=0
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.connection(28) [C18] closing socket: 1
2023-06-03T00:02:35.453Z+00:00 [debug] envoy.conn_handler(28) [C18] adding to cleanup list
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.filter(28) [C20] new tcp proxy session
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.filter(28) [C20] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.connection(28) [C21] connecting to 10.244.0.9:20000
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.connection(28) [C21] connection in progress
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.conn_handler(28) [C20] new connection from 127.0.0.1:49390
2023-06-03T00:02:37.601Z+00:00 [debug] envoy.connection(28) [C21] connected
2023-06-03T00:02:37.602Z+00:00 [debug] envoy.connection(28) [C21] Async cert validation completed
2023-06-03T00:02:37.602Z+00:00 [debug] envoy.pool(28) [C21] attaching to next stream
2023-06-03T00:02:37.602Z+00:00 [debug] envoy.pool(28) [C21] creating stream
2023-06-03T00:02:37.602Z+00:00 [debug] envoy.filter(28) [C20] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:37.604Z+00:00 [debug] envoy.connection(28) [C21] SSL shutdown: rc=0
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.connection(28) [C21] remote close
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.connection(28) [C21] closing socket: 0
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.pool(28) [C21] client disconnected, failure reason:
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.pool(28) [C21] destroying stream: 0 remaining
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.connection(28) [C20] closing data_to_write=0 type=0
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.connection(28) [C20] closing socket: 1
2023-06-03T00:02:37.605Z+00:00 [debug] envoy.conn_handler(28) [C20] adding to cleanup list
2023-06-03T00:02:38.124Z+00:00 [debug] envoy.main(18) flushing stats
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.filter(28) [C22] new tcp proxy session
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.filter(28) [C22] Creating connection to cluster local_app
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.connection(28) [C23] connecting to 127.0.0.1:0
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.connection(28) [C23] connection in progress
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.conn_handler(28) [C22] new connection from 10.244.0.1:44662
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.connection(28) [C22] closing socket: 0
2023-06-03T00:02:38.836Z+00:00 [debug] envoy.pool(28) cancelling pending stream
2023-06-03T00:02:38.837Z+00:00 [debug] envoy.connection(28) [C23] closing data_to_write=0 type=1
2023-06-03T00:02:38.837Z+00:00 [debug] envoy.connection(28) [C23] closing socket: 1
2023-06-03T00:02:38.837Z+00:00 [debug] envoy.pool(28) [C23] client disconnected, failure reason:
2023-06-03T00:02:38.837Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:38.837Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:38.837Z+00:00 [debug] envoy.conn_handler(28) [C22] adding to cleanup list
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.filter(28) [C24] new tcp proxy session
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.filter(28) [C24] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.connection(28) [C25] connecting to 10.244.0.9:20000
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.connection(28) [C25] connection in progress
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.conn_handler(28) [C24] new connection from 127.0.0.1:49398
2023-06-03T00:02:39.759Z+00:00 [debug] envoy.connection(28) [C25] connected
2023-06-03T00:02:39.760Z+00:00 [debug] envoy.connection(28) [C25] Async cert validation completed
2023-06-03T00:02:39.760Z+00:00 [debug] envoy.pool(28) [C25] attaching to next stream
2023-06-03T00:02:39.760Z+00:00 [debug] envoy.pool(28) [C25] creating stream
2023-06-03T00:02:39.760Z+00:00 [debug] envoy.filter(28) [C24] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:39.761Z+00:00 [debug] envoy.connection(28) [C25] SSL shutdown: rc=0
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.connection(28) [C25] remote close
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.connection(28) [C25] closing socket: 0
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.pool(28) [C25] client disconnected, failure reason:
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.pool(28) [C25] destroying stream: 0 remaining
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.connection(28) [C24] closing data_to_write=0 type=0
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.connection(28) [C24] closing socket: 1
2023-06-03T00:02:39.762Z+00:00 [debug] envoy.conn_handler(28) [C24] adding to cleanup list
2023-06-03T00:02:41.908Z+00:00 [debug] envoy.filter(29) [C26] new tcp proxy session
2023-06-03T00:02:41.909Z+00:00 [debug] envoy.filter(29) [C26] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:41.909Z+00:00 [debug] envoy.misc(29) Allocating TCP conn pool
2023-06-03T00:02:41.909Z+00:00 [debug] envoy.pool(29) trying to create new connection
2023-06-03T00:02:41.909Z+00:00 [debug] envoy.pool(29) creating a new connection (connecting=0)
2023-06-03T00:02:41.909Z+00:00 [debug] envoy.connection(29) [C27] connecting to 10.244.0.9:20000
2023-06-03T00:02:41.909Z+00:00 [debug] envoy.connection(29) [C27] connection in progress
2023-06-03T00:02:41.909Z+00:00 [debug] envoy.conn_handler(29) [C26] new connection from 127.0.0.1:48292
2023-06-03T00:02:41.910Z+00:00 [debug] envoy.connection(29) [C27] connected
2023-06-03T00:02:41.911Z+00:00 [debug] envoy.connection(29) [C27] Async cert validation completed
2023-06-03T00:02:41.911Z+00:00 [debug] envoy.pool(29) [C27] attaching to next stream
2023-06-03T00:02:41.911Z+00:00 [debug] envoy.pool(29) [C27] creating stream
2023-06-03T00:02:41.911Z+00:00 [debug] envoy.filter(29) [C26] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:41.913Z+00:00 [debug] envoy.connection(29) [C27] SSL shutdown: rc=0
2023-06-03T00:02:41.914Z+00:00 [debug] envoy.connection(29) [C27] remote close
2023-06-03T00:02:41.914Z+00:00 [debug] envoy.connection(29) [C27] closing socket: 0
2023-06-03T00:02:41.914Z+00:00 [debug] envoy.pool(29) [C27] client disconnected, failure reason:
2023-06-03T00:02:41.915Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:41.915Z+00:00 [debug] envoy.pool(29) [C27] destroying stream: 0 remaining
2023-06-03T00:02:41.915Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:41.915Z+00:00 [debug] envoy.connection(29) [C26] closing data_to_write=0 type=0
2023-06-03T00:02:41.915Z+00:00 [debug] envoy.connection(29) [C26] closing socket: 1
2023-06-03T00:02:41.915Z+00:00 [debug] envoy.conn_handler(29) [C26] adding to cleanup list
2023-06-03T00:02:43.124Z+00:00 [debug] envoy.main(18) flushing stats
2023-06-03T00:02:44.053Z+00:00 [debug] envoy.filter(28) [C28] new tcp proxy session
2023-06-03T00:02:44.053Z+00:00 [debug] envoy.filter(28) [C28] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:44.053Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:44.053Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:44.053Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:44.053Z+00:00 [debug] envoy.connection(28) [C29] connecting to 10.244.0.9:20000
2023-06-03T00:02:44.054Z+00:00 [debug] envoy.connection(28) [C29] connection in progress
2023-06-03T00:02:44.054Z+00:00 [debug] envoy.conn_handler(28) [C28] new connection from 127.0.0.1:48294
2023-06-03T00:02:44.054Z+00:00 [debug] envoy.connection(28) [C29] connected
2023-06-03T00:02:44.054Z+00:00 [debug] envoy.connection(28) [C29] Async cert validation completed
2023-06-03T00:02:44.055Z+00:00 [debug] envoy.pool(28) [C29] attaching to next stream
2023-06-03T00:02:44.055Z+00:00 [debug] envoy.pool(28) [C29] creating stream
2023-06-03T00:02:44.055Z+00:00 [debug] envoy.filter(28) [C28] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:44.056Z+00:00 [debug] envoy.connection(28) [C29] SSL shutdown: rc=0
2023-06-03T00:02:44.056Z+00:00 [debug] envoy.connection(28) [C29] remote close
2023-06-03T00:02:44.056Z+00:00 [debug] envoy.connection(28) [C29] closing socket: 0
2023-06-03T00:02:44.056Z+00:00 [debug] envoy.pool(28) [C29] client disconnected, failure reason:
2023-06-03T00:02:44.056Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:44.056Z+00:00 [debug] envoy.pool(28) [C29] destroying stream: 0 remaining
2023-06-03T00:02:44.057Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:44.057Z+00:00 [debug] envoy.connection(28) [C28] closing data_to_write=0 type=0
2023-06-03T00:02:44.057Z+00:00 [debug] envoy.connection(28) [C28] closing socket: 1
2023-06-03T00:02:44.057Z+00:00 [debug] envoy.conn_handler(28) [C28] adding to cleanup list
2023-06-03T00:02:46.215Z+00:00 [debug] envoy.filter(29) [C30] new tcp proxy session
2023-06-03T00:02:46.215Z+00:00 [debug] envoy.filter(29) [C30] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:46.215Z+00:00 [debug] envoy.misc(29) Allocating TCP conn pool
2023-06-03T00:02:46.215Z+00:00 [debug] envoy.pool(29) trying to create new connection
2023-06-03T00:02:46.215Z+00:00 [debug] envoy.pool(29) creating a new connection (connecting=0)
2023-06-03T00:02:46.215Z+00:00 [debug] envoy.connection(29) [C31] connecting to 10.244.0.9:20000
2023-06-03T00:02:46.216Z+00:00 [debug] envoy.connection(29) [C31] connection in progress
2023-06-03T00:02:46.216Z+00:00 [debug] envoy.conn_handler(29) [C30] new connection from 127.0.0.1:48300
2023-06-03T00:02:46.217Z+00:00 [debug] envoy.connection(29) [C31] connected
2023-06-03T00:02:46.218Z+00:00 [debug] envoy.connection(29) [C31] Async cert validation completed
2023-06-03T00:02:46.218Z+00:00 [debug] envoy.pool(29) [C31] attaching to next stream
2023-06-03T00:02:46.218Z+00:00 [debug] envoy.pool(29) [C31] creating stream
2023-06-03T00:02:46.218Z+00:00 [debug] envoy.filter(29) [C30] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:46.219Z+00:00 [debug] envoy.connection(29) [C31] SSL shutdown: rc=0
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.connection(29) [C31] remote close
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.connection(29) [C31] closing socket: 0
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.pool(29) [C31] client disconnected, failure reason:
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.pool(29) [C31] destroying stream: 0 remaining
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.connection(29) [C30] closing data_to_write=0 type=0
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.connection(29) [C30] closing socket: 1
2023-06-03T00:02:46.220Z+00:00 [debug] envoy.conn_handler(29) [C30] adding to cleanup list
2023-06-03T00:02:48.125Z+00:00 [debug] envoy.main(18) flushing stats
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.filter(28) [C32] new tcp proxy session
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.filter(28) [C32] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.connection(28) [C33] connecting to 10.244.0.9:20000
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.connection(28) [C33] connection in progress
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.conn_handler(28) [C32] new connection from 127.0.0.1:48316
2023-06-03T00:02:48.401Z+00:00 [debug] envoy.connection(28) [C33] connected
2023-06-03T00:02:48.404Z+00:00 [debug] envoy.connection(28) [C33] Async cert validation completed
2023-06-03T00:02:48.404Z+00:00 [debug] envoy.pool(28) [C33] attaching to next stream
2023-06-03T00:02:48.404Z+00:00 [debug] envoy.pool(28) [C33] creating stream
2023-06-03T00:02:48.404Z+00:00 [debug] envoy.filter(28) [C32] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:48.407Z+00:00 [debug] envoy.connection(28) [C33] SSL shutdown: rc=0
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.connection(28) [C33] remote close
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.connection(28) [C33] closing socket: 0
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.pool(28) [C33] client disconnected, failure reason:
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.pool(28) [C33] destroying stream: 0 remaining
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.connection(28) [C32] closing data_to_write=0 type=0
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.connection(28) [C32] closing socket: 1
2023-06-03T00:02:48.408Z+00:00 [debug] envoy.conn_handler(28) [C32] adding to cleanup list
2023-06-03T00:02:48.836Z+00:00 [debug] envoy.filter(28) [C34] new tcp proxy session
2023-06-03T00:02:48.836Z+00:00 [debug] envoy.filter(28) [C34] Creating connection to cluster local_app
2023-06-03T00:02:48.836Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:48.836Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:48.836Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:48.837Z+00:00 [debug] envoy.connection(28) [C35] connecting to 127.0.0.1:0
2023-06-03T00:02:48.838Z+00:00 [debug] envoy.connection(28) [C35] connection in progress
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.conn_handler(28) [C34] new connection from 10.244.0.1:56350
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.connection(28) [C34] closing socket: 0
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.pool(28) cancelling pending stream
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.connection(28) [C35] closing data_to_write=0 type=1
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.connection(28) [C35] closing socket: 1
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.pool(28) [C35] client disconnected, failure reason:
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:48.839Z+00:00 [debug] envoy.conn_handler(28) [C34] adding to cleanup list
2023-06-03T00:02:50.558Z+00:00 [debug] envoy.filter(28) [C36] new tcp proxy session
2023-06-03T00:02:50.558Z+00:00 [debug] envoy.filter(28) [C36] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:50.558Z+00:00 [debug] envoy.misc(28) Allocating TCP conn pool
2023-06-03T00:02:50.558Z+00:00 [debug] envoy.pool(28) trying to create new connection
2023-06-03T00:02:50.558Z+00:00 [debug] envoy.pool(28) creating a new connection (connecting=0)
2023-06-03T00:02:50.559Z+00:00 [debug] envoy.connection(28) [C37] connecting to 10.244.0.9:20000
2023-06-03T00:02:50.559Z+00:00 [debug] envoy.connection(28) [C37] connection in progress
2023-06-03T00:02:50.559Z+00:00 [debug] envoy.conn_handler(28) [C36] new connection from 127.0.0.1:48322
2023-06-03T00:02:50.559Z+00:00 [debug] envoy.connection(28) [C37] connected
2023-06-03T00:02:50.560Z+00:00 [debug] envoy.connection(28) [C37] Async cert validation completed
2023-06-03T00:02:50.560Z+00:00 [debug] envoy.pool(28) [C37] attaching to next stream
2023-06-03T00:02:50.560Z+00:00 [debug] envoy.pool(28) [C37] creating stream
2023-06-03T00:02:50.560Z+00:00 [debug] envoy.filter(28) [C36] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:50.562Z+00:00 [debug] envoy.connection(28) [C37] SSL shutdown: rc=0
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.connection(28) [C37] remote close
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.connection(28) [C37] closing socket: 0
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.pool(28) [C37] client disconnected, failure reason:
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.pool(28) [C37] destroying stream: 0 remaining
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.pool(28) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.connection(28) [C36] closing data_to_write=0 type=0
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.connection(28) [C36] closing socket: 1
2023-06-03T00:02:50.563Z+00:00 [debug] envoy.conn_handler(28) [C36] adding to cleanup list
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.filter(29) [C38] new tcp proxy session
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.filter(29) [C38] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.misc(29) Allocating TCP conn pool
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.pool(29) trying to create new connection
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.pool(29) creating a new connection (connecting=0)
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.connection(29) [C39] connecting to 10.244.0.9:20000
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.connection(29) [C39] connection in progress
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.conn_handler(29) [C38] new connection from 127.0.0.1:39956
2023-06-03T00:02:52.709Z+00:00 [debug] envoy.connection(29) [C39] connected
2023-06-03T00:02:52.710Z+00:00 [debug] envoy.connection(29) [C39] Async cert validation completed
2023-06-03T00:02:52.710Z+00:00 [debug] envoy.pool(29) [C39] attaching to next stream
2023-06-03T00:02:52.710Z+00:00 [debug] envoy.pool(29) [C39] creating stream
2023-06-03T00:02:52.710Z+00:00 [debug] envoy.filter(29) [C38] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:52.712Z+00:00 [debug] envoy.connection(29) [C39] SSL shutdown: rc=0
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.connection(29) [C39] remote close
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.connection(29) [C39] closing socket: 0
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.pool(29) [C39] client disconnected, failure reason:
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.pool(29) [C39] destroying stream: 0 remaining
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.connection(29) [C38] closing data_to_write=0 type=0
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.connection(29) [C38] closing socket: 1
2023-06-03T00:02:52.714Z+00:00 [debug] envoy.conn_handler(29) [C38] adding to cleanup list
2023-06-03T00:02:53.127Z+00:00 [debug] envoy.main(18) flushing stats
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.filter(29) [C40] new tcp proxy session
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.filter(29) [C40] Creating connection to cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.misc(29) Allocating TCP conn pool
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.pool(29) trying to create new connection
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.pool(29) creating a new connection (connecting=0)
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.connection(29) [C41] connecting to 10.244.0.9:20000
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.connection(29) [C41] connection in progress
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.conn_handler(29) [C40] new connection from 127.0.0.1:39964
2023-06-03T00:02:54.842Z+00:00 [debug] envoy.connection(29) [C41] connected
2023-06-03T00:02:54.843Z+00:00 [debug] envoy.connection(29) [C41] Async cert validation completed
2023-06-03T00:02:54.843Z+00:00 [debug] envoy.pool(29) [C41] attaching to next stream
2023-06-03T00:02:54.843Z+00:00 [debug] envoy.pool(29) [C41] creating stream
2023-06-03T00:02:54.843Z+00:00 [debug] envoy.filter(29) [C40] TCP:onUpstreamEvent(), requestedServerName:
2023-06-03T00:02:54.845Z+00:00 [debug] envoy.connection(29) [C41] SSL shutdown: rc=0
2023-06-03T00:02:54.845Z+00:00 [debug] envoy.connection(29) [C41] remote close
2023-06-03T00:02:54.845Z+00:00 [debug] envoy.connection(29) [C41] closing socket: 0
2023-06-03T00:02:54.846Z+00:00 [debug] envoy.pool(29) [C41] client disconnected, failure reason:
2023-06-03T00:02:54.846Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:54.846Z+00:00 [debug] envoy.pool(29) [C41] destroying stream: 0 remaining
2023-06-03T00:02:54.846Z+00:00 [debug] envoy.pool(29) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:54.846Z+00:00 [debug] envoy.connection(29) [C40] closing data_to_write=0 type=0
2023-06-03T00:02:54.846Z+00:00 [debug] envoy.connection(29) [C40] closing socket: 1
2023-06-03T00:02:54.846Z+00:00 [debug] envoy.conn_handler(29) [C40] adding to cleanup list
2023-06-03T00:02:55.122Z+00:00 [debug] envoy.conn_handler(18) [C42] new connection from 127.0.0.1:51380
2023-06-03T00:02:55.123Z+00:00 [debug] envoy.http(18) [C42] new stream
2023-06-03T00:02:55.123Z+00:00 [debug] envoy.http(18) [C42][S4544074838802117489] request headers complete (end_stream=true):
':authority', '127.0.0.1:50840'
':path', '/config_dump?format=json'
':method', 'GET'
'user-agent', 'Go-http-client/1.1'
'accept-encoding', 'gzip'

2023-06-03T00:02:55.123Z+00:00 [debug] envoy.http(18) [C42][S4544074838802117489] request end stream
2023-06-03T00:02:55.123Z+00:00 [debug] envoy.connection(18) [C42] current connecting state: false
2023-06-03T00:02:55.123Z+00:00 [debug] envoy.admin(18) [C42][S4544074838802117489] request complete: path: /config_dump?format=json
2023-06-03T00:02:55.129Z+00:00 [debug] envoy.http(18) [C42][S4544074838802117489] drain closing connection
2023-06-03T00:02:55.129Z+00:00 [debug] envoy.http(18) [C42][S4544074838802117489] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'application/json'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Sat, 03 Jun 2023 00:02:55 GMT'
'server', 'envoy'
'connection', 'close'

2023-06-03T00:02:55.129Z+00:00 [debug] envoy.misc(18) nextChunk: response.length=73582 more_data=false end_stream=true
2023-06-03T00:02:55.129Z+00:00 [debug] envoy.http(18) [C42][S4544074838802117489] Codec completed encoding stream.
2023-06-03T00:02:55.140Z+00:00 [debug] envoy.conn_handler(18) [C43] new connection from 127.0.0.1:51396
2023-06-03T00:02:55.143Z+00:00 [debug] envoy.http(18) [C43] new stream
2023-06-03T00:02:55.143Z+00:00 [debug] envoy.http(18) [C43][S3845022082479536213] request headers complete (end_stream=true):
':authority', '127.0.0.1:50840'
':path', '/clusters?format=json'
':method', 'GET'
'user-agent', 'Go-http-client/1.1'
'accept-encoding', 'gzip'

2023-06-03T00:02:55.143Z+00:00 [debug] envoy.http(18) [C43][S3845022082479536213] request end stream
2023-06-03T00:02:55.143Z+00:00 [debug] envoy.connection(18) [C43] current connecting state: false
2023-06-03T00:02:55.143Z+00:00 [debug] envoy.admin(18) [C43][S3845022082479536213] request complete: path: /clusters?format=json
2023-06-03T00:02:55.143Z+00:00 [debug] envoy.http(18) [C43][S3845022082479536213] drain closing connection
2023-06-03T00:02:55.144Z+00:00 [debug] envoy.http(18) [C43][S3845022082479536213] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'application/json'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Sat, 03 Jun 2023 00:02:55 GMT'
'server', 'envoy'
'connection', 'close'

2023-06-03T00:02:55.144Z+00:00 [debug] envoy.misc(18) nextChunk: response.length=3712 more_data=false end_stream=true
2023-06-03T00:02:55.144Z+00:00 [debug] envoy.http(18) [C43][S3845022082479536213] Codec completed encoding stream.
2023-06-03T00:02:55.223Z+00:00 [debug] envoy.connection(18) [C42] closing data_to_write=0 type=2
2023-06-03T00:02:55.223Z+00:00 [debug] envoy.connection(18) [C42] closing socket: 1
2023-06-03T00:02:55.223Z+00:00 [debug] envoy.conn_handler(18) [C42] adding to cleanup list
2023-06-03T00:02:55.244Z+00:00 [debug] envoy.connection(18) [C43] closing data_to_write=0 type=2
2023-06-03T00:02:55.244Z+00:00 [debug] envoy.connection(18) [C43] closing socket: 1
2023-06-03T00:02:55.244Z+00:00 [debug] envoy.conn_handler(18) [C43] adding to cleanup list
2023-06-03T00:02:56.121Z+00:00 [debug] envoy.conn_handler(18) [C44] new connection from 127.0.0.1:51410
2023-06-03T00:02:56.150Z [INFO]  consul-dataplane.lifecycle: shutdown grace period timeout reached
2023-06-03T00:02:56.150Z [DEBUG] consul-dataplane: stopping proxy connection draining, starting graceful shutdown of Envoy proxy
2023-06-03T00:02:56.151Z+00:00 [debug] envoy.conn_handler(18) [C45] new connection from 127.0.0.1:51416
2023-06-03T00:02:56.151Z+00:00 [debug] envoy.http(18) [C45] new stream
2023-06-03T00:02:56.151Z+00:00 [debug] envoy.http(18) [C45][S13448045597211269508] request headers complete (end_stream=true):
':authority', '127.0.0.1:19000'
':path', '/quitquitquit'
':method', 'POST'
'user-agent', 'Go-http-client/1.1'
'content-length', '0'
'content-type', 'text/plain'
'accept-encoding', 'gzip'

2023-06-03T00:02:56.151Z+00:00 [debug] envoy.http(18) [C45][S13448045597211269508] request end stream
2023-06-03T00:02:56.151Z+00:00 [debug] envoy.connection(18) [C45] current connecting state: false
2023-06-03T00:02:56.151Z+00:00 [debug] envoy.admin(18) [C45][S13448045597211269508] request complete: path: /quitquitquit
2023-06-03T00:02:56.151Z+00:00 [info] envoy.main(18) shutting down server instance
2023-06-03T00:02:56.151Z+00:00 [debug] envoy.http(18) [C45][S13448045597211269508] drain closing connection
2023-06-03T00:02:56.152Z+00:00 [debug] envoy.http(18) [C45][S13448045597211269508] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'text/plain; charset=UTF-8'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Sat, 03 Jun 2023 00:02:56 GMT'
'server', 'envoy'
'connection', 'close'

2023-06-03T00:02:56.152Z+00:00 [debug] envoy.misc(18) nextChunk: response.length=3 more_data=false end_stream=true
2023-06-03T00:02:56.152Z+00:00 [debug] envoy.http(18) [C45][S13448045597211269508] Codec completed encoding stream.
2023-06-03T00:02:56.152Z+00:00 [info] envoy.main(18) main dispatch loop exited
2023-06-03T00:02:56.152Z+00:00 [debug] envoy.main(28) worker exited dispatch loop
2023-06-03T00:02:56.152Z+00:00 [debug] envoy.grpc(28) Joining completionThread
2023-06-03T00:02:56.152Z+00:00 [debug] envoy.grpc(30) completionThread exiting
2023-06-03T00:02:56.152Z+00:00 [debug] envoy.grpc(28) Joined completionThread
2023-06-03T00:02:56.153Z+00:00 [debug] envoy.upstream(28) shutting down thread local cluster manager
2023-06-03T00:02:56.153Z+00:00 [debug] envoy.main(29) worker exited dispatch loop
2023-06-03T00:02:56.153Z+00:00 [debug] envoy.grpc(29) Joining completionThread
2023-06-03T00:02:56.154Z+00:00 [debug] envoy.grpc(31) completionThread exiting
2023-06-03T00:02:56.154Z+00:00 [debug] envoy.grpc(29) Joined completionThread
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.upstream(29) shutting down thread local cluster manager
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.main(18) flushing stats
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.init(18) ClusterImplBase destroyed
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.init(18) init manager Cluster consul-dataplane destroyed
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.init(18) ClusterImplBase destroyed
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.init(18) init manager Cluster static-server.default.dc1.internal.919c5dd1-b830-1543-3e8e-c2c510a33275.consul destroyed
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.init(18) ClusterImplBase destroyed
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.init(18) init manager Cluster local_app destroyed
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.connection(18) [C45] closing data_to_write=241 type=1
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.connection(18) [C45] closing socket: 1
2023-06-03T00:02:56.155Z+00:00 [debug] envoy.conn_handler(18) [C45] adding to cleanup list
2023-06-03T00:02:56.156Z+00:00 [debug] envoy.connection(18) [C44] closing data_to_write=0 type=1
2023-06-03T00:02:56.156Z+00:00 [debug] envoy.connection(18) [C44] closing socket: 1
2023-06-03T00:02:56.156Z+00:00 [debug] envoy.conn_handler(18) [C44] adding to cleanup list
2023-06-03T00:02:56.157Z+00:00 [debug] envoy.grpc(18) Joining completionThread
2023-06-03T00:02:56.157Z+00:00 [debug] envoy.grpc(22) completionThread exiting
2023-06-03T00:02:56.157Z+00:00 [debug] envoy.grpc(18) Joined completionThread
2023-06-03T00:02:56.157Z+00:00 [debug] envoy.upstream(18) shutting down thread local cluster manager
2023-06-03T00:02:56.157Z+00:00 [debug] envoy.connection(18) [C0] closing data_to_write=0 type=1
2023-06-03T00:02:56.157Z+00:00 [debug] envoy.connection(18) [C0] closing socket: 1
2023-06-03T00:02:56.158Z+00:00 [debug] envoy.client(18) [C0] disconnect. resetting 1 pending requests
2023-06-03T00:02:56.158Z+00:00 [debug] envoy.client(18) [C0] request reset
2023-06-03T00:02:56.158Z+00:00 [debug] envoy.pool(18) [C0] destroying stream: 0 remaining
2023-06-03T00:02:56.158Z+00:00 [debug] envoy.router(18) [C0][S7076816018391924453] upstream reset: reset reason: connection termination, transport failure reason:
2023-06-03T00:02:56.158Z+00:00 [warning] envoy.config(18) DeltaAggregatedResources gRPC config stream to consul-dataplane closed: 13,
2023-06-03T00:02:56.158Z+00:00 [debug] envoy.config(18) gRPC update for type.googleapis.com/envoy.config.listener.v3.Listener failed
2023-06-03T00:02:56.158Z+00:00 [debug] envoy.pool(18) [C0] client disconnected, failure reason:
2023-06-03T00:02:56.160Z+00:00 [debug] envoy.pool(18) invoking idle callbacks - is_draining_for_deletion_=false
2023-06-03T00:02:56.161Z+00:00 [info] envoy.main(18) exiting
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.init(18) RunHelper destroyed
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.main(18) destroying listener manager
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.init(18) target LDS destroyed
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.main(18) destroying dispatcher worker_1
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.main(18) destroying dispatcher worker_0
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.init(18) Listener-local-init-watcher public_listener:10.244.0.10:20000 destroyed
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.init(18) init manager Listener-local-init-manager public_listener:10.244.0.10:20000 15401925844802677188 destroyed
2023-06-03T00:02:56.161Z+00:00 [debug] envoy.init(18) target Listener-init-target public_listener:10.244.0.10:20000 destroyed
2023-06-03T00:02:56.162Z+00:00 [debug] envoy.init(18) Listener-local-init-watcher static-server:127.0.0.1:1234 destroyed
2023-06-03T00:02:56.162Z+00:00 [debug] envoy.init(18) init manager Listener-local-init-manager static-server:127.0.0.1:1234 7713589232072738267 destroyed
2023-06-03T00:02:56.162Z+00:00 [debug] envoy.init(18) target Listener-init-target static-server:127.0.0.1:1234 destroyed
2023-06-03T00:02:56.162Z+00:00 [debug] envoy.main(18) destroyed listener manager
2023-06-03T00:02:56.165Z+00:00 [debug] envoy.main(18) destroying dispatcher workers_guarddog_thread
2023-06-03T00:02:56.165Z+00:00 [debug] envoy.main(18) destroying dispatcher main_thread_guarddog_thread
2023-06-03T00:02:56.165Z+00:00 [debug] envoy.init(18) init manager RTDS destroyed
2023-06-03T00:02:56.165Z+00:00 [debug] envoy.init(18) RTDS destroyed
2023-06-03T00:02:56.165Z+00:00 [debug] envoy.main(18) destroying access logger /dev/null
2023-06-03T00:02:56.166Z+00:00 [debug] envoy.main(18) destroyed access loggers
2023-06-03T00:02:56.166Z+00:00 [debug] envoy.main(18) destroying dispatcher main_thread
2023-06-03T00:02:56.166Z+00:00 [debug] envoy.init(18) init manager Server destroyed
2023-06-03T00:02:56.173Z [INFO]  consul-dataplane: envoy process exited: error=<nil>
2023-06-03T00:02:56.173Z [INFO]  consul-dataplane.server-connection-manager: stopping
2023-06-03T00:02:56.173Z [DEBUG] consul-dataplane.server-connection-manager: backoff: retry after=673.154092ms
2023-06-03T00:02:56.173Z [DEBUG] consul-dataplane.server-connection-manager: aborting: error="context canceled"
2023/06/03 00:02:56 envoy proxy exited unexpectedly

logs from execing into pod and attempting to dial out while shutting down (before changing acceptance tests to expect no errors)

        command.go:100: 2023-06-02T20:05:06-04:00 Running command kubectl with args [--namespace default exec static-client-845bd94679-n55tp -c static-client -- curl -vvvsSf http://localhost:1234]
        command.go:179: 2023-06-02T20:05:06-04:00 *   Trying 127.0.0.1:1234...
        command.go:179: 2023-06-02T20:05:06-04:00 * Connected to localhost (127.0.0.1) port 1234 (#0)
        command.go:179: 2023-06-02T20:05:06-04:00 > GET / HTTP/1.1
        command.go:179: 2023-06-02T20:05:06-04:00 > Host: localhost:1234
        command.go:179: 2023-06-02T20:05:06-04:00 > User-Agent: curl/7.81.0
        command.go:179: 2023-06-02T20:05:06-04:00 > Accept: */*
        command.go:179: 2023-06-02T20:05:06-04:00 >
        command.go:179: 2023-06-02T20:05:06-04:00 "hello world"
        command.go:179: 2023-06-02T20:05:06-04:00 * Mark bundle as not supporting multiuse
        command.go:179: 2023-06-02T20:05:06-04:00 < HTTP/1.1 200 OK
        command.go:179: 2023-06-02T20:05:06-04:00 < X-App-Name: http-echo
        command.go:179: 2023-06-02T20:05:06-04:00 < X-App-Version: 0.2.4
        command.go:179: 2023-06-02T20:05:06-04:00 < Date: Sat, 03 Jun 2023 00:05:06 GMT
        command.go:179: 2023-06-02T20:05:06-04:00 < Content-Length: 14
        command.go:179: 2023-06-02T20:05:06-04:00 < Content-Type: text/plain; charset=utf-8
        command.go:179: 2023-06-02T20:05:06-04:00 <
        command.go:179: 2023-06-02T20:05:06-04:00 { [14 bytes data]
        command.go:179: 2023-06-02T20:05:06-04:00 * Connection #0 to host localhost left intact
        connect_proxy_lifecycle_test.go:119: assertions.go:262:
                Error Trace:    /Users/mikemorris/Projects/consul-k8s/acceptance/tests/connect/connect_proxy_lifecycle_test.go:121
                                                        /Users/mikemorris/go/pkg/mod/github.com/hashicorp/consul/[email protected]/testutil/retry/retry.go:151
                                                        /Users/mikemorris/go/pkg/mod/github.com/hashicorp/consul/[email protected]/testutil/retry/retry.go:152
                                                        /Users/mikemorris/go/pkg/mod/github.com/hashicorp/consul/[email protected]/testutil/retry/retry.go:111
                                                        /Users/mikemorris/Projects/consul-k8s/acceptance/tests/connect/connect_proxy_lifecycle_test.go:119
                Error:          An error is expected but got nil.

How I expect reviewers to test this PR:

Audit that the Helm and annotation support is correct, has a sensible implementation and UX, and that configuration setting has sufficient test coverage.
Confirm the fixed expected behavior as exercised by the updated acceptance test is now correct.

Checklist:

Tests added
CHANGELOG entry added

HashiCorp engineers only, community PRs should not add a changelog entry.
Entries should use present tense (e.g. Add support for...)

Overview of commits

e152660

hashicorp-cla · 2023-06-27T17:55:32Z

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

temp seems not to be a GitHub user.
You need a GitHub account to be able to sign the CLA. If you already have a GitHub account, please add the email address used for this commit to your account.

_{Have you signed the CLA already but the status is still pending? Recheck it.}

curtbushko · 2023-06-27T18:01:39Z

Will do manual backport.

no-op commit due to failed cherry-picking

1fa6b4d

hc-github-team-consul-core force-pushed the backport/acceptance/proxy-lifecycle-shutdown-fixed/hopelessly-next-kodiak branch from d488609 to 7b70bb5 Compare June 27, 2023 17:55

hc-github-team-consul-core assigned curtbushko Jun 27, 2023

hc-github-team-consul-core requested a review from curtbushko June 27, 2023 17:55

curtbushko closed this Jun 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport of helm: add configuration for proxy lifecycle management into release/1.0.x #2465

Backport of helm: add configuration for proxy lifecycle management into release/1.0.x #2465

hc-github-team-consul-core commented Jun 27, 2023

hashicorp-cla commented Jun 27, 2023

curtbushko commented Jun 27, 2023

Backport of helm: add configuration for proxy lifecycle management into release/1.0.x #2465

Backport of helm: add configuration for proxy lifecycle management into release/1.0.x #2465

Conversation

hc-github-team-consul-core commented Jun 27, 2023

Backport

Changes proposed in this PR:

How I've tested this PR:

Logs

How I expect reviewers to test this PR:

hashicorp-cla commented Jun 27, 2023

curtbushko commented Jun 27, 2023