Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport of Filter api-gateway cache logging to reduce log output on server disconnect into release/1.2.x #2893

Conversation

hc-github-team-consul-core

Backport

This PR is auto-generated from #2880 to be assessed for backporting due to the inclusion of the label backport/1.2.x.

The below text is copied from the body of the original PR.


Changes proposed in this PR:

  • I noticed this when I was working on a customer issue and it made debugging more difficult
  • I was working on an issue that was reproduced by deleting the consul-server-0 pod and the api-gateway logging spit out about 20k log lines in about 5 seconds
  • I don't think we need to log these errors out at the api-gateway level as other parts of the controller will log a similar error out but less frequently.
  • The customer will still see that the consul server is unreachable in the logs (see below)

Instead of 20k of these:

2023-08-31T14:33:54.861Z ERROR error fetching config entries for kind: http-route {"error": "Get "http://10.244.0.8:8500/v1/config/http-route?dc=dc1&index=10\": dial tcp 10.244.0.8:8500: connect: connection refused"}

They still see:

2023-08-31T14:41:43.031Z ERROR Reconciler error {"controller": "endpoints", "controllerGroup": "", "controllerKind": "Endpoints", "Endpoints": {"name":"consul-ui","namespace":"consul"}, "namespace": "consul", "name": "consul-ui", "reconcileID": "ce66b001-0173-40d1-95ab-a7a5363a6612", "error": "1 error occurred:\n\t* Get "http://:8500/v1/catalog/nodes?dc=dc1&filter=Meta%5B%22synthetic-node%22%5D+%3D%3D+%22true%22+&ns=%2A": dial tcp: lookup : no such host\n\n"}

How I've tested this PR:

  • build locally, delete consul-server-0 and look at the connect injector logs.

How I expect reviewers to test this PR:

👀

Checklist:


Overview of commits

@hc-github-team-consul-core hc-github-team-consul-core merged commit 653e7fd into release/1.2.x Sep 5, 2023
22 of 32 checks passed
@hc-github-team-consul-core hc-github-team-consul-core deleted the backport/reduce-gateway-noise/reliably-vocal-foal branch September 5, 2023 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants