Backport of Datadog Integration into release/1.2.x

[hc-github-team-consul-core]

Backport

This PR is auto-generated from #3407 to be assessed for backporting due to the inclusion of the label backport/1.2.x.

🚨

Warning automatic cherry-pick of commits failed. If the first commit failed,
you will see a blank no-op commit below. If at least one commit succeeded, you
will see the cherry-picked commits up to, not including, the commit where
the merge conflict occurred.

The person who merged in the original PR is:
@natemollica-nm
This person should manually cherry-pick the original PR into a new backport PR,
and close this one when the manual backport PR is merged in.

merge conflict error: POST https://api.github.com/repos/hashicorp/consul-k8s/merges: 409 Merge conflict []

The below text is copied from the body of the original PR.

---

Changes proposed in this PR

PR Resubmitted from Branch vice Fork for Automation Testing

• Expose several metrics specific features on consul-k8s to include:
  • agent enable_debug
  • agent telemetry.disable_hostname
  • agent telemetry.enable_host_metrics
  • agent telemetry.prefix_filter
  • agent telemetry.dogstatsd_addr
  • agent telemetry.dogstatsd_tags
• Introduce a means to ease the integration and operation of integrating with Datadog Agent metrics collection via fail-safe helm override value parameters. Overrides are intended to allow operators to easily configure 1 of the 3 following methods of monitoring Consul with Datadog on Kubernetes:
  • DogStatsD via one of either "UDP" or "UDS" transport protocols
  • OpenMetrics via Datadogs Autodiscovery feature to scrape the /v1/agent/metrics?format=prometheus endpoint
  • Datadog + Consul Integration Feature standard checks:
    • Serf events and member flaps
    • The Raft protocol
    • DNS performance
    • API Endpoints Health Checks:
      • /v1/agent/metrics?format=prometheus
      • /v1/agent/self
      • /v1/status/leader
      • /v1/status/peers
      • /v1/catalog/services
      • /v1/health/service
      • /v1/health/state/any
      • /v1/coordinate/datacenters
      • /v1/coordinate/nodes
• Introduces server-acl-init token creation for OpenMetrics and Datadog Consul Integration check methods allowing default minimal acl token permission generation for Datadog agent usage as necessary.

How I've tested this PR

• New ACL Token Testing as outline in the CONTRIBUTING.md steps.
• Deployment and testing of local consul-dev (main) and consul-k8s-control-plane-dev (datadog-integration branch) images on k3d test cluster for each scenario. Test repository here.
• Verification of helm templating for new value overrides added as instructed in CONTRIBUTING.md steps. bats ./charts/consul/test/unit --jobs 8 - ran successfully for all tests.

How I expect reviewers to test this PR

• Assess the need for additional unit testing creation and verification.
• If possible:
  • Reach out with any question/concerns or reasons for PR push-back.
  • Verification of fail-safe interlocks between the 3 methods of integration mentioned above.
  • Verification of ACL policy implementation.

Checklist

• Tests added
• CHANGELOG entry added

---

Overview of commits

• 8b6a9d8 - d27ea03 - 8a922e5 - e792599 - 3b27810 - f66f0f3 - 2dc237b - e4a234a - 6c7d488 - 8e7fd4c - f88bf52 - 96692a2 - ae4ac1a - d01568a - 307991b - a139223 - 2d3f39b - 5559f02 - 436884b - 0c4a8b4 - 85abcc8

[hashicorp-cla]

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

---

temp seems not to be a GitHub user.
You need a GitHub account to be able to sign the CLA. If you already have a GitHub account, please add the email address used for this commit to your account.

_{Have you signed the CLA already but the status is still pending? Recheck it.}

[natemollica-nm]

Closing in favor of #3626