Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize AWS RDS CPU Utilization values before metadata API call #39664

Merged
merged 3 commits into from
May 24, 2024

Conversation

gpop63
Copy link
Contributor

@gpop63 gpop63 commented May 22, 2024

Overview

We should normalize CPU Utilization values before making the metadata API call. Normalizing these metric values before, ensures that they are correctly scaled regardless of the API call outcome.

Bug noticed by @agithomas

When the API call fails the CPU Utilization value is left untransformed and metadata (db_instance.*) is missing as expected.

Normalization successful

"rds": {
        "write": {
          "iops": 32.992212962548756
        },
        "database_connections": 202.5,
        "disk_queue_depth": 3.8251273585859344,
        "latency": {
          "read": 0.000597923599665421,
          "write": 0.001494035012286908
        },
        "freeable_memory": {
          "bytes": 24291670016
        },
        "free_storage": {
          "bytes": 1936544964608
        },
        "read": {
          "iops": 6147.187137265246
        },
        "throughput": {
          "network_receive": 382272.82916163537,
          "network_transmit": 12146763.734239174,
          "read": 211789547.4894744,
          "write": 276755.216951128
        },
        "swap_usage": {
          "bytes": 4340457472
        },
        "db_instance": {
          "class": "db.r5.4xlarge",
          "engine_name": "oracle-ee",
          "arn": "arn:aws:rds:eu-west-1:x:db:prod-zonda-mea-master",
          "status": "available",
          "identifier": "prod-zonda-mea-master"
        },
        "cpu": {
          "total": {
            "pct": 0.141125
          }
        }
      },

Normalization failed

      "rds": {
        "latency": {
          "write": 0.00373913188801891,
          "read": 0.0033277979724311226
        },
        "swap_usage": {
          "bytes": 4340457472
        },
        "disk_queue_depth": 29.583544356294787,
        "freeable_memory": {
          "bytes": 24227618816
        },
        "database_connections": 205,
        "free_storage": {
          "bytes": 1936431253504
        },
        "read": {
          "iops": 8572.673917114498
        },
        "throughput": {
          "read": 407276872.41836303,
          "write": 2831619.3420464424,
          "network_receive": 422334.8679408032,
          "network_transmit": 29206122.435347874
        },
        "write": {
          "iops": 153.26444758516493
        },
        "cpu": {
          "total": {
            "pct": 15.412756538452992
          }
        }
      }

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 22, 2024
@gpop63 gpop63 requested a review from agithomas May 22, 2024 11:21
@gpop63 gpop63 self-assigned this May 22, 2024
Copy link
Contributor

mergify bot commented May 22, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @gpop63? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@agithomas agithomas requested a review from a team May 22, 2024 11:22
@agithomas agithomas added the Team:Service-Integrations Label for the Service Integrations team label May 22, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 22, 2024
@gpop63 gpop63 marked this pull request as ready for review May 22, 2024 16:21
@gpop63 gpop63 requested a review from a team as a code owner May 22, 2024 16:21
Copy link
Contributor

@agithomas agithomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@agithomas
Copy link
Contributor

@elastic/obs-ds-hosted-services can you please look into the PR when you get some time?

Copy link
Contributor

@zmoog zmoog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch @agithomas! LGTM.

@gpop63 gpop63 merged commit 2d687ac into elastic:main May 24, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Service-Integrations Label for the Service Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants