CloudWatch Input Plugin Throttling should be configurable

[tmonk42]

Feature Request

#1197 added throttling for the CloudWatch Input Plugin. The requests per second is set to a hardcoded "10". This should be a config option to hand the case of a higher limit for a specific account or lower limit if there are other tools querying metrics.

Proposal:

Add new configuration option for inputs.cloudwatch "ratelimit" with a default of 10.
Update https://github.com/influxdata/telegraf/blob/master/plugins/inputs/cloudwatch/cloudwatch.go#L178 to use this config option

Current behavior:

Maximum requests per second is hardcoded at 10.

Desired behavior:

Maximum requests per second can be set to values higher and lower than 10

Use case: [Why is this important (helps with prioritizing requests)]

Some account can have this rate limit increased, and if there are a large number of metrics to be gathered, being able to collect at faster than 10/sec is valuable. Some account have multiple tools collecting CloudWatch metrics and might need to collect slower.

Edit: After a bit more digging, it looks like the rate limit is per namespace you define in telegraf.conf, while the AWS API rate limit is global. Ie: if you want to collect metrics from AWS/EC2, AWS/ELB, and AWS/EBS, each of those will be limited at 10 reqs/sec, which with all three running at the same time results in a maximum of 30 reqs/sec and the AWS API is limited to 10 reqs/sec resulting lots of API throttling and missed metrics. If you could configure the rate limit and you were querying metrics for multiple namespaces, you could do split up the 10req/sec among your Namespaces to keep the total under 10.

[sparrc]

closed by #1673