Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add azure storage metricset #15342

Merged
merged 18 commits into from
Jan 13, 2020
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -588,6 +588,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Add overview dashboard for AWS SNS module {pull}14977[14977]
- Add `index` option to all modules to specify a module-specific output index. {pull}15100[15100]
- Add a `system/service` metricset for systemd data. {pull}14206[14206]
- Add azure `storage` metricset in order to retrieve metric values for storage accounts. {issue}14548[14548] {pull}15342[15342]
- Add cost warnings for the azure module. {pull}15356[15356]

*Packetbeat*
Expand Down
62 changes: 36 additions & 26 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2131,84 +2131,84 @@ azure module



*`azure.namespace`*::
+
--
The namespace selected
[float]
=== resource

The resource specified

type: keyword

--

*`azure.subscription_id`*::
*`azure.resource.name`*::
+
--
The subscription ID
The name of the resource


type: keyword

--

*`azure.dimensions.*`*::
*`azure.resource.type`*::
+
--
Azure metric dimensions.
The type of the resource


type: object
type: keyword

--

[float]
=== resource
*`azure.resource.group`*::
+
--
The resource group

The resource specified

type: keyword

--

*`azure.resource.name`*::
*`azure.resource.tags.*`*::
+
--
The name of the resource
Azure resource tags.


type: keyword
type: object

--

*`azure.resource.type`*::
*`azure.namespace`*::
+
--
The type of the resource
The namespace selected


type: keyword

--

*`azure.resource.group`*::
*`azure.subscription_id`*::
+
--
The resource group
The subscription ID


type: keyword

--

*`azure.resource.tags.*`*::
*`azure.dimensions.*`*::
+
--
Azure resource tags.
Azure metric dimensions.


type: object

--

*`azure.resource.compute_vm.*.*`*::
*`azure.compute_vm.*.*`*::
+
--
compute_vm
Expand All @@ -2218,7 +2218,7 @@ type: object

--

*`azure.resource.compute_vm_scaleset.*.*`*::
*`azure.compute_vm_scaleset.*.*`*::
+
--
compute_vm_scaleset
Expand All @@ -2235,12 +2235,22 @@ monitor



*`azure.resource.monitor.metrics.*.*`*::
*`azure.monitor.metrics.*.*`*::
+
--
Metrics returned.


type: object

--

*`azure.storage.*.*`*::
+
--
storage account


type: object

--
Expand Down
19 changes: 19 additions & 0 deletions metricbeat/docs/modules/azure.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ so the `period` for `compute_vm` metricset should be `300s` or multiples of `30
This metricset will collect metrics from the virtual machine scalesets, these metrics will have a timegrain every 5 minutes,
so the `period` for `compute_vm_scaleset` metricset should be `300s` or multiples of `300s`.

[float]
=== `storage`
This metricset will collect metrics from the storage accounts, these metrics will have a timegrain every 5 minutes,
so the `period` for `storage` metricset should be `300s` or multiples of `300s`.


[float]
== Additional notes about metrics and costs
Expand Down Expand Up @@ -130,6 +135,16 @@ metricbeat.modules:
client_secret: '${AZURE_CLIENT_SECRET:""}'
tenant_id: '${AZURE_TENANT_ID:""}'
subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'

- module: azure
metricsets:
- storage
enabled: true
period: 300s
client_id: '${AZURE_CLIENT_ID:""}'
client_secret: '${AZURE_CLIENT_SECRET:""}'
tenant_id: '${AZURE_TENANT_ID:""}'
subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'
----

[float]
Expand All @@ -143,9 +158,13 @@ The following metricsets are available:

* <<metricbeat-metricset-azure-monitor,monitor>>

* <<metricbeat-metricset-azure-storage,storage>>

include::azure/compute_vm.asciidoc[]

include::azure/compute_vm_scaleset.asciidoc[]

include::azure/monitor.asciidoc[]

include::azure/storage.asciidoc[]

23 changes: 23 additions & 0 deletions metricbeat/docs/modules/azure/storage.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
////
This file is generated! See scripts/mage/docs_collector.go
////

[[metricbeat-metricset-azure-storage]]
=== azure storage metricset

beta[]

include::../../../../x-pack/metricbeat/module/azure/storage/_meta/docs.asciidoc[]


==== Fields

For a description of each field in the metricset, see the
<<exported-fields-azure,exported fields>> section.

Here is an example document generated by this metricset:

[source,json]
----
include::../../../../x-pack/metricbeat/module/azure/storage/_meta/data.json[]
----
3 changes: 2 additions & 1 deletion metricbeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,10 @@ This file is generated! See scripts/mage/docs_collector.go
|<<metricbeat-metricset-aws-sqs,sqs>>
|<<metricbeat-metricset-aws-usage,usage>> beta[]
|<<metricbeat-module-azure,azure>> beta[] |image:./images/icon-yes.png[Prebuilt dashboards are available] |
.3+| .3+| |<<metricbeat-metricset-azure-compute_vm,compute_vm>> beta[]
.4+| .4+| |<<metricbeat-metricset-azure-compute_vm,compute_vm>> beta[]
|<<metricbeat-metricset-azure-compute_vm_scaleset,compute_vm_scaleset>> beta[]
|<<metricbeat-metricset-azure-monitor,monitor>> beta[]
|<<metricbeat-metricset-azure-storage,storage>> beta[]
|<<metricbeat-module-beat,Beat>> |image:./images/icon-no.png[No prebuilt dashboards] |
.2+| .2+| |<<metricbeat-metricset-beat-state,state>>
|<<metricbeat-metricset-beat-stats,stats>>
Expand Down
1 change: 1 addition & 0 deletions x-pack/metricbeat/include/list.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions x-pack/metricbeat/metricbeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,16 @@ metricbeat.modules:
tenant_id: '${AZURE_TENANT_ID:""}'
subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'

- module: azure
metricsets:
- storage
enabled: true
period: 300s
client_id: '${AZURE_CLIENT_ID:""}'
client_secret: '${AZURE_CLIENT_SECRET:""}'
tenant_id: '${AZURE_TENANT_ID:""}'
subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'

#--------------------------------- Beat Module ---------------------------------
- module: beat
metricsets:
Expand Down
10 changes: 10 additions & 0 deletions x-pack/metricbeat/module/azure/_meta/config.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,13 @@
client_secret: '${AZURE_CLIENT_SECRET:""}'
tenant_id: '${AZURE_TENANT_ID:""}'
subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'

- module: azure
metricsets:
- storage
enabled: true
period: 300s
client_id: '${AZURE_CLIENT_ID:""}'
client_secret: '${AZURE_CLIENT_SECRET:""}'
tenant_id: '${AZURE_TENANT_ID:""}'
subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'
11 changes: 11 additions & 0 deletions x-pack/metricbeat/module/azure/_meta/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,14 @@
# tenant_id: '${AZURE_TENANT_ID:""}'
# subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'
# refresh_list_interval: 600s

#- module: azure
# metricsets:
# - storage
# enabled: true
# period: 300s
# client_id: '${AZURE_CLIENT_ID:""}'
# client_secret: '${AZURE_CLIENT_SECRET:""}'
# tenant_id: '${AZURE_TENANT_ID:""}'
# subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'
# refresh_list_interval: 600s
5 changes: 5 additions & 0 deletions x-pack/metricbeat/module/azure/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,11 @@ so the `period` for `compute_vm` metricset should be `300s` or multiples of `30
This metricset will collect metrics from the virtual machine scalesets, these metrics will have a timegrain every 5 minutes,
so the `period` for `compute_vm_scaleset` metricset should be `300s` or multiples of `300s`.

[float]
=== `storage`
This metricset will collect metrics from the storage accounts, these metrics will have a timegrain every 5 minutes,
so the `period` for `storage` metricset should be `300s` or multiples of `300s`.


[float]
== Additional notes about metrics and costs
Expand Down
64 changes: 32 additions & 32 deletions x-pack/metricbeat/module/azure/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,44 +4,44 @@
azure module
release: beta
fields:
- name: azure
type: group
description: >
fields:
- name: namespace
type: keyword
description: >
The namespace selected
- name: subscription_id
type: keyword
description: >
The subscription ID
- name: dimensions.*
type: object
object_type: keyword
object_type_mapping_type: "*"
description: >
Azure metric dimensions.
- name: resource
type: group
description: >
The resource specified
fields:
- name: name
type: keyword
- name: azure
type: group
description: >
fields:
- name: resource
type: group
description: >
The name of the resource
- name: type
The resource specified
fields:
- name: name
type: keyword
description: >
The name of the resource
- name: type
type: keyword
description: >
The type of the resource
- name: group
type: keyword
description: >
The resource group
- name: tags.*
type: object
object_type: keyword
object_type_mapping_type: "*"
description: >
Azure resource tags.
- name: namespace
type: keyword
description: >
The type of the resource
- name: group
The namespace selected
- name: subscription_id
type: keyword
description: >
The resource group
- name: tags.*
The subscription ID
- name: dimensions.*
type: object
object_type: keyword
object_type_mapping_type: "*"
description: >
Azure resource tags.
Azure metric dimensions.
Loading