-
Notifications
You must be signed in to change notification settings - Fork 102
query and ingest in depth
Table of Contents
- Send Numeric Metrics with Curl
- Send Aggregated Metrics with Curl
- Retrieve Numeric Metrics with Curl
- Test Metrics Rollups with Python Script
- Metrics List Query
The current API version is '2.0'.
The deprecated v1.0 endpoints still work but will be replaced in a future release of Blueflood.
The following is the curl command for Blueflood API. The tenant-id is how Blueflood supports multi-tenant. Metric names under different tenant-id will always be treated as different metrics even if they are identical.
curl -i -X POST 'http://localhost:19000/v2.0/tenant-id/ingest' -d \
'[
{
"collectionTime": 1376509892612,
"ttlInSeconds": 172800,
"metricValue": 66,
"metricName": "example.metric.one"
},
{
"collectionTime": 1376509892612,
"ttlInSeconds": 172800,
"metricValue": 66,
"metricName": "example.metric.two"
},
{
"collectionTime": 1376509892612,
"ttlInSeconds": 172800,
"metricValue": 66,
"metricName": "example.metric.three"
}
]'
Blueflood accepts aggregated metrics, such as the ones sent by statsd. The following is the curl command for sending aggregated metrics to Blueflood for the tenant with id 123456.
curl -i -XPOST 'http://localhost:19000/v2.0/123456/ingest/aggregated' -d '
{
"tenantId": "123456",
"timestamp": 111111111,
"gauges": [
{
"name": "gauge_name",
"value": 42
},
{
"name":"another_gauge",
"value": 4343
}
],
"counters": [
{
"name": "counter_name",
"value": 32,
"rate": 2.32
},
{
"name": "another_counter",
"value": 4424,
"rate": 52.1
}
],
"timers": [
{
"name": "timer_name",
"count": 32,
"rate": 2.3,
"min": 1,
"max": 5,
"sum": 21,
"avg": 2.1,
"median": 3,
"std": 1.01,
"histogram": {
"bin_50": 0,
"bin_100": 0,
"bin_inf": 0
}
}
],
"sets": [
{
"name": "set_name",
"values": ["foo", "bar", "baz"]
},
{
"name": "another_set",
"values": ["boo", "far", "zab"]
}
]
}'
curl -i -X GET 'http://localhost:20000/v2.0/tenant-id/views/example.metric.one?from=1376509892611&to=1376509892613&points=200'
This will retrieve the metrics data as
HTTP/1.1 200 OK
Content-Length: 232
{
"values": [
{
"numPoints": 1,
"timestamp": 1376509892612,
"average": 66
}
],
"metadata": {
"limit": null,
"next_href": null,
"count": 1,
"marker": null
}
}
You can filter stats displayed by appending a select parameter to the API request like so:
curl -i -X GET 'http://localhost:20000/v1.0/tenant-id/experimental/views/metric_data/example.metric.one?from=1376509892611&to=1376509892613&points=200&select=average,min,numPoints'
The available fields are: average, variance, min, max, numPoints
You can also get data for a list of metrics. The maximum number of metrics that can be fetched is set to 100 by default (configurable). Here is the curl command to get two metrics:
curl -i -X POST 'http://localhost:20000/v2.0/ac12a36b5c-5313-11e3-ba22-000c29da7d83/views?from=1384975594392&to=1385178594392&points=100' -d \
'[
'met.42f287ca-5313-11e3-99ba-000c29da7d83',
'met.12a3a658-5313-11e3-ba22-000c29da7d83'
]'
Results:
HTTP/1.1 200 OK
Content-Length: 749
{
"metrics": [
{
"unit": "seconds",
"metric": "met.12a3a658-5313-11e3-ba22-000c29da7d83",
"data": [
{
"numPoints": 97,
"timestamp": 1385074800000,
"average": 48
},
{
"numPoints": 3,
"timestamp": 1385078400000,
"average": 54
}
],
"type": "number"
},
{
"unit": "seconds",
"metric": "met.42f287ca-5313-11e3-99ba-000c29da7d83",
"data": [
{
"numPoints": 94,
"timestamp": 1385074800000,
"average": 54
},
{
"numPoints": 6,
"timestamp": 1385078400000,
"average": 62
}
],
"type": "number"
}
]
}
An alternate way to interact with blueflood is to use the [ingest.py] (https://github.com/rackerlabs/blueflood/blob/master/demo/ingest.py) and [retrieve.py] (https://github.com/rackerlabs/blueflood/blob/master/demo/retrieve.py) scripts to ingest data into blueflood and retrieve aggregated metrics.
If you run [ingest.py] (https://github.com/rackerlabs/blueflood/blob/master/demo/ingest.py) against the blueflood instance that you set up, it will generate large number of metric data and push them into Blueflood. The following is an example of output. Yours will have different metric name and tenant id.
NOTE: These scripts currently use the v1.0 endpoints, which are deprecated.
> ./ingest.py
[
{
"collectionTime": 1378333729069,
"ttlInSeconds": 172800,
"metricValue": 98,
"unit": "seconds",
"metricName": "met.2211ecf0-15c2-11e3-a505-bc764e10e18e"
},
{
"collectionTime": 1378333759069,
"ttlInSeconds": 172800,
"metricValue": 19,
"unit": "seconds",
"metricName": "met.2211ecf0-15c2-11e3-a505-bc764e10e18e"
},
...
{
"collectionTime": 1378336699069,
"ttlInSeconds": 172800,
"metricValue": 3,
"unit": "seconds",
"metricName": "met.2211ecf0-15c2-11e3-a505-bc764e10e18e"
}
]
http://localhost:19000/v1.0/ac2210dd10-15c2-11e3-a505-bc764e10e18e/experimental/metrics
Writing metrics for tenant: ac2210dd10-15c2-11e3-a505-bc764e10e18e, metric name: met.2211ecf0-15c2-11e3-a505-bc764e10e18e, start: 1378333729069, end: 1378336729069
Response from server <Response [200]>
To retrive the generated data with retrieve.py script, use the following command (assuming port number 20000):
./retrieve.py --host localhost --port 20000 --metric met.2211ecf0-15c2-11e3-a505-bc764e10e18e --tenant ac2210dd10-15c2-11e3-a505-bc764e10e18e --from 1378233729069 --to 1378436729069 --points 100
The output has the information about the metric name and the tenant ID under which the data are generated. It also prints out the command to retrieve the data using the retrieve.py. You can change the time range to decide see the change on the data being returned.
>./retrieve.py --host localhost --port 20000 --metric met.2211ecf0-15c2-11e3-a505-bc764e10e18e --tenant ac2210dd10-15c2-11e3-a505-bc764e10e18e --from 1378233729069 --to 1378436729069 --points 100
http://localhost:20000/v1.0/ac2210dd10-15c2-11e3-a505-bc764e10e18e/experimental/views/metric_data/met.2211ecf0-15c2-11e3-a505-bc764e10e18e?from=1378233729069&to=1378436729069&points=100
{
"values": [
{
"numPoints": 63,
"timestamp": 1378332000000,
"average": 51
},
{
"numPoints": 37,
"timestamp": 1378335600000,
"average": 36
}
],
"metadata": {
"limit": null,
"next_href": null,
"count": 2,
"marker": null
}
}
You can change the number of points and see how you get different granularities of rolled up data. The retrieve script also prints out the URL for the retrieve API. You can use curl and add additional parameters that are not supported by the retrieve metrics.
curl -i -X GET 'http://localhost:20000/v2.0/ac2210dd10-15c2-11e3-a505-bc764e10e18e/views/met.2211ecf0-15c2-11e3-a505-bc764e10e18e?from=1378233729069&to=1378436729069&points=100&select=average,variance'
HTTP/1.1 200 OK
Content-Length: 354
{
"values": [
{
"timestamp": 1378332000000,
"variance": 858.74,
"average": 49
},
{
"timestamp": 1378335600000,
"unit": "seconds",
"variance": 694.67,
"average": 36
}
],
"metadata": {
"limit": null,
"next_href": null,
"count": 2,
"marker": null
}
}
You can list all metrics of a given tenant if you have ElasticSearch set up. You can also do wildcard searches on metric names. See examples below.
curl -X GET http://localhost:20000/v2.0/tenant-id/metrics/search?query=*
Results follow:
[
{
"metric": "example.metric.one"
},
{
"metric": "example.metric.two"
},
{
"metric": "example.metric.three"
}
]
Few more examples of wildcard searches are:
curl -X GET http://localhost:20000/v2.0/tenant-id/metrics/search?query=example.*
curl -X GET http://localhost:20000/v2.0/tenant-id/metrics/search?query=example.metric.*