...
Custom Metrics
...
SPM gives ability for users to extend standard reports using custom metrics.
Terminology
- A "metric" is a distinct triple of (metric name, filter1 name, filter2 name). Some examples:
- ("cpuIdle", server="server1", null) vs. ("cpuIdle", null, server="server1") – these are 2 distinct metrics
- ("cpuSteal", server="server1", null) vs. ("cpuIdle", server="server1", null) – these are 2 distinct metrics
- ("diskRead", server="server1", device="/dev/xxx") vs. ("diskRead", server="server1", device="/dev/yyy") – this is the same 1 metric with two different values for filter2 named "device"
- ("diskWrite", server="server1", device="/dev/xxx") vs. ("diskWrite", server="server1", device="/dev/yyy") – this is the same 1 metric with two different values for filter2 named "device"
- ("diskRead", server="server1", device="/dev/xxx") vs. ("diskWrite", server="server1", device="/dev/xxx") – these are 2 distinct metrics
...
- A "data point" is the value associated with some (metric, filter1, filter2) combination. Some examples:
- ("cpuIdle", server="server1", null) = 90
- ("cpuIdle", server="server1", null) = 90 – this is a distinct data point for the same metric as in the previous line
Pushing Metrics
Request format
Custom metrics can be sent to receiver via HTTP as POST request.
Request parameters:
Name | Description |
---|---|
host | host name |
token | application token |
Request body format:
Code Block | ||
---|---|---|
| ||
[timestamp]\tcustom\t[metric_name]\t[filter1_value]\t[filter2_value]\t[metric_value]\t[aggregation_type]
.... |
Placeholder | Description | Limitations |
---|---|---|
timestamp | Number of milliseconds passed since Epoch (1 Jan 1970) | |
metric_name | Name of metric | Length > 0 and <= 255 |
filter1_value | Value of first filter | Length > 0 and <= 255 |
filter2_value | Value of second filter | Length > 0 and <= 255 |
metric_value | Value of metric (Double) | |
aggregation_type | Type of aggregation | avg,min,max,sum |
Curl example
Code Block | ||
---|---|---|
| ||
TIMESTAMP=$(($(date +%s)*1000))
METRIC=`echo -n "$TIMESTAMP\tcustom\tconsumed_coffee\toffice-1\tkitchen-2\t100.0\tsum"`
TOKEN=xxxxxxxx-759b-46bf-a9d3-xxxxxxxxxxxx
curl -H "Content-Type: text/plain" -X POST -d "$METRIC" "http://apps.sematext.com/spm-receiver/receive?host=`hostname`&token=$TOKEN" |
Ruby example
Example using gem "httparty"
Code Block | ||
---|---|---|
| ||
require 'httparty'
class MetricSerializer
def serialize(options)
"#{options[:timestamp]}\tcustom\t#{options[:name]}\t#{options[:filter1]}\t#{options[:filter2]}\t#{options[:value]}\t#{options[:agg]}"
end
end
class MetricSender
include HTTParty
def initialize(options)
raise "receiver_url is required" unless options[:receiver_url]
raise "host is required" unless options[:host]
raise "token is required" unless options[:token]
self.class.default_options[:base_uri] = options[:receiver_url]
@serializer = MetricSerializer.new
@host = options[:host]
@token = options[:token]
@version = "1.9.0"
end
def send(options)
current_time = (Time.now.to_f * 1000).to_i
merged_options = {
:timestamp => current_time,
:agg => 'avg'
}.merge(options)
s = @serializer.serialize(merged_options)
puts s
post_metric(s)
end
private
def post_metric(body)
post_options = {
:query => {:host => @host, :token => @token, :v => @version},
:headers => {'Content-Type' => 'text/plain; charset=utf-8'},
:body => body
}
self.class.post('receive', post_options)
end
end
RECEIVER_OPTIONS = {
:receiver_url => "http://appssematext.com/spm-receiver/",
:host => "katrin",
:token => "xxxxxxxx-759b-46bf-a9d3-xxxxxxxxxxxx"
}
sender = MetricSender.new(RECEIVER_OPTIONS)
sender.send :name => "coffee_consumed", :filter1 => "office-1", :filter2 => "kitchen-2", :value => 100.0, :agg => "sum"
|
Comments
Any frequency limitations depend on AF implementation.
TODO (refer to AF frequency limits)
...
Introduction
Data point is represented using parameters:
- timestamp - Time since "Epoch" in milliseconds
- name - Name of metric
- value - Double value
- aggregation type -min, max, avg, sum
- filter1 value - String value for filter1
- filter2 value - String value for filter2
Custom metrics report displays values, that aggregated using 'aggregation type' for selected granularity. For example, sent values 1.0, 2.0 and 3.0 during one minute with average ('avg') aggregation type will be represented by one point with value 3.0 on plot for this minute.
Filter1 and filter2 values can be used to organize complex metrics, for example to measure registered users count by gender and account type. In registered users example data points can be specified as:
Code Block | ||
---|---|---|
| ||
{
"name" : "registered-users-count",
"value" : 42,
"aggregation" : "sum",
"filter1" : "user.gender=male",
"filter2" : "account.type=free"
}
{
"name" : "registered-users-count",
"value" : 24,
"aggregation" : "sum",
"filter1" : "user.gender=female",
"filter2" : "account.type=free"
}
{
"name" : "registered-users-count",
"value" : 10,
"aggregation" : "sum",
"filter1" : "user.gender=female",
"filter2" : "account.type=paid"
} |
In report page selected values in one filter are threated as 'OR' and selected values in both as 'AND'. For example to display count of female users in filter1 value 'female' should be selected, to display female paid users in filter1 should be selected 'female' and in filter2 'paid'.
We recomend to use 'parameter.name=value' format to define filter values.
Custom Metrics API
SPM provides REST API for sending data points for custom metrics:
http://spm-receiver.sematext.com/receiver/custom/receive.[format]?token=[spm app token]
List of tokens per application available at settings page: https://apps.sematext.com/users-web/home.do?homeTab=spm
Supported two formats: 'json' and 'raw'.
Json format
Url: http://spm-receiver.sematext.com/receiver/custom/receive.json?token=[spm app token]
Content-type: application/json
Method: POST
Json body:
Field name | Field type | Constraints |
---|---|---|
datapoints | array of 'data point' | elements count <= 100 |
Data point:
Field name | Description | Field type | Constraints | Required |
---|---|---|---|---|
timestamp | Time in milliseconds since "Epoch" | Long | No | |
name | Metric name | String | length > 0 and <= 255 | Yes |
value | Data point value | Double | Yes | |
aggregation | Aggregation type | String | "min", "max", "avg", "sum" | Yes |
filter1 | Value for filter1 | String | length > 0 and <= 255 | No |
filter2 | Value for filter2 | String | length > 0 and <= 255 | No |
Example:
Code Block | ||
---|---|---|
| ||
{
"datapoints" : [
{
"timestamp" : 1369669889927,
"name" : "registered-users-count",
"value" : 1,
"aggregation" : "sum",
"filter1" : "user.gender=male",
"filter2" : "account.type=free"
},
{
"name" : "registered-users-count",
"value" : 1,
"aggregation" : "sum",
"filter1" : "user.gender=female",
"filter2" : "account.type=paid"
}
]
} |
To send metrics using curl, save example above to file named "datapoints.json",
Code Block | ||
---|---|---|
| ||
curl -v -H 'Content-type: application/json' -d @datapoints.json http://spm-receiver.sematext.com/receiver/custom/receive.json?token=[spm app token] |
Raw format
Url: http://spm-receiver.sematext.com/receiver/custom/receive.raw?token=[spm app token]
Content-type: text/plan
Method: POST
Each line is tab delimited sequence of fields:
(timestamp)\t(name)\t(value)\t(aggregation type)\t(filter1)\t(filter2)
Lines are separated by new line character ('\n'). Limit is 100 lines per request.
Field name | Description | Field type | Constraints | Required |
---|---|---|---|---|
timestamp | Time in milliseconds since "Epoch" | Long | No | |
name | Metric name | String | length > 0 and <= 255 | Yes |
value | Data point value | Double | Yes | |
aggregation | Aggregation type | String | "min", "max", "avg", "sum" | Yes |
filter1 | Value for filter1 | String | length > 0 and <= 255 | No |
filter2 | Value for filter2 | String | length > 0 and <= 255 | No |
Example:
Code Block | ||
---|---|---|
| ||
1369671381221 registered-users-count 1 sum user.gender=male account.type=free
1369671381221 registered-users-count 1 sum user.gender=female account.type=paid |
To post metrics using curl save example above (take care about tabs) to datapoints.raw.
Code Block | ||
---|---|---|
| ||
curl -v -H 'Content-type: text/plain' -d @datapoints.raw http://spm-receiver.sematext.com/receiver/custom/receive.raw?token=[spm app token] |
Java API
Sematext-metrics is open source library for sending custom metrics from java application. Please refer to README for quick start.
Limitations
There are limitations by datapoints count per month and metrics count mer month. Please refer to our page with plans description.