Functions

Functions are used to transform, combine, and perform computations on series data. Functions are applied using the Composer interface or by manipulating the target parameters in the Render API.

Usage

Most functions are applied to one series list. Functions with the parameter *seriesLists can take an arbitrary number of series lists. To pass multiple series lists to a function which only takes one, use the group() function.

List of functions

absolute(seriesList)

Takes one metric or a wildcard seriesList and applies the mathematical abs function to each datapoint transforming it to its absolute value.

Example:

&target=absolute(Server.instance01.threads.busy)
&target=absolute(Server.instance*.threads.busy)
add(seriesList, constant)

Takes one metric or a wildcard seriesList followed by a constant, and adds the constant to each datapoint. Also works for negative numbers.

Example:

&target=add(Server.instance01.threads.busy, 10)
&target=add(Server.instance*.threads.busy, 10)
aggregate(seriesList, func, xFilesFactor=None)

Aggregate series using the specified function.

Example:

&target=aggregate(host.cpu-[0-7].cpu-{user,system}.value, "sum")

This would be the equivalent of

&target=sumSeries(host.cpu-[0-7].cpu-{user,system}.value)

This function can be used with aggregation functions average (or avg), avg_zero, median, sum (or total), min, max, diff, stddev, count, range (or rangeOf) , multiply & last (or current).

aggregateLine(seriesList, func='average', keepStep=False)

Takes a metric or wildcard seriesList and draws a horizontal line based on the function applied to each series.

If the optional keepStep parameter is set to True, the result will have the same time period and step as the source series.

Note: By default, the graphite renderer consolidates data points by averaging data points over time. If you are using the ‘min’ or ‘max’ function for aggregateLine, this can cause an unusual gap in the line drawn by this function and the data itself. To fix this, you should use the consolidateBy() function with the same function argument you are using for aggregateLine. This will ensure that the proper data points are retained and the graph should line up correctly.

Example:

&target=aggregateLine(server01.connections.total, 'avg')
&target=aggregateLine(server*.connections.total, 'avg')
aggregateSeriesLists(seriesListFirstPos, seriesListSecondPos, func, xFilesFactor=None)

Iterates over a two lists and aggregates using specified function list1[0] to list2[0], list1[1] to list2[1] and so on. The lists will need to be the same length

Position of seriesList matters. For example using “sum” function aggregateSeriesLists(list1[0..n], list2[0..n], "sum") it would find sum for each member of the list list1[0] + list2[0], list1[1] + list2[1], list1[n] + list2[n].

Example:

&target=aggregateSeriesLists(mining.{carbon,graphite,diamond}.extracted,mining.{carbon,graphite,diamond}.shipped, 'sum')

An example above would be the same as running aggregate for each member of the list:

?target=aggregate(mining.carbon.extracted,mining.carbon.shipped, 'sum')
&target=aggregate(mining.graphite.extracted,mining.graphite.shipped, 'sum')
&target=aggregate(mining.diamond.extracted,mining.diamond.shipped, 'sum')

This function can be used with aggregation functions average (or avg), avg_zero, median, sum (or total), min, max, diff, stddev, count, range (or rangeOf) , multiply & last (or current).

aggregateWithWildcards(seriesList, func, *positions)

Call aggregator after inserting wildcards at the given position(s).

Example:

&target=aggregateWithWildcards(host.cpu-[0-7].cpu-{user,system}.value, "sum", 1)

This would be the equivalent of

&target=sumSeries(host.cpu-[0-7].cpu-user.value)&target=sumSeries(host.cpu-[0-7].cpu-system.value)
# or
&target=aggregate(host.cpu-[0-7].cpu-user.value,"sum")&target=aggregate(host.cpu-[0-7].cpu-system.value,"sum")

This function can be used with all aggregation functions supported by aggregate: average, median, sum, min, max, diff, stddev, range & multiply.

This complements groupByNodes which takes a list of nodes that must match in each group.

alias(seriesList, newName)

Takes one metric or a wildcard seriesList and a string in quotes. Prints the string instead of the metric name in the legend.

&target=alias(Sales.widgets.largeBlue,"Large Blue Widgets")
aliasByMetric(seriesList)

Takes a seriesList and applies an alias derived from the base metric name.

&target=aliasByMetric(carbon.agents.graphite.creates)
aliasByNode(seriesList, *nodes)

Takes a seriesList and applies an alias derived from one or more “node” portion/s of the target name or tags. Node indices are 0 indexed.

&target=aliasByNode(ganglia.*.cpu.load5,1)

Each node may be an integer referencing a node in the series name or a string identifying a tag.

&target=seriesByTag("name=~cpu.load.*", "server=~server[1-9]+", "datacenter=dc1")|aliasByNode("datacenter", "server", 1)

# will produce output series like
# dc1.server1.load5, dc1.server2.load5, dc1.server1.load10, dc1.server2.load10
aliasByTags(seriesList, *tags)

Takes a seriesList and applies an alias derived from one or more tags and/or nodes

&target=seriesByTag("name=cpu")|aliasByTags("server","name")

This is an alias for aliasByNode.

aliasQuery(seriesList, search, replace, newName)

Performs a query to alias the metrics in seriesList.

&target=aliasQuery(channel.power.*,"channel\.power\.([0-9]+)","channel.frequency.\1", "Channel %d MHz")

The series in seriesList will be aliased by first translating the series names using the search & replace parameters, then using the last value of the resulting series to construct the alias using sprintf-style syntax.

aliasSub(seriesList, search, replace)

Runs series names through a regex search/replace.

&target=aliasSub(ip.*TCP*,"^.*TCP(\d+)","\1")
alpha(seriesList, alpha)

Assigns the given alpha transparency setting to the series. Takes a float value between 0 and 1.

applyByNode(seriesList, nodeNum, templateFunction, newName=None)

Takes a seriesList and applies some complicated function (described by a string), replacing templates with unique prefixes of keys from the seriesList (the key is all nodes up to the index given as nodeNum).

If the newName parameter is provided, the name of the resulting series will be given by that parameter, with any “%” characters replaced by the unique prefix.

Example:

&target=applyByNode(servers.*.disk.bytes_free,1,"divideSeries(%.disk.bytes_free,sumSeries(%.disk.bytes_*))")

Would find all series which match servers.*.disk.bytes_free, then trim them down to unique series up to the node given by nodeNum, then fill them into the template function provided (replacing % by the prefixes).

Additional Examples:

Given keys of

  • stats.counts.haproxy.web.2XX
  • stats.counts.haproxy.web.3XX
  • stats.counts.haproxy.web.5XX
  • stats.counts.haproxy.microservice.2XX
  • stats.counts.haproxy.microservice.3XX
  • stats.counts.haproxy.microservice.5XX

The following will return the rate of 5XX’s per service:

applyByNode(stats.counts.haproxy.*.*XX, 3, "asPercent(%.5XX, sumSeries(%.*XX))", "%.pct_5XX")

The output series would have keys stats.counts.haproxy.web.pct_5XX and stats.counts.haproxy.microservice.pct_5XX.

areaBetween(seriesList)

Draws the vertical area in between the two series in seriesList. Useful for visualizing a range such as the minimum and maximum latency for a service.

areaBetween expects exactly one argument that results in exactly two series (see example below). The order of the lower and higher values series does not matter. The visualization only works when used in conjunction with areaMode=stacked.

Most likely use case is to provide a band within which another metric should move. In such case applying an alpha(), as in the second example, gives best visual results.

Example:

&target=areaBetween(service.latency.{min,max})&areaMode=stacked

&target=alpha(areaBetween(service.latency.{min,max}),0.3)&areaMode=stacked

If for instance, you need to build a seriesList, you should use the group function, like so:

&target=areaBetween(group(minSeries(a.*.min),maxSeries(a.*.max)))
asPercent(seriesList, total=None, *nodes)

Calculates a percentage of the total of a wildcard series. If total is specified, each series will be calculated as a percentage of that total. If total is not specified, the sum of all points in the wildcard series will be used instead.

A list of nodes can optionally be provided, if so they will be used to match series with their corresponding totals following the same logic as groupByNodes.

When passing nodes the total parameter may be a series list or None. If it is None then for each series in seriesList the percentage of the sum of series in that group will be returned.

When not passing nodes, the total parameter may be a single series, reference the same number of series as seriesList or be a numeric value.

Example:

# Server01 connections failed and succeeded as a percentage of Server01 connections attempted
&target=asPercent(Server01.connections.{failed,succeeded}, Server01.connections.attempted)

# For each server, its connections failed as a percentage of its connections attempted
&target=asPercent(Server*.connections.failed, Server*.connections.attempted)

# For each server, its connections failed and succeeded as a percentage of its connections attemped
&target=asPercent(Server*.connections.{failed,succeeded}, Server*.connections.attempted, 0)

# apache01.threads.busy as a percentage of 1500
&target=asPercent(apache01.threads.busy,1500)

# Server01 cpu stats as a percentage of its total
&target=asPercent(Server01.cpu.*.jiffies)

# cpu stats for each server as a percentage of its total
&target=asPercent(Server*.cpu.*.jiffies, None, 0)

When using nodes, any series or totals that can’t be matched will create output series with names like asPercent(someSeries,MISSING) or asPercent(MISSING,someTotalSeries) and all values set to None. If desired these series can be filtered out by piping the result through |exclude("MISSING") as shown below:

&target=asPercent(Server{1,2}.memory.used,Server{1,3}.memory.total,0)

# will produce 3 output series:
# asPercent(Server1.memory.used,Server1.memory.total) [values will be as expected]
# asPercent(Server2.memory.used,MISSING) [all values will be None]
# asPercent(MISSING,Server3.memory.total) [all values will be None]

&target=asPercent(Server{1,2}.memory.used,Server{1,3}.memory.total,0)|exclude("MISSING")

# will produce 1 output series:
# asPercent(Server1.memory.used,Server1.memory.total) [values will be as expected]

Each node may be an integer referencing a node in the series name or a string identifying a tag.

Note

When total is a seriesList, specifying nodes to match series with the corresponding total series will increase reliability.

averageAbove(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant N. Out of all metrics passed, draws only the metrics with an average value above N for the time period specified.

Example:

&target=averageAbove(server*.instance*.threads.busy,25)

Draws the servers with average values above 25.

averageBelow(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant N. Out of all metrics passed, draws only the metrics with an average value below N for the time period specified.

Example:

&target=averageBelow(server*.instance*.threads.busy,25)

Draws the servers with average values below 25.

averageOutsidePercentile(seriesList, n)

Removes series lying inside an average percentile interval

averageSeries(*seriesLists)

Short Alias: avg()

Takes one metric or a wildcard seriesList. Draws the average value of all metrics passed at each time.

Example:

&target=averageSeries(company.server.*.threads.busy)

This is an alias for aggregate with aggregation average.

averageSeriesWithWildcards(seriesList, *position)

Categorizes the provided series in groups by name, by ignoring (“wildcarding”) the given position(s) and calls averageSeries on each group. Important: the introduction of wildcards only happens after retrieving the input.

Example:

&target=averageSeriesWithWildcards(host.cpu-[0-7].cpu-{user,system}.value, 1)

This would be the equivalent of

&target=averageSeries(host.cpu-[0-7].cpu-user.value)&target=averageSeries(host.cpu-[0-7].cpu-system.value)

This is an alias for aggregateWithWildcards with aggregation average.

cactiStyle(seriesList, system=None, units=None)

Takes a series list and modifies the aliases to provide column aligned output with Current, Max, and Min values in the style of cacti. Optionally takes a “system” value to apply unit formatting in the same style as the Y-axis, or a “unit” string to append an arbitrary unit suffix.

&target=cactiStyle(ganglia.*.net.bytes_out,"si")
&target=cactiStyle(ganglia.*.net.bytes_out,"si","b")

A possible value for system is si, which would express your values in multiples of a thousand. A second option is to use binary which will instead express your values in multiples of 1024 (useful for network devices).

Column alignment of the Current, Max, Min values works under two conditions: you use a monospace font such as terminus and use a single cactiStyle call, as separate cactiStyle calls are not aware of each other. In case you have different targets for which you would like to have cactiStyle to line up, you can use group() to combine them before applying cactiStyle, such as:

&target=cactiStyle(group(metricA,metricB))
changed(seriesList)

Takes one metric or a wildcard seriesList. Output 1 when the value changed, 0 when null or the same

Example:

&target=changed(Server01.connections.handled)
color(seriesList, theColor)

Assigns the given color to the seriesList

Example:

&target=color(collectd.hostname.cpu.0.user, 'green')
&target=color(collectd.hostname.cpu.0.system, 'ff0000')
&target=color(collectd.hostname.cpu.0.idle, 'gray')
&target=color(collectd.hostname.cpu.0.idle, '6464ffaa')
compressPeriodicGaps(seriesList)

Tries to intelligently remove periodic None’s from series, recalculating start, stop and step values. You can use summarize(seriesList, ‘<desired step>’, ‘last’) function for that also, but this function trying to guess desired step automatically. Can be used in case of fix metric with improper resolution. Especially useful for derivative functions, which are not working with series with regular gaps.

Example:

&target=compressPeriodicGaps(Server.instance01.threads.busy)
consolidateBy(seriesList, consolidationFunc)

Takes one metric or a wildcard seriesList and a consolidation function name.

Valid function names are ‘sum’, ‘average’/’avg’, ‘min’, ‘max’, ‘first’ & ‘last’.

When a graph is drawn where width of the graph size in pixels is smaller than the number of datapoints to be graphed, Graphite consolidates the values to to prevent line overlap. The consolidateBy() function changes the consolidation function from the default of ‘average’ to one of ‘sum’, ‘max’, ‘min’, ‘first’, or ‘last’. This is especially useful in sales graphs, where fractional values make no sense and a ‘sum’ of consolidated values is appropriate.

&target=consolidateBy(Sales.widgets.largeBlue, 'sum')
&target=consolidateBy(Servers.web01.sda1.free_space, 'max')
constantLine(value)

Takes a float F.

Draws a horizontal line at value F across the graph.

Example:

&target=constantLine(123.456)
countSeries(*seriesLists)

Draws a horizontal line representing the number of nodes found in the seriesList.

&target=countSeries(carbon.agents.*.*)
cumulative(seriesList)

Takes one metric or a wildcard seriesList.

When a graph is drawn where width of the graph size in pixels is smaller than the number of datapoints to be graphed, Graphite consolidates the values to to prevent line overlap. The cumulative() function changes the consolidation function from the default of ‘average’ to ‘sum’. This is especially useful in sales graphs, where fractional values make no sense and a ‘sum’ of consolidated values is appropriate.

Alias for consolidateBy(series, 'sum')

&target=cumulative(Sales.widgets.largeBlue)
currentAbove(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant N. Out of all metrics passed, draws only the metrics whose value is above N at the end of the time period specified.

Example:

&target=currentAbove(server*.instance*.threads.busy,50)

Draws the servers with more than 50 busy threads.

currentBelow(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant N. Out of all metrics passed, draws only the metrics whose value is below N at the end of the time period specified.

Example:

&target=currentBelow(server*.instance*.threads.busy,3)

Draws the servers with less than 3 busy threads.

dashed(seriesList, dashLength=5)

Takes one metric or a wildcard seriesList, followed by a float F.

Draw the selected metrics with a dotted line with segments of length F If omitted, the default length of the segments is 5.0

Example:

&target=dashed(server01.instance01.memory.free,2.5)
delay(seriesList, steps)

This shifts all samples later by an integer number of steps. This can be used for custom derivative calculations, among other things. Note: this will pad the early end of the data with None for every step shifted.

This complements other time-displacement functions such as timeShift and timeSlice, in that this function is indifferent about the step intervals being shifted.

Example:

&target=divideSeries(server.FreeSpace,delay(server.FreeSpace,1))

This computes the change in server free space as a percentage of the previous free space.

derivative(seriesList)

This is the opposite of the integral function. This is useful for taking a running total metric and calculating the delta between subsequent data points.

This function does not normalize for periods of time, as a true derivative would. Instead see the perSecond() function to calculate a rate of change over time.

Example:

&target=derivative(company.server.application01.ifconfig.TXPackets)

Each time you run ifconfig, the RX and TXPackets are higher (assuming there is network traffic.) By applying the derivative function, you can get an idea of the packets per minute sent or received, even though you’re only recording the total.

diffSeries(*seriesLists)

Subtracts series 2 through n from series 1.

Example:

&target=diffSeries(service.connections.total,service.connections.failed)

To diff a series and a constant, one should use offset instead of (or in addition to) diffSeries

Example:

&target=offset(service.connections.total,-5)

&target=offset(diffSeries(service.connections.total,service.connections.failed),-4)

This is an alias for aggregate with aggregation diff.

diffSeriesLists(seriesListFirstPos, seriesListSecondPos)

Iterates over a two lists and subtracts series lists 2 through n from series 1 list1[0] to list2[0], list1[1] to list2[1] and so on. The lists will need to be the same length

Example:

&target=diffSeriesLists(mining.{carbon,graphite,diamond}.extracted,mining.{carbon,graphite,diamond}.shipped)

An example above would be the same as running diffSeries for each member of the list:

?target=diffSeries(mining.carbon.extracted,mining.carbon.shipped)
&target=diffSeries(mining.graphite.extracted,mining.graphite.shipped)
&target=diffSeries(mining.diamond.extracted,mining.diamond.shipped)

This is an alias for aggregateSeriesLists with aggregation diff.

divideSeries(dividendSeriesList, divisorSeries)

Takes a dividend metric and a divisor metric and draws the division result. A constant may not be passed. To divide by a constant, use the scale() function (which is essentially a multiplication operation) and use the inverse of the dividend. (Division by 8 = multiplication by 1/8 or 0.125)

Example:

&target=divideSeries(Series.dividends,Series.divisors)
divideSeriesLists(dividendSeriesList, divisorSeriesList)

Iterates over a two lists and divides list1[0] by list2[0], list1[1] by list2[1] and so on. The lists need to be the same length

drawAsInfinite(seriesList)

Takes one metric or a wildcard seriesList. If the value is zero, draw the line at 0. If the value is above zero, draw the line at infinity. If the value is null or less than zero, do not draw the line.

Useful for displaying on/off metrics, such as exit codes. (0 = success, anything else = failure.)

Example:

drawAsInfinite(Testing.script.exitCode)
events(*tags)

Returns the number of events at this point in time. Usable with drawAsInfinite.

Example:

&target=events("tag-one", "tag-two")
&target=events("*")

Returns all events tagged as “tag-one” and “tag-two” and the second one returns all events.

exclude(seriesList, pattern)

Takes a metric or a wildcard seriesList, followed by a regular expression in double quotes. Excludes metrics that match the regular expression.

Example:

&target=exclude(servers*.instance*.threads.busy,"server02")
exp(seriesList)

Raise e to the power of the datapoint, where e = 2.718281… is the base of natural logarithms.

Example:

&target=exp(Server.instance01.threads.busy)
exponentialMovingAverage(seriesList, windowSize)

Takes a series of values and a window size and produces an exponential moving average utilizing the following formula:

ema(current) = constant * (Current Value) + (1 - constant) * ema(previous)

The Constant is calculated as:

constant = 2 / (windowSize + 1)

The first period EMA uses a simple moving average for its value.

Example:

&target=exponentialMovingAverage(*.transactions.count, 10)
&target=exponentialMovingAverage(*.transactions.count, '-10s')
fallbackSeries(seriesList, fallback)

Takes a wildcard seriesList, and a second fallback metric. If the wildcard does not match any series, draws the fallback metric.

Example:

&target=fallbackSeries(server*.requests_per_second, constantLine(0))

Draws a 0 line when server metric does not exist.

filterSeries(seriesList, func, operator, threshold)

Takes one metric or a wildcard seriesList followed by a consolidation function, an operator and a threshold. Draws only the metrics which match the filter expression.

Example:

&target=filterSeries(system.interface.eth*.packetsSent, 'max', '>', 1000)

This would only display interfaces which has a peak throughput higher than 1000 packets/min.

Supported aggregation functions: average, median, sum, min, max, diff, stddev, range, multiply & last.

Supported operators: =, !=, >, >=, < & <=.

grep(seriesList, pattern)

Takes a metric or a wildcard seriesList, followed by a regular expression in double quotes. Excludes metrics that don’t match the regular expression.

Example:

&target=grep(servers*.instance*.threads.busy,"server02")
group(*seriesLists)

Takes an arbitrary number of seriesLists and adds them to a single seriesList. This is used to pass multiple seriesLists to a function which only takes one

groupByNode(seriesList, nodeNum, callback='average')

Takes a serieslist and maps a callback to subgroups within as defined by a common node

&target=groupByNode(ganglia.by-function.*.*.cpu.load5,2,"sumSeries")

Would return multiple series which are each the result of applying the “sumSeries” function to groups joined on the second node (0 indexed) resulting in a list of targets like

sumSeries(ganglia.by-function.server1.*.cpu.load5),sumSeries(ganglia.by-function.server2.*.cpu.load5),...

Node may be an integer referencing a node in the series name or a string identifying a tag.

This is an alias for using groupByNodes with a single node.

groupByNodes(seriesList, callback, *nodes)

Takes a serieslist and maps a callback to subgroups within as defined by multiple nodes

&target=groupByNodes(ganglia.server*.*.cpu.load*,"sum",1,4)

Would return multiple series which are each the result of applying the “sum” aggregation to groups joined on the nodes’ list (0 indexed) resulting in a list of targets like

sumSeries(ganglia.server1.*.cpu.load5),sumSeries(ganglia.server1.*.cpu.load10),sumSeries(ganglia.server1.*.cpu.load15),sumSeries(ganglia.server2.*.cpu.load5),sumSeries(ganglia.server2.*.cpu.load10),sumSeries(ganglia.server2.*.cpu.load15),...

This function can be used with all aggregation functions supported by aggregate: average, median, sum, min, max, diff, stddev, range & multiply.

Each node may be an integer referencing a node in the series name or a string identifying a tag.

&target=seriesByTag("name=~cpu.load.*", "server=~server[1-9]+", "datacenter=~dc[1-9]+")|groupByNodes("average", "datacenter", 1)

# will produce output series like
# dc1.load5, dc2.load5, dc1.load10, dc2.load10

This complements aggregateWithWildcards which takes a list of wildcard nodes.

groupByTags(seriesList, callback, *tags)

Takes a serieslist and maps a callback to subgroups within as defined by multiple tags

&target=seriesByTag("name=cpu")|groupByTags("average","dc")

Would return multiple series which are each the result of applying the “averageSeries” function to groups joined on the specified tags resulting in a list of targets like

averageSeries(seriesByTag("name=cpu","dc=dc1")),averageSeries(seriesByTag("name=cpu","dc=dc2")),...

This function can be used with all aggregation functions supported by aggregate: average (or avg), avg_zero, median, sum (or total), min, max, diff, stddev, count, range (or rangeOf) , multiply & last (or current).

highest(seriesList, n=1, func='average')

Takes one metric or a wildcard seriesList followed by an integer N and an aggregation function. Out of all metrics passed, draws only the N metrics with the highest aggregated value over the time period specified.

Example:

&target=highest(server*.instance*.threads.busy,5,'max')

Draws the 5 servers with the highest number of busy threads.

highestAverage(seriesList, n)

Takes one metric or a wildcard seriesList followed by an integer N. Out of all metrics passed, draws only the top N metrics with the highest average value for the time period specified.

Example:

&target=highestAverage(server*.instance*.threads.busy,5)

Draws the top 5 servers with the highest average value.

This is an alias for highest with aggregation average.

highestCurrent(seriesList, n)

Takes one metric or a wildcard seriesList followed by an integer N. Out of all metrics passed, draws only the N metrics with the highest value at the end of the time period specified.

Example:

&target=highestCurrent(server*.instance*.threads.busy,5)

Draws the 5 servers with the highest busy threads.

This is an alias for highest with aggregation current.

highestMax(seriesList, n)

Takes one metric or a wildcard seriesList followed by an integer N.

Out of all metrics passed, draws only the N metrics with the highest maximum value in the time period specified.

Example:

&target=highestMax(server*.instance*.threads.busy,5)

Draws the top 5 servers who have had the most busy threads during the time period specified.

This is an alias for highest with aggregation max.

hitcount(seriesList, intervalString, alignToInterval=False)

Estimate hit counts from a list of time series.

This function assumes the values in each time series represent hits per second. It calculates hits per some larger interval such as per day or per hour. This function is like summarize(), except that it compensates automatically for different time scales (so that a similar graph results from using either fine-grained or coarse-grained records) and handles rarely-occurring events gracefully.

holtWintersAberration(seriesList, delta=3, bootstrapInterval='7d', seasonality='1d')

Performs a Holt-Winters forecast using the series as input data and plots the positive or negative deviation of the series data from the forecast.

holtWintersConfidenceArea(seriesList, delta=3, bootstrapInterval='7d', seasonality='1d')

Performs a Holt-Winters forecast using the series as input data and plots the area between the upper and lower bands of the predicted forecast deviations.

holtWintersConfidenceBands(seriesList, delta=3, bootstrapInterval='7d', seasonality='1d')

Performs a Holt-Winters forecast using the series as input data and plots upper and lower bands with the predicted forecast deviations.

holtWintersForecast(seriesList, bootstrapInterval='7d', seasonality='1d')

Performs a Holt-Winters forecast using the series as input data. Data from bootstrapInterval (one week by default) previous to the series is used to bootstrap the initial forecast.

identity(name)

Identity function: Returns datapoints where the value equals the timestamp of the datapoint. Useful when you have another series where the value is a timestamp, and you want to compare it to the time of the datapoint, to render an age

Example:

&target=identity("The.time.series")

This would create a series named “The.time.series” that contains points where x(t) == t.

integral(seriesList)

This will show the sum over time, sort of like a continuous addition function. Useful for finding totals or trends in metrics that are collected per minute.

Example:

&target=integral(company.sales.perMinute)

This would start at zero on the left side of the graph, adding the sales each minute, and show the total sales for the time period selected at the right side, (time now, or the time specified by ‘&until=’).

integralByInterval(seriesList, intervalUnit)

This will do the same as integral() funcion, except resetting the total to 0 at the given time in the parameter “from” Useful for finding totals per hour/day/week/..

Example:

&target=integralByInterval(company.sales.perMinute, "1d")&from=midnight-10days

This would start at zero on the left side of the graph, adding the sales each minute, and show the evolution of sales per day during the last 10 days.

interpolate(seriesList, limit=inf)

Takes one metric or a wildcard seriesList, and optionally a limit to the number of ‘None’ values to skip over. Continues the line with the last received value when gaps (‘None’ values) appear in your data, rather than breaking your line.

Example:

&target=interpolate(Server01.connections.handled)
&target=interpolate(Server01.connections.handled, 10)
invert(seriesList)

Takes one metric or a wildcard seriesList, and inverts each datapoint (i.e. 1/x).

Example:

&target=invert(Server.instance01.threads.busy)
isNonNull(seriesList)

Takes a metric or wildcard seriesList and counts up the number of non-null values. This is useful for understanding the number of metrics that have data at a given point in time (i.e. to count which servers are alive).

Example:

&target=isNonNull(webapp.pages.*.views)

Returns a seriesList where 1 is specified for non-null values, and 0 is specified for null values.

keepLastValue(seriesList, limit=inf)

Takes one metric or a wildcard seriesList, and optionally a limit to the number of ‘None’ values to skip over. Continues the line with the last received value when gaps (‘None’ values) appear in your data, rather than breaking your line.

Example:

&target=keepLastValue(Server01.connections.handled)
&target=keepLastValue(Server01.connections.handled, 10)
legendValue(seriesList, *valueTypes)

Takes one metric or a wildcard seriesList and a string in quotes. Appends a value to the metric name in the legend. Currently one or several of: last, avg, total, min, max. The last argument can be si (default) or binary, in that case values will be formatted in the corresponding system.

&target=legendValue(Sales.widgets.largeBlue, 'avg', 'max', 'si')
limit(seriesList, n)

Takes one metric or a wildcard seriesList followed by an integer N.

Only draw the first N metrics. Useful when testing a wildcard in a metric.

Example:

&target=limit(server*.instance*.memory.free,5)

Draws only the first 5 instance’s memory free.

lineWidth(seriesList, width)

Takes one metric or a wildcard seriesList, followed by a float F.

Draw the selected metrics with a line width of F, overriding the default value of 1, or the &lineWidth=X.X parameter.

Useful for highlighting a single metric out of many, or having multiple line widths in one graph.

Example:

&target=lineWidth(server01.instance01.memory.free,5)
linearRegression(seriesList, startSourceAt=None, endSourceAt=None)

Graphs the linear regression function by least squares method.

Takes one metric or a wildcard seriesList, followed by a quoted string with the time to start the line and another quoted string with the time to end the line. The start and end times are inclusive (default range is from to until). See from / until in the Render API for examples of time formats. Datapoints in the range is used to regression.

Example:

&target=linearRegression(Server.instance01.threads.busy, '-1d')
&target=linearRegression(Server.instance*.threads.busy, "00:00 20140101","11:59 20140630")
logarithm(seriesList, base=10)

Takes one metric or a wildcard seriesList, a base, and draws the y-axis in logarithmic format. If base is omitted, the function defaults to base 10.

Example:

&target=log(carbon.agents.hostname.avgUpdateTime,2)
logit(seriesList)

Takes one metric or a wildcard seriesList and applies the logit function log(x / (1 - x)) to each datapoint.

Example:

&target=logit(Server.instance01.threads.busy)
&target=logit(Server.instance*.threads.busy)
lowest(seriesList, n=1, func='average')

Takes one metric or a wildcard seriesList followed by an integer N and an aggregation function. Out of all metrics passed, draws only the N metrics with the lowest aggregated value over the time period specified.

Example:

&target=lowest(server*.instance*.threads.busy,5,'min')

Draws the 5 servers with the lowest number of busy threads.

lowestAverage(seriesList, n)

Takes one metric or a wildcard seriesList followed by an integer N. Out of all metrics passed, draws only the bottom N metrics with the lowest average value for the time period specified.

Example:

&target=lowestAverage(server*.instance*.threads.busy,5)

Draws the bottom 5 servers with the lowest average value.

This is an alias for lowest with aggregation average.

lowestCurrent(seriesList, n)

Takes one metric or a wildcard seriesList followed by an integer N. Out of all metrics passed, draws only the N metrics with the lowest value at the end of the time period specified.

Example:

&target=lowestCurrent(server*.instance*.threads.busy,5)

Draws the 5 servers with the least busy threads right now.

This is an alias for lowest with aggregation current.

mapSeries(seriesList, *mapNodes)

Short form: map()

Takes a seriesList and maps it to a list of seriesList. Each seriesList has the given mapNodes in common.

Note

This function is not very useful alone. It should be used with reduceSeries()

mapSeries(servers.*.cpu.*,1) =>

  [
    servers.server1.cpu.*,
    servers.server2.cpu.*,
    ...
    servers.serverN.cpu.*
  ]

Each node may be an integer referencing a node in the series name or a string identifying a tag.

maxSeries(*seriesLists)

Takes one metric or a wildcard seriesList. For each datapoint from each metric passed in, pick the maximum value and graph it.

Example:

&target=maxSeries(Server*.connections.total)

This is an alias for aggregate with aggregation max.

maximumAbove(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant n. Draws only the metrics with a maximum value above n.

Example:

&target=maximumAbove(system.interface.eth*.packetsSent,1000)

This would only display interfaces which sent more than 1000 packets/min.

maximumBelow(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant n. Draws only the metrics with a maximum value below n.

Example:

&target=maximumBelow(system.interface.eth*.packetsSent,1000)

This would only display interfaces which sent less than 1000 packets/min.

minMax(seriesList)

Applies the popular min max normalization technique, which takes each point and applies the following normalization transformation to it: normalized = (point - min) / (max - min).

Example:

&target=minMax(Server.instance01.threads.busy)
minSeries(*seriesLists)

Takes one metric or a wildcard seriesList. For each datapoint from each metric passed in, pick the minimum value and graph it.

Example:

&target=minSeries(Server*.connections.total)

This is an alias for aggregate with aggregation min.

minimumAbove(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant n. Draws only the metrics with a minimum value above n.

Example:

&target=minimumAbove(system.interface.eth*.packetsSent,1000)

This would only display interfaces which sent more than 1000 packets/min.

minimumBelow(seriesList, n)

Takes one metric or a wildcard seriesList followed by a constant n. Draws only the metrics with a minimum value below or equal to n.

Example:

&target=minimumBelow(system.interface.eth*.packetsSent,1000)

This would only display interfaces which at one point sent less than 1000 packets/min.

mostDeviant(seriesList, n)

Takes one metric or a wildcard seriesList followed by an integer N. Draws the N most deviant metrics. To find the deviants, the standard deviation (sigma) of each series is taken and ranked. The top N standard deviations are returned.

Example:
&target=mostDeviant(server*.instance*.memory.free, 5)

Draws the 5 instances furthest from the average memory free.

movingAverage(seriesList, windowSize, xFilesFactor=None)

Graphs the moving average of a metric (or metrics) over a fixed number of past points, or a time interval.

Takes one metric or a wildcard seriesList followed by a number N of datapoints or a quoted string with a length of time like ‘1hour’ or ‘5min’ (See from / until in the Render API for examples of time formats), and an xFilesFactor value to specify how many points in the window must be non-null for the output to be considered valid. Graphs the average of the preceeding datapoints for each point on the graph.

Example:

&target=movingAverage(Server.instance01.threads.busy,10)
&target=movingAverage(Server.instance*.threads.idle,'5min')
movingMax(seriesList, windowSize, xFilesFactor=None)

Graphs the moving maximum of a metric (or metrics) over a fixed number of past points, or a time interval.

Takes one metric or a wildcard seriesList followed by a number N of datapoints or a quoted string with a length of time like ‘1hour’ or ‘5min’ (See from / until in the Render API for examples of time formats), and an xFilesFactor value to specify how many points in the window must be non-null for the output to be considered valid. Graphs the maximum of the preceeding datapoints for each point on the graph.

Example:

&target=movingMax(Server.instance01.requests,10)
&target=movingMax(Server.instance*.errors,'5min')
movingMedian(seriesList, windowSize, xFilesFactor=None)

Graphs the moving median of a metric (or metrics) over a fixed number of past points, or a time interval.

Takes one metric or a wildcard seriesList followed by a number N of datapoints or a quoted string with a length of time like ‘1hour’ or ‘5min’ (See from / until in the Render API for examples of time formats), and an xFilesFactor value to specify how many points in the window must be non-null for the output to be considered valid. Graphs the median of the preceeding datapoints for each point on the graph.

Example:

&target=movingMedian(Server.instance01.threads.busy,10)
&target=movingMedian(Server.instance*.threads.idle,'5min')
movingMin(seriesList, windowSize, xFilesFactor=None)

Graphs the moving minimum of a metric (or metrics) over a fixed number of past points, or a time interval.

Takes one metric or a wildcard seriesList followed by a number N of datapoints or a quoted string with a length of time like ‘1hour’ or ‘5min’ (See from / until in the Render API for examples of time formats), and an xFilesFactor value to specify how many points in the window must be non-null for the output to be considered valid. Graphs the minimum of the preceeding datapoints for each point on the graph.

Example:

&target=movingMin(Server.instance01.requests,10)
&target=movingMin(Server.instance*.errors,'5min')
movingSum(seriesList, windowSize, xFilesFactor=None)

Graphs the moving sum of a metric (or metrics) over a fixed number of past points, or a time interval.

Takes one metric or a wildcard seriesList followed by a number N of datapoints or a quoted string with a length of time like ‘1hour’ or ‘5min’ (See from / until in the Render API for examples of time formats), and an xFilesFactor value to specify how many points in the window must be non-null for the output to be considered valid. Graphs the sum of the preceeding datapoints for each point on the graph.

Example:

&target=movingSum(Server.instance01.requests,10)
&target=movingSum(Server.instance*.errors,'5min')
movingWindow(seriesList, windowSize, func='average', xFilesFactor=None)

Graphs a moving window function of a metric (or metrics) over a fixed number of past points, or a time interval.

Takes one metric or a wildcard seriesList, a number N of datapoints or a quoted string with a length of time like ‘1hour’ or ‘5min’ (See from / until in the Render API for examples of time formats), a function to apply to the points in the window to produce the output, and an xFilesFactor value to specify how many points in the window must be non-null for the output to be considered valid. Graphs the output of the function for the preceeding datapoints for each point on the graph.

Example:

&target=movingWindow(Server.instance01.threads.busy,10)
&target=movingWindow(Server.instance*.threads.idle,'5min','median',0.5)

Note

xFilesFactor follows the same semantics as in Whisper storage schemas. Setting it to 0 (the default) means that only a single value in a given interval needs to be non-null, setting it to 1 means that all values in the interval must be non-null. A setting of 0.5 means that at least half the values in the interval must be non-null.

multiplySeries(*seriesLists)

Takes two or more series and multiplies their points. A constant may not be used. To multiply by a constant, use the scale() function.

Example:

&target=multiplySeries(Series.dividends,Series.divisors)

This is an alias for aggregate with aggregation multiply.

multiplySeriesLists(seriesListFirstPos, seriesListSecondPos)

Iterates over a two lists and subtracts series lists 2 through n from series 1 list1[0] to list2[0], list1[1] to list2[1] and so on. The lists will need to be the same length

Example:

&target=multiplySeriesLists(mining.{carbon,graphite,diamond}.extracted,mining.{carbon,graphite,diamond}.shipped)

An example above would be the same as running multiplySeries for each member of the list:

?target=multiplySeries(mining.carbon.extracted,mining.carbon.shipped)
&target=multiplySeries(mining.graphite.extracted,mining.graphite.shipped)
&target=multiplySeries(mining.diamond.extracted,mining.diamond.shipped)

This is an alias for aggregateSeriesLists with aggregation multiply.

multiplySeriesWithWildcards(seriesList, *position)

Categorizes the provided series in groups by name, by ignoring (“wildcarding”) the given position(s) and calls multiplySeries on each group. Important: the introduction of wildcards only happens after retrieving the input.

Example:

&target=multiplySeriesWithWildcards(web.host-[0-7].{avg-response,total-request}.value, 2)

This would be the equivalent of

&target=multiplySeries(web.host-0.{avg-response,total-request}.value)&target=multiplySeries(web.host-1.{avg-response,total-request}.value)...

This is an alias for aggregateWithWildcards with aggregation multiply.

nPercentile(seriesList, n)

Returns n-percent of each series in the seriesList.

nonNegativeDerivative(seriesList, maxValue=None, minValue=None)

Same as the derivative function above, but ignores datapoints that trend down. Useful for counters that increase for a long time, then wrap or reset. (Such as if a network interface is destroyed and recreated by unloading and re-loading a kernel module, common with USB / WiFi cards.

By default, a null value is returned in place of negative datapoints. When maxValue is supplied, the missing value is computed as if the counter had wrapped at maxValue. When minValue is supplied, the missing value is computed as if the counter had wrapped to minValue.

Example:

&target=nonNegativederivative(company.server.application01.ifconfig.TXPackets)
offset(seriesList, factor)

Takes one metric or a wildcard seriesList followed by a constant, and adds the constant to each datapoint.

Example:

&target=offset(Server.instance01.threads.busy,10)
offsetToZero(seriesList)

Offsets a metric or wildcard seriesList by subtracting the minimum value in the series from each datapoint.

Useful to compare different series where the values in each series may be higher or lower on average but you’re only interested in the relative difference.

An example use case is for comparing different round trip time results. When measuring RTT (like pinging a server), different devices may come back with consistently different results due to network latency which will be different depending on how many network hops between the probe and the device. To compare different devices in the same graph, the network latency to each has to be factored out of the results. This is a shortcut that takes the fastest response (lowest number in the series) and sets that to zero and then offsets all of the other datapoints in that series by that amount. This makes the assumption that the lowest response is the fastest the device can respond, of course the more datapoints that are in the series the more accurate this assumption is.

Example:

&target=offsetToZero(Server.instance01.responseTime)
&target=offsetToZero(Server.instance*.responseTime)
perSecond(seriesList, maxValue=None, minValue=None)

NonNegativeDerivative adjusted for the series time interval This is useful for taking a running total metric and showing how many requests per second were handled.

The optional minValue and maxValue parameters have the same meaning as in nonNegativeDerivative.

Example:

&target=perSecond(company.server.application01.ifconfig.TXPackets)

Each time you run ifconfig, the RX and TXPackets are higher (assuming there is network traffic.) By applying the perSecond function, you can get an idea of the packets per second sent or received, even though you’re only recording the total.

percentileOfSeries(seriesList, n, interpolate=False)

percentileOfSeries returns a single series which is composed of the n-percentile values taken across a wildcard series at each point. Unless interpolate is set to True, percentile values are actual values contained in one of the supplied series.

pieAverage(series)

Return the average

pieMaximum(series)

Return the maximum

pieMinimum(series)

Return the minimum

pow(seriesList, factor)

Takes one metric or a wildcard seriesList followed by a constant, and raises the datapoint by the power of the constant provided at each point.

Example:

&target=pow(Server.instance01.threads.busy,10)
&target=pow(Server.instance*.threads.busy,10)
powSeries(*seriesLists)

Takes two or more series and pows their points. A constant line may be used.

Example:

&target=powSeries(Server.instance01.app.requests, Server.instance01.app.replies)
randomWalkFunction(name, step=60)

Short Alias: randomWalk()

Returns a random walk starting at 0. This is great for testing when there is no real data in whisper.

Example:

&target=randomWalk("The.time.series")

This would create a series named “The.time.series” that contains points where x(t) == x(t-1)+random()-0.5, and x(0) == 0. Accepts optional second argument as ‘step’ parameter (default step is 60 sec)

rangeOfSeries(*seriesLists)

Takes a wildcard seriesList. Distills down a set of inputs into the range of the series

Example:

&target=rangeOfSeries(Server*.connections.total)

This is an alias for aggregate with aggregation rangeOf.

reduceSeries(seriesLists, reduceFunction, reduceNode, *reduceMatchers)

Short form: reduce()

Takes a list of seriesLists and reduces it to a list of series by means of the reduceFunction.

Reduction is performed by matching the reduceNode in each series against the list of reduceMatchers. Then each series is passed to the reduceFunction as arguments in the order given by reduceMatchers. The reduceFunction should yield a single series.

The resulting list of series are aliased so that they can easily be nested in other functions.

Example: Map/Reduce asPercent(bytes_used,total_bytes) for each server

Assume that metrics in the form below exist:

servers.server1.disk.bytes_used
servers.server1.disk.total_bytes
servers.server2.disk.bytes_used
servers.server2.disk.total_bytes
servers.server3.disk.bytes_used
servers.server3.disk.total_bytes
...
servers.serverN.disk.bytes_used
servers.serverN.disk.total_bytes

To get the percentage of disk used for each server:

reduceSeries(mapSeries(servers.*.disk.*,1),"asPercent",3,"bytes_used","total_bytes") =>

  alias(asPercent(servers.server1.disk.bytes_used,servers.server1.disk.total_bytes),"servers.server1.disk.reduce.asPercent"),
  alias(asPercent(servers.server2.disk.bytes_used,servers.server2.disk.total_bytes),"servers.server2.disk.reduce.asPercent"),
  alias(asPercent(servers.server3.disk.bytes_used,servers.server3.disk.total_bytes),"servers.server3.disk.reduce.asPercent"),
  ...
  alias(asPercent(servers.serverN.disk.bytes_used,servers.serverN.disk.total_bytes),"servers.serverN.disk.reduce.asPercent")

In other words, we will get back the following metrics:

servers.server1.disk.reduce.asPercent
servers.server2.disk.reduce.asPercent
servers.server3.disk.reduce.asPercent
...
servers.serverN.disk.reduce.asPercent

See also

mapSeries()

removeAbovePercentile(seriesList, n)

Removes data above the nth percentile from the series or list of series provided. Values above this percentile are assigned a value of None.

removeAboveValue(seriesList, n)

Removes data above the given threshold from the series or list of series provided. Values above this threshold are assigned a value of None.

removeBelowPercentile(seriesList, n)

Removes data below the nth percentile from the series or list of series provided. Values below this percentile are assigned a value of None.

removeBelowValue(seriesList, n)

Removes data below the given threshold from the series or list of series provided. Values below this threshold are assigned a value of None.

removeBetweenPercentile(seriesList, n)

Removes series that do not have an value lying in the x-percentile of all the values at a moment

removeEmptySeries(seriesList, xFilesFactor=None)

Takes one metric or a wildcard seriesList. Out of all metrics passed, draws only the metrics with not empty data

Example:

&target=removeEmptySeries(server*.instance*.threads.busy)

Draws only live servers with not empty data.

xFilesFactor follows the same semantics as in Whisper storage schemas. Setting it to 0 (the default) means that only a single value in the series needs to be non-null for it to be considered non-empty, setting it to 1 means that all values in the series must be non-null. A setting of 0.5 means that at least half the values in the series must be non-null.

roundFunction(seriesList, precision=None)

Takes one metric or a wildcard seriesList optionally followed by a precision, and rounds each datapoint to the specified precision.

Example:

&target=round(Server.instance01.threads.busy)
&target=round(Server.instance01.threads.busy,2)
scale(seriesList, factor)

Takes one metric or a wildcard seriesList followed by a constant, and multiplies the datapoint by the constant provided at each point.

Example:

&target=scale(Server.instance01.threads.busy,10)
&target=scale(Server.instance*.threads.busy,10)
scaleToSeconds(seriesList, seconds)

Takes one metric or a wildcard seriesList and returns “value per seconds” where seconds is a last argument to this functions.

Useful in conjunction with derivative or integral function if you want to normalize its result to a known resolution for arbitrary retentions

secondYAxis(seriesList)

Graph the series on the secondary Y axis.

seriesByTag(*tagExpressions)

Returns a SeriesList of series matching all the specified tag expressions.

Example:

&target=seriesByTag("tag1=value1","tag2!=value2")

Returns a seriesList of all series that have tag1 set to value1, AND do not have tag2 set to value2.

Tags specifiers are strings, and may have the following formats:

tag=spec    tag value exactly matches spec
tag!=spec   tag value does not exactly match spec
tag=~value  tag value matches the regular expression spec
tag!=~spec  tag value does not match the regular expression spec

Any tag spec that matches an empty value is considered to match series that don’t have that tag.

At least one tag spec must require a non-empty value.

Regular expression conditions are treated as being anchored at the start of the value.

See querying tagged series for more detail.

setXFilesFactor(seriesList, xFilesFactor)

Short form: xFilesFactor()

Takes one metric or a wildcard seriesList and an xFilesFactor value between 0 and 1

When a series needs to be consolidated, this sets the fraction of values in an interval that must not be null for the consolidation to be considered valid. If there are not enough values then None will be returned for that interval.

&target=xFilesFactor(Sales.widgets.largeBlue, 0.5)
&target=Servers.web01.sda1.free_space|consolidateBy('max')|xFilesFactor(0.5)

The xFilesFactor set via this function is used as the default for all functions that accept an xFilesFactor parameter, all functions that aggregate data across multiple series and/or intervals, and maxDataPoints consolidation.

A default for the entire render request can also be set using the xFilesFactor query parameter.

Note

xFilesFactor follows the same semantics as in Whisper storage schemas. Setting it to 0 (the default) means that only a single value in a given interval needs to be non-null, setting it to 1 means that all values in the interval must be non-null. A setting of 0.5 means that at least half the values in the interval must be non-null.

sigmoid(seriesList)

Takes one metric or a wildcard seriesList and applies the sigmoid function 1 / (1 + exp(-x)) to each datapoint.

Example:

&target=sigmoid(Server.instance01.threads.busy)
&target=sigmoid(Server.instance*.threads.busy)
sinFunction(name, amplitude=1, step=60)

Short Alias: sin()

Just returns the sine of the current time. The optional amplitude parameter changes the amplitude of the wave.

Example:

&target=sin("The.time.series", 2)

This would create a series named “The.time.series” that contains sin(x)*2. Accepts optional second argument as ‘amplitude’ parameter (default amplitude is 1) Accepts optional third argument as ‘step’ parameter (default step is 60 sec)

smartSummarize(seriesList, intervalString, func='sum', alignTo=None)

Smarter version of summarize.

The alignToFrom boolean parameter has been replaced by alignTo and no longer has any effect. Alignment can be to years, months, weeks, days, hours, and minutes.

This function can be used with aggregation functions average, median, sum, min, max, diff, stddev, count, range, multiply & last.

sortBy(seriesList, func='average', reverse=False)

Takes one metric or a wildcard seriesList followed by an aggregation function and an optional reverse parameter.

Returns the metrics sorted according to the specified function.

Example:

&target=sortBy(server*.instance*.threads.busy,'max')

Draws the servers in ascending order by maximum.

sortByMaxima(seriesList)

Takes one metric or a wildcard seriesList.

Sorts the list of metrics in descending order by the maximum value across the time period specified. Useful with the &areaMode=all parameter, to keep the lowest value lines visible.

Example:

&target=sortByMaxima(server*.instance*.memory.free)
sortByMinima(seriesList)

Takes one metric or a wildcard seriesList.

Sorts the list of metrics by the lowest value across the time period specified, including only series that have a maximum value greater than 0.

Example:

&target=sortByMinima(server*.instance*.memory.free)
sortByName(seriesList, natural=False, reverse=False)

Takes one metric or a wildcard seriesList. Sorts the list of metrics by the metric name using either alphabetical order or natural sorting. Natural sorting allows names containing numbers to be sorted more naturally, e.g: - Alphabetical sorting: server1, server11, server12, server2 - Natural sorting: server1, server2, server11, server12

sortByTotal(seriesList)

Takes one metric or a wildcard seriesList.

Sorts the list of metrics in descending order by the sum of values across the time period specified.

squareRoot(seriesList)

Takes one metric or a wildcard seriesList, and computes the square root of each datapoint.

Example:

&target=squareRoot(Server.instance01.threads.busy)
stacked(seriesLists, stackName='__DEFAULT__')

Takes one metric or a wildcard seriesList and change them so they are stacked. This is a way of stacking just a couple of metrics without having to use the stacked area mode (that stacks everything). By means of this a mixed stacked and non stacked graph can be made

It can also take an optional argument with a name of the stack, in case there is more than one, e.g. for input and output metrics.

Example:

&target=stacked(company.server.application01.ifconfig.TXPackets, 'tx')
stddevSeries(*seriesLists)

Takes one metric or a wildcard seriesList. Draws the standard deviation of all metrics passed at each time.

Example:

&target=stddevSeries(company.server.*.threads.busy)

This is an alias for aggregate with aggregation stddev.

stdev(seriesList, points, windowTolerance=0.1)

Takes one metric or a wildcard seriesList followed by an integer N. Draw the Standard Deviation of all metrics passed for the past N datapoints. If the ratio of null points in the window is greater than windowTolerance, skip the calculation. The default for windowTolerance is 0.1 (up to 10% of points in the window can be missing). Note that if this is set to 0.0, it will cause large gaps in the output anywhere a single point is missing.

Example:

&target=stdev(server*.instance*.threads.busy,30)
&target=stdev(server*.instance*.cpu.system,30,0.0)
substr(seriesList, start=0, stop=0)

Takes one metric or a wildcard seriesList followed by 1 or 2 integers. Assume that the metric name is a list or array, with each element separated by dots. Prints n - length elements of the array (if only one integer n is passed) or n - m elements of the array (if two integers n and m are passed). The list starts with element 0 and ends with element (length - 1).

Example:

&target=substr(carbon.agents.hostname.avgUpdateTime,2,4)

The label would be printed as “hostname.avgUpdateTime”.

sumSeries(*seriesLists)

Short form: sum()

This will add metrics together and return the sum at each datapoint. (See integral for a sum over time)

Example:

&target=sum(company.server.application*.requestsHandled)

This would show the sum of all requests handled per minute (provided requestsHandled are collected once a minute). If metrics with different retention rates are combined, the coarsest metric is graphed, and the sum of the other metrics is averaged for the metrics with finer retention rates.

This is an alias for aggregate with aggregation sum.

sumSeriesLists(seriesListFirstPos, seriesListSecondPos)

Iterates over a two lists and subtracts series lists 2 through n from series 1 list1[0] to list2[0], list1[1] to list2[1] and so on. The lists will need to be the same length

Example:

&target=sumSeriesLists(mining.{carbon,graphite,diamond}.extracted,mining.{carbon,graphite,diamond}.shipped)

An example above would be the same as running sumSeries for each member of the list:

?target=sumSeries(mining.carbon.extracted,mining.carbon.shipped)
&target=sumSeries(mining.graphite.extracted,mining.graphite.shipped)
&target=sumSeries(mining.diamond.extracted,mining.diamond.shipped)

This is an alias for aggregateSeriesLists with aggregation sum.

sumSeriesWithWildcards(seriesList, *position)

Categorizes the provided series in groups by name, by ignoring (“wildcarding”) the given position(s) and calls sumSeries on each group. Important: the introduction of wildcards only happens after retrieving the input.

Example:

&target=sumSeriesWithWildcards(host.cpu-[0-7].cpu-{user,system}.value, 1)

This would be the equivalent of

&target=sumSeries(host.cpu-[0-7].cpu-user.value)&target=sumSeries(host.cpu-[0-7].cpu-system.value)

This is an alias for aggregateWithWildcards with aggregation sum.

summarize(seriesList, intervalString, func='sum', alignToFrom=False)

Summarize the data into interval buckets of a certain size.

By default, the contents of each interval bucket are summed together. This is useful for counters where each increment represents a discrete event and retrieving a “per X” value requires summing all the events in that interval.

Specifying ‘average’ instead will return the mean for each bucket, which can be more useful when the value is a gauge that represents a certain value in time.

This function can be used with aggregation functions average, median, sum, min, max, diff, stddev, count, range, multiply & last.

By default, buckets are calculated by rounding to the nearest interval. This works well for intervals smaller than a day. For example, 22:32 will end up in the bucket 22:00-23:00 when the interval=1hour.

Passing alignToFrom=true will instead create buckets starting at the from time. In this case, the bucket for 22:32 depends on the from time. If from=6:30 then the 1hour bucket for 22:32 is 22:30-23:30.

Example:

&target=summarize(counter.errors, "1hour") # total errors per hour
&target=summarize(nonNegativeDerivative(gauge.num_users), "1week") # new users per week
&target=summarize(queue.size, "1hour", "avg") # average queue size per hour
&target=summarize(queue.size, "1hour", "max") # maximum queue size during each hour
&target=summarize(metric, "13week", "avg", true)&from=midnight+20100101 # 2010 Q1-4
threshold(value, label=None, color=None)

Takes a float F, followed by a label (in double quotes) and a color. (See bgcolor in the Render API for valid color names & formats.)

Draws a horizontal line at value F across the graph.

Example:

&target=threshold(123.456, "omgwtfbbq", "red")
timeFunction(name, step=60)

Short Alias: time()

Just returns the timestamp for each X value. T

Example:

&target=time("The.time.series")

This would create a series named “The.time.series” that contains in Y the same value (in seconds) as X. Accepts optional second argument as ‘step’ parameter (default step is 60 sec)

timeShift(seriesList, timeShift, resetEnd=True, alignDST=False)

Takes one metric or a wildcard seriesList, followed by a quoted string with the length of time (See from / until in the Render API for examples of time formats).

Draws the selected metrics shifted in time. If no sign is given, a minus sign ( - ) is implied which will shift the metric back in time. If a plus sign ( + ) is given, the metric will be shifted forward in time.

Will reset the end date range automatically to the end of the base stat unless resetEnd is False. Example case is when you timeshift to last week and have the graph date range set to include a time in the future, will limit this timeshift to pretend ending at the current time. If resetEnd is False, will instead draw full range including future time.

Because time is shifted by a fixed number of seconds, comparing a time period with DST to a time period without DST, and vice-versa, will result in an apparent misalignment. For example, 8am might be overlaid with 7am. To compensate for this, use the alignDST option.

Useful for comparing a metric against itself at a past periods or correcting data stored at an offset.

Example:

&target=timeShift(Sales.widgets.largeBlue,"7d")
&target=timeShift(Sales.widgets.largeBlue,"-7d")
&target=timeShift(Sales.widgets.largeBlue,"+1h")
timeSlice(seriesList, startSliceAt, endSliceAt='now')

Takes one metric or a wildcard metric, followed by a quoted string with the time to start the line and another quoted string with the time to end the line. The start and end times are inclusive. See from / until in the Render API for examples of time formats.

Useful for filtering out a part of a series of data from a wider range of data.

Example:

&target=timeSlice(network.core.port1,"00:00 20140101","11:59 20140630")
&target=timeSlice(network.core.port1,"12:00 20140630","now")
timeStack(seriesList, timeShiftUnit='1d', timeShiftStart=0, timeShiftEnd=7)

Takes one metric or a wildcard seriesList, followed by a quoted string with the length of time (See from / until in the Render API for examples of time formats). Also takes a start multiplier and end multiplier for the length of time

create a seriesList which is composed the original metric series stacked with time shifts starting time shifts from the start multiplier through the end multiplier

Useful for looking at history, or feeding into averageSeries or stddevSeries.

Example:

&target=timeStack(Sales.widgets.largeBlue,"1d",0,7)    # create a series for today and each of the previous 7 days
toLowerCase(seriesList, *pos)

Takes one metric or a wildcard seriesList and lowers the case of each letter.

Optionally, a letter position to lower case can be specified, in which case only the letter at the specified position gets lower-cased. The position parameter may be given multiple times. The position parameter may be negative to define a position relative to the end of the metric name.

toUpperCase(seriesList, *pos)

Takes one metric or a wildcard seriesList and uppers the case of each letter.

Optionally, a letter position to upper case can be specified, in which case only the letter at the specified position gets upper-cased. The position parameter may be given multiple times. The position parameter may be negative to define a position relative to the end of the metric name.

transformNull(seriesList, default=0, referenceSeries=None)

Takes a metric or wildcard seriesList and replaces null values with the value specified by default. The value 0 used if not specified. The optional referenceSeries, if specified, is a metric or wildcard series list that governs which time intervals nulls should be replaced. If specified, nulls are replaced only in intervals where a non-null is found for the same interval in any of referenceSeries. This method compliments the drawNullAsZero function in graphical mode, but also works in text-only mode.

Example:

&target=transformNull(webapp.pages.*.views,-1)

This would take any page that didn’t have values and supply negative 1 as a default. Any other numeric value may be used as well.

unique(*seriesLists)

Takes an arbitrary number of seriesLists and returns unique series, filtered by name.

Example:

&target=unique(mostDeviant(server.*.disk_free,5),lowestCurrent(server.*.disk_free,5))

Draws servers with low disk space, and servers with highly deviant disk space, but never the same series twice.

useSeriesAbove(seriesList, value, search, replace)

Compares the maximum of each series against the given value. If the series maximum is greater than value, the regular expression search and replace is applied against the series name to plot a related metric

e.g. given useSeriesAbove(ganglia.metric1.reqs,10,’reqs’,’time’), the response time metric will be plotted only when the maximum value of the corresponding request/s metric is > 10

&target=useSeriesAbove(ganglia.metric1.reqs,10,"reqs","time")
verticalLine(ts, label=None, color=None)

Takes a timestamp string ts.

Draws a vertical line at the designated timestamp with optional ‘label’ and ‘color’. Supported timestamp formats include both relative (e.g. -3h) and absolute (e.g. 16:00_20110501) strings, such as those used with from and until parameters. When set, the ‘label’ will appear in the graph legend.

Note: Any timestamps defined outside the requested range will raise a ‘ValueError’ exception.

Example:

&target=verticalLine("12:3420131108","event","blue")
&target=verticalLine("16:00_20110501","event")
&target=verticalLine("-5mins")
weightedAverage(seriesListAvg, seriesListWeight, *nodes)

Takes a series of average values and a series of weights and produces a weighted average for all values. The corresponding values should share one or more zero-indexed nodes and/or tags.

Example:

&target=weightedAverage(*.transactions.mean,*.transactions.count,0)

Each node may be an integer referencing a node in the series name or a string identifying a tag.

Function Plugins

Function plugins can define additional functions for use in render calls.

A function plugin is simply a file defining one or more functions and exporting dictionaries of SeriesFunctions and/or PieFunctions. When Graphite loads the plugin it will add functions in SeriesFunctions and/or PieFunctions to the list of available functions.

Each exposed function must accept at least a requestContext and seriesList parameter, and may accept additional parameters as needed.

requestContext will be a dictionary as defined in graphite.render.views.renderView(), seriesList will be a list of TimeSeries objects.

from graphite.functions.params import Param, ParamTypes

def toUpperCase(requestContext, seriesList):
  """Custom function that changes series names to UPPERCASE"""
  for series in seriesList:
    series.name = series.name.upper()
  return seriesList

# optionally set the group attribute
toUpperCase.group = 'Custom'
toUpperCase.params = [
  Param('seriesList', ParamTypes.seriesList, required=True),
]

SeriesFunctions = {
  'upper': toUpperCase,
}

Each function can have a docstring, .group, and .params attributes defined, these are used in the function API output as hints for query builders.

The .group attribute is the group name as a string, the .params attribute is a list of parameter definitions.

Each parameter definition is Param object, the Param constructor accepts the following arguments (note that requestContext is not included in the list of parameters):

  • name: The name of the parameter
  • paramtype: The parameter type, one of:
    • ParamTypes.aggFunc: An aggregation function name
    • ParamTypes.boolean: True/False
    • ParamTypes.date: A date specification
    • ParamTypes.float: A float value
    • ParamTypes.integer: An integer value
    • ParamTypes.interval: An interval specifier like 1h, 1d, etc
    • ParamTypes.intOrInterval: An integer or interval specifier
    • ParamTypes.node: A node number
    • ParamTypes.nodeOrTag: A node number or tag name
    • ParamTypes.series: A single series
    • ParamTypes.seriesList: A list of series
    • ParamTypes.seriesLists: A list of seriesLists
    • ParamTypes.string: A string value
    • ParamTypes.tag: A tag name
  • required: Set to True for required parameters
  • default: Default value for optional parameters
  • multiple: Set to True for parameters that accept multiple instances (defined with * in Python)
  • options: A list of available values for parameters that accept only a defined list
  • suggestions: A list of suggested values for parameters that accept free-form values

Custom plugin files may be placed in the /opt/graphite/webapp/graphite/functions/custom folder and will be loaded automatically when graphite starts.

To load a packaged function plugin module, add it to the FUNCTION_PLUGINS setting:

FUNCTION_PLUGINS = [
  'some.function_plugin',
]

Function API

You can use the HTTP api to get a list of available functions, or the details of a specific function.

To get a list of available functions:

$ curl -s "http://graphite/functions?pretty=1"

{
  "absolute": {
    "description": "<function description>",
    "function": "absolute(seriesList)",
    "group": "Transform",
    "module": "graphite.render.functions",
    "name": "absolute",
    "params": [
      {
        "name": "seriesList",
        "required": true,
        "type": "seriesList"
      }
    ]
  },
  <more functions...>
}

If the parameter grouped=1 is passed, the returned list will be organized by group:

$ curl -s "http://graphite/functions?pretty=1&grouped=1"

{
  "Alias": {
    <alias functions...>
  },
  <more groups...>
}

To get the definition of a specific function:

$ curl -s "http://graphite/functions/absolute?pretty=1"

{
  "description": "<function description>",
  "function": "absolute(seriesList)",
  "group": "Transform",
  "module": "graphite.render.functions",
  "name": "absolute",
  "params": [
    {
      "name": "seriesList",
      "required": true,
      "type": "seriesList"
    }
  ]
}