Filter Custom Metrics
You can filter custom metrics in the following ways:
Using the configurable patterns
Using
log
and identifying the custom metrics that are exceeding limits
After you identify those key custom metrics that must be received, use the new include
and exclude
filtering parameters to make sure you receive them before the metrics limit is hit.
Including and Excluding Metrics
Here is an example configuration entry that would be put into the agent configuration file, /opt/draios/etc/dragent.yaml
.
metrics_filter:
- include: test.*
- exclude: test.*
- include: haproxy.backend.*
- exclude: haproxy.*
- exclude: redis.*
Given the config entry above, this is the action for these metrics:
test.\*
: send
haproxy.backend.request
: send
haproxy.frontend.bytes
: drop
redis.keys
: drop
When the agent reads the metrics, they are filtered according to the configured filters and the filtering rule order. It is the first rule that matches is applied. Threfore, in this example, the inclusion item for test.\*
will be applied, and the second exclude'
rule for the same exact metric entry will be ignored.
Logging Accepted and Dropped Metrics
Logging is disabled by default. You can enable logging to see which metrics are accepted or dropped by adding the following configuration entry into the dragent.yaml
file:
metrics_excess_log: true
When logging of excess metrics is enabled, logging occurs at INFO-level,
every 30 seconds and lasts for 10 seconds. The entries that can be seen
in /opt/draios/logs/draios.log
will be formatted like this:
+/-[type] [metric included/excluded]: metric.name (filter: +/-[metric.filter])
+
indicates included
and -
indicates excluded
.
The first +
or -
, followed by type
provides an easy way to quickly scan the list of metrics and spot which are included or excluded .
The second entry specifies the metric type: statsd
, app\_check
, service\_check
, or jmx
.
A third entry spells out whether included
or excluded
, followed by the metric name. Finally, the last entry, in parentheses, gives information about the filter applied and its effect.
With this example filter rule set:
metrics_filter:
- include: mongo.statsd.net*
- exclude: mongo.statsd.*
You might see the following INFO-level log entries (timestamps stripped):
-[statsd] metric excluded: mongo.statsd.vsize (filter: -[mongo.statsd.*])
+[statsd] metric included: mongo.statsd.netIn (filter: +[mongo.statsd.net*])
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.