Filter Custom Metrics

You can filter custom metrics in the following ways:

• Using the configurable patterns

• Using log and identifying the custom metrics that are exceeding limits

After you identify those key custom metrics that must be received, use the new include and exclude filtering parameters to make sure you receive them before the metrics limit is hit.

Including and Excluding Metrics

Here is an example configuration entry that would be put into the agent configuration file, /opt/draios/etc/dragent.yaml.

metrics_filter:
  - include: test.*
  - exclude: test.*
  - include: haproxy.backend.*
  - exclude: haproxy.*
  - exclude: redis.*

Given the config entry above, this is the action for these metrics:

test.\* : send

haproxy.backend.request: send

haproxy.frontend.bytes : drop

redis.keys : drop

When the agent reads the metrics, they are filtered according to the configured filters and the filtering rule order. It is the first rule that matches is applied. Threfore, in this example, the inclusion item for test.\* will be applied, and the second exclude' rule for the same exact metric entry will be ignored.

Logging Accepted and Dropped Metrics

Logging is disabled by default. You can enable logging to see which metrics are accepted or dropped by adding the following configuration entry into the dragent.yaml file:

metrics_excess_log: true

When logging of excess metrics is enabled, logging occurs at INFO-level, every 30 seconds and lasts for 10 seconds. The entries that can be seen in /opt/draios/logs/draios.log will be formatted like this:

+/-[type] [metric included/excluded]: metric.name (filter: +/-[metric.filter])

+ indicates included and - indicates excluded.

The first + or -, followed by type provides an easy way to quickly scan the list of metrics and spot which are included or excluded .

The second entry specifies the metric type: statsd, app\_check, service\_check, or jmx.

A third entry spells out whether included or excluded, followed by the metric name. Finally, the last entry, in parentheses, gives information about the filter applied and its effect.

With this example filter rule set:

metrics_filter:
  - include: mongo.statsd.net*
  - exclude: mongo.statsd.*

You might see the following INFO-level log entries (timestamps stripped):

-[statsd] metric excluded: mongo.statsd.vsize (filter: -[mongo.statsd.*])
+[statsd] metric included: mongo.statsd.netIn (filter: +[mongo.statsd.net*])