Limit Prometheus Metric Collection

The metric limit is dictated by the Sysdig backend and the enforced limit is consumed by the Sysdig agent. Additionally, the agent also imposes a limit on the number of metrics read from a Prometheus metric endpoint transmitted to the metric store. As an administrator, you have the ability to override the limit by configuring the agent’s dragent.yaml file provided that the value does not exceed the Sysdig metric limits.

Metrics Limit

As of agent v10.0.0, the new metrics limit per agent is:

  • Custom metrics: 10,000

  • Prometheus metrics: 8000

    Set other custom metric limits to zero to increase the Prometheus metrics limit to 10,000.

Configuring the Agent

The limit on the number of Prometheus metrics that the Sysdig process and store can be controlled by certain parameters in the dragent.yaml file.

Given below are the relevant settings and the defaults. Changes on the values will take effect after restarting the agent:

prometheus:
  max_tags_per_metric: 20
  max_metrics_per_process: 1000
  max_metrics: 1000

Note

The max_tags_per_metric and max_metrics_per_process parameters have been deprecated in agent v10.0.0.

max_metrics

The maximum number of Prometheus metrics that the agent can consume from the target. The default is 1000. The maximum limit is 10,000 on agent v10.0.0 and above. The maximum limit is 3000 on agent versions below 10.0.0.

max_metrics_per_process

This parameter is deprecated in agent v10.0.0.

The maximum number of Prometheus metrics that the agent can read from a single process. The default is -1 (infinity). The limit is imposed by the value of max_metrics.

max_tags_per_metric

This parameter is deprecated in agent v10.0.0.

It is the maximum number of Prometheus metrics that the agent will save from a single scraped target.