Collect Prometheus Metrics

Sysdig supports collecting, storing, and querying Prometheus native metrics and labels. You can use Sysdig in the same way that you use Prometheus and leverage Prometheus Query Language (PromQL) to create dashboards and alerts. Sysdig is compatible with Prometheus HTTP API to query your monitoring data programmatically using PromQL and extend Sysdig to other platforms like Grafana.

From a metric collection standpoint, a lightweight Prometheus server is directly embedded into the Sysdig agent to facilitate metric collection. This also supports targets, instances, and jobs with filtering and relabeling using Prometheus syntax. You can configure the agent to identify these processes that expose Prometheus metric endpoints on its own host and send it to the Sysdig collector for storing and further processing.

This document uses metric and time series interchangeably. The description of configuration parameters refers to “metric”, but in strict Prometheus terms, those imply time series. That is, applying a limit of 100 metrics implies applying a limit on time series, where all the time series data might not have the same metric name.

The Prometheus product itself does not necessarily have to be installed for Prometheus metrics collection.

See the Sysdig agent versions and compatibility with Prometheus features:

  • Sysdig agent v10.5.0 and above: Promscrape V2 is used that supports Prometheus native service discovery. This option is controlled by the prom_service_discovery parameter in the dragent.yaml file. Additionally, a default prometheus.yaml file has been added with Kubernetes pod discovery rules to use when native Prometheus service discovery is enabled.

  • Sysdig agent v10.0.0 and above: promscrape, a lightweight Prometheus server, by default is used for scraping Prometheus metrics. This feature is controlled by the use_promscrape parameter and is enabled by default.

  • Sysdig agent v9.8.0 to v10.0: promscrape , a lightweight Prometheus server, is introduced in v9.8.0 to scrape Prometheus metrics. You must enable use_promscrape in the dragent.yaml file to use this method.

  • Sysdig agent v0.70.0 and above: Provides rich support for automatically collecting metrics from Prometheus exporters .

The following topics describe in detail how to configure the Sysdig agent for service discovery, metrics collection, and further processing.

Learn More

See the following blog posts for additional context on the Prometheus metric and how such metrics are typically used.

Last modified October 6, 2021