Enabling Histogram Metrics

By default, the Agent does not scrape details for the histogram metrics reported by exporters. To enable this, include the additional histograms option:

  enabled: true
  histograms: true

Note that the way Prometheus histograms are typically reported by exporters, they are cumulative across buckets and also report accumulations within each bucket range since the exporter started. To make this data easier to visualize, the Sysdig Monitor application presents histogram data as a delta over the selected time range. For instance, below we've had our sample Prometheus exporter running for over an hour, accumulating data in a histogram metric called random:

# curl
# HELP random Random sleep
# TYPE random histogram
random_bucket{le="0.005",} 33.0
random_bucket{le="0.01",} 74.0
random_bucket{le="0.025",} 180.0
random_bucket{le="0.05",} 404.0
random_bucket{le="0.075",} 639.0
random_bucket{le="0.1",} 848.0
random_bucket{le="0.25",} 2145.0
random_bucket{le="0.5",} 4307.0
random_bucket{le="0.75",} 6371.0
random_bucket{le="1.0",} 8592.0
random_bucket{le="2.5",} 8619.0
random_bucket{le="5.0",} 8619.0
random_bucket{le="7.5",} 8619.0
random_bucket{le="10.0",} 8619.0
random_bucket{le="+Inf",} 8619.0

However in the Sysdig Monitor interface, if we select the metric and a time range such as 10 minutes, we'll see the per-bucket distribution for only that time range. Hover your mouse over a particular bar to see the bucket range. If this histogram metric is being reported from multiple sources, you will see the contribution to the total from each source based on your choice of Segment by. In this case we are segmenting by hostname and only have one contributing source, called "ubuntu16".