Sysdig Documentation

Elasticsearch

Elasticsearch is an open-source, distributed, document storage and search engine that stores and retrieves data structures in near real-time.Elasticsearch represents data in the form of structured JSON documents, and makes full-text search accessible via RESTful API and web clients for languages like PHP, Python, and Ruby. It’s also elastic in the sense that it’s easy to scale horizontally—simply add more nodes to distribute the load. If Elasticsearch is installed on your environment, the Sysdig agent will automatically connect in most of the cases. See the Default Configuration, below.

The Sysdig Agent automatically collects default metrics. You can also edit the configuration to collect Primary Shard stats.

This page describes the default configuration settings, how to edit the configuration to collect additional information, the metrics available for integration, and a sample result in the Sysdig Monitor UI.

Elasticsearch Setup

Elasticsearch is ready to expose metrics without any special configuration.

Sysdig Agent Configuration

Review how to Edit dragent.yaml to Integrate or Modify Application Checks.

Default Configuration

By default, Sysdig's dragent.default.yaml uses the following code to connect with Elasticsearch and collect basic metrics.

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200

For more metrics, you may need to change the elasticsearch default setting in dragent.yaml:

Warning

Remember! Never edit dragent.default.yaml directly; always edit only dragent.yaml.

Example 1: Agent authentication to Elasticsearch cluster with auth

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: https://sysdigcloud-elasticsearch:9200
      username: readonly
      password: some_password
      ssl_verify: false

Example 2: Enable Primary shard Statistics

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200
      pshard_stats : true

pshard-specific Metrics

Enable pshard_stats to monitor the following additional metrics:

Metric Name

elasticsearch.primaries.flush.total

elasticsearch.primaries.flush.total.time

elasticsearch.primaries.docs.count

elasticsearch.primaries.docs.deleted

elasticsearch.primaries.get.current

elasticsearch.primaries.get.exists.time

elasticsearch.primaries.get.exists.total

elasticsearch.primaries.get.missing.time

elasticsearch.primaries.get.missing.total

elasticsearch.primaries.get.time

elasticsearch.primaries.get.total

elasticsearch.primaries.indexing.delete.current

elasticsearch.primaries.indexing.delete.time

elasticsearch.primaries.indexing.delete.total

elasticsearch.primaries.indexing.index.current

elasticsearch.primaries.indexing.index.time

elasticsearch.primaries.indexing.index.total

elasticsearch.primaries.merges.current

elasticsearch.primaries.merges.current.docs

elasticsearch.primaries.merges.current.size

elasticsearch.primaries.merges.total

elasticsearch.primaries.merges.total.docs

elasticsearch.primaries.merges.total.size

elasticsearch.primaries.merges.total.time

elasticsearch.primaries.refresh.total

elasticsearch.primaries.refresh.total.time

elasticsearch.primaries.search.fetch.current

elasticsearch.primaries.search.fetch.time

elasticsearch.primaries.search.fetch.total

elasticsearch.primaries.search.query.current

elasticsearch.primaries.search.query.time

elasticsearch.primaries.search.query.total

elasticsearch.primaries.store.size

Example 3: Enable Primary shard Statistics for Master Node only

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200
      pshard_stats_master_node_only: true

Note that this option takes precedence over the pshard_stats option (above). This means that if the following configuration were put into place, only the pshard_stats_master_node_only option would be respected:

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200
      pshard_stats: true
      pshard_stats_master_node_only: true

All Available Metrics

With the default settings and the pshard setting, the total available metrics are listed here: Elasticsearch Metrics.

Result in the Monitor UI

373981615.png
373981603.png