Elasticsearch

Elasticsearch is an open-source, distributed, document storage and search engine that stores and retrieves data structures in near real-time. Elasticsearch represents data in the form of structured JSON documents and makes full-text search accessible via RESTful API and web clients for languages like PHP, Python, and Ruby. It’s also elastic in the sense that it’s easy to scale horizontally—simply add more nodes to distribute the load. If Elasticsearch is installed on your environment, the Sysdig agent will automatically connect in most of the cases. See the Default Configuration, below.

The Sysdig Agent automatically collects default metrics. You can also edit the configuration to collect Primary Shard stats.

This page describes the default configuration settings, how to edit the configuration to collect additional information, the metrics available for integration, and a sample result in the Sysdig Monitor UI.

Elasticsearch Setup

Elasticsearch is ready to expose metrics without any special configuration.

Sysdig Agent Configuration

Review how to edit dragent.yaml to Integrate or Modify Application Checks.

Default Configuration

By default, Sysdig’s dragent.default.yaml uses the following code to connect with Elasticsearch and collect basic metrics.

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200

For more metrics, you may need to change the elasticsearch default setting in dragent.yaml:

Example 1: Agent authentication to Elasticsearch Cluster with Authentication

Password Authentication

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: https://sysdigcloud-elasticsearch:9200
      username: readonly
      password: some_password
      ssl_verify: false

Certificate Authentication

app_checks:
   - name: elasticsearch
     check_module: elastic
     pattern:
       port: 9200
       comm: java
     conf:
       url: https://localhost:9200
       ssl_cert: /tmp/certs/ssl.crt
       ssl_key: /tmp/certs/ssl.key
       ssl_verify: true

ssl_cert: Path to the certificate chain used for validating the authenticity of the Elasticsearch server.

ssl_key: Path to the certificate key used for authenticating to the Elasticsearch server.

Example 2: Enable Primary shard Statistics

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200
      pshard_stats : true

pshard-specific Metrics

Enable pshard_stats to monitor the following additional metrics:

Example 3: Enable Primary shard Statistics for Master Node only

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200
      pshard_stats_master_node_only: true

Note that this option takes precedence over the pshard_stats option (above). This means that if the following configuration were put into place, only the pshard_stats_master_node_only option would be respected:

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: http://localhost:9200
      pshard_stats: true
      pshard_stats_master_node_only: true

All Available Metrics

With the default settings and the pshard setting, the total available metrics are listed here: Elasticsearch Metrics.

Result in the Monitor UI