fluentd

Fluentd is an open source data collector, which allows unifying data collection and consumption to better use and understand data. Fluentd structures data as JSON as much as possible, to unify all facets of processing log data: collecting, filtering, buffering, and outputting logs across multiple sources and destinations. If Fluentd is installed on your environment, the Sysdig agent will automatically connect. See See the Default Configuration section, below. The Sysdig agent automatically collects default metrics.

This page describes the default configuration settings, how to edit the configuration to collect additional information, the metrics available for integration, and a sample result in the Sysdig Monitor UI.

Fluentd Setup

Fluentd can be installed as a package (.deb, .rpm, etc) depending on the OS flavor, or it can be deployed in a Docker container. Fluentd installation is documented here. For the examples on this page, a .deb package installation is used.

After installing Fluentd, add following lines in fluentd.conf :

<source>
  @type monitor_agent
  bind 0.0.0.0
  port 24220
</source>

Sysdig Agent Configuration

Review how to Edit dragent.yaml to Integrate or Modify Application Checks.

Default Configuration

By default, Sysdig’sdragent.default.yaml uses the following code to connect with Fluentd and collect default metrics.

(If you use a non-standard port for monitor_agent , you can configure it as usual in the agent config file dragent.yaml.)

  - name: fluentd
    pattern:
      comm: fluentd
    conf:
      monitor_agent_url: http://localhost:24220/api/plugins.json

Example

To generate the metric data, it is necessary to generate some logs through an application. In the following example, HTTP is used. (For more information, see Life of a Fluentd event.)

Execute the following command on in the Fluentd environment:

$ curl -i -X POST -d 'json={"action":"login","user":2}' http://localhost:8888/test.cycle

Expected output: (Note: Here the status code is 200 OK, as HTTP traffic is successfully generated; it will vary per application.)

HTTP/1.1 200 OK
Content-type: text/plain
Connection: Keep-Alive
Content-length: 0

Metrics Available

See fluentd Metrics.

Result in the Monitor UI