Tuning Sysdig Agent

The resource requirements for the Sysdig agent are subjective to the size and load of the host. Increased activity equates to higher resource requirements. At a minimum, the agent requires 2% of the total CPU and 512 MiB of memory.

You might see 5 to 20 KiB/s of bandwidth consumed. Different variables can increase the throughput required. For example:

  • The number of metrics

  • The number of events

  • Kubernetes objects

  • Products and features enabled

When a Sysdig Capture is being collected, you can expect to see a spike in the bandwidth while the capture file is being ingested.

Sysdig does not recommend placing bandwidth shaping or caps on the agent to ensure that data is sent to the Sysdig Collection service.

In general, in larger clusters, the agent requires more memory, and in servers with a high number of cores, the agent requires more CPU cores to monitor all the system calls. You will use CPU cores on the host and the Kubernetes nodes visible to the agent as proxies for the rate of events processed in the agent.

Similarly, there are different factors that are at play, and considering all the factors, we recommend the following:

Small: CPU core count <= 8. Kubernetes nodes <=10

Medium: 8 < CPU core count <= 32. 10 < Kubernetes nodes <= 100

Large: CPU core count > 32. Kubernetes nodes > 100

While you can expect the behavior with the given numbers to be better than simply using the default values, Sysdig cannot guarantee that resource allocation will be correct for all the cases.

Cluster Size

Small

Medium

Large

Kubernetes CPU Request

1

3

5

Kubernetes CPU Limit

1

3

5

Kubernetes Memory Request

1024 MB

3072 MB

6144 MB

Kubernetes Memory Limit

1024 MB

3072 MB

6144 MB

Dragent Memory Watchdog

512 MB

1024 MB

2048 MB

Cointerface Memory Watchdog

512 MB

2048 MB

4096 MB

Note that the agent has its own memory watchdog to prevent runaway memory consumption on the host in case of memory leaks. The default values of the watchdog are specified in the following agent configuration file.

watchdog:
  max_memory_usage_mb: 1024
  max_memory_usage_subprocesses:
    sdchecks: 128
    sdjagent: 256
    mountedfs_reader: 32
    statsite_forwarder: 32
    cointerface: 256

All the values are given in MiB.

For example, to match the agent watchdog settings with large values, the agent configuration would be:

watchdog:
  max_memory_usage_mb: 2048
  max_memory_usage_subprocesses:
    sdchecks: 128
    sdjagent: 256
    mountedfs_reader: 32
    statsite_forwarder: 32
    cointerface: 4096