Tuning Sysdig Agent
The resource requirements for the Sysdig agent are subjective to the size and load of the host. Increased activity equates to higher resource requirements. At a minimum, the agent requires 2% of the total CPU and 512 MiB of memory.
You might see 5 to 20 KiB/s of bandwidth consumed. Different variables can increase the throughput required. For example:
The number of metrics
The number of events
Products and features enabled
When a Sysdig Capture is being collected, you can expect to see a spike in the bandwidth while the capture file is being ingested.
Sysdig does not recommend placing bandwidth shaping or caps on the agent to ensure that data is sent to the Sysdig Collection service.
In general, in larger clusters, the agent requires more memory, and in servers with a high number of cores, the agent requires more CPU cores to monitor all the system calls. You will use CPU cores on the host and the Kubernetes nodes visible to the agent as proxies for the rate of events processed in the agent.
Similarly, there are different factors that are at play, and considering all the factors, we recommend the following:
Small: CPU core count <= 8. Kubernetes nodes <=10
Medium: 8 < CPU core count <= 32. 10 < Kubernetes nodes <= 100
Large: CPU core count > 32. Kubernetes nodes > 100
While you can expect the behavior with the given numbers to be better than simply using the default values, Sysdig cannot guarantee that resource allocation will be correct for all the cases.
|Kubernetes CPU Request||1||3||5|
|Kubernetes CPU Limit||1||3||5|
|Kubernetes Memory Request||1024 MB||3072 MB||6144 MB|
|Kubernetes Memory Limit||1024 MB||3072 MB||6144 MB|
|Dragent Memory Watchdog||512 MB||1024 MB||2048 MB|
|Cointerface Memory Watchdog||512 MB||2048 MB||4096 MB|
Note that the agent has its own memory watchdog to prevent runaway memory consumption on the host in case of memory leaks. The default values of the watchdog are specified in the following agent configuration file.
watchdog: max_memory_usage_mb: 1024 max_memory_usage_subprocesses: sdchecks: 128 sdjagent: 256 mountedfs_reader: 32 statsite_forwarder: 32 cointerface: 512 promscrape: 640
The recommended value for
promscrape depends on the amount of timeseries and label data that required to be scraped on a particular node. The cluster size does not have an effect on
promscrape memory usage.
max_memory_usage_mb corresponds to the dragent process in the agent.
All the values are given in MiB.
For example, use the following agent configuration to match the agent watchdog settings with large values:
watchdog: max_memory_usage_mb: 2048 max_memory_usage_subprocesses: sdchecks: 128 sdjagent: 256 mountedfs_reader: 32 statsite_forwarder: 32 cointerface: 4096 promscrape: 640