Agent Installation
Sysdig agents are delivered as either a container or a service and can be deployed with or without an orchestrator such as Kubernetes or Mesos.
A quick install involves just a few lines of code from the Getting Started wizard copied into a shell. The complete install instructions address checking for and installing kernel headers if needed, any prerequisite permissions settings for particular environments, and tips about updating the configuration files after initial installation.
Plan the Installation
Host Requirements | Review the platforms, runtimes, Linux distributions, orchestration, browsers, etc. that are supported. |
Access key | An agent access key is provided with a Sysidg trial |
Installation Options | Different ways in which you can install Sysdig agent. |
Troubleshooting Agents | Troubleshooting tips for agent installation, tuning agents, and compiling kernel modules. |
Installation Options
In the default mode of agent installation, you install the agent package as two containers, each container responsible for different functions as given below. The agent-slim
reduces the surface area of attack for potential vulnerabilities and is, therefore, more secure.
agent-kmodule
: Responsible for downloading and building the kernel
module. The image is short-lived. The container exits after the
kernel module is loaded. The transient nature of the container
reduces the time and opportunities for exploiting any potential
vulnerabilities present in the container image.
Prerequisites: The package depends on Dynamic Kernel Module
Support (DKMS) and requires the compiler and kernel headers
installed if you are using the agent-kmodule
to build the kernel
probe. Alternatively, you can use it without the kernel headers. In
such cases, the agent-kmodule
will attempt to download a pre-built
kernel probe if it is present in the Sysdig probe repository.
The module contains:
agent-slim
: Responsible for running the agent module once the
kernel module has been loaded. Slim agent functions the same way as the regular agent and retains the feature parity.
Use the instruction below to install agent on your chosen environment:
Legacy Agent: The legacy agent can be run as a single container or a service. It includes the components for downloading and building the kernel module, as well as for gathering and reporting on a wide variety of pre-defined metrics. For more information, see Installing Agent as a Single Container.
Helm
Helm is the preferred way of installing Sysdig agent. It is used in most cloud environments, for example, Amazon EKS or EC2 on AWS Cloud or AWS Outpost, EC2, and Azure AKS.
Manual
With the Getting Started wizard, you can copy a simple line of code to deploy agents in a variety of environments.
Behind the scenes, the wizard auto-detects and completes configuration items such as the required access key and port information. The wizard can also be launched from the Start a Free Trial button at sysdig.com.
After the first install, Sysdig Secure and Monitor users can access the wizard at any time from the Rocket icon on the navigation bar.
1 - Quick Install Sysdig Agent
Sysdig provides you with quick-install commands pre-filled with some of your environment variables to get started with Sysdig agent. You choose the deployment type and Sysdig gives you auto-generated commands to ease your installation experience.
Access from Get Started or Data Sources
Log in as the administrator to Sysdig Monitor or Sysdig Secure.
Select the Get Started page in Monitor or the Data Sources page in Sysdig Secure.
Click Install the Agent, select the appropriate deployment type, and copy the auto-generated code, filling in remaining variable values as required.
Sample Usage
Kubernetes
Helm is the recommended option for installing agents on Kubernetes. For the latest helm-based installation instructions and configuration options, see sysdig-deploy.
If you cannot utilize helm, we also provide a script
which will download and apply Kubernetes manifests to deploy the agent as a DaemonSet. The script requires curl
and kubectl
installed in the $PATH
on the host in which it is run.
Sysdig Monitor
kubectl create ns sysdig-agent
helm repo add sysdig https://charts.sysdig.com
helm repo update
helm install sysdig sysdig/sysdig-deploy
--namespace sysdig-agent \
--set global.sysdig.accessKey=ACCESS_KEY \
--set global.sysdig.region=REGION \
--set global.clusterConfig.name=CLUSTER_NAME \
--set nodeAnalyzer.enabled=false \
curl -s https://download.sysdig.com/stable/install-agent-kubernetes | sudo bash -s -- \
--access_key <ACCESS_KEY> \
--collector <COLLECTOR_ADDRESS> --collector_port <COLLECTOR_PORT> \
Sysdig Secure
kubectl create ns sysdig-agent
helm repo add sysdig https://charts.sysdig.com
helm repo update
helm install sysdig sysdig/sysdig-deploy
--namespace sysdig-agent \
--set global.sysdig.accessKey=ACCESS_KEY \
--set global.sysdig.region=REGION \
--set global.clusterConfig.name=CLUSTER_NAME \
--set global.kspm.deploy=true \
--set nodeAnalyzer.secure.vulnerabilityManagement.newEngineOnly=true \
--set nodeAnalyzer.nodeAnalyzer.benchmarkRunner.deploy=false \
curl -s https://download.sysdig.com/stable/install-agent-kubernetes | sudo bash -s -- \
--access_key <ACCESS_KEY> \
--collector <COLLECTOR_ADDRESS> --collector_port <COLLECTOR_PORT> \
--nodeanalyzer --api_endpoint <SECURE_ENDPOINT_ADDRESS>
Docker
#Build and load the kernel module
docker run -it --privileged --rm --name sysdig-agent-kmodule \
-v /usr:/host/usr:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
quay.io/sysdig/agent-kmodule
#Configure the Kernel module to load during system boot.
sudo mkdir -p /etc/modules-load.d
sudo bash -c "echo sysdigcloud-probe > /etc/modules-load.d/sysdigcloud-probe.conf"
#Run the sysdig-agent
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=<ACCESS_KEY> \
-e COLLECTOR=<COLLECTOR_URL> \
-e SECURE=true \
-e TAGS=<LIST_OF_TAGS> \
-e ADDITIONAL_CONF= <LIST_OF_CONFIG> \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
--shm-size=512m \
quay.io/sysdig/agent-slim
Linux
curl -s https://download.sysdig.com/stable/install-agent | sudo bash -c -- \
--access_key [-t | --tags <value>] [-c | --collector <value>] \
[-cp | --collector_port <value>] [-s | --secure <value>] \
[-cc | --check_certificate] [-ac | --additional_conf <value>] \
[-b | --bpf] [-h | --help]
curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- \
--access_key <ACCESS_KEY> --collector <COLLECTOR_URL> \
--secure true
Configuration Options
Option | Description |
---|
Access Key | The agent access key. You can retrieve this from Settings > Agent Installation in either Sysdig Monitor or Sysdig Secure. |
Tags | Enter meaningful tags you want applied to your instances. |
Collector | The collector URL for Sysdig Monitor or Sysdig Secure. This value is region-dependent in SaaS and is auto-completed on the Get Started page in the UI. It is a custom value in on-prem installations. See SaaS Regions and IP Ranges. |
Collector Port | The default is 6443. |
Region | The SaaS region for the agents. For example, us1 , us2 , and au1 . See Regions and IP Ranges for more information. |
Secure | Use a secure SSL/TLS connection to send metrics to the collector. This option is enabled by default. |
Check Certificate | Disables strong SSL certificate check for Sysdig Monitor on-premises installation. |
Additional Conf | Optional. Use this option to provide custom configuration values to the agent as environment variables. If provided, the value will be appended to agent configuration file. For example, file log configuration. |
BBF | Enables eBPF probe. |
KSPM | The configuration specific to KSPM. |
Node Analyzer | Installs image analyzer, benchmarks, and host scanning components. For more information, see Node Analyzer. |
2 - Agent Installation Requirements
Sysdig agents can be installed on a wide array of Linux hosts. Check
your environment to ensure it meets the minimum supported platform,
operating system, runtime, and orchestration requirements and uses the
appropriate installation instructions.
Versioning Scheme
We recommend that you use the latest version of the agent. Sysdig supports n-3 versions back based on the minor number. For example, if the latest release is v12.0.0
, we will support n-3 versions back, up to v11.2.0
.
End of Support
Sysdig agents that are older than version 0.85.1, released October 1, 2018, will no longer connect to the Sysdig US-East SaaS platform with default agent values.
Going forward all the agent releases will have a 3-year deprecation policy. This implies:
Sysdig Support might not be able to help you troubleshoot or address the problems with agents past the deprecation date.
Sysdig will no longer provide prebuilt kernel probe binaries for these agent releases. You need to build the kernel probe binaries on the fly by using the hosts kernel headers.
These changes is effective starting Sysdig agent v12.1.0.
Agent Installation Requirements
Tested and Supported
Sysdig agent has been tested on the following list of latest Kubernetes versions. The matrix provides a single view into the supported operating systems, architecture, and runtime versions for different flavors of Kubernetes orchestrators.
Supported
Sysdig agent is supported on the following orchestration platforms:
If you are not using an orchestrator in your environment, follow the instructions for Agent Install Non-Orchestrated.
Note: Installing the Sysdig agent into a namespace managed by Istio and configured for sidecar auto-injection is not supported. For example, setting kubectl label namespace sysdig-agent istio-injection=enabled
. Because the agent behaves more like a host component, it is required to be part of the host PID and network namespace to function correctly. Due to this requirement, deploying the Sysdig agent in Istio with an Envoy sidecar is not supported. However, running the Sysdig agent in a non-injected namespace where Istio is installed and managing other namespaces is fully supported. See Istio integration for more details on using the Sysdig agent to monitor Istio control plane and sidecar metrics.
Linux Distributions and Kernels
Tested and Supported
Sysdig agent (installed as a service) has been tested on the following list of latest linux distros:
Operating System | Architecture |
---|
Amazon Linux 2 | x86_64 |
Amazon Linux 2022 | x86_64 |
Debian GNU/Linux 10 (buster) | x86_64 |
Debian GNU/Linux 11 (bullseye) | x86_64 |
Fedora Linux 35 (Cloud Edition) | x86_64 |
Red Hat Enterprise Linux 8.7 (Ootpa) | x86_64 |
Red Hat Enterprise Linux 9.1 (Plow) | x86_64 |
Ubuntu 18.04 LTS (Bionic Beaver) | x86_64 |
Ubuntu 20.04 LTS (Focal Fossa) | x86_64 |
Ubuntu 22.04 LTS (Jammy Jellyfish) | x86_64 |
Supported
Sysdig agent is supported on the following Linux distributions:
Core Set | Debian Ubuntu Ubuntu (Amazon) CentOS Red Hat Enterprise Linux (RHEL) SuSE Linux Enterprise Server RHEL CoreOS (RHCOS) Fedora Fedora CoreOS Linux Mint Amazon Linux Amazon Linux v2 Amazon Bottlerocket Google Container Optimized OS (COS) Oracle Linux (UEH) Oracle Linux (RHCK)
|
AWS EC2 | Amazon Linux 2 Amazon Bottlerocket Core set (see above)
|
GCP | |
Azure | |
Container Runtimes
Sysdig agent supports the detection of the following:
- Docker
- LXC
- CRI-O
- containerd
- Podman
- Mesos
Tested and Supported for Docker
Sysdig agent has been tested on the following list of latest linux distros:
Operating System | Architecture |
---|
Amazon Linux 2 | x86_64, arm64 |
Amazon Linux 2022 | x86_64, arm64 |
Debian GNU/Linux 10 (buster) | x86_64, arm64 |
Debian GNU/Linux 11 (bullseye) | x86_64, arm64 |
Fedora Linux 35 (Cloud Edition) | x86_64, arm64 |
Fedora Linux 36 (Cloud Edition) | x86_64, arm64 |
Red Hat Enterprise Linux 7.9 (Maipo) | x86_64 |
Red Hat Enterprise Linux 8.4 (Ootpa) | zlinux |
Red Hat Enterprise Linux 8.7 (Ootpa) | x86_64, arm64 |
Red Hat Enterprise Linux 9.1 (Plow) | x86_64, arm64 |
SLES 15 SP4 | x86_64, arm64 |
Ubuntu 18.04 LTS (Bionic Beaver) | x86_64, arm64 |
Ubuntu 20.04 LTS (Focal Fossa) | x86_64, arm64 |
Ubuntu 22.04 LTS (Jammy Jellyfish) | x86_64, arm64 |
Prerequisites for Podman Environments
Sysdig agent supports running as a
Podman container.
Enable Podman API Service for all the users.
The agent will not able to collect Podman-managed container
metadata, such as the container name, if the API service is not
enabled.
Secure rules and policies that depend on container metadata other
than the container ID will not work.
Pausing and terminating containers will not work because Policy
actions for Podman are not supported.
The containers started as a non-root user will have the
podman_owner_uid
label associated with it if the API service is
enabled for that user. The value of podman_owner_uid
will be the
numeric user ID corresponding to the user that started the
container.
Container Registries
Quay.io
For example, to pull the latest agent container from Quay.io:
docker pull quay.io/sysdig/agent
CPU Architectures
x86
Supported Agent Containers
agent
agent-slim
agent-kmodule
ARM (aarch64)
Supported kernel versions are v4.17 and above
Unsupported Features
- Pre-built probes
- Activity Audit
- Sysdig agent installation using the
agent
container
s390x (zLinux)
Unsupported Features
Probes
No support for pre-built probes on zLinux.
For kernel instrumentation, use the kernel module. eBPF probes are not supported on zLinux.
Captures
Capture is not supported on zLinux.
Legacy Agent Installation
Sysdig agent installation using agent
container is not supported.
Java Versions and Vendors
Sysdig agent supports the following:
- Java versions: v7 and above
- Vendors: Oracle, OpenJDK
For Java-based applications (Cassandra, Elasticsearch, Kafka, Tomcat,
Zookeeper and etc.), the Sysdig agent requires the Java runtime
environment (JRE) to be installed to poll for metrics (beans).
If the Docker-container-based Sysdig agent is installed, the JRE is
installed alongside the agent binaries and no further dependencies
exist. However, if you are installing the service-based agent
(non-container) and you do not see the JVM/JMX metrics reporting, your
host may not have the JRE installed or it may not be installed in the
expected location: usr/bin/java
Minimum Resource Requirements
The resource requirements of the agent are subjective to the size and
load of the host— more activity equates to more resources required.
It is typical to see between 5-20KiB/s of bandwidth consumed—different
variables can increase the throughput required such as the number of
metrics, events, Kubernetes objects, and which products and features are
enabled. When a Sysdig Capture is being collected, you can expect to see
a spike in bandwidth while the capture file is being ingested.
We do not recommend placing bandwidth shaping or caps on the agent to
ensure data can be sent to our collection service.
Additional Requirements
Access key
The installation of the Sysdig agent requires an access key.
This key and the agent installation instructions are presented to you after
activating your account and using a web-based wizard upon initial login.
The same information can also be found in the
Settings > Agent Installation
menu of the web interface after logging
in. See Agent Installation: Overview and
Key for details.
Network connection
A Sysdig agent (containerized or native) is installed into each host
being monitored and will need to be able to connect to the Sysdig
Monitor backend servers to report host metrics. The agent must be able
to reach the Sysdig Collector addresses. For example, for US East, it is
‘collector.sysdigcloud.com’ (via
multiple IPs) over port tcp/6443
. See Sysdig Collector
Ports for supported ports
for other regions.
The agent supports the HTTP proxy for communicating with Sysdig backend
components. For more information, see Enable HTTP Proxy for
Agents.
3 - Agent Install: Kubernetes
The recommended method to monitor Kubernetes environments is to deploy the Sysdig agent using the helm chart. Alternatively, you can install the agent container using DaemonSet. This section helps you install the agent in both the methods.
Installing the agent using helm or as a daemonSet will deploy agent containers on every node in your Kubernetes environment. Once the agent is installed, Sysdig Monitor automatically begins monitoring all of your hosts, apps, pods, and services and automatically connects to the Kubernetes API server to pull relevant metadata about the environment. If licensed, Sysdig Secure launches with default policies that you can view and configure to suit your needs. You can access the front-end web interfaces for Sysdig Monitor and Sysdig Secure immediately.
Sysdig supports monitoring numerous Kubernetes platforms, including the following:
Prerequisites
A supported distribution: See Agent Installation Requirements for details.
Kubernetes v1.9+: The agent installation on Kubernetes requires
v1.9 or higher because the APIs used to fetch kubernetes
metadata are only present in v1.9+.
Sysdig account and access key: Request a trial or full account
at Sysdig.com and click the Activate Account button.
The Get Started page in Sysdig Monitor or the Data Sources page in Secure provide an access key.
Port 6443 open for outbound traffic: The agent communicates
with the collector on port 6443. If you are using a firewall,
you must open port 6443 for outbound traffic for the agent.
Kernel headers installed: If a prebuilt kernel probe is not
available for your kernel, the kernel headers must be installed
in order to build the kernel probe.
kubectl installed: All of the installation methods utilize
kubectl
to install the agent in the cluster.
The Sysdig agent requires kernel header files to install successfully on a Kubernetes cluster. If the hosts in your environment match the pre-compiled kernel modules available from Sysdig, no special action is required.
In some cases, the nodes in your Kubernetes environment might use Unix versions that do not match the provided headers, and the agent might fail to install correctly. In those cases, you must install the kernel headers manually on each node.
To do so:
For Debian-style distributions, run the command:
apt-get -y install linux-headers-$(uname -r)
For RHEL-style distributions, run the command:
yum -y install kernel-devel-$(uname -r)
For more information on troubleshooting, see About Kernel Headers and the Kernel Module.
Kubernetes Environments
Some Kubernetes environments require special configuration options to deploy the agent. If you’re installing in one of the following environments, follow the guides specific to those environments to deploy the agent. Otherwise, continue with this topic.
Installation
Helm
Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.
Script
Sysdig also provides a script that you can use to install the agent as a DaemonSet.
Installation
Download the script and make it executable.
wget https://download.sysdig.com/stable/install-agent-kubernetes
chmod +x install-agent-kubernetes
Run the script to install the agent as a DaemonSet.
./install-agent-kubernetes -a <ACCESS_KEY> -c <COLLECTOR_URL> -cn <CLUSTER_NAME>
Options
-a
| The agent access key. You can retrieve this from Settings > Agent Installation in either Sysdig Monitor or Sysdig Secure. |
-t
| The list of tags to identify the host where the agent is installed. For example: role:webserver , location:europe , role:webserver . |
-c
| The collector URL for Sysdig Monitor or Sysdig Secure. This value is region-dependent in SaaS and is auto-completed on the Get Started page or Data Sources page in the UI. It is a custom value in on-prem installations. |
-cp
| The collector port. The default is 6443. |
-cn
| If a value is provided, the cluster will be identified with the name provided |
-s
| Use a secure SSL/TLS connection to send metrics to the collector. This option is enabled by default. |
-cc
| Enable strong SSL certificate check. The default is true. |
-ns
| If a value is provided, the agent will be deployed to the specified namespace/project. The default is sysdig-agent . |
-op
| If provided, perform the agent installation using the OpenShift command line. |
-ac
| If a value is provided, the additional configuration will be appended to the agent configuration file. |
-av
| If a version is provided, use the specified agent version. The default is the latest version. |
-r
| If a value is provided, the daemonset, configmap, cluster role binding, service acccount and secret associated with the Sysdig Agent will be removed from the specified namespace. |
-ae
| The api_endpoint is the region-dependent domain for the Sysdig product, without the protocol. E.g. secure.sysdig.com , us2.app.sysdig.com , eu1.app.sysdig.com |
-h
| Print this usage and exit. |
Sysdig Secure Only | |
-na
| If provided, will install the Node Analyzer tools. It is an error to set both -ia and -na. |
-ds
| The docker socket for Image Analyzer. |
-cs
| The CRI socket for Image Analyzer. |
-cv
| The custom volume for Image Analyzer. |
-h
| Print this usage and exit. |
-b
| Required in AWS Bottlerocket nodes to determine whether the eBPF should be built. Alternatively, you can use `--bpf`. |
Sysdig Secure Only (Legacy) These values apply to the Node Image Analyzer (v1) in Sysdig Secure. | |
-am
| The Analysis Manager endpoint for Sysdig Secure. |
-ia
| If provided, will install the Node Image Analyzer (v1). It is an error to set both -ia and -na. The v1 Node Image Analyzer will be deprecated and replaced by the NA tools. |
Manifests
To deploy agents using Kubernetes manifests, you can download manifest files, edit them as required, and deploy them using kubectl
.
Download the sample files:
sysdig-agent-clusterrole.yaml
sysdig-agent-daemonset-v2.yaml
sysdig-agent-configmap.yaml
sysdig-agent-service.yaml
Create a namespace for the Sysdig agent.
Note: You can use whatever name you prefer. This example uses
sysdig-agent
for both the namespace and the service account. The default service account name was automatically defined in
sysdig-agent-daemonset-v2.yaml, at
the line:
serviceAccount: sysdig-agent
kubectl create ns sysdig-agent
Create a secret key:
kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
Create a cluster role and service account, and define the cluster
role binding that grants the Sysdig agent rules in the cluster role:
kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
Edit sysdig-agent-configmap.yaml
to add the collector address
,
port
, and the SSL/TLS
information:
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
For SaaS, find the collector address for your region.
For On-prem, enter the collector endpoint defined in your
environment.
check_certificate
should be set to false
if a self-signed
certificate or private, and a CA-signed cert is used. See Set Up SSL Connectivity to the Backend for more information.
Apply the sysdig-agent-configmap.yaml
file:
kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
Apply the sysdig-agent-service.yaml
file:
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
This allows the agent to receive Kubernetes audit events from the
Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.
Apply the daemonset-v2.yaml
file :
kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
Additional Options
Verify Metrics in Sysdig Monitor
Log in to Sysdig Monitor to verify that the agent deployed and the metrics are detected and collected appropriately.
The steps below give one way to do the check.
Access Sysdig Monitor:
SaaS: See SaaS Regions and IP
Ranges and identify the
correct domain URL associated with your Sysdig application and
region. For example, for US East, the URL is
https://app.sysdigcloud.com.
For other regions, the format is https://<region>.app.sysdig.com)
.
Replace ; with the region where your Sysidig
application is hosted. For example, for Sysdig Monitor in the EU,
you use https://eu1.app.sysdig.com.
Log in with your Sysdig user name and password.
Select the Explore
tab to see if metrics are displayed.
Determine the Kube State Metrics you want to collect.
To verify that kube state metrics and cluster name are working correctly, select the Explore tab and see if your cluster is listed.
Kubernetes metadata (pods, deployments etc.) appear a minute or two
later than the nodes/containers themselves; if pod names do not appear
immediately, wait and retry the Explore view.
If agents are disconnecting, there could be an issue with your MAC
addresses. See Troubleshooting Agent Installation for tips.
Verify Agent Installation in Sysdig Secure
Log in to Sysdig Secure.
See SaaS Regions and IP Ranges and identify the
correct domain URL associated with your Sysdig application and region.
Navigate to Integrations > Sysdig Agents.
The Sysdig Agents page list all the agents installed your environment.
For more information, see Sysdig Agents.
3.1.1 - GKE Autopilot
Autopilot is an operation mode for creating and managing clusters in GKE. In brief, with Autopilot, Google configures and manages the underlying node infrastructure for you. This topic helps you use helm to install Sysdig agent on a GKE cluster installed in Autopilot mode.
NodeAnalyzer is not supported on Autopilot environments.
Prerequisites
Install a GKE cluster in Autopilot mode.
Connect the GKE cluster.
Install your workload.
Deploy Sysdig Agent
Sysdig recommends using Helm to install Sysdig agent in kubernetes environments. After connecting to the GKE cluster, use the sysdig-deplpy chart to install Sysdig agent.
To customize the configuration of the agent, see the Sysdig Agent Helm Chart.
Verify Agent Installation in Sysdig Secure
Log in to Sysdig Secure.
See SaaS Regions and IP Ranges and identify the
correct domain URL associated with your Sysdig application and region.
Navigate to Integrations > Sysdig Agents.
The Sysdig Agents page list all the agents installed your environment.
For more information, see Sysdig Agents.
Verify Metrics on the Sysdig Monitor UI
Log in to Sysdig Monitor to verify that the agent deployed and the metrics are detected and collected appropriately.
Given below is one way to do so.
Access Sysdig Monitor:
SaaS: See SaaS Regions and IP Ranges and identify the
correct domain URL associated with your Sysdig application and
region. For example, for US East, the URL is
https://app.sysdigcloud.com.
For other regions, the format is https://<region>.app.sysdig.com
.
Replace <region>
with the region where your Sysdig
application is hosted. For example, for Sysdig Monitor in the EU,
you use
https://eu1.app.sysdig.com.
Log in with your Sysdig user name and password.
Select the Explore
tab to see if metrics are displayed.
Verify that kube state metrics and cluster name are working correctly: select the Explore
tab and create a grouping by kube_cluster_name
and
kube_pod_name
.
Select an individual container or pod to see the details.
3.1.2 - GKE Standard
Google Kubernetes Engine (GKE) is a managed
environment for running Kubernetes in Google Cloud, in order to deploy
containerized applications. Sysdig supports all flavors of GKE, including Ubuntu and GKE’s default
Container-Optimized OS(COS).
GKE COS environments require eBPF probe to support agent installation.
Preparation
Open Port 6443 for Agent Egress
Because GKE uses stateful firewalls, you must actively open port 6443
for the Sysdig agent outbound traffic.
In earlier versions, the Sysdig Agent connected to port 6666. This
behavior has been deprecated, as the Sysdig agent now connects to port
6443.
GKE COS/eBPF-Specific Requirements
Linux kernel version >= 4.14.
When performing the installation steps, you will add one additional
parameter to install the eBPF probe. See Step 7. Note that
only the eBPF probe is supported in GKE COS environments.
Prerequisites
You can review Agent Install: Kubernetes and the Agent Installation Requirements for additional context, if desired.
Installation Steps
Helm
Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.
Manifests
To deploy agents using Kubernetes manifests, you can download the manifest files, edit them as required, and deploy them using kubectl
.
Download the sample files:
sysdig-agent-clusterrole.yaml
sysdig-agent-daemonset-v2.yaml
sysdig-agent-configmap.yaml
sysdig-agent-service.yaml
Create a namespace to use for the Sysdig agent.
You can use whatever name you want. In this document, we used
sysdig-agent
for both the namespace and the service account.
kubectl create ns sysdig-agent
Create a secret key:
kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
If you are running Kubernetes 1.6 or higher, you must grant your user the ability to create roles in Kubernetes by running the following command:
kubectl create clusterrolebinding your-user-cluster-admin-binding --clusterrole=cluster-admin --user=your.google.cloud.email@example.org
See Google documentation
for more information.
Create a service account for the Sysdig agent using the
clusterrole.yaml
file.
The Sysdig agent must be granted read-only access to certain
Kubernetes APIs, which the agent uses to populate metadata and
provide component metrics.
You can use the Sysdig-provided, sysdig-agent-clusterrole.yaml
file. Deploying this file creates
a cluster role and service account in Kubernetes, and defines cluster role binding that grants the Sysdig agent rules in the cluster role.
Run the following commands (using whatever namespace you’ve defined in
Step 2):
kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
Edit sysdig-agent-configmap.yaml
to add the collector address
,
port
, and the SSL/TLS
information :
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
Apply the sysdig-agent-configmap.yaml
file:
kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
FOR GKE COS ONLY: To enable the eBPF probe required for COS,
uncomment the following parameters in
sysdig-agent-daemonset-v2.yaml
under the env section:
env:
- name: SYSDIG_BPF_PROBE
value: ""
Apply the sysdig-agent-service.yaml
file:
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
This allows the agent to receive Kubernetes audit events from the
Kubernetes API server. See Kubernetes Audit Logging for information
on enabling Kubernetes audit logging.
Apply the daemonset-v2.yaml
file:
kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.
Next Steps
You can continue with instructions in Additional Options.
3.2 - Steps for OKE
Oracle Kubernetes Engine (OKE) is a managed environment for running Kubernetes in Oracle Cloud, in order to deploy containerized applications. As of Sysdig agent version 12.0.1, Sysdig supports all flavors of OKE.
OKE environments require eBPF probe to support agent installation.
The instructions below describe a standard OKE agent install and call out the special steps needed to install the eBPF probe.
Preparation
Open Port 6443 for Agent Egress
Because OKE uses stateful firewalls, you must actively open port 6443
for the Sysdig agent outbound traffic.
OKE by default allows network access to the sysdig Agent on 6443, but ensure that firewall rules are open and the agent can connect to the Sysdig backends.
eBPF-Specific Requirements
Linux kernel version >= 4.14.
When performing the installation steps, you will add one additional
parameter to install the eBPF probe. See Step 7, below.
Installation Steps
Identify the appropriate endpoint depending on your Sysdig account region. For more information, see SaaS Regions and IP Ranges. More info here https://docs.sysdig.com/en/docs/administration/saas-regions-and-ip-ranges/
After making clear which region your account belongs to, please choose one of the following methods:
Helm
Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.
Manifests
To deploy agents using Kubernetes manifests, you can downloadmanifest files, edit them as required, and deploy them using kubectl
.
Download
the sample files:
sysdig-agent-clusterrole.yaml
sysdig-agent-daemonset-v2.yaml
sysdig-agent-configmap.yaml
sysdig-agent-service.yaml
Create a namespace to use for the Sysdig agent.
Notes: You can use whatever name you want. In this document, we used
sysdig-agent
for both the namespace and the service account.
kubectl create ns sysdig-agent
Create a secret key:
kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
If you are running Kubernetes 1.6 or higher, you must create a service account for the Sysdig agent by using the clusterrole.yaml
file.
The Sysdig agent must be granted read-only access to certain
Kubernetes APIs, which the agent uses to populate metadata and
provide component metrics.
You can use the Sysdig-provided sysdig-agent-clusterrole.yaml
file. Deploying it create a cluster role and service account in Kubernetes, and defines cluster role binding that grants the Sysdig agent rules in the cluster role.
Run the following commands by using the namespace you’ve defined in Step 2:
kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
Edit sysdig-agent-configmap.yaml
to add the collector address
,
port
, and the SSL/TLS
information :
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
Apply the sysdig-agent-configmap.yaml
file:
kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
To enable the eBPF probe uncomment the following parameters in
sysdig-agent-daemonset-v2.yaml
under the env section:
env:
- name: SYSDIG_BPF_PROBE
value: ""
Apply the sysdig-agent-service.yaml
file:
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
This allows the agent to receive Kubernetes audit events from the
Kubernetes API server. See Kubernetes Audit
Logging for information
on enabling Kubernetes audit logging.
Apply the daemonset-v2.yaml
file:
kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.
Next Steps
You can continue with instructions in Additional Options.
3.3 - Steps for OpenShift
You can review Agent Install: Kubernetes and the Agent Installation Requirements for additional context, if desired.
RHCOS environments require eBPF probe to support agent installation.
Preparation
RHCOS/eBPF-Specific Requirements
- Linux kernel version 4.14 or above.
- When performing the installation steps, you will add one additional parameter to install the eBPF probe. See Step 7, below.
The Sysdig agent requires kernel header files to install successfully on a Kubernetes cluster. If the hosts in your environment match the pre-compiled kernel modules available from Sysdig, no special action is required.
In some cases, the nodes in your Kubernetes environment might use Unix versions that do not match the provided headers, and the agent might fail to install correctly. In those cases, you must install the kernel headers manually on each node.
To do so:
For RHEL-style distributions, run:
yum -y install kernel-devel-$(uname -r)
For more information on troubleshooting, see About Kernel Headers and the Kernel Module.
If you are using Red Hat OpenShift, these steps are required. They
describe how to create a project, assign and label the node selector,
create a privileged service account, and add it to a cluster role.
Copy/Paste Sample Code Block
In the example code, this document uses sysdig-agent
for the
PROJECT NAME (-n
) and the SERVICE ACCOUNT (-z
).
You can copy and paste the code as is, or follow the steps below to customize your naming conventions.
oc adm new-project sysdig-agent --node-selector=''
oc project sysdig-agent
oc create serviceaccount sysdig-agent
oc adm policy add-scc-to-user privileged -n sysdig-agent -z sysdig-agent -z node-analyzer
oc adm policy add-cluster-role-to-user cluster-reader -n sysdig-agent -z sysdig-agent -z node-analyzer
Customize the Code
You can use your own Project name and Service Account name if desired.
Note that if you use a different Service Account name, you will
need to edit the default service account in the Sysdig Installation
Steps, below.
Create a new OpenShift project for the Sysdig agent deployment and
use an empty string for the node selector:
oc adm new-project PROJECT-NAME --node-selector=""
Change to the new OpenShift Project for the Sysdig agent deployment:
Create a service account for the project:
oc create serviceaccount SERVICE-ACCOUNT
Add the service account to privileged Security Context
Constraints:
oc adm policy add-scc-to-user privileged -n PROJECT-NAME -z SERVICE-ACCOUNT -z node-analyzer
Add the service account to the cluster-reader
Cluster Role:
oc adm policy add-cluster-role-to-user cluster-reader -n PROJECT-NAME -z SERVICE-ACCOUNT -z node-analyzer
Installation
Helm
Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.
Manifests
Download
the sample files:
sysdig-agent-daemonset-v2.yaml
sysdig-agent-clusterrole.yaml
sysdig-agent-configmap.yaml
sysdig-agent-service.yaml
Create the sysdig-agent
cluster role and assign it to the service account:
oc apply -f sysdig-agent-clusterrole.yaml
oc adm policy add-cluster-role-to-user sysdig-agent -n PROJECT-NAME -z SERVICE-ACCOUNT
Create a secret key:
oc create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n PROJECT-NAME
If you created a service account name other than sysdig-agent
:
Edit sysdig-agent-daemonset-v2.yaml
to provide your custom value:``
serviceAccount: SERVICE-ACCOUNT
Edit sysdig-agent-configmap.yaml
to add the collector address
,
port
, and the SSL/TLS
information:
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
- For SaaS, find the collector address for your
region.
- For On-prem, enter the collector endpoint defined in your
environment.
check_certificate
should be set to false
if a self-signed
certificate or private, CA-signed cert is used. See also Set Up SSL Connectivity to the Backend.
Apply the sysdig-agent-configmap.yaml
file:
oc apply -f sysdig-agent-configmap.yaml -n PROJECT-NAME
FOR RHCOS ONLY: To enable the eBPF probe required for COS, uncomment the following parameters in sysdig-agent-daemonset-v2.yaml
under the env section:`
env:
- name: SYSDIG_BPF_PROBE
value: ""
Apply the sysdig-agent-service.yaml
file:
oc apply -f sysdig-agent-service.yaml -n PROJECT-NAME
This allows the agent to receive Kubernetes audit events from the
Kubernetes API server. See Kubernetes Audit
Logging for information
on enabling Kubernetes audit logging.
Apply the daemonset-v2.yaml
file:
oc apply -f sysdig-agent-daemonset-v2.yaml -n PROJECT-NAME
The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.
Next Steps
You can continue with instructions in Additional Options.
3.4 - Steps for Rancher
Preparation
General Requirements
You can review Agent Install: Kubernetes | GKE | OpenShift | IBM and the Agent Installation Requirements for additional context, if desired.
The Sysdig agent requires a kernel module in order to be installed
successfully on a host. On RancherOS distributions, the Unix version
does not match the provided headers, and the agent might fail to install
correctly. Therefore, you must install the kernel headers manually.
For RancherOS distributions, the kernel headers are available in the
form of a system service and therefore are enabled using the ros
service command:
$ sudo ros service enable kernel-headers-system-docker
$ sudo ros service up -d kernel-headers-system-docker
Some cloud hosting service providers supply pre-configured Linux
instances with customized kernels. You may need to contact your
provider’s support desk for instructions on obtaining appropriate header
files, or for installing the distribution’s default kernel.
Installation
Helm
Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.
Manifests
To deploy agents using Kubernetes manifests, you can download the manifest files, edit them as required, and deploy them using kubectl
.
Download
the sample files:
sysdig-agent-clusterrole.yaml
sysdig-agent-daemonset-v2.yaml
sysdig-agent-configmap.yaml
sysdig-agent-service.yaml
Create a namespace to use for the Sysdig agent.
You can use whatever naming you prefer. This document uses sysdig-agent
for both the namespace and the service account.
The default service account name was automatically defined in sysdig-agent-daemonset-v2.yaml
, at the line: serviceAccount: sysdig-agent.
kubectl create ns sysdig-agent
Create a secret key:
kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
Create a cluster role and service account, and define the cluster
role binding that grants the Sysdig agent rules in the cluster role:
kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
Edit sysdig-agent-configmap.yaml
to add the collector address
,
port
, and the SSL/TLS
information:
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
Apply the sysdig-agent-configmap.yaml
file:
kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
Apply the sysdig-agent-service.yaml
file:
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.
Apply the daemonset-v2.yaml
file :
kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.
Next Steps
You can continue with instructions in Additional Options.
3.5 - Steps for MKE
Mirantis Kubernetes Engine (MKE) formerly Docker Enterprise, is a managed environment for running Kubernetes to deploy containerized applications. As of Sysdig agent version 12.0.1, Sysdig supports all flavors of MKE.
MKE environments require eBPF probe to support agent installation.
The instructions below describe a standard MKE agent install and call out the special steps needed to install the eBPF probe.
Preparation
eBPF-Specific Requirements
Linux kernel versions 4.14 or above.
eBPF probe parameter, --set ebpf.enabled=true
to install eBPF probe. See the instructions given below.
Installation Steps
Identify the appropriate endpoint depending on your Sysdig account region. For more information, see SaaS Regions and IP Ranges.
Helm
Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.
Make sure to add the eBPF parameter to the helm command:
--set ebpf.enabled=true
Next Steps
You can continue with instructions in Additional Options.
3.6 - Using Node Leases
The Sysdig agent uses Kubernetes
Lease
to control how and when connections are made to the Kubernetes API
Server. This mechanism prevents overloading the Kubernetes API server
with connection requests during agent bootup.
Kubernetes node leases are automatically created for agent version
12.0.0 and above. On versions prior to 12.0.0, you must configure node
leases as given in the KB article.
Prerequisites
Types of Leases
The agent creates the following leases:
Cold Start
During boot up, the Sysdig agent connects to the Kubernetes API server
to retrieve Kubernetes metadata and build a cache. The cold-start
leases control the number of agents that build up this cache at any
given time. An agent will grab a lease, build its cache, and then
release the lease so that another agent can build its cache. This
mechanism prevents agents from creating a “boot storm” which can
overwhelm the API server in large clusters.
Delegation
In Kubernetes environments, two agents are marked as delegated
in each
cluster. The delegated
agents are the designated agents to request
more data from the API server and produce KubeState metrics. The
delegation
leases will not be released until the agent is terminated.
View Leases
To view the leases, run the following:
$ kubectl get leases -n sysdig-agent
You will see an output similar to the following:
NAME HOLDER AGE
cold-start-0 20m
cold-start-1 20m
cold-start-2 21m
cold-start-3 ip-10-20-51-167 21m
cold-start-4 21m
cold-start-5 21m
cold-start-6 20m
cold-start-7 21m
cold-start-8 20m
cold-start-9 ip-10-20-51-166 21m
delegation-0 ip-10-20-52-53 21m
delegation-1 ip-10-20-51-98 21m
Troubleshoot Leases
Verify Configuration
When lease-based delegation is working as expected, the agent logs show
one of the following:
Getting pods only for node <node>
Getting pods for all nodes.
Both (occasionally on the delegated nodes)
Run the following to confirm that it is working:
$ kubectl logs sysdig-agent-9l2gf -n sysdig-agent | grep -i "getting pods"
The configuration is working as expected if the output on a pod is
similar to the following:
2021-05-05 02:48:32.877, 15732.15765, Information, cointerface[15738]: Only getting pods for node ip-10-20-51-166.ec2.internal
Unable to Create Leases
The latest Sysdig ClusterRole is required for the agent to create
leases. If you do not have the latest ClusterRole or if you have not
configured the ClusterRole correctly, the logs show the following error:
Error, lease_pool_manager[2989554]: Cannot access leases objects: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:sysdig-agent:sysdig-agent" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "sysdig-agent"
Contact Sysdig Support for help.
Optional Agent Configuration
Several configuration options exist for leases. It is recommended to not
change the default settings unless prompted by Sysdig Customer Support.
k8s_coldstart:
enabled: <true/false>
| true above agent versions 12.0.0
| When true, the agent will attempt to create cold-start leases to control the number of agents which are allowed to build their cache at one time. |
k8s_coldstart:
max_parallel_cold_start: <int>
| 10 | The number of cold-start leases to be created. This is the number of agents that can connect to the API Server simultaneously during agent initialization. |
k8s_coldstart:
namespace: <string>
| sysdig-agent
| The namespace to be created. This shouldn't be needed in agent version 12.0.0 because the DownwardAPI in the ClusterRole will provide the appropriate namespace. |
k8s_coldstart:
enforce_leader_election: <true/false>
| false
| When true , the agent will not fall back to the previous method if it cannot create leases.This can be useful if the previous method caused API Server problems. |
k8s_delegation_election: <true/false>
| true above agent versions 12.0.0
| When true , the agent will create delegation leases to control which set of agents generate global cluster metrics. |
4 - Agent Install: Non-Orchestrated
This section describes how to install the Sysdig agent directly on a
Linux host, without using an orchestrator, such as Kubernetes or Mesos.
The agent can be installed in two ways:
The steps for each flavor differ slightly depending on whether you are
using the SaaS or on-premises version of the Sysdig platform.
If you are installing the Sysdig agent in an environment that has Kubernetes, use the Agent Install: Kubernetes instructions instead.
Prerequisites
See Agent Installation Requirements for information on the following:
Supported Linux distributions
Network connection
Sysdig access key
Cloud service providers (AWS, Google, and Microsoft Azure) and
any steps you may need to configure to integrate the Sysdig
agent.
kernel headers: The Sysdig agent requires kernel header files in
order to install successfully on a host, and the agent is delivered with
precompiled headers. If the hosts in your environment match the kernel
versions included with the agent, no special action is needed .In some cases, the hosts in your environment may use Unix versions that do not match the provided headers, and the agent may fail to install correctly. In those cases, you must install the kernel headers manually. See About Kernel Headers and the Kernel Module for details.
Run any commands as root or with the sudo
command.
Retrieve the Sysdig access key.
Collect the configuration parameters.
Configuration Options
Option | Description |
---|
ACCESS_KEY | The agent access key. You can retrieve this from Settings > Agent Installation in either Sysdig Monitor or Sysdig Secure. |
tags | The list of tags for the host where the agent is installed. For example: role:webserver , location:europe , role:webserver |
COLLECTOR | The collector URL for Sysdig Monitor or Sysdig Secure. This value is region-dependent in SaaS and is auto-completed on the Get Started page in the Monitor UI or Data Sources page in Secure. It is a custom value in on-prem installations. See SaaS Regions and IP Ranges. |
collector_port | The default is 6443. |
SECURE | Use a secure SSL/TLS connection to send metrics to the collector. This option is enabled by default. |
CHECK_CERTIFICATE | (On-prem) Determines strong SSL certificate check for Sysdig Monitor on-premises installation. Set to true when using SSL/TLS to connect to the collector service to ensure that a valid SSL/TLS certificate is installed. For more information, see Set Up SSL Connectivity to the Backend. |
ADDITIONAL_CONF | Optional. Use this option to provide custom configuration values to the agent as environment variables. If provided, will be appended to agent configuration file. For example, For example, file log configuration. |
bpf | Enables eBPF probe. The path to the probe file that is either built or downloaded. |
Installing Agent Using Containers
The Sysdig agent can be deployed as a docker container.
The commands below can also be copied from the Get Started page. In that case, your access key will already be included in the command automatically.
SaaS
Installing As Two Containers
The agent is installed by running sysdig/agent-kmodule
, followed by running sysdig/agent-slim
. See Installation Options for description about agent-slim
and agent-kmodule
.
Every host restart requires subsequent running of agent-kmodule
and
agent-slim
containers.
Collect the configuration parameters.
Build and load the kernel module:
If you are not using eBPF, use the following:
docker run -it --privileged --rm --name sysdig-agent-kmodule \
-v /usr:/host/usr:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules \
quay.io/sysdig/agent-kmodule
If you are using eBPF use the following:
docker run -it --privileged --rm --name sysdig-agent-kmodule \
-e SYSDIG_BPF_PROBE="" \
-v /etc/os-release:/host/etc/os-release:ro \
-v /root/.sysdig:/root/.sysdig \
-v /usr:/host/usr:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
quay.io/sysdig/agent-kmodule
Configure kernel module to load during system boot.
If you are not using eBPF, use the following commands to configure the Linux system to automatically load the kernel module during system boot.
$ sudo mkdir -p /etc/modules-load.d
$ sudo bash -c "echo sysdigcloud-probe > /etc/modules-load.d/sysdigcloud-probe.conf"
Run the agent module providing the access key and, optionally, user-defined tags:
If you are not using eBPF, use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
[-e TAGS=[TAGS]]
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
--shm-size=512m \
quay.io/sysdig/agent-slim
If you are using eBPF use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host\
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
[-e TAGS=[TAGS]]
-e SYSDIG_BPF_PROBE="" \
-v /sys/kernel/debug:/sys/kernel/debug:ro \
-v /root/.sysdig:/root/.sysdig \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
--shm-size=512m \
quay.io/sysdig/agent-slim
Installing As Single Container (Legacy)
Collect the configuration parameters.
Run the agent container providing the access key and, optionally, user-defined tags:
If you are not using eBPF, use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host\
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-e TAGS=[TAGS] \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
--shm-size=512m \
quay.io/sysdig/agent
If you are using eBPF use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host\
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-e TAGS=[TAGS] \
-e SYSDIG_BPF_PROBE="" \
-v /sys/kernel/debug:/sys/kernel/debug:ro \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
--shm-size=512m \
quay.io/sysdig/agent
On-Premises
Installing As Two Containers
Collect the configuration parameters:
Build and load the kernel module:
If you are not using eBPF, use the following:
docker run -it --privileged --rm --name sysdig-agent-kmodule \
-v /usr:/host/usr:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules \
quay.io/sysdig/agent-kmodule
If you are using eBPF use the following:
docker run -it --privileged --rm --name sysdig-agent-kmodule \
-e SYSDIG_BPF_PROBE="" \
-v /etc/os-release:/host/etc/os-release:ro \
-v /root/.sysdig:/root/.sysdig \
-v /usr:/host/usr:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
quay.io/sysdig/agent-kmodule
Configure kernel module to load during system boot.
If you are not using eBPF, use the following commands to configure the Linux system to automatically load the kernel module during system boot.
$ sudo mkdir -p /etc/modules-load.d
$ sudo bash -c "echo sysdigcloud-probe > /etc/modules-load.d/sysdigcloud-probe.conf"
Run the agent module providing the access key and, optionally, user-defined tags:
If you are not using eBPF, use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
[-e TAGS=[TAGS]]
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
--shm-size=512m \
quay.io/sysdig/agent-slim
If you are using eBPF use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
[-e TAGS=[TAGS]]
-e SYSDIG_BPF_PROBE="" \
-v /sys/kernel/debug:/sys/kernel/debug:ro \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
--shm-size=512m \
quay.io/sysdig/agent-slim
Installing As Single Container (Legacy)
Collect the configuration parameters.
Run the agent module providing the access key and, optionally, user-defined tags:
If you are not using eBPF, use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
[-e TAGS=[TAGS]]
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
--shm-size=512m \
quay.io/sysdig/agent
If you are using eBPF use the following:
docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
[-e TAGS=[TAGS]]
-e SYSDIG_BPF_PROBE="" \
-v /sys/kernel/debug:/sys/kernel/debug:ro \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
--shm-size=512m \
quay.io/sysdig/agent
Installing Agent as a Service on Linux Host
Use these instructions to install the agent on the host itself, not in a
container. Install on each host in the environment.
The command lines below can also be copy/pasted from the Welcome wizard
or the Settings>Agent Installation
page in the Sysdig Monitor
interface.
In that case, your access key will already be included in the command
automatically.
The Sysdig agent depends on several python modules, some of which might
not be installed on the hosts where the agent is running as a service.
When the required dependencies are not available, the sdchecks
component in the agent will report errors in the log files, such as:
>> Error, sdchecks[0] ModuleNotFoundError: No module named 'posix_ipc'
To address these errors, install the missing modules using the
pip install
command.
SaaS
Run the following command:
curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- --access_key [ACCESS_KEY] --collector [COLLECTOR_ADDRESS] [--tags [TAGS]]
Where
[ACCESS_KEY]
is your unique agent access key string. For example, 1234-your-key-here-1234.
TAGS
is an optional list of user-defined agent tags. For example, role:webserver,location:europe
.
See SaaS Regions and IP Ranges to find the collector endpoint for your region.
Restart the agent and start the service:
sudo systemctl enable dragent
On-Premises
Run the following command:
curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- --access_key [ACCESS_KEY] --collector [COLLECTOR_ADDRESS] --secure true --check_certificate true [--tags [TAGS]]
For configuration parameters, see Configuration Options.
Restart the agent and start the service:
sudo systemctl enable dragent
Connect to the Sysdig Backend via Static IPs (SaaS only)
Sysdig provides a list of static IP addresses that can be whitelisted in
a Sysdig environment, allowing users to establish a network connection
to the Sysdig backend without opening complete network connectivity.
This is done by setting the Collector IP to
collector-static.sysdigcloud.com:
user@host:~$ docker run --name sysdig-agent \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e TAGS=[TAGS] \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
-e COLLECTOR=collector-static.sysdigcloud.com \
-e COLLECTOR_PORT=6443 \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
--shm-size=512m \
quay.io/sysdig/agent-slim
Guidelines for Manual Agent Installation
In the following cases, we recommend that you manually install the agent.
See Agent Install: Manual Linux Installation for more information.
5 - Agent Install: Manual Linux Installation
Manual installation of the native Linux agent is recommended in the following cases:
Full control over the deployment process
Integration with configuration management tools
Custom kernel
Unsupported distribution (within Debian/Fedora flavors)
Otherwise, you may want to follow Agent Install: Non-Orchestrated
Note: If you are installing the Sysdig agent in an orchestrated infrastructure such as Kubernetes, Mesos/Marathon, use the respective
Installation Guides:
Prerequisites
Review the Agent Installation Requirements.
Collect the configuration parameters:
ACCESS_KEY
: Your unique access key string.
Inability to retrieve the key indicates that the administrator of your instance might have it turned off for non-admin users. Contact your Sysdig administrator to receive the key. If you still have issues please contact Sysdig Support.
TAGS
: The optional parameter you can use to list one or more tags for this host. Tagging your hosts is highly recommended. Agent Tags allow you to sort nodes of your infrastructure into custom groups in Sysdig Monitor. Replace the [TAGS] parameter above with a comma-separated list of
TAG_NAME:TAG_VALUE
. For example: role:webserver,location:europe
COLLECTOR
: The collector URL for Sysdig Monitor or Sysdig Secure. This value is region-dependent in SaaS and is auto-completed on the Get Started page in the Monitor UI or the Data Sources page in Secure. It is a custom value in on-prem installations.
SECURE
: Use a secure SSL/TLS connection to send metrics to the collector. This option is enabled by default.
Run the commands as root or with sudo.
Installation
Follow the instructions for the appropriate Linux distribution:
Debian, Ubuntu
Use the Sysdig Monitor GPG key, configure the apt repository, and update the package list:
curl -s https://download.sysdig.com/DRAIOS-GPG-KEY.public | apt-key add -
curl -s -o /etc/apt/sources.list.d/draios.list http://download.sysdig.com/stable/deb/draios.list
apt-get update
Install kernel development files.
Note: The following command might not work with every kernel. Ensure that you
customize the name of the package properly.
apt-get -y install linux-headers-$(uname -r)
Install, configure, and restart the Sysdig agent.
apt-get -y install draios-agent
echo customerid: ACCESS_KEY >> /opt/draios/etc/dragent.yaml
echo tags: [TAGS] >> /opt/draios/etc/dragent.yaml
echo collector: COLLECTOR_URL >> /opt/draios/etc/dragent.yaml
echo ssl: true >> /opt/draios/etc/dragent.yaml
service dragent restart
See Prerequisites for the configuration parameters required.
CentOS, RHEL, Fedora, Amazon AMI, Amazon Linux 2
Trust the Sysdig Monitor GPG key, configure the yum repository.
$ rpm --import https://download.sysdig.com/DRAIOS-GPG-KEY.public
$ curl -s -o /etc/yum.repos.d/draios.repo http://download.sysdig.com/stable/rpm/draios.repo
Install the EPEL repository.
Note: The following command is required only if DKMS is not available in
the distribution. You can verify if DKMS is available with
yum list dkms
.
The command below contains a sample release number; be sure to
update with the correct release.
$ rpm -i http://mirror.us.leaseweb.net/epel/6/i386/epel-release-6-8.noarch.rpm
Install kernel development files.
Note: The following command might not work with every kernel. Make sure to
customize the name of the package properly.
$ yum -y install kernel-devel-$(uname -r)
Install, configure, and start the Sysdig agent.
$ yum -y install draios-agent
$ echo customerid: ACCESS_KEY >> /opt/draios/etc/dragent.yaml
$ echo tags: [TAGS] >> /opt/draios/etc/dragent.yaml
$ echo collector: COLLECTOR_URL >> /opt/draios/etc/dragent.yaml
$ echo ssl: true >> /opt/draios/etc/dragent.yaml
$ echo secure: true >> /opt/draios/etc/dragent.yaml
$ sudo systemctl enable dragent
$ sudo systemctl start dragent
See Prerequisites for the configuration parameters required.
If you using a non-systemd Linux distribution, use the service command to start dragent
.
$ service dragent restart
Other Linux Distributions
The Sysdig agent is unsupported outside of the Debian, Fedora, and Amazon distributions.
6 - Agent Install: Amazon ECS
Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that helps to easily deploy, manage, and scale containerized applications.
This section describes how to install the Sysdig agent container on each underlying host in your ECS cluster. Once installed, the agent will automatically begin monitoring all of your hosts, service and tasks.
These instructions are valid only for ECS clusters using EC2 instances. For information on ECS Fargate clusters, see AWS Fargate Serverless Agents.
Installation
To install Sysdig agent on ECS, do the following:
Create an ECS task definition for the Sysdig agent.
Register the task definition in your AWS account.
Create a service with the previous task definition to run the Sysdig agent in each of the nodes of your ECS cluster.
Create an ECS Task Definition
- Collect the following configuration parameters:
ACCESS_KEY
: The agent access key. You can retrieve this from Settings > Agent Installation in either Sysdig Monitor or Sysdig Secure.COLLECTOR
: Use the collector address for your region. For more information, see SaaS Regions and IP Ranges.TAGS
: The list of tags for the host where the agent is installed. For example: role:webserver
, location:europe
, role:webserver
- Use the above values to customize the JSON snippet below and save it as a file named
sysdig-agent-ecs.json
.
Note that memory and cpu have both been set to 1024, depending on the size of your cluster you might want to tune those values.
{
"family": "sysdig-agent-ecs",
"containerDefinitions": [
{
"name": "sysdig-agent",
"image": "quay.io/sysdig/agent-slim",
"cpu": 1024,
"memory": 1024,
"privileged": true,
"environment": [
{
"name": "ACCESS_KEY",
"value": "$ACCESS_KEY"
},
{
"name": "COLLECTOR",
"value": "$COLLECTOR"
},
{
"name": "TAGS",
"value": "$TAG1,TAG2"
}
],
"mountPoints": [
{
"readOnly": true,
"containerPath": "/host/boot",
"sourceVolume": "boot"
},
{
"containerPath": "/host/dev",
"sourceVolume": "dev"
},
{
"readOnly": true,
"containerPath": "/host/lib/modules",
"sourceVolume": "modules"
},
{
"readOnly": true,
"containerPath": "/host/proc",
"sourceVolume": "proc"
},
{
"containerPath": "/host/var/run/docker.sock",
"sourceVolume": "sock"
},
{
"readOnly": true,
"containerPath": "/host/usr",
"sourceVolume": "usr"
}
],
"dependsOn": [
{
"containerName": "sysdig-agent-kmodule",
"condition": "SUCCESS"
}
]
},
{
"name": "sysdig-agent-kmodule",
"image": "quay.io/sysdig/agent-kmodule",
"memory": 512,
"privileged": true,
"essential": false,
"mountPoints": [
{
"readOnly": true,
"containerPath": "/host/boot",
"sourceVolume": "boot"
},
{
"containerPath": "/host/dev",
"sourceVolume": "dev"
},
{
"readOnly": true,
"containerPath": "/host/lib/modules",
"sourceVolume": "modules"
},
{
"readOnly": true,
"containerPath": "/host/proc",
"sourceVolume": "proc"
},
{
"containerPath": "/host/var/run/docker.sock",
"sourceVolume": "sock"
},
{
"readOnly": true,
"containerPath": "/host/usr",
"sourceVolume": "usr"
}
]
}
],
"pidMode": "host",
"networkMode": "host",
"volumes": [
{
"name": "sock",
"host": {
"sourcePath": "/var/run/docker.sock"
}
},
{
"name": "dev",
"host": {
"sourcePath": "/dev/"
}
},
{
"name": "proc",
"host": {
"sourcePath": "/proc/"
}
},
{
"name": "boot",
"host": {
"sourcePath": "/boot/"
}
},
{
"name": "modules",
"host": {
"sourcePath": "/lib/modules/"
}
},
{
"name": "usr",
"host": {
"sourcePath": "/usr/"
}
}
],
"requiresCompatibilities": [
"EC2"
]
}
Register a Task Definition
Once your task definition is ready, ensure that you register it in your AWS account:
aws ecs register-task-definition \
--cli-input-json file://sysdig-agent-ecs.json
Run the Agent as an ECS Service
Using the ECS task definition you have created, create a service in the cluster that you want to monitor with Sysdig.
aws ecs create-service \
--cluster $CLUSTER_NAME \
--service-name sysdig-agent-svc \
--launch-type EC2 \
--task-definition sysdig-agent-ecs \
--scheduling-strategy DAEMON
With the agent installed, Sysdig will begin auto-discovering your containers and other resources of your ECS environment.
Using ECS Anywhere
If you’re using ECS Anywhere, change the launch type to EXTERNAL
when the service is created.
aws ecs create-service \
--cluster $CLUSTER_NAME \
--service-name sysdig-agent-svc \
--launch-type EXTERNAL \
--task-definition sysdig-agent-ecs \
--scheduling-strategy DAEMON
Enable Log Driver
You can send the logs from the containers running the ECS tasks to the log groups in CloudWatch Logs. You can send agent container log files to AWS by enabling the log driver, awslogs
. To do so:
Add the following section to each of the container definitions you’ve created above:
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "$YOUR_LOG_GROUP",
"awslogs-region": "$AWS_REGION",
"awslogs-stream-prefix": "sysdig"
}
Update your task definition and the service to enable the logs.
7 - Agent Install: IKS (IBM Cloud Monitoring)
IBM Cloud maintains the documentation for Sysdig agent installation on IBM Cloud Kubernetes Service (IKS).
For more information, see the IBM Cloud Monitoring
documentation:
8 - Agent Install: Mesos | Marathon | DCOS
Marathon is the container orchestration platform for Mesosphere’s
Datacenter Operating System (DC/OS) and Apache Mesos.
This guide describes how to install the Sysdig agent container on each
underlying host in your Mesos cluster. Once installed, the agent will
automatically connect to the Mesos and Marathon APIs to pull relevant
metadata about the environment and will begin monitoring all of your
hosts, apps, containers, and frameworks.
Prerequisites
Installation
In this three-part installation, you:
Deploy the Sysdig agent on all Mesos Agent (Slave) nodes,
either automatically or by creating and posting a .json
file to
the leader Marathon API server.
Deploy the Sysdig agent on the Mesos Master nodes.
Special configuration steps: modify the Sysdig agent config file to
monitor Marathon instances.
Deploy the Sysdig Agent on Mesos Agent Nodes
Preferred Option: Automatic install (DC/OS 1.11+)
If you’re using DC/OS 1.8 or higher, then you can find Sysdig agent in the
Mesosphere Universe marketplace and install it from there.
It will automatically deploy the Sysdig agent container on each of your
Mesos Agent nodes as a Marathon app.
Proceed to Deploy the Sysdig Agent.
Alternate Option: Post a .json file
If you are using a version of DC/OS earlier than 1.8 then:
Create a JSON file for Marathon, in the following format. See configuration parameters for details.
{
"backoffFactor": 1.15,
"backoffSeconds": 1,
"constraints": [
[
"hostname",
"UNIQUE"
]
],
"container": {
"docker": {
"forcePullImage": true,
"image": "sysdig/agent",
"parameters": [],
"privileged": true
},
"type": "DOCKER",
"volumes": [
{
"containerPath": "/host/var/run/docker.sock",
"hostPath": "/var/run/docker.sock",
"mode": "RW"
},
{
"containerPath": "/host/dev",
"hostPath": "/dev",
"mode": "RW"
},
{
"containerPath": "/host/proc",
"hostPath": "/proc",
"mode": "RO"
},
{
"containerPath": "/host/boot",
"hostPath": "/boot",
"mode": "RO"
},
{
"containerPath": "/host/lib/modules",
"hostPath": "/lib/modules",
"mode": "RO"
},
{
"containerPath": "/host/usr",
"hostPath": "/usr",
"mode": "RO"
}
]
},
"cpus": 1,
"deployments": [],
"disk": 0,
"env": {
"ACCESS_KEY": "ACCESS_KEY=YOUR-ACCESS-KEY-HERE",
"CHECK_CERT": "false",
"SECURE": "true",
"TAGS": "example_tag:example_value",
"name": "sdc-agent",
"pid": "host",
"role": "monitoring",
"shm-size": "350m"
},
"executor": "",
"gpus": 0,
"id": "/sysdig-agent",
"instances": 1,
"killSelection": "YOUNGEST_FIRST",
"labels": {},
"lastTaskFailure": {
"appId": "/sysdig-agent",
"host": "YOUR-HOST",
"message": "Container exited with status 70",
"slaveId": "1fa6f2fc-95b0-445f-8b97-7f91c1321250-S2",
"state": "TASK_FAILED",
"taskId": "sysdig-agent.3bb0759d-3fa3-11e9-b446-c60a7a2ee871",
"timestamp": "2019-03-06T00:03:16.234Z",
"version": "2019-03-06T00:01:57.182Z"
},
"maxLaunchDelaySeconds": 3600,
"mem": 850,
"networks": [
{
"mode": "host"
}
],
"portDefinitions": [
{
"name": "default",
"port": 10101,
"protocol": "tcp"
}
],
"requirePorts": false,
"tasks": [
{
"appId": "/sysdig-agent",
"healthCheckResults": [],
"host": "YOUR-HOST-IP",
"id": "sysdig-agent.0d5436f4-3fa4-11e9-b446-c60a7a2ee871",
"ipAddresses": [
{
"ipAddress": "YOUR-HOST-IP",
"protocol": "IPv4"
}
],
"localVolumes": [],
"ports": [
4764
],
"servicePorts": [],
"slaveId": "1fa6f2fc-95b0-445f-8b97-7f91c1321250-S2",
"stagedAt": "2019-03-06T00:09:04.232Z",
"startedAt": "2019-03-06T00:09:06.912Z",
"state": "TASK_RUNNING",
"version": "2019-03-06T00:09:04.182Z"
}
],
"tasksHealthy": 0,
"tasksRunning": 1,
"tasksStaged": 0,
"tasksUnhealthy": 0,
"unreachableStrategy": {
"expungeAfterSeconds": 0,
"inactiveAfterSeconds": 0
},
"upgradeStrategy": {
"maximumOverCapacity": 1,
"minimumHealthCapacity": 1
},
"version": "2019-03-06T00:09:04.182Z",
"versionInfo": {
"lastConfigChangeAt": "2019-03-06T00:09:04.182Z",
"lastScalingAt": "2019-03-06T00:09:04.182Z"
}
}
See Environment Variables for Agent Config File for the Sysdig name:value
definitions.
Complete the “cpus
”, “mem
” and “labels
” (i.e. Marathon labels)
entries to fit the capacity and requirements of the cluster
environment.
Update the created.json
file to the leader Marathon API server:
$ $curl -X POST http://$(hostname -i):8080/v2/apps -d @sysdig.json -H "Content-type: application/json"
Deploy the Sysdig Agent on Master Nodes
After deploying the agent to the Mesos Agent nodes, you will install
agents on each of the Mesos Master nodes as well.
If any cluster node has both Mesos Master and Mesos Agent roles, do not
perform this installation step on that node. It already will have a
Sysdig agent installed from the procedure in step A. Running duplicate
Sysdig agents on a node will cause errors.
Use the Agent Install: Non-Orchestrated instructions to install the agent directly on each of your Mesos Master nodes.
When the Sysdig agent is successfully installed on the master nodes, it
will automatically connect to the local Mesos and Marathon (if
available) API servers via http://localhost:5050
and
http://localhost:8080
respectively, to collect cluster configuration
and current state metadata in addition to host metrics.
Additional Configuration
In certains situations, you may need to add additional configurations to
the dragent.yaml
file:
Descriptions and examples are shown below.
Sysdig Agent Unable to Run on the Mesos API Server
Mesos allows multiple masters. If the API server can not be instrumented
with a Sysdig agent, simply delegate ONE other node with an agent
installed to remotely receive infrastructure information from the API
server.
NOTE: If you manually configure the agent to point to a master with a
static configuration file entry, then automatic detection/following of
leader changes will no longer be enabled.
Add the following Mesos parameter to the delegated agent’s
dragent.yaml
file to allow it to connect to the remote API server and
authenticate, either by:
a. Directly editing dragent.yaml
on the host, or
b. Converting the YAML code to a single-line format and adding it as an
ADDITIONAL_CONF
argument in a Docker command.
See Understanding the Agent Configuration for details.
Specify the API server’s connection method, address, and port. Also
specify credentials if necessary.
YAML example:
mesos_state_uri: http://[acct:passwd@][hostname][:port]
marathon_uris:
- http://[acct:passwd@][hostname][:port]
Although marathon_uris:
is an array, currently only a single “root”
Marathon framework per cluster is supported. Multiple side-by-side
Marathon frameworks should not be configured in order for our agent to
function properly. Multiple side-by-side “root” Marathon frameworks on
the same cluster are currently not supported. The only supported
multiple-Marathon configuration is with one “root” Marathon and other
Marathon frameworks as its apps.
Mesos API Server Requires Authentication
If the agent is installed on the API server but the API server uses a
different port or requires authentication, those parameters must be
explicitly specified.
Add the following Mesos parameters to the API server’s dragent.yaml
to
make it connect to the API server and authenticate with any unique
account and password, either by:
a. Directly editing dragent.yaml
on the host, or
b. Converting the YAML code to a single-line format and adding it as an
ADDITIONAL_CONF
argument in a Docker command.
See Understanding the Agent Configuration for details.
Specify the API server’s protocol, user credentials, and port:
mesos_state_uri: http://[username:password@][hostname][:port]
marathon_uris:
- http://[acct:passwd@][hostname][:port]
*HTTPS protocol is also supported.
In troubleshooting cases where auto-detection and reporting of your
Mesos infrastructure needs to be temporarily turned off in a designated
agent:
Comment out the Mesos parameter entries in the agent’s dragent.yaml
file.
Example parameters to disable: mesos_state_uri, marathon_uris
If the agent is running on the API server (Master node) and
auto-detecting a default configuration, you can add the line:
mesos_autodetect: false
either directly in the dragent.yaml file or as an ADDITIONAL_CONF
parameter in a Docker command.
Restart the agent.
9 - Airgapped Agent Installation
Airgapped environments are those that do not have network access to the public internet.
At startup, the agent will try to compile its own version of the probes, provided kernel header packages are installed on the host. Failing that, the agent will try to download pre-compiled probes, sysdigcloud-probe-<suffix>.ko
or sysdigcloud-probe-bpf-<suffix>.o
, from the Sysdig download site over the internet.
In an airgapped environemnt, you cannot download these artifacts. Therefore, before installing the agent, you will have to compile sysdigcloud-probe-<suffix>
for each kernel version in your environment, and make it available to the installed agents through an internally accessible URL.
Prerequisites
- A machine with internet access where you can download the required artifacts
- A machine in your airgapped environment where you can build your probes
- Tool to transfer artifacts to the machine in your airgapped environment
- Docker installed
Overview
Sysdig provides a tool, named the probe builder, to help you build the probes for different kernels and for a specific agent version. After downloading the required artifacts on a machine connected to the internet, you can copy them to an airgapped host, build your own probes, and make them available to your agent installations.
On a Machine with Internet Connectivity
Prepare the Sysdig Probe Builder Images
On a machine with internet connectivity, build the Sysdig probe builder container images and create a tar file of the images.
Get the probe builder source code from the repository:
$ git clone https://github.com/draios/probe-builder
Build the container image for the probe builder:
$ docker build -t airgap/sysdig-probe-builder probe-builder/
Build the images for each supported distribution-compiler combination:
$ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock airgap/sysdig-probe-builder:latest -P -b airgap/
Running this command will create a different image tag for each supported combination of distribution-compiler
, with the distro-compiler information suffixed to the image name, airgap/sysdig-probe-builder
. For example, airgap/sysdig-probe-builder:centos-gcc4.8
.
Save all the above images to a tar archive:
$ docker save airgap/sysdig-probe-builder | gzip > builders.tar.gz
(optional) If you are building probes for the Ubuntu kernels, you will also need an ubuntu:latest
image on your airgapped host. You can build it as follows:
$ docker pull ubuntu
$ docker save ubuntu | gzip > ubuntu.tar.gz
Download the Kernel Packages
Download your kernel packages. For more information, see Download Kernel Packages.
Download Probe Source Code
You need to download the probe source code for a specific agent version you want to build your probes for.
For example, for agent version 12.0.0
you would use:
$ git clone https://github.com/draios/agent-libs
$ cd agent-libs
$ git archive agent/12.0.0 --prefix sysdig/ | gzip > sysdig.tar.gz
Transfer the Downloaded Files
Copy the artifacts you have built to the airgapped host machine:
builders.tar.gz
ubuntu.tar.gz
(if needed, see above)sysdig.tar.gz
- Kernel packages
On the Airgapped Host
Load the Builder Images
$ zcat builders.tar.gz | docker load
Unpack the Sysdig Source
Running this command will create the sysdig/
directory in the current directory.
Move the Kernel Packages to a Dedicated Location
Make sure you have all the downloaded kernel package artifacts in a single directory, /directory-containing-kernel-packages/
, for each distribution you want to support.
Run the Probe Builder
Now that you have all your requirements in place, you can run the main probe builder:
$ docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /a-directory-with-some-free-space/:/workspace \
-v /wherever-you-unpacked/sysdig/:/sysdig \
-v /directory-containing-kernel-packages/:/kernels \
airgap/sysdig-probe-builder:latest -B -b airgap/ -- \
-p sysdigcloud-probe -v 12.0.0 -k CustomCentOS
The probes will appear in /a-directory-with-some-free-space/output
.
That directory can be served over HTTP and the URL to the server used as SYSDIG_PROBE_URL
when loading the module (e.g. agent-kmodule
container). As an example, the following sections describe how you can deploy your own nginx
server within your cluster and upload your probes there.
Serve Your Pre-Compiled Probes
Set up a local repository to host the pre-compiled kernel module.
Here is an example with nginx:
$ docker run --rm -v /a-directory-with-some-free-space/output:/usr/share/nginx/html/stable/sysdig-probe-binaries -p 80:80 nginx
Note down the URL and use it as the SYSDIG_PROBE_URL
while installing the agent.
See Run the Probe Builder.
Run the Probe Builder
$ docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /sysdigcloud-probe/:/workspace \
-v /wherever-you-unpacked/sysdig/:/sysdig \
-v /directory-containing-kernel-packages/:/kernels \
airgap/sysdig-probe-builder:latest -B -b airgap/ -- \
-p sysdigcloud-probe -v 12.0.0 -k CustomCentOS
The probes will appear in /sysdigcloud-probe/output
. This directory can be served over HTTP and the URL to the server used as SYSDIG_PROBE_URL
when loading the module. For example, agent-kmodule container
.
Install Agent in a Docker Environment
Install Sysdig agent by pointing SYSDIG_PROBE_URL
to the local
repository:
For docker-based installations:
$ docker run -d --name sysdig-agent --restart always --privileged --net host --pid host -e ACCESS_KEY=WWWWW-YYYY-XXXX-ZZZZ-123456789 -e SECURE=true -e SYSDIG_PROBE_URL=http://www.mywebserver.net:80/ -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro --shm-size=512m sysdig/agent
Where -e SYSDIG_PROBE_URL=http://www.mywebserver:80/
is the local nginx
web server with the loaded module.
Note: To use HTTPS communication with a self-signed or untrusted certificate, use the -e SYSDIG_PROBE_INSECURE_DOWNLOAD=true
environment variable in the above command.
Check the agent log. If the installation is successful, you will see a message as follows:
Evaluating override of environment variables
Trying to download precompiled module from http://mywebserver:80/stable/sysdig-probe-binaries/sysdigcloud-probe-<version>
Download succeeded
Continue with the instructions in Agent Install:
Non-Orchestrated.
Install Agent in a Kubernetes Environment
Open your agent daemonset and update the SYSDIG_PROBE_URL
to point
to the local repository:
- name: SYSDIG_PROBE_URL
value: http://www.mywebserver:80/
If you would like to use secure communication with a self-signed or
untrusted certificate, apply the SYSDIG_PROBE_INSECURE_DOWNLOAD
environment variable.
- name: SYSDIG_PROBE_INSECURE_DOWNLOAD
value: true
Continue with the instructions in Agent Install:
Kubernetes.
10 - Identify Agent Version
Use one of the following methods to determine the version of the agents
installed in your environment:
Sysdig Secure
The Sysdig Agents page displays all the agents deployed your environment. You can determine the version number as well as the agents that are up-to-date, out of date, or approaching being out of date.
- Log in to Sysdig Secure.
- Select Integrations > Sysdig Agents.
For more information, see Sysdig Agents.
Sysdig Monitor
Explore
Segmenting metrics by using agent.version
shows the installed versions
of agents in your environment. For example, segment the uptime
metric
across your environment by using agent.version
. Hover over the graph
to see the list of agent versions.

The image shows the list of agent versions in
n/a.
Dashboard
Use the Sysdig Agent Health Dashboard to determine the agent
versions:
Log in to the Sysdig Monitor.
Select Dashboards and expand Host Infrastructure Dashboards.
Open the Sysdig Agent Health & Status template or create your
own from the template.

The Sysdig Agent and Health & Status Dashboard shows the agent
version corresponding to each host in your environment.