Agent Installation

• 1: Agent Installation Requirements
• 2: Quick Install Sysdig Agent
• 3: Agent Install: Kubernetes
• 3.1: Steps for GKE
• 3.1.1: GKE Autopilot
• 3.1.2: GKE Standard
• 3.2: Steps for OKE
• 3.3: Steps for OpenShift
• 3.4: Steps for Rancher
• 3.5: Steps for MKE
• 3.6: Using Node Leases
• 4: Agent Install: Non-Orchestrated
• 5: Agent Install: Manual Linux Installation
• 6: Agent Install: Amazon ECS
• 7: Agent Install: IKS (IBM Cloud Monitoring)
• 8: Agent Install: Mesos | Marathon | DCOS
• 9: Airgapped Agent Installation
• 10: Identify Agent Version

1 - Agent Installation Requirements

Sysdig agents can be installed on a wide array of Linux hosts. Check your environment to ensure it meets the minimum supported platform, operating system, runtime, and orchestration requirements and uses the appropriate installation instructions.

Versioning Scheme

We recommend that you use the latest version of the agent. Sysdig supports n-3 versions back based on the minor number. For example, if the latest release is v12.0.0, we will support n-3 versions back, up to v11.2.0.

End of Support

Sysdig agents that are older than version 0.85.1, released October 1, 2018, will no longer connect to the Sysdig US-East SaaS platform with default agent values.

Going forward all the agent releases will have a 3-year deprecation policy. This implies:

• Sysdig Support might not be able to help you troubleshoot or address the problems with agents past the deprecation date.

• Sysdig will no longer provide prebuilt kernel probe binaries for these agent releases. You need to build the kernel probe binaries on the fly by using the hosts kernel headers.

  These changes is effective starting Sysdig agent v12.1.0.

Agent Installation Requirements

Orchestration Platforms

Support Matrix for Kubernetes

Sysdig agent version 12.8.1 has been tested on the following list of latest Kubernetes versions. The matrix provides a single view into the supported operating systems, architecture, and runtime versions for different flavors of Kubernetes orchestrators.

(Beta) Additional Orchestration Platforms

If you are not using an orchestrator in your environment, follow the instructions for Agent Install Non-Orchestrated.

Note: Installing the Sysdig agent into a namespace managed by Istio and configured for sidecar auto-injection is not supported. For example, setting kubectl label namespace sysdig-agent istio-injection=enabled. Because the agent behaves more like a host component, it is required to be part of the host PID and network namespace to function correctly. Due to this requirement, deploying the Sysdig agent in Istio with an Envoy sidecar is not supported. However, running the Sysdig agent in a non-injected namespace where Istio is installed and managing other namespaces is fully supported. See Istio integration for more details on using the Sysdig agent to monitor Istio control plane and sidecar metrics.

Linux Distributions and Kernels

Support Matrix for Linux Distributions

Sysdig agent version 12.8.1 (installed as a service) has been tested on the following list of latest linux distros:

(Beta) Linux Distributions

Sysdig agent is supported on the following Linux distributions:

Container Runtimes

Sysdig agent supports the detection of the following:

• Docker
• LXC
• CRI-O
• containerd
• Podman
• Mesos

Support Matrix for Docker

Sysdig agent version 12.8.1 has been tested on the following list of latest linux distros:

Prerequisites for Podman Environments

Sysdig agent supports running as a Podman container.

• Enable Podman API Service for all the users.

  The agent will not able to collect Podman-managed container metadata, such as the container name, if the API service is not enabled.

• Secure rules and policies that depend on container metadata other than the container ID will not work.

• Pausing and terminating containers will not work because Policy actions for Podman are not supported.

• The containers started as a non-root user will have the podman_owner_uid label associated with it if the API service is enabled for that user. The value of podman_owner_uid will be the numeric user ID corresponding to the user that started the container.

Container Registries

Quay.io

For example, to pull the latest agent container from Quay.io:

docker pull quay.io/sysdig/agent

CPU Architectures

x86

Supported Agent Containers

• agent
• agent-slim
• agent-kmodule

ARM (aarch64)

Supported kernel versions are v4.17 and above

Unsupported Features

• Pre-built probes
• Activity Audit
• Sysdig agent installation using the agent container

s390x (zLinux)

Unsupported Features

Probes

No support for pre-built probes on zLinux. For kernel instrumentation, use the kernel module. eBPF probes are not supported on zLinux.

Captures

Capture is not supported on zLinux.

Legacy Agent Installation

Sysdig agent installation using agent container is not supported.

Java Versions and Vendors

Sysdig agent supports the following:

• Java versions: v7 and above
• Vendors: Oracle, OpenJDK

For Java-based applications (Cassandra, Elasticsearch, Kafka, Tomcat, Zookeeper and etc.), the Sysdig agent requires the Java runtime environment (JRE) to be installed to poll for metrics (beans).

If the Docker-container-based Sysdig agent is installed, the JRE is installed alongside the agent binaries and no further dependencies exist. However, if you are installing the service-based agent (non-container) and you do not see the JVM/JMX metrics reporting, your host may not have the JRE installed or it may not be installed in the expected location: usr/bin/java

Minimum Resource Requirements

The resource requirements of the agent are subjective to the size and load of the host— more activity equates to more resources required.

It is typical to see between 5-20KiB/s of bandwidth consumed—different variables can increase the throughput required such as the number of metrics, events, Kubernetes objects, and which products and features are enabled. When a Sysdig Capture is being collected, you can expect to see a spike in bandwidth while the capture file is being ingested.

We do not recommend placing bandwidth shaping or caps on the agent to ensure data can be sent to our collection service. For more information, see Tuning Sysdig Agent.

Additional Requirements

Access key

The installation of the Sysdig agent requires an access key.

This key and the agent installation instructions are presented to you after activating your account and using a web-based wizard upon initial login.

The same information can also be found in the Settings > Agent Installation menu of the web interface after logging in. See Agent Installation: Overview and Key for details.

Network connection

A Sysdig agent (containerized or native) is installed into each host being monitored and will need to be able to connect to the Sysdig Monitor backend servers to report host metrics. The agent must be able to reach the Sysdig Collector addresses. For example, for US East, it is ‘collector.sysdigcloud.com’ (via multiple IPs) over port tcp/6443 . See Sysdig Collector Ports for supported ports for other regions.

The agent supports the HTTP proxy for communicating with Sysdig backend components. For more information, see Enable HTTP Proxy for Agents.

2 - Quick Install Sysdig Agent

Sysdig provides you with quick-install commands pre-filled with some of your environment variables to get started with Sysdig agent. You choose the deployment type and Sysdig gives you auto-generated command to ease your installation experience.

Access from Get Started Pages

1. Log in as the administrator to Sysdig Monitor or Sysdig Secure.

2. Select the Get Started page.

   

3. Click Install the Agent, select the appropriate deployment type, and copy the auto-generated code, filling in remaining variable values as required.

Sample Usage

Kubernetes

Helm

Helm is the recommended option for installing agents on Kubernetes.

Example

The shell commands below will create a new Kubernetes namespace called sysdig-agent and deploy the agent with a Helm release name of sysdig. Be sure to replace the configuration options with the values specific to your setup.

SaaS

kubectl create ns sysdig-agent
helm repo add sysdig https://charts.sysdig.com
helm repo update
helm install sysdig-agent \
    --namespace=sysdig-agent \
    --set global.sysdig.accessKey=`1234-your-key-here-1234` \
    --set global.sysdig.region='us1' \
    --set global.clusterConfig.name='my_cluster' \
    --set agent.sysdig.settings.tags='linux:ubuntu,dept:dev,local:nyc' \
    sysdig/sysdig-deploy

On-Prem

kubectl create ns sysdig-agent
helm repo add sysdig https://charts.sysdig.com
helm repo update
helm install sysdig-agent \
    --namespace=`dev` \
    --set global.sysdig.accessKey=`1234-your-key-here-1234` \
    --set agent.collectorSettings.collectorHost='mycollector.elb.us-west-1.amazonaws.com' \
    --set agent.collectorSettings.collectorPort=`6443` \
    --set agent.sysdig.settings.tags='linux:ubuntu,dept:dev,local:nyc' \
    --set agent.sysdig.settings.k8s_cluster_name='my_cluster' \
    sysdig/sysdig-deploy

Options

For the latest helm-based installation instructions and configuration options, see sysdig-deploy.

Script

If you cannot utilize helm, we also provide a script which will download and apply Kubernetes manifests to deploy the agent as a DaemonSet. The script requires curl and kubectl installed in the $PATH on the host in which it is run.

install-agent-kubernetes \
[-a | --access_key <value>] [-t | --tags <value>] \
[-c | --collector <value>] [-cp | --collector_port <value>] [-s | --secure <value>] \
[-cc | --check_certificate <value>] [-ns | --namespace | --project <value>] \
[-ac | --additional_conf <value>] [-op | --openshift] [-as | --agent_slim] \
[-av | --agent_version <value>] [-ae | --api_endpoint <value> ] [-na | --nodeanalyzer ] \
[-ia | --imageanalyzer ] [-am | --analysismanager <value>] [-ds | --dockersocket <value>] \
[-cs | --crisocket <value>] [-cv | --customvolume <value>] \
[-cn | --cluster_name <value>] [-r | --remove ] [-h | --help]

Example

curl -s https://download.sysdig.com/stable/install-agent-kubernetes | sudo bash -s -- \
--access_key <ACCESS_KEY>  \
--collector <COLLECTOR_ADDRESS> --collector_port <COLLECTOR_PORT> \
--nodeanalyzer --api_endpoint <SECURE_ENDPOINT_ADDRESS>

Options

For the complete configuration options, see Agent Install:Kubernetes.

Docker

Install agent-kmodule

docker run -it --privileged --rm --name sysdig-agent-kmodule \
  -v /usr:/host/usr:ro \
  -v /boot:/host/boot:ro \
  -v /lib/modules:/host/lib/modules:ro \
  quay.io/sysdig/agent-kmodule

Install agent-slim

docker run -d --name sysdig-agent \
  --restart always \
  --privileged \
  --net host \
  --pid host \
  -e ACCESS_KEY=<ACCESS_KEY> \
  -e COLLECTOR=<COLLECTOR_URL> \
  -e SECURE=true \
  [-e TAGS=<LIST_OF_TAGS>] \
  -e ADDITIONAL_CONF= <LIST_OF_CONFIG> \
  -v /var/run/docker.sock:/host/var/run/docker.sock \
  -v /dev:/host/dev \
  -v /proc:/host/proc:ro \
  -v /boot:/host/boot:ro \
  --shm-size=512m \
  quay.io/sysdig/agent-slim

Example

Install agent-kmodule

docker run -it --privileged --rm --name sysdig-agent-kmodule \
  -v /usr:/host/usr:ro \
  -v /boot:/host/boot:ro \
  -v /lib/modules:/host/lib/modules:ro \
  quay.io/sysdig/agent-kmodule

Install agent-slim

docker run \
  --name sysdig-agent \
  --privileged \
  --net host \
  --pid host \
  -e ACCESS_KEY=1234-your-key-here-1234  \
  -e COLLECTOR=mycollector.elb.us-west-1.amazonaws.com \
  -e COLLECTOR_PORT=6443 \
  -e CHECK_CERTIFICATE=false \
  -e TAGS=my_tag:some_value \
  -e ADDITIONAL_CONF="log:\n file_priority: debug\n console_priority: error" \
  -v /var/run/docker.sock:/host/var/run/docker.sock \
  -v /dev:/host/dev \
  -v /proc:/host/proc:ro \
  -v /boot:/host/boot:ro \
  -v /lib/modules:/host/lib/modules:ro \
  -v /usr:/host/usr:ro \
  --shm-size=350m \
quay.io/sysdig/agent-slim

Options

Linux

$ curl -s https://download.sysdig.com/stable/install-agent | sudo bash -c -- \
--access_key [-t | --tags <value>] [-c | --collector <value>] \
[-cp | --collector_port <value>] [-s | --secure <value>] \
[-cc | --check_certificate]  [-ac | --additional_conf <value>] \
[-b | --bpf] [-h | --help]

Example

curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- \
--access_key <ACCESS_KEY> --collector collector-staging.sysdigcloud.com \
--secure true

Options

3 - Agent Install: Kubernetes

The recommended method to monitor Kubernetes environments is to deploy the Sysdig agent using the helm chart. Alternatively, you can install the agent container using DaemonSet. This section helps you install the agent in both the methods.

Installing the agent using helm or as a daemonSet will deploy agent containers on every node in your Kubernetes environment. Once the agent is installed, Sysdig Monitor automatically begins monitoring all of your hosts, apps, pods, and services and automatically connects to the Kubernetes API server to pull relevant metadata about the environment. If licensed, Sysdig Secure launches with default policies that you can view and configure to suit your needs. You can access the front-end web interfaces for Sysdig Monitor and Sysdig Secure immediately.

Sysdig supports monitoring numerous Kubernetes platforms, including the following:

• Amazon environments (EKS,EC2, ECS)
• Google Kubernetes Engine (GKE)
• Red Hat OpenShift
• Oracle Kubernetes Engine (OKE)
• IBM Cloud Kubernetes Service (IKS)
• Rancher
• Mirantis Kubernetes Engine (MKE)
• Azure Container Service (AKS)

Prerequisites

• A supported distribution: See Agent Installation Requirements for details.

• Kubernetes v1.9+: The agent installation on Kubernetes requires v1.9 or higher because the APIs used to fetch kubernetes metadata are only present in v1.9+.

• Sysdig account and access key: Request a trial or full account at Sysdig.com and click the Activate Account button. The Getting Started Wizard provides an access key.

• Port 6443 open for outbound traffic: The agent communicates with the collector on port 6443. If you are using a firewall, you must open port 6443 for outbound traffic for the agent.

• Kernel headers installed: If a prebuilt kernel probe is not available for your kernel, the kernel headers must be installed in order to build the kernel probe.

• kubectl installed: All of the installation methods utilize kubectl to install the agent in the cluster.

Kernel Headers

The Sysdig agent requires kernel header files to install successfully on a Kubernetes cluster. If the hosts in your environment match the pre-compiled kernel modules available from Sysdig, no special action is required.

In some cases, the nodes in your Kubernetes environment might use Unix versions that do not match the provided headers, and the agent might fail to install correctly. In those cases, you must install the kernel headers manually on each node.

To do so:

For Debian-style distributions, run the command:

apt-get -y install linux-headers-$(uname -r)

For RHEL-style distributions, run the command:

yum -y install kernel-devel-$(uname -r)

For more information on troubleshooting, see About Kernel Headers and the Kernel Module.

Kubernetes Environments

Some Kubernetes environments require special configuration options to deploy the agent. If you’re installing in one of the following environments, follow the guides specific to those environments to deploy the agent. Otherwise, continue with this topic.

• IBM Cloud Kubernetes Service (IKS)
• Google Kubernetes Engine (GKE)
• Oracle Kubernetes Engine (OKE)
• OpenShift
• Rancher

Installation

Helm

Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.

Script

Sysdig also provides a script that you can use to install the agent as a DaemonSet.

Installation

1. Download the script and make it executable.

    wget https://download.sysdig.com/stable/install-agent-kubernetes
    chmod +x install-agent-kubernetes
   

2. Run the script to install the agent as a DaemonSet.

   ./install-agent-kubernetes -a <ACCESS_KEY> -c <COLLECTOR_URL> -cn <CLUSTER_NAME>
   

Options

Manifests

To deploy agents using Kubernetes manifests, you can download manifest files, edit them as required, and deploy them using kubectl.

1. Download the sample files:

   • sysdig-agent-clusterrole.yaml

   • sysdig-agent-daemonset-v2.yaml

   • sysdig-agent-configmap.yaml

   • sysdig-agent-service.yaml

2. Create a namespace for the Sysdig agent.

   Note: You can use whatever name you prefer. This example uses sysdig-agent for both the namespace and the service account. The default service account name was automatically defined in sysdig-agent-daemonset-v2.yaml, at the line: serviceAccount: sysdig-agent

   kubectl create ns sysdig-agent
   

3. Create a secret key:

   kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
   

4. Create a cluster role and service account, and define the cluster role binding that grants the Sysdig agent rules in the cluster role:

   kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
   kubectl create serviceaccount sysdig-agent -n sysdig-agent
   kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
   

5. Edit sysdig-agent-configmap.yaml to add the collector address , port , and the SSL/TLS information:

   collector:
   collector_port:
   ssl: #true or false
   check_certificate: #true or false
   

   • For SaaS, find the collector address for your region.

   • For On-prem, enter the collector endpoint defined in your environment.

   • check_certificate should be set to false if a self-signed certificate or private, and a CA-signed cert is used. See Set Up SSL Connectivity to the Backend for more information.

6. Apply the sysdig-agent-configmap.yaml file:

   kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
   

7. Apply the sysdig-agent-service.yaml file:

   kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
   

   This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

8. Apply the daemonset-v2.yaml file :

   kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
   

Additional Options

Verify Metrics in Sysdig Monitor

Log in to Sysdig Monitor to verify that the agent deployed and the metrics are detected and collected appropriately.

The steps below give one way to do the check.

1. Access Sysdig Monitor:

   SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for US East, the URL is https://app.sysdigcloud.com.

   For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.

   Log in with your Sysdig user name and password.

2. Select the Explore tab to see if metrics are displayed.

3. Determine the Kube State Metrics you want to collect.

4. To verify that kube state metrics and cluster name are working correctly, select the Explore tab and see if your cluster is listed.

   

Kubernetes metadata (pods, deployments etc.) appear a minute or two later than the nodes/containers themselves; if pod names do not appear immediately, wait and retry the Explore view.

If agents are disconnecting, there could be an issue with your MAC addresses. See Troubleshooting Agent Installation for tips.

Connect to the Sysdig Backend via Static IPs (SaaS only)

Sysdig provides a list of static IP addresses that can be whitelisted in a Sysdig environment, allowing users to establish a network connection to the Sysdig backend without opening complete network connectivity. This is done by setting the Collector IP to collector-static.sysdigcloud.com.

The sysdig-agent-configmap.yaml file can be edited either locally or using the edit command in Kubernetes.

To configure the collector IP in a Kubernetes SaaS instance:

1. Open sysdig-agent-configmap.yaml in a text editor.

2. Uncomment the following lines:

   • collector:

   • collector_port

3. Set the collector: value to collector-static.sysdigcloud.com

4. Set the collector_port: value to 6443

5. Save the file.

The example file below shows how the sysdig-agent-configmap.yaml file should look after configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-agent
data:
  dragent.yaml: |
    ### Agent tags
    # tags: linux:ubuntu,dept:dev,local:nyc

    #### Sysdig Software related config ####

    # Sysdig collector address
    collector: collector-static.sysdigcloud.com

    # Collector TCP port
    collector_port: 6443

    # Whether collector accepts ssl/TLS
    ssl: true

    # collector certificate validation
    ssl_verify_certificate: true

    # Sysdig Secure
    security:
      enabled: true

    #######################################
    # new_k8s: true
    # k8s_cluster_name: production    

3.1 - Steps for GKE

• Installing on GKE Standard
• Installing on GKE Autopilot

3.1.1 - GKE Autopilot

Autopilot is an operation mode for creating and managing clusters in GKE. In brief, with Autopilot, Google configures and manages the underlying node infrastructure for you. This topic helps you use helm to install Sysdig agent on a GKE cluster installed in Autopilot mode.

NodeAnalyzer is not supported on Autopilot environments.

Prerequisites

1. Install a GKE cluster in Autopilot mode.

2. Connect the GKE cluster.

3. Install your workload.

Deploy Sysdig Agent

Sysdig recommends using Helm to install Sysdig agent in kubernetes environments. After connecting to the GKE cluster, use the sysdig-deplpy chart to install Sysdig agent.

To customize the configuration of the agent, see the Sysdig Agent Helm Chart.

Verify Metrics on the Sysdig Monitor UI

Log in to Sysdig Monitor to verify that the agent deployed and the metrics are detected and collected appropriately.

Given below is one way to do so.

1. Access Sysdig Monitor:

   SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for US East, the URL is https://app.sysdigcloud.com.

   For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysdig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.

   Log in with your Sysdig user name and password.

2. Select the Explore tab to see if metrics are displayed.

3. Verify that kube state metrics and cluster name are working correctly: select the Explore tab and create a grouping by kube_cluster_name and kube_pod_name.

4. Select an individual container or pod to see the details.

3.1.2 - GKE Standard

Google Kubernetes Engine (GKE) is a managed environment for running Kubernetes in Google Cloud, in order to deploy containerized applications. Sysdig supports all flavors of GKE, including Ubuntu and GKE’s default Container-Optimized OS(COS).

GKE COS environments require eBPF probe to support agent installation.

Preparation

Open Port 6443 for Agent Egress

Because GKE uses stateful firewalls, you must actively open port 6443 for the Sysdig agent outbound traffic.

In earlier versions, the Sysdig Agent connected to port 6666. This behavior has been deprecated, as the Sysdig agent now connects to port 6443.

GKE COS/eBPF-Specific Requirements

• Linux kernel version >= 4.14.

• When performing the installation steps, you will add one additional parameter to install the eBPF probe. See Step 7. Note that only the eBPF probe is supported in GKE COS environments.

Prerequisites

You can review Agent Install: Kubernetes and the Agent Installation Requirements for additional context, if desired.

Installation Steps

Helm

Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.

Manifests

To deploy agents using Kubernetes manifests, you can download the manifest files, edit them as required, and deploy them using kubectl.

1. Download the sample files:

   • sysdig-agent-clusterrole.yaml

   • sysdig-agent-daemonset-v2.yaml

   • sysdig-agent-configmap.yaml

   • sysdig-agent-service.yaml

2. Create a namespace to use for the Sysdig agent.

   You can use whatever name you want. In this document, we used sysdig-agent for both the namespace and the service account.

   kubectl create ns sysdig-agent
   

3. Create a secret key:

   kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
   

4. If you are running Kubernetes 1.6 or higher, you must grant your user the ability to create roles in Kubernetes by running the following command:

   kubectl create clusterrolebinding your-user-cluster-admin-binding --clusterrole=cluster-admin --user=your.google.cloud.email@example.org
   

   See Google documentation for more information.

   Create a service account for the Sysdig agent using the clusterrole.yaml file.

   The Sysdig agent must be granted read-only access to certain Kubernetes APIs, which the agent uses to populate metadata and provide component metrics.

   You can use the Sysdig-provided, sysdig-agent-clusterrole.yaml file. Deploying this file creates a cluster role and service account in Kubernetes, and defines cluster role binding that grants the Sysdig agent rules in the cluster role.

   Run the following commands (using whatever namespace you’ve defined in Step 2):

   kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
   kubectl create serviceaccount sysdig-agent -n sysdig-agent
   kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
   

5. Edit sysdig-agent-configmap.yaml to add the collector address, port, and the SSL/TLS information :

   collector:
   collector_port:
   ssl: #true or false
   check_certificate: #true or false
   

   • For SaaS, find the collector address for your region.

   • For On-prem, enter the collector endpoint defined in your environment.

   • check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used.

     See Set Up SSL Connectivity to the Backend for more information.

6. Apply the sysdig-agent-configmap.yaml file:

   kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
   

7. FOR GKE COS ONLY: To enable the eBPF probe required for COS, uncomment the following parameters in  sysdig-agent-daemonset-v2.yaml under the env section:

   env:
     - name: SYSDIG_BPF_PROBE
       value: ""
   

8. Apply the sysdig-agent-service.yaml file:

   kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
   

   This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

9. Apply the daemonset-v2.yaml file:

   kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
   

The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.

Next Steps

You can continue with instructions in Verify Metrics in Sysdig Monitor and optionally, Connect to Sysdig Backend.

3.2 - Steps for OKE

Oracle Kubernetes Engine (OKE) is a managed environment for running Kubernetes in Oracle Cloud, in order to deploy containerized applications. As of Sysdig agent version 12.0.1, Sysdig supports all flavors of OKE.

OKE environments require eBPF probe to support agent installation.

The instructions below describe a standard OKE agent install and call out the special steps needed to install the eBPF probe.

Preparation

Open Port 6443 for Agent Egress

Because OKE uses stateful firewalls, you must actively open port 6443 for the Sysdig agent outbound traffic.

OKE by default allows network access to the sysdig Agent on 6443, but ensure that firewall rules are open and the agent can connect to the Sysdig backends.

eBPF-Specific Requirements

• Linux kernel version >= 4.14.

• When performing the installation steps, you will add one additional parameter to install the eBPF probe. See Step 7, below.

Installation Steps

Identify the appropriate endpoint depending on your Sysdig account region. For more information, see SaaS Regions and IP Ranges. More info here https://docs.sysdig.com/en/docs/administration/saas-regions-and-ip-ranges/

After making clear which region your account belongs to, please choose one of the following methods:

Helm

Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.

Manifests

To deploy agents using Kubernetes manifests, you can downloadmanifest files, edit them as required, and deploy them using kubectl.

1. Download the sample files:

   • sysdig-agent-clusterrole.yaml

   • sysdig-agent-daemonset-v2.yaml

   • sysdig-agent-configmap.yaml

   • sysdig-agent-service.yaml

2. Create a namespace to use for the Sysdig agent.

   Notes: You can use whatever name you want. In this document, we used sysdig-agent for both the namespace and the service account.

   kubectl create ns sysdig-agent
   

3. Create a secret key:

   kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
   

4. If you are running Kubernetes 1.6 or higher, you must create a service account for the Sysdig agent by using the clusterrole.yaml file.

   The Sysdig agent must be granted read-only access to certain Kubernetes APIs, which the agent uses to populate metadata and provide component metrics.

   You can use the Sysdig-provided sysdig-agent-clusterrole.yaml file. Deploying it create a cluster role and service account in Kubernetes, and defines cluster role binding that grants the Sysdig agent rules in the cluster role.

   Run the following commands by using the namespace you’ve defined in Step 2:

   kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
   kubectl create serviceaccount sysdig-agent -n sysdig-agent
   kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
   

5. Edit sysdig-agent-configmap.yaml to add the collector address, port, and the SSL/TLS information :

   collector:
   collector_port:
   ssl: #true or false
   check_certificate: #true or false
   

   • For SaaS, find the collector address for your region.

   • For On-prem, enter the collector endpoint defined in your environment.

   • check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Set Up SSL Connectivity to the Backend.

6. Apply the sysdig-agent-configmap.yaml file:

   kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
   

7. To enable the eBPF probe uncomment the following parameters in  sysdig-agent-daemonset-v2.yaml under the env section:

   env:
     - name: SYSDIG_BPF_PROBE
       value: ""
   

8. Apply the sysdig-agent-service.yaml file:

   kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
   

   This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

9. Apply the daemonset-v2.yaml file:

   kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
   

   The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.

Next Steps

You can continue with instructions in Verify Metrics in Sysdig Monitor and optionally, Connect to Sysdig Backend.

3.3 - Steps for OpenShift

You can review Agent Install: Kubernetes and the Agent Installation Requirements for additional context, if desired.

RHCOS environments require eBPF probe to support agent installation.

Preparation

RHCOS/eBPF-Specific Requirements

• Linux kernel version 4.14 or above.
• When performing the installation steps, you will add one additional parameter to install the eBPF probe. See Step 7, below.

Kernel Headers

The Sysdig agent requires kernel header files to install successfully on a Kubernetes cluster. If the hosts in your environment match the pre-compiled kernel modules available from Sysdig, no special action is required.

In some cases, the nodes in your Kubernetes environment might use Unix versions that do not match the provided headers, and the agent might fail to install correctly. In those cases, you must install the kernel headers manually on each node.

To do so:

For Debian-style distributions, run the command:

apt-get -y install linux-headers-$(uname -r)

For RHEL-style distributions, run the command:

yum -y install kernel-devel-$(uname -r)

For more information on troubleshooting, see About Kernel Headers and the Kernel Module.

Configure for OpenShift

If you are using Red Hat OpenShift, these steps are required. They describe how to create a project, assign and label the node selector, create a privileged service account, and add it to a cluster role.

Copy/Paste Sample Code Block

In the example code, this document uses sysdig-agent for the PROJECT NAME (-n) and the SERVICE ACCOUNT (-z).

You can copy and paste the code as is, or follow the steps below to customize your naming conventions.

oc adm new-project sysdig-agent --node-selector=''
oc project sysdig-agent
oc create serviceaccount sysdig-agent
oc adm policy add-scc-to-user privileged -n sysdig-agent -z sysdig-agent -z node-analyzer
oc adm policy add-cluster-role-to-user cluster-reader -n sysdig-agent -z sysdig-agent -z node-analyzer

Customize the Code

You can use your own Project name and Service Account name if desired.

Note that if you use a different Service Account name, you will need to edit the default service account in the Sysdig Installation Steps, below.

1. Create a new OpenShift project for the Sysdig agent deployment and use an empty string for the node selector:

   oc adm new-project PROJECT-NAME --node-selector=""
   

2. Change to the new OpenShift Project for the Sysdig agent deployment:

   oc project PROJECT-NAME
   

3. Create a service account for the project:

   oc create serviceaccount SERVICE-ACCOUNT
   

4. Add the service account to privileged Security Context Constraints:

   oc adm policy add-scc-to-user privileged -n PROJECT-NAME -z SERVICE-ACCOUNT -z node-analyzer
   

5. Add the service account to the cluster-reader Cluster Role:

   oc adm policy add-cluster-role-to-user cluster-reader -n PROJECT-NAME -z SERVICE-ACCOUNT -z node-analyzer
   

Installation

Helm

Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.

Manifests

1. Download the sample files:

   • sysdig-agent-daemonset-v2.yaml

   • sysdig-agent-clusterrole.yaml

   • sysdig-agent-configmap.yaml

   • sysdig-agent-service.yaml

2. Create the sysdig-agent cluster role and assign it to the service account:

    oc apply -f sysdig-agent-clusterrole.yaml
    oc adm policy add-cluster-role-to-user sysdig-agent -n PROJECT-NAME -z SERVICE-ACCOUNT
   

3. Create a secret key:

   oc create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n PROJECT-NAME
   

4. If you created a service account name other than sysdig-agent: Edit sysdig-agent-daemonset-v2.yamlto provide your custom value:``

   serviceAccount: SERVICE-ACCOUNT
   

5. Edit sysdig-agent-configmap.yaml to add the collector address, port, and the SSL/TLS information:

   collector:
   collector_port:
   ssl: #true or false
   check_certificate: #true or false
   

   • For SaaS, find the collector address for your region.
   • For On-prem, enter the collector endpoint defined in your environment.
   • check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Set Up SSL Connectivity to the Backend.

6. Apply the sysdig-agent-configmap.yaml file:

   oc apply -f sysdig-agent-configmap.yaml -n PROJECT-NAME
   

7. FOR RHCOS ONLY: To enable the eBPF probe required for COS, uncomment the following parameters in sysdig-agent-daemonset-v2.yaml under the env section:`

   env:
     - name: SYSDIG_BPF_PROBE
       value: ""
   

8. Apply the sysdig-agent-service.yaml file:

   oc apply -f sysdig-agent-service.yaml -n PROJECT-NAME
   

   This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

9. Apply the daemonset-v2.yaml file:

   oc apply -f sysdig-agent-daemonset-v2.yaml -n PROJECT-NAME
   

   The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.

Next Steps

You can continue with instructions in Verify Metrics in Sysdig Monitor and optionally, Connect to Sysdig Backend.

3.4 - Steps for Rancher

Preparation

General Requirements

You can review Agent Install: Kubernetes | GKE | OpenShift | IBM and the Agent Installation Requirements for additional context, if desired.

Kernel Headers

The Sysdig agent requires a kernel module in order to be installed successfully on a host. On RancherOS distributions, the Unix version does not match the provided headers, and the agent might fail to install correctly. Therefore, you must install the kernel headers manually.

For RancherOS distributions, the kernel headers are available in the form of a system service and therefore are enabled using the ros service command:

$ sudo ros service enable kernel-headers-system-docker
$ sudo ros service up -d kernel-headers-system-docker

Some cloud hosting service providers supply pre-configured Linux instances with customized kernels. You may need to contact your provider’s support desk for instructions on obtaining appropriate header files, or for installing the distribution’s default kernel.

Installation

Helm

Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.

Manifests

To deploy agents using Kubernetes manifests, you can download the manifest files, edit them as required, and deploy them using kubectl.

1. Download the sample files:

   • sysdig-agent-clusterrole.yaml

   • sysdig-agent-daemonset-v2.yaml

   • sysdig-agent-configmap.yaml

   • sysdig-agent-service.yaml

2. Create a namespace to use for the Sysdig agent.

   You can use whatever naming you prefer. This document uses sysdig-agent for both the namespace and the service account.

   The default service account name was automatically defined in sysdig-agent-daemonset-v2.yaml, at the line: serviceAccount: sysdig-agent.

   kubectl create ns sysdig-agent
   

3. Create a secret key:

   kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent
   

4. Create a cluster role and service account, and define the cluster role binding that grants the Sysdig agent rules in the cluster role:

   kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
   kubectl create serviceaccount sysdig-agent -n sysdig-agent
   kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
   

5. Edit sysdig-agent-configmap.yaml to add the collector address , port , and the SSL/TLS information:

   collector:
   collector_port:
   ssl: #true or false
   check_certificate: #true or false
   

   • For SaaS, find the collector address for your region.

   • For On-prem, enter the collector endpoint defined in your environment.

   • check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Set Up SSL Connectivity to the Backend.

6. Apply the sysdig-agent-configmap.yaml file:

    kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
   

7. Apply the sysdig-agent-service.yaml file:

   kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
   

This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

8. Apply the daemonset-v2.yaml file :

   kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent
   

The agents will be deployed and you can see some metrics in the Sysdig Monitor UI.

Next Steps

You can continue with instructions in Verify Metrics in Sysdig Monitor and optionally, Connect to Sysdig Backend.

3.5 - Steps for MKE

Mirantis Kubernetes Engine (MKE) formerly Docker Enterprise, is a managed environment for running Kubernetes to deploy containerized applications. As of Sysdig agent version 12.0.1, Sysdig supports all flavors of MKE.

MKE environments require eBPF probe to support agent installation.

The instructions below describe a standard MKE agent install and call out the special steps needed to install the eBPF probe.

Preparation

eBPF-Specific Requirements

• Linux kernel versions 4.14 or above.

• eBPF probe parameter, --set ebpf.enabled=true to install eBPF probe. See the instructions given below.

Installation Steps

Identify the appropriate endpoint depending on your Sysdig account region. For more information, see SaaS Regions and IP Ranges.

Helm

Sysdig recommends using helm charts to install Sysdig agent in Kubernetes environments. For the latest chart and installation instructions, see sysdig-deploy.

Make sure to add the eBPF parameter to the helm command:

--set ebpf.enabled=true

Next Steps

You can continue with instructions in Verify Metrics in Sysdig Monitor and optionally, Connect to Sysdig Backend.

3.6 - Using Node Leases

The Sysdig agent uses Kubernetes Lease to control how and when connections are made to the Kubernetes API Server. This mechanism prevents overloading the Kubernetes API server with connection requests during agent bootup.

Kubernetes node leases are automatically created for agent version 12.0.0 and above. On versions prior to 12.0.0, you must configure node leases as given in the KB article.

Prerequisites

• Sysdig Agent v11.3.0 or above

• Kubernetes v1.14 or above

Types of Leases

The agent creates the following leases:

Cold Start

During boot up, the Sysdig agent connects to the Kubernetes API server to retrieve Kubernetes metadata and build a cache. The cold-start leases control the number of agents that build up this cache at any given time. An agent will grab a lease, build its cache, and then release the lease so that another agent can build its cache. This mechanism prevents agents from creating a “boot storm” which can overwhelm the API server in large clusters.

Delegation

In Kubernetes environments, two agents are marked as delegated in each cluster. The delegated agents are the designated agents to request more data from the API server and produce KubeState metrics. The delegation leases will not be released until the agent is terminated.

View Leases

To view the leases, run the following:

$ kubectl get leases -n sysdig-agent

You will see an output similar to the following:

NAME           HOLDER             AGE
cold-start-0                      20m
cold-start-1                      20m
cold-start-2                      21m
cold-start-3   ip-10-20-51-167    21m
cold-start-4                      21m
cold-start-5                      21m
cold-start-6                      20m
cold-start-7                      21m
cold-start-8                      20m
cold-start-9   ip-10-20-51-166   21m
delegation-0   ip-10-20-52-53    21m
delegation-1   ip-10-20-51-98    21m

Troubleshoot Leases

Verify Configuration

When lease-based delegation is working as expected, the agent logs show one of the following:

• Getting pods only for node <node>

• Getting pods for all nodes.

• Both (occasionally on the delegated nodes)

Run the following to confirm that it is working:

$ kubectl logs sysdig-agent-9l2gf -n sysdig-agent | grep -i "getting pods"

The configuration is working as expected if the output on a pod is similar to the following:

2021-05-05 02:48:32.877, 15732.15765, Information, cointerface[15738]: Only getting pods for node ip-10-20-51-166.ec2.internal

Unable to Create Leases

The latest Sysdig ClusterRole is required for the agent to create leases. If you do not have the latest ClusterRole or if you have not configured the ClusterRole correctly, the logs show the following error:

Error, lease_pool_manager[2989554]: Cannot access leases objects: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:sysdig-agent:sysdig-agent" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "sysdig-agent"

Contact Sysdig Support for help.

Optional Agent Configuration

Several configuration options exist for leases. It is recommended to not change the default settings unless prompted by Sysdig Customer Support.

k8s_coldstart:
  enabled: <true/false>

k8s_coldstart:
  max_parallel_cold_start: <int>

k8s_coldstart:
  namespace: <string>

k8s_coldstart:
  enforce_leader_election: <true/false>

k8s_delegation_election: <true/false>

4 - Agent Install: Non-Orchestrated

This section describes how to install the Sysdig agent directly on a Linux host, without using an orchestrator, such as Kubernetes or Mesos.

The agent can be installed in two ways:

• As a standard container

• As a non-containerized service

The steps for each flavor differ slightly depending on whether you are using the SaaS or on-premises version of the Sysdig platform.

If you are installing the Sysdig agent in an environment that has Kubernetes, use the Agent Install: Kubernetes instructions instead.

Prerequisites

• See Agent Installation Requirements for information on the following:

  • Supported Linux distributions

  • Network connection

  • Sysdig access key

  • Cloud service providers (AWS, Google, and Microsoft Azure) and any steps you may need to configure to integrate the Sysdig agent.

• kernel headers: The Sysdig agent requires kernel header files in order to install successfully on a host, and the agent is delivered with precompiled headers. If the hosts in your environment match the kernel versions included with the agent, no special action is needed .In some cases, the hosts in your environment may use Unix versions that do not match the provided headers, and the agent may fail to install correctly. In those cases, you must install the kernel headers manually. See About Kernel Headers and the Kernel Module for details.

• Run any commands as root or with the sudo command.

• Retrieve the Sysdig access key.

• Collect the configuration parameters.

Configuration Options

Installing Agent Using Containers

The Sysdig agent can be deployed as a docker container.

The commands below can also be copied from the Get Started page. In that case, your access key will already be included in the command automatically.

SaaS

Installing As Two Containers

The agent is installed by running sysdig/agent-kmodule, followed by running sysdig/agent-slim. See Installation Options for description about agent-slim and agent-kmodule.

Every host restart requires subsequent running of agent-kmodule and agent-slim containers.

1. Collect the configuration parameters.

2. Build and load the kernel module:

   If you are not using eBPF, use the following:

   docker run -it --privileged --rm --name sysdig-agent-kmodule \
   -v /usr:/host/usr:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules \
   quay.io/sysdig/agent-kmodule
   

   If you are using eBPF use the following:

   docker run -it --privileged --rm --name sysdig-agent-kmodule \
   -e SYSDIG_BPF_PROBE="" \
   -v /etc/os-release:/host/etc/os-release:ro \
   -v /root/.sysdig:/root/.sysdig \
   -v /usr:/host/usr:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   quay.io/sysdig/agent-kmodule
   

3. Configure kernel module to load during system boot.

   If you are not using eBPF, use the following commands to configure the Linux system to automatically load the kernel module during system boot.

   $ sudo mkdir -p /etc/modules-load.d
   $ sudo bash -c "echo sysdigcloud-probe > /etc/modules-load.d/sysdigcloud-probe.conf"
   

4. Run the agent module providing the access key and, optionally, user-defined tags:

   If you are not using eBPF, use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host \
   -e ACCESS_KEY=[ACCESS_KEY] \
   -e COLLECTOR=[COLLECTOR_ADDRESS] \
   [-e TAGS=[TAGS]]
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   --shm-size=512m \
   quay.io/sysdig/agent-slim
   

   If you are using eBPF use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host\
   -e ACCESS_KEY=[ACCESS_KEY] \
   -e COLLECTOR=[COLLECTOR_ADDRESS] \
   [-e TAGS=[TAGS]]
   -e SYSDIG_BPF_PROBE="" \
   -v /sys/kernel/debug:/sys/kernel/debug:ro \
   -v /root/.sysdig:/root/.sysdig \
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   --shm-size=512m \
   quay.io/sysdig/agent-slim
   

Installing As Single Container (Legacy)

1. Collect the configuration parameters.

2. Run the agent container providing the access key and, optionally, user-defined tags:

   If you are not using eBPF, use the following:

   If you are not using eBPF, use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host\
    -e ACCESS_KEY=[ACCESS_KEY] \
    -e COLLECTOR=[COLLECTOR_ADDRESS] \
   -e TAGS=[TAGS] \
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   -v /usr:/host/usr:ro \
   --shm-size=512m \
   quay.io/sysdig/agent
   

   If you are using eBPF use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host\
    -e ACCESS_KEY=[ACCESS_KEY] \
    -e COLLECTOR=[COLLECTOR_ADDRESS] \
   -e TAGS=[TAGS] \
   -e SYSDIG_BPF_PROBE="" \
   -v /sys/kernel/debug:/sys/kernel/debug:ro \
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   -v /usr:/host/usr:ro \
   --shm-size=512m \
   quay.io/sysdig/agent
   

On-Premises

Installing As Two Containers

1. Collect the configuration parameters:

2. Build and load the kernel module:

   If you are not using eBPF, use the following:

   docker run -it --privileged --rm --name sysdig-agent-kmodule \
   -v /usr:/host/usr:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules \
   quay.io/sysdig/agent-kmodule
   

   If you are using eBPF use the following:

   docker run -it --privileged --rm --name sysdig-agent-kmodule \
   -e SYSDIG_BPF_PROBE="" \
   -v /etc/os-release:/host/etc/os-release:ro \
   -v /root/.sysdig:/root/.sysdig \
   -v /usr:/host/usr:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   quay.io/sysdig/agent-kmodule
   

3. Configure kernel module to load during system boot.

   If you are not using eBPF, use the following commands to configure the Linux system to automatically load the kernel module during system boot.

   $ sudo mkdir -p /etc/modules-load.d
   $ sudo bash -c "echo sysdigcloud-probe > /etc/modules-load.d/sysdigcloud-probe.conf"
   

4. Run the agent module providing the access key and, optionally, user-defined tags:

   If you are not using eBPF, use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host \
   -e ACCESS_KEY=[ACCESS_KEY] \
   -e COLLECTOR=[COLLECTOR_ADDRESS] \
   -e SECURE=true \
   -e CHECK_CERTIFICATE=true \
   [-e TAGS=[TAGS]]
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   -v /usr:/host/usr:ro \
   --shm-size=512m \
   quay.io/sysdig/agent-slim
   

   If you are using eBPF use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host \
   -e ACCESS_KEY=[ACCESS_KEY] \
   -e COLLECTOR=[COLLECTOR_ADDRESS] \
   -e SECURE=true \
   -e CHECK_CERTIFICATE=true \
   [-e TAGS=[TAGS]]
   -e SYSDIG_BPF_PROBE="" \
   -v /sys/kernel/debug:/sys/kernel/debug:ro \
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   -v /usr:/host/usr:ro \
   --shm-size=512m \
   quay.io/sysdig/agent-slim
   

Installing As Single Container (Legacy)

1. Collect the configuration parameters.

2. Run the agent module providing the access key and, optionally, user-defined tags:

   If you are not using eBPF, use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host \
   -e ACCESS_KEY=[ACCESS_KEY] \
   -e COLLECTOR=[COLLECTOR_ADDRESS] \
   -e SECURE=true \
   -e CHECK_CERTIFICATE=true \
   [-e TAGS=[TAGS]]
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   -v /usr:/host/usr:ro \
   --shm-size=512m \
   quay.io/sysdig/agent
   

   If you are using eBPF use the following:

   docker run -d --name sysdig-agent \
   --restart always \
   --privileged \
   --net host \
   --pid host \
   -e ACCESS_KEY=[ACCESS_KEY] \
   -e COLLECTOR=[COLLECTOR_ADDRESS] \
   -e SECURE=true \
   -e CHECK_CERTIFICATE=true \
   [-e TAGS=[TAGS]]
   -e SYSDIG_BPF_PROBE="" \
   -v /sys/kernel/debug:/sys/kernel/debug:ro \
   -v /var/run/docker.sock:/host/var/run/docker.sock \
   -v /dev:/host/dev \
   -v /proc:/host/proc:ro \
   -v /boot:/host/boot:ro \
   -v /lib/modules:/host/lib/modules:ro \
   -v /usr:/host/usr:ro \
   --shm-size=512m \
   quay.io/sysdig/agent
   

Installing Agent as a Service on Linux Host

Use these instructions to install the agent on the host itself, not in a container. Install on each host in the environment.

The command lines below can also be copy/pasted from the Welcome wizard or the Settings>Agent Installation page in the Sysdig Monitor interface.

In that case, your access key will already be included in the command automatically.

 >> Error, sdchecks[0] ModuleNotFoundError: No module named 'posix_ipc'

SaaS

1. Run the following command:

   curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- --access_key [ACCESS_KEY] --collector [COLLECTOR_ADDRESS] [--tags [TAGS]]
   

   Where [ACCESS_KEY] is your unique agent access key string. For example, 1234-your-key-here-1234. TAGS is an optional list of user-defined agent tags. For example, role:webserver,location:europe.

   See SaaS Regions and IP Ranges to find the collector endpoint for your region.

2. Restart the agent and start the service:

   sudo systemctl enable dragent
   

On-Premises

1. Run the following command:

   curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- --access_key [ACCESS_KEY] --collector [COLLECTOR_ADDRESS] --secure true --check_certificate true [--tags [TAGS]]
   

   For configuration parameters, see Configuration Options.

2. Restart the agent and start the service:

   sudo systemctl enable dragent
   

Connect to the Sysdig Backend via Static IPs (SaaS only)

Sysdig provides a list of static IP addresses that can be whitelisted in a Sysdig environment, allowing users to establish a network connection to the Sysdig backend without opening complete network connectivity. This is done by setting the Collector IP to collector-static.sysdigcloud.com:

user@host:~$ docker run --name sysdig-agent \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e TAGS=[TAGS] \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
-e COLLECTOR=collector-static.sysdigcloud.com \
-e COLLECTOR_PORT=6443 \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
--shm-size=512m \
quay.io/sysdig/agent-slim

Guidelines for Manual Agent Installation

In the following cases, we recommend that you manually install the agent.

• Full control over the deployment process

• Integration with configuration management tools

• Custom kernel

• Unsupported distribution

See Agent Install: Manual Linux Installation for more information.

5 - Agent Install: Manual Linux Installation

Manual installation of the native Linux agent is recommended in the following cases:

• Full control over the deployment process

• Integration with configuration management tools

• Custom kernel

• Unsupported distribution (within Debian/Fedora flavors)

Otherwise, you may want to follow Agent Install: Non-Orchestrated

Note: If you are installing the Sysdig agent in an orchestrated infrastructure such as Kubernetes, Mesos/Marathon, use the respective Installation Guides:

• Agent Install: Kubernetes

• Agent Install: Mesos | Marathon | DCOS

Prerequisites

• Review the Agent Installation Requirements.

• Collect the configuration parameters:

  • ACCESS_KEY: Your unique access key string. Inability to retrieve the key indicates that the administrator of your instance might have it turned off for non-admin users. Contact your Sysdig administrator to receive the key. If you still have issues please contact Sysdig Support.

  • TAGS: The optional parameter you can use to list one or more tags for this host. Tagging your hosts is highly recommended. Agent Tags allow you to sort nodes of your infrastructure into custom groups in Sysdig Monitor. Replace the [TAGS] parameter above with a comma-separated list of TAG_NAME:TAG_VALUE. For example: role:webserver,location:europe

  • COLLECTOR: The collector URL for Sysdig Monitor or Sysdig Secure. This value is region-dependent in SaaS and is auto-completed on the Get Started page in the UI. It is a custom value in on-prem installations.

  • SECURE: Use a secure SSL/TLS connection to send metrics to the collector. This option is enabled by default.

• Run the commands as root or with sudo.

Installation

Follow the instructions for the appropriate Linux distribution:

Debian, Ubuntu

1. Use the Sysdig Monitor GPG key, configure the apt repository, and update the package list:

   curl -s https://download.sysdig.com/DRAIOS-GPG-KEY.public | apt-key add -
   curl -s -o /etc/apt/sources.list.d/draios.list http://download.sysdig.com/stable/deb/draios.list
   apt-get update
   

2. Install kernel development files.

   Note: The following command might not work with every kernel. Ensure that you customize the name of the package properly.

   apt-get -y install linux-headers-$(uname -r)
   

3. Install, configure, and restart the Sysdig agent.

   apt-get -y install draios-agent
   echo customerid: ACCESS_KEY >> /opt/draios/etc/dragent.yaml
   echo tags: [TAGS] >> /opt/draios/etc/dragent.yaml
   echo collector: COLLECTOR_URL >> /opt/draios/etc/dragent.yaml
   echo ssl: true >> /opt/draios/etc/dragent.yaml
   service dragent restart
   

   See Prerequisites for the configuration parameters required.

CentOS, RHEL, Fedora, Amazon AMI, Amazon Linux 2

1. Trust the Sysdig Monitor GPG key, configure the yum repository.

   $ rpm --import https://download.sysdig.com/DRAIOS-GPG-KEY.public
   $ curl -s -o /etc/yum.repos.d/draios.repo http://download.sysdig.com/stable/rpm/draios.repo
   

2. Install the EPEL repository.

   Note: The following command is required only if DKMS is not available in the distribution. You can verify if DKMS is available with yum list dkms.

   The command below contains a sample release number; be sure to update with the correct release.

   $ rpm -i http://mirror.us.leaseweb.net/epel/6/i386/epel-release-6-8.noarch.rpm
   

3. Install kernel development files.

   Note: The following command might not work with every kernel. Make sure to customize the name of the package properly.

   $ yum -y install kernel-devel-$(uname -r)
   

4. Install, configure, and start the Sysdig agent.

   $ yum -y install draios-agent
   $ echo customerid: ACCESS_KEY >> /opt/draios/etc/dragent.yaml
   $ echo tags: [TAGS] >> /opt/draios/etc/dragent.yaml
   $ echo collector: COLLECTOR_URL >> /opt/draios/etc/dragent.yaml
   $ echo ssl: true >> /opt/draios/etc/dragent.yaml
   $ echo secure: true >> /opt/draios/etc/dragent.yaml
   $ sudo systemctl enable dragent
   $ sudo systemctl start dragent
   

   See Prerequisites for the configuration parameters required.

5. If you using a non-systemd Linux distribution, use the service command to start dragent.

   $ service dragent restart
   

Other Linux Distributions

The Sysdig agent is unsupported outside of the Debian, Fedora, and Amazon distributions.

6 - Agent Install: Amazon ECS

Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that helps to easily deploy, manage, and scale containerized applications.

This section describes how to install the Sysdig agent container on each underlying host in your ECS cluster. Once installed, the agent will automatically begin monitoring all of your hosts, service and tasks.

Installation

To install Sysdig agent on ECS, do the following:

• Create an ECS task definition for the Sysdig agent.

• Register the task definition in your AWS account.

• Create a service with the previous task definition to run the Sysdig agent in each of the nodes of your ECS cluster.

Create an ECS Task Definition

1. Collect the following configuration parameters:

• ACCESS_KEY: The agent access key. You can retrieve this from Settings > Agent Installation in either Sysdig Monitor or Sysdig Secure.
• COLLECTOR: Use the collector address for your region. For more information, see SaaS Regions and IP Ranges.
• TAGS: The list of tags for the host where the agent is installed. For example: role:webserver, location:europe, role:webserver

2. Use the above values to customize the JSON snippet below and save it as a file named sysdig-agent-ecs.json.

Note that memory and cpu have both been set to 1024, depending of the size of your cluster you might want to tune those values. See Tuning Sysdig Agent for more information.

```yaml
{
  "family": "sysdig-agent-ecs",
  "containerDefinitions": [
    {
      "name": "sysdig-agent",
      "image": "quay.io/sysdig/agent-slim",
      "cpu": 1024,
      "memory": 1024,
      "privileged": true,
      "environment": [
        {
          "name": "ACCESS_KEY",
          "value": "$ACCESS_KEY"
        },
        {
          "name": "COLLECTOR",
          "value": "$COLLECTOR"
        },
	      {
          "name": "TAGS",
          "value": "$TAG1,TAG2"
        }
      ],
      "mountPoints": [
        {
          "readOnly": true,
          "containerPath": "/host/boot",
          "sourceVolume": "boot"
        },
        {
          "containerPath": "/host/dev",
          "sourceVolume": "dev"
        },
        {
          "readOnly": true,
          "containerPath": "/host/lib/modules",
          "sourceVolume": "modules"
        },
        {
          "readOnly": true,
          "containerPath": "/host/proc",
          "sourceVolume": "proc"
        },
        {
          "containerPath": "/host/var/run/docker.sock",
          "sourceVolume": "sock"
        },
        {
          "readOnly": true,
          "containerPath": "/host/usr",
          "sourceVolume": "usr"
        }
      ],
      "dependsOn": [
        {
          "containerName": "sysdig-agent-kmodule",
          "condition": "SUCCESS"
        }
      ]
    },
    {
      "name": "sysdig-agent-kmodule",
      "image": "quay.io/sysdig/agent-kmodule",
      "memory": 512,
      "privileged": true,
      "essential": false,
      "mountPoints": [
        {
          "readOnly": true,
          "containerPath": "/host/boot",
          "sourceVolume": "boot"
        },
        {
          "containerPath": "/host/dev",
          "sourceVolume": "dev"
        },
        {
          "readOnly": true,
          "containerPath": "/host/lib/modules",
          "sourceVolume": "modules"
        },
        {
          "readOnly": true,
          "containerPath": "/host/proc",
          "sourceVolume": "proc"
        },
        {
          "containerPath": "/host/var/run/docker.sock",
          "sourceVolume": "sock"
        },
        {
          "readOnly": true,
          "containerPath": "/host/usr",
          "sourceVolume": "usr"
        }
      ]
    }
  ],
  "pidMode": "host",
  "networkMode": "host",
  "volumes": [
    {
      "name": "sock",
      "host": {
        "sourcePath": "/var/run/docker.sock"
      }
    },
    {
      "name": "dev",
      "host": {
        "sourcePath": "/dev/"
      }
    },
    {
      "name": "proc",
      "host": {
        "sourcePath": "/proc/"
      }
    },
    {
      "name": "boot",
      "host": {
        "sourcePath": "/boot/"
      }
    },
    {
      "name": "modules",
      "host": {
        "sourcePath": "/lib/modules/"
      }
    },
    {
      "name": "usr",
      "host": {
        "sourcePath": "/usr/"
      }
    }
  ],
  "requiresCompatibilities": [
    "EC2"
  ]
}
```

Register a Task Definition

Once your task definition is ready, ensure that you register it in your AWS account:

aws ecs register-task-definition \
    --cli-input-json file://sysdig-agent-ecs.json

Run the Agent as an ECS Service

Using the ECS task definition you have created, create a service in the cluster that you want to monitor with Sysdig.

aws ecs create-service \
    --cluster $CLUSTER_NAME \
    --service-name sysdig-agent-svc \
    --launch-type EC2 \
    --task-definition sysdig-agent-ecs \
    --scheduling-strategy DAEMON

With the agent installed, Sysdig will begin auto-discovering your containers and other resources of your ECS environment.

Using ECS Anywhere

If you’re using ECS Anywhere, change the launch type to EXTERNAL when the service is created.

aws ecs create-service \
    --cluster $CLUSTER_NAME \
    --service-name sysdig-agent-svc \
    --launch-type EXTERNAL \
    --task-definition sysdig-agent-ecs \
    --scheduling-strategy DAEMON

Enable Log Driver

You can send the logs from the containers running the ECS tasks to the log groups in CloudWatch Logs. You can send agent container log files to AWS by enabling the log driver, awslogs. To do so:

1. Add the following section to each of the container definitions you’ve created above:

               "logConfiguration": {
                   "logDriver": "awslogs",
                   "options": {
                       "awslogs-group": "$YOUR_LOG_GROUP",
                       "awslogs-region": "$AWS_REGION",
                       "awslogs-stream-prefix": "sysdig"
                   }
   

2. Update your task definition and the service to enable the logs.

7 - Agent Install: IKS (IBM Cloud Monitoring)

IBM Cloud maintains the documentation for Sysdig agent installation on IBM Cloud Kubernetes Service (IKS).

For more information, see the IBM Cloud Monitoring documentation:

• Getting Started

• Deploying a Monitoring Agent on a Standard Kubernetes Cluster

• Forwarding cluster and app metrics to IBM Cloud Monitoring

8 - Agent Install: Mesos | Marathon | DCOS

Marathon is the container orchestration platform for Mesosphere’s Datacenter Operating System (DC/OS) and Apache Mesos.

This guide describes how to install the Sysdig agent container on each underlying host in your Mesos cluster. Once installed, the agent will automatically connect to the Mesos and Marathon APIs to pull relevant metadata about the environment and will begin monitoring all of your hosts, apps, containers, and frameworks.

Prerequisites

• Review the Agent Installation Requirements.

• Collect the configuration parameters. You will need them to create the JSON file.

  • ACCESS_KEY: Your unique access key string. Inability to retrieve the key indicates that the administrator of your instance might have it turned off for non-admin users. Contact your Sysdig administrator to receive the key. If you still have issues please contact Sysdig Support.

  • COLLECTOR: The collector URL for Sysdig Monitor or Sysdig Secure. This value is region-dependent in SaaS and is auto-completed on the Get Started page in the UI. It is a custom value in on-prem installations. See SaaS Regions and IP Ranges.

  • COLLECTOR_PORT: The default is 6443. It is used in environments with Sysdig’s on-premises backend installed.

  • SECURE: Use a secure SSL/TLS connection to send metrics to the collector. It is used in environments with Sysdig’s on-premises backend installed.

  • CHECK_CERT: Determines strong SSL certificate check for Sysdig Monitor on-premises installation. Set to true when using SSL/TLS to connect to the collector service to ensure that a valid SSL/TLS certificate is installed. It is used in environments with Sysdig’s on-premises backend installed.

Installation

In this three-part installation, you:

• Deploy the Sysdig agent on all Mesos Agent (Slave) nodes, either automatically or by creating and posting a .json file to the leader Marathon API server.

• Deploy the Sysdig agent on the Mesos Master nodes.

• Special configuration steps: modify the Sysdig agent config file to monitor Marathon instances.

Deploy the Sysdig Agent on Mesos Agent Nodes

Preferred Option: Automatic install (DC/OS 1.11+)

If you’re using DC/OS 1.8 or higher, then you can find Sysdig agent in the Mesosphere Universe marketplace and install it from there.

It will automatically deploy the Sysdig agent container on each of your Mesos Agent nodes as a Marathon app.

Proceed to Deploy the Sysdig Agent.

Alternate Option: Post a .json file

If you are using a version of DC/OS earlier than 1.8 then:

1. Create a JSON file for Marathon, in the following format. See configuration parameters for details.

   {
     "backoffFactor": 1.15,
     "backoffSeconds": 1,
     "constraints": [
       [
         "hostname",
         "UNIQUE"
       ]
     ],
     "container": {
       "docker": {
         "forcePullImage": true,
         "image": "sysdig/agent",
         "parameters": [],
         "privileged": true
       },
       "type": "DOCKER",
       "volumes": [
         {
           "containerPath": "/host/var/run/docker.sock",
           "hostPath": "/var/run/docker.sock",
           "mode": "RW"
         },
         {
           "containerPath": "/host/dev",
           "hostPath": "/dev",
           "mode": "RW"
         },
         {
           "containerPath": "/host/proc",
           "hostPath": "/proc",
           "mode": "RO"
         },
         {
           "containerPath": "/host/boot",
           "hostPath": "/boot",
           "mode": "RO"
         },
         {
           "containerPath": "/host/lib/modules",
           "hostPath": "/lib/modules",
           "mode": "RO"
         },
         {
           "containerPath": "/host/usr",
           "hostPath": "/usr",
           "mode": "RO"
         }
       ]
     },
     "cpus": 1,
     "deployments": [],
     "disk": 0,
     "env": {
       "ACCESS_KEY": "ACCESS_KEY=YOUR-ACCESS-KEY-HERE",
       "CHECK_CERT": "false",
       "SECURE": "true",
       "TAGS": "example_tag:example_value",
       "name": "sdc-agent",
       "pid": "host",
       "role": "monitoring",
       "shm-size": "350m"
     },
     "executor": "",
     "gpus": 0,
     "id": "/sysdig-agent",
     "instances": 1,
     "killSelection": "YOUNGEST_FIRST",
     "labels": {},
     "lastTaskFailure": {
       "appId": "/sysdig-agent",
       "host": "YOUR-HOST",
       "message": "Container exited with status 70",
       "slaveId": "1fa6f2fc-95b0-445f-8b97-7f91c1321250-S2",
       "state": "TASK_FAILED",
       "taskId": "sysdig-agent.3bb0759d-3fa3-11e9-b446-c60a7a2ee871",
       "timestamp": "2019-03-06T00:03:16.234Z",
       "version": "2019-03-06T00:01:57.182Z"
     },
     "maxLaunchDelaySeconds": 3600,
     "mem": 850,
     "networks": [
       {
         "mode": "host"
       }
     ],
     "portDefinitions": [
       {
         "name": "default",
         "port": 10101,
         "protocol": "tcp"
       }
     ],
     "requirePorts": false,
     "tasks": [
       {
         "appId": "/sysdig-agent",
         "healthCheckResults": [],
         "host": "YOUR-HOST-IP",
         "id": "sysdig-agent.0d5436f4-3fa4-11e9-b446-c60a7a2ee871",
         "ipAddresses": [
           {
             "ipAddress": "YOUR-HOST-IP",
             "protocol": "IPv4"
           }
         ],
         "localVolumes": [],
         "ports": [
           4764
         ],
         "servicePorts": [],
         "slaveId": "1fa6f2fc-95b0-445f-8b97-7f91c1321250-S2",
         "stagedAt": "2019-03-06T00:09:04.232Z",
         "startedAt": "2019-03-06T00:09:06.912Z",
         "state": "TASK_RUNNING",
         "version": "2019-03-06T00:09:04.182Z"
       }
     ],
     "tasksHealthy": 0,
     "tasksRunning": 1,
     "tasksStaged": 0,
     "tasksUnhealthy": 0,
     "unreachableStrategy": {
       "expungeAfterSeconds": 0,
       "inactiveAfterSeconds": 0
     },
     "upgradeStrategy": {
       "maximumOverCapacity": 1,
       "minimumHealthCapacity": 1
     },
     "version": "2019-03-06T00:09:04.182Z",
     "versionInfo": {
       "lastConfigChangeAt": "2019-03-06T00:09:04.182Z",
       "lastScalingAt": "2019-03-06T00:09:04.182Z"
     }
   }
   

   See Environment Variables for Agent Config File for the Sysdig name:value definitions.

   Complete the “cpus”, “mem” and “labels” (i.e. Marathon labels) entries to fit the capacity and requirements of the cluster environment.

2. Update the created.json file to the leader Marathon API server:

   $ $curl -X POST http://$(hostname -i):8080/v2/apps -d @sysdig.json -H "Content-type: application/json"
   

Deploy the Sysdig Agent on Master Nodes

After deploying the agent to the Mesos Agent nodes, you will install agents on each of the Mesos Master nodes as well.

Use the Agent Install: Non-Orchestrated instructions to install the agent directly on each of your Mesos Master nodes.

When the Sysdig agent is successfully installed on the master nodes, it will automatically connect to the local Mesos and Marathon (if available) API servers via http://localhost:5050 and http://localhost:8080 respectively, to collect cluster configuration and current state metadata in addition to host metrics.

Additional Configuration

In certains situations, you may need to add additional configurations to the dragent.yaml file:

• If the Sysdig agent cannot be run directly on the Mesos API server.

• If the API server is protected with a username/password.

Descriptions and examples are shown below.

Sysdig Agent Unable to Run on the Mesos API Server

Mesos allows multiple masters. If the API server can not be instrumented with a Sysdig agent, simply delegate ONE other node with an agent installed to remotely receive infrastructure information from the API server.

NOTE: If you manually configure the agent to point to a master with a static configuration file entry, then automatic detection/following of leader changes will no longer be enabled.

Add the following Mesos parameter to the delegated agent’s dragent.yaml file to allow it to connect to the remote API server and authenticate, either by:

a. Directly editing dragent.yaml on the host, or

b. Converting the YAML code to a single-line format and adding it as an ADDITIONAL_CONF argument in a Docker command.

See Understanding the Agent Configuration for details.

Specify the API server’s connection method, address, and port. Also specify credentials if necessary.

YAML example:

mesos_state_uri: http://[acct:passwd@][hostname][:port]
marathon_uris:
  - http://[acct:passwd@][hostname][:port]

Mesos API Server Requires Authentication

If the agent is installed on the API server but the API server uses a different port or requires authentication, those parameters must be explicitly specified.

Add the following Mesos parameters to the API server’s dragent.yaml to make it connect to the API server and authenticate with any unique account and password, either by:

a. Directly editing dragent.yaml on the host, or

b. Converting the YAML code to a single-line format and adding it as an ADDITIONAL_CONF argument in a Docker command.

See Understanding the Agent Configuration for details.

Specify the API server’s protocol, user credentials, and port:

mesos_state_uri: http://[username:password@][hostname][:port]
marathon_uris:
  - http://[acct:passwd@][hostname][:port]

*HTTPS protocol is also supported.

Troubleshooting: Turning Off Metadata Reception

In troubleshooting cases where auto-detection and reporting of your Mesos infrastructure needs to be temporarily turned off in a designated agent:

1. Comment out the Mesos parameter entries in the agent’s dragent.yaml file.

   Example parameters to disable: mesos_state_uri, marathon_uris

2. If the agent is running on the API server (Master node) and auto-detecting a default configuration, you can add the line:

   mesos_autodetect: false

   either directly in the dragent.yaml file or as an ADDITIONAL_CONF parameter in a Docker command.

3. Restart the agent.

9 - Airgapped Agent Installation

Airgapped environments are those that do not have network access to the public internet.

At startup, the agent will try to compile its own version of the probes, provided kernel header packages are installed on the host. Failing that, the agent will try to download pre-compiled probes, sysdigcloud-probe-<suffix>.ko or sysdigcloud-probe-bpf-<suffix>.o, from the Sysdig download site over the internet.

In an airgapped environemnt, you cannot download these artifacts. Therefore, before installing the agent, you will have to compile sysdigcloud-probe-<suffix> for each kernel version in your environment, and make it available to the installed agents through an internally accessible URL.

Prerequisites

• A machine with internet access where you can download the required artifacts
• A machine in your airgapped environment where you can build your probes
• Tool to transfer artifacts to the machine in your airgapped environment
• Docker installed

Overview

Sysdig provides a tool, named the probe builder, to help you build the probes for different kernels and for a specific agent version. After downloading the required artifacts on a machine connected to the internet, you can copy them to an airgapped host, build your own probes, and make them available to your agent installations.

On a Machine with Internet Connectivity

Prepare the Sysdig Probe Builder Images

On a machine with internet connectivity, build the Sysdig probe builder container images and create a tar file of the images.

1. Get the probe builder source code from the repository:

   $ git clone https://github.com/draios/probe-builder
   

2. Build the container image for the probe builder:

   $ docker build -t airgap/sysdig-probe-builder probe-builder/
   

3. Build the images for each supported distribution-compiler combination:

   $ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock airgap/sysdig-probe-builder:latest -P -b airgap/
   

   Running this command will create a different image tag for each supported combination of distribution-compiler, with the distro-compiler information suffixed to the image name, airgap/sysdig-probe-builder. For example, airgap/sysdig-probe-builder:centos-gcc4.8.

4. Save all the above images to a tar archive:

   $ docker save airgap/sysdig-probe-builder | gzip > builders.tar.gz
   

5. (optional) If you are building probes for the Ubuntu kernels, you will also need an ubuntu:latest image on your airgapped host. You can build it as follows:

   $ docker pull ubuntu
   $ docker save ubuntu | gzip > ubuntu.tar.gz
   

Download the Kernel Packages

Download your kernel packages. For more information, see Download Kernel Packages.

Download Probe Source Code

You need to download the probe source code for a specific agent version you want to build your probes for.

For example, for agent version 12.0.0 you would use:

$ git clone https://github.com/draios/agent-libs
$ cd agent-libs
$ git archive agent/12.0.0 --prefix sysdig/ | gzip > sysdig.tar.gz

Transfer the Downloaded Files

Copy the artifacts you have built to the airgapped host machine:

• builders.tar.gz
• ubuntu.tar.gz (if needed, see above)
• sysdig.tar.gz
• Kernel packages

On the Airgapped Host

Load the Builder Images

$ zcat builders.tar.gz | docker load

Unpack the Sysdig Source

$ tar xzf sysdig.tar.gz

Running this command will create the sysdig/ directory in the current directory.

Move the Kernel Packages to a Dedicated Location

Make sure you have all the downloaded kernel package artifacts in a single directory, /directory-containing-kernel-packages/, for each distribution you want to support.

Run the Probe Builder

Now that you have all your requirements in place, you can run the main probe builder:

$ docker run --rm \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /a-directory-with-some-free-space/:/workspace \
  -v /wherever-you-unpacked/sysdig/:/sysdig \
  -v /directory-containing-kernel-packages/:/kernels \
  airgap/sysdig-probe-builder:latest -B -b airgap/ -- \
  -p sysdigcloud-probe -v 12.0.0 -k CustomCentOS

The probes will appear in /a-directory-with-some-free-space/output. That directory can be served over HTTP and the URL to the server used as SYSDIG_PROBE_URL when loading the module (e.g. agent-kmodule container). As an example, the following sections describe how you can deploy your own nginx server within your cluster and upload your probes there.

Serve Your Pre-Compiled Probes

Set up a local repository to host the pre-compiled kernel module. Here is an example with nginx:

$ docker run --rm -v /a-directory-with-some-free-space/output:/usr/share/nginx/html/stable/sysdig-probe-binaries -p 80:80 nginx

Note down the URL and use it as the SYSDIG_PROBE_URL while installing the agent.

​ See Run the Probe Builder.

Run the Probe Builder

$ docker run --rm \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /sysdigcloud-probe/:/workspace \
  -v /wherever-you-unpacked/sysdig/:/sysdig \
  -v /directory-containing-kernel-packages/:/kernels \
  airgap/sysdig-probe-builder:latest -B -b airgap/ -- \
  -p sysdigcloud-probe -v 12.0.0 -k CustomCentOS

The probes will appear in /sysdigcloud-probe/output. This directory can be served over HTTP and the URL to the server used as SYSDIG_PROBE_URL when loading the module. For example, agent-kmodule container.

Install Agent in a Docker Environment

1. Install Sysdig agent by pointing SYSDIG_PROBE_URL to the local repository:

   For docker-based installations:

   $ docker run -d --name sysdig-agent --restart always --privileged --net host --pid host -e ACCESS_KEY=WWWWW-YYYY-XXXX-ZZZZ-123456789 -e SECURE=true -e SYSDIG_PROBE_URL=http://www.mywebserver.net:80/ -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro --shm-size=512m sysdig/agent
   

   Where -e SYSDIG_PROBE_URL=http://www.mywebserver:80/ is the local nginx web server with the loaded module.

   Note: To use HTTPS communication with a self-signed or untrusted certificate, use the -e SYSDIG_PROBE_INSECURE_DOWNLOAD=true environment variable in the above command.

2. Check the agent log. If the installation is successful, you will see a message as follows:

   Evaluating override of environment variables
   
   Trying to download precompiled module from http://mywebserver:80/stable/sysdig-probe-binaries/sysdigcloud-probe-<version>
   
   Download succeeded
   

3. Continue with the instructions in Agent Install: Non-Orchestrated.

Install Agent in a Kubernetes Environment

1. Open your agent daemonset and update the SYSDIG_PROBE_URL to point to the local repository:

   - name: SYSDIG_PROBE_URL
     value: http://www.mywebserver:80/
   

   If you would like to use secure communication with a self-signed or untrusted certificate, apply the SYSDIG_PROBE_INSECURE_DOWNLOAD environment variable.

   - name: SYSDIG_PROBE_INSECURE_DOWNLOAD
     value: true
   

2. Continue with the instructions in Agent Install: Kubernetes.

10 - Identify Agent Version

Use one of the following methods to determine the version of the agents installed in your environment:

Explore

Segmenting metrics by using agent.version shows the installed versions of agents in your environment. For example, segment the uptime metric across your environment by using agent.version . Hover over the graph to see the list of agent versions.

The image shows the list of agent versions in n/a.

Dashboard

Use the Sysdig Agent Health Dashboard to determine the agent versions:

1. Log in to the Sysdig Monitor.

2. Select Dashboards and expand Host Infrastructure Dashboards.

3. Open the Sysdig Agent Health & Status template or create your own from the template.

   

   The Sysdig Agent and Health & Status Dashboard shows the agent version corresponding to each host in your environment.