This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Documentation

1.1:

1.1.1:

1.1.1.1:

1.1.1.1.1:

1.1.1.1.2:

1.1.1.2:

1.1.1.3:

1.1.1.3.1:

1.1.1.3.2:

1.1.1.3.3:

1.1.1.3.4:

1.1.1.3.5:

1.1.1.4:

1.1.1.5:

1.1.1.6:

1.1.1.7:

1.1.1.8:

1.1.1.9:

1.1.1.10:

1.1.1.11:

1.1.1.12:

1.1.2:

1.1.2.1:

1.1.2.2:

1.1.2.2.1:

1.1.2.2.2:

1.1.2.2.3:

1.1.2.3:

1.1.2.4:

1.1.2.4.1:

1.1.2.4.2:

1.1.2.4.3:

1.1.2.4.4:

1.1.2.4.4.1:

1.1.2.4.5:

1.1.2.4.6:

1.1.2.4.7:

1.1.2.5:

1.1.2.6:

1.1.2.7:

1.1.2.8:

1.1.2.8.1:

1.1.2.8.2:

1.1.2.8.3:

1.1.2.9:

1.1.2.10:

1.1.3:

1.1.4:

1.2:

1.2.1:

1.2.1.1:

1.3:

1.4:

1.4.1:

1.4.2:

1.4.3:

1.5:

1.6:

1.7:

2.1:

2.1.1:

2.1.1.1:

2.1.2:

2.1.2.1:

2.1.2.2:

2.1.3:

2.1.3.1:

2.1.3.1.1:

2.1.3.1.2:

2.1.3.1.3:

2.1.3.1.4:

2.1.3.1.5:

2.1.3.1.6:

2.1.3.1.7:

2.1.3.1.8:

2.1.3.1.9:

2.1.3.1.10:

2.1.3.1.11:

2.1.3.2:

2.1.4:

2.1.4.1:

2.1.4.2:

2.1.5:

2.1.6:

2.1.7:

2.1.8:

2.1.8.1:

2.1.8.2:

2.1.9:

2.1.9.1:

2.1.9.2:

2.1.9.2.1:

2.1.9.2.2:

2.1.9.2.3:

2.1.9.2.4:

2.1.9.3:

2.1.9.3.1:

2.1.9.3.2:

2.1.9.3.3:

2.1.9.3.4:

2.1.9.4:

2.1.9.5:

2.1.10:

2.2:

2.3:

2.4:

2.5:

2.5.1:

2.5.1.1:

2.5.1.2:

2.5.1.3:

2.5.2:

2.5.2.1:

2.5.2.1.1:

2.5.2.2:

2.5.2.3:

2.5.2.4:

2.5.2.5:

2.5.2.5.1:

2.5.2.5.2:

2.5.2.5.3:

2.5.2.6:

2.5.3:

2.5.3.1:

2.5.3.2:

2.5.3.3:

2.5.3.4:

2.5.3.5:

2.5.3.6:

2.5.3.7:

2.5.4:

2.5.5:

2.5.6:

2.5.7:

2.5.7.1:

2.5.7.2:

2.5.7.2.1:

2.5.7.2.2:

2.5.7.2.3:

2.5.7.2.4:

2.5.7.3:

2.5.7.3.1:

2.5.7.3.2:

2.5.7.3.3:

2.5.7.3.4:

2.5.7.4:

2.5.7.4.1:

2.5.7.4.2:

2.5.7.4.3:

2.5.7.4.4:

2.5.7.5:

2.5.7.6:

2.6:

2.6.1:

2.6.2:

2.6.2.1:

2.6.2.2:

2.6.2.3:

2.6.3:

3.1:

3.1.1:

3.2:

3.2.1:

3.2.2:

3.2.3:

3.2.4:

3.3:

3.3.1:

3.3.2:

3.3.3:

3.3.4:

3.3.5:

3.3.6:

3.3.7:

3.3.8:

3.4:

3.4.1:

3.4.2:

3.4.3:

3.4.4:

3.4.5:

3.4.6:

3.4.7:

3.5:

3.5.1:

3.5.2:

3.5.2.1:

3.5.2.2:

3.5.2.3:

3.5.2.4:

3.5.3:

3.5.3.1:

3.5.3.2:

3.5.3.2.1:

3.5.3.2.2:

3.5.3.3:

3.5.3.3.1:

3.5.3.3.2:

3.5.3.4:

3.5.3.5:

3.5.3.6:

3.5.3.7:

3.5.3.8:

3.5.3.9:

3.5.3.10:

3.5.3.11:

3.5.3.12:

3.5.3.13:

3.5.3.14:

3.5.3.14.1:

3.5.3.14.2:

3.5.3.14.3:

3.5.3.15:

3.5.3.16:

3.5.3.17:

3.5.3.17.1:

3.5.3.17.2:

3.5.3.18:

3.5.3.19:

3.5.3.20:

3.5.3.21:

3.5.3.22:

3.5.3.23:

3.5.3.24:

3.5.3.25:

3.5.4:

3.5.4.1:

3.5.4.2:

3.5.5:

3.5.6:

3.5.6.1:

3.5.6.1.1:

3.5.6.1.2:

3.5.6.1.3:

3.5.6.1.4:

3.5.6.1.5:

3.5.6.1.6:

3.5.6.1.7:

3.5.6.1.8:

3.5.7:

3.5.8:

3.5.9:

3.5.10:

3.5.11:

3.5.11.1:

3.5.11.2:

3.5.12:

3.5.13:

3.5.14:

3.5.15:

3.5.16:

3.6:

3.6.1:

3.6.2:

3.6.3:

3.6.4:

3.6.4.1:

3.6.4.2:

3.6.4.3:

3.6.4.4:

3.6.5:

3.6.5.1:

3.6.5.2:

3.6.5.2.1:

3.6.5.2.2:

3.6.5.2.3:

3.6.5.2.4:

3.6.5.2.5:

3.6.5.2.6:

3.6.5.3:

3.6.5.3.1:

3.6.5.3.2:

3.6.5.3.3:

3.6.5.3.4:

3.6.5.3.5:

3.6.6:

3.6.6.1:

3.6.6.2:

3.6.6.3:

3.6.6.4:

3.6.6.4.1:

3.6.6.4.2:

3.6.7:

3.7:

3.7.1:

3.7.2:

3.7.3:

3.7.4:

3.7.5:

3.7.6:

3.7.7:

3.8:

3.8.1:

3.8.2:

3.8.3:

3.8.4:

3.8.5:

3.8.6:

3.8.7:

3.8.8:

3.8.9:

3.9:

3.9.1:

3.9.2:

3.10:

3.10.1:

3.10.1.1:

3.10.1.2:

3.10.1.3:

3.10.1.4:

3.10.2:

3.10.2.1:

3.10.2.2:

3.10.2.3:

3.10.2.4:

3.10.2.5:

3.10.2.6:

3.10.2.7:

3.10.2.8:

3.10.2.9:

3.10.2.10:

3.10.3:

3.10.4:

3.10.5:

3.10.6:

3.10.7:

3.10.8:

3.10.8.1:

3.10.8.2:

3.10.8.3:

3.10.8.4:

3.10.8.5:

3.10.8.6:

3.10.8.7:

3.10.8.8:

3.10.8.9:

3.10.8.10:

3.10.8.11:

3.10.8.12:

3.10.8.13:

3.10.8.14:

3.10.8.15:

3.10.8.16:

3.10.8.17:

3.10.8.18:

3.10.8.19:

3.10.8.20:

3.10.8.21:

3.10.8.22:

3.10.8.23:

3.10.8.24:

3.10.8.25:

3.10.8.26:

3.10.8.27:

3.10.9:

3.10.10:

3.10.11:

4.1:

4.2:

4.3:

4.3.1:

4.3.2:

4.3.3:

4.3.4:

4.3.4.1:

4.3.4.2:

4.3.4.3:

4.4:

4.5:

4.5.1:

4.5.1.1:

4.5.2:

4.5.3:

4.5.4:

4.5.4.1:

4.5.4.2:

4.5.4.3:

4.5.5:

4.5.6:

4.5.6.1:

4.5.7:

4.5.8:

4.5.9:

4.5.10:

4.5.10.1:

4.5.11:

4.5.12:

4.6:

4.6.1:

4.6.2:

4.7:

4.7.1:

4.7.2:

4.7.3:

4.7.3.1:

4.7.4:

4.7.5:

4.7.6:

4.8:

4.9:

4.9.1:

4.9.1.1:

4.9.1.2:

4.9.1.3:

4.9.1.4:

4.9.1.5:

4.9.1.6:

4.9.1.7:

4.9.1.8:

4.9.1.9:

4.9.1.10:

4.9.2:

4.9.3:

4.10:

4.10.1:

4.10.1.1:

4.10.2:

4.10.3:

4.11:

4.11.1:

4.12:

4.12.1:

4.12.1.1:

4.12.2:

4.12.2.1:

4.12.3:

4.12.3.1:

4.13:

4.13.1:

4.13.2:

5.1:

5.1.1:

5.1.2:

5.2:

5.3:

5.4:

5.5:

5.6:

5.7:

5.7.1:

5.8:

5.8.1:

5.8.2:

5.8.3:

5.9:

6.1:

6.2:

6.3:

6.4:

6.5:

6.6:

6.7:

Welcome to the Sysdig Documentation Hub!

Explore Sysdig’s brand-new documentation site using the table of contents placed at the left of this paragraph.

We encourage you to send your feedback at doc-feedback@sysdig.com.

1 -

Installation

This guide describes deployment options for various Sysdig components:

Sysdig Agents (also called Host Agents)
Serverless Agents used for container-based cloud environments such as Fargate
Sysdig Secure for cloud connection components: CIS Benchmarks, threat detection based on cloud provider native logs and compute resources and registry image scanning.
Rapid Response component, allowing designated Advanced Users to remote connect into a host (available for Sysdig Secure on-prem only)
Node Analyzer: Multi-Feature Installation for benchmarks, host scanning, and the image analyzer.
Admission Controller: Installation for enhanced scanning, where container images that do not fulfill the configured admission policies will be rejected from the cluster before being assigned to a node and allowed to run.

1.1 -

Sysdig Agent

Sysdig agents are simple to deploy and upgrade, and out of the box, they will gather and report on a wide variety of pre-defined metrics. Agent installation can be as simple as copying and pasting a few lines of code from a Wizard prompt, or you can follow step-by-step instructions to check supported environments and distributions, review installation options, and customize the agent configuration file post-install.

About Sysdig Agent

Sysdig agent is a lightweight host component that processes syscall, creates capture files, and performs auditing and compliance. It is a platform that supports monitoring, securing, and troubleshooting cloud-native environments

In addition, the agent performs the following:

Metrics processing, analysis, aggregation, and transmission
Policy monitoring and alert notification through bidirectional communication with the Sysdig collector
Integration with third-party software for consolidating customer ecosystem data
Full assimilation into containerized and orchestrated environment

Data Processing

A top-level flow for monitoring is illustrated in the following diagram.

The following agent components gather relevant data:
- Syscall
- StatsD
- JMX
- Promscrape
Store the metrics for analysis.
Sends the metrics in every second for analysis and aggregation.
Build the 10-second roll-up data.
Send the data every tenth second for serialization
Send the serialized metrics to the Sysdig collector

Learn More

Host Requirements: Determine supported platforms and system requirements for installing Sysdig Agent.
Agent Licensing: Retrieve the agent access key and understand the subscription model.
Installing Agent: Install Sysdig agents on supported platforms.
Configuring Agent: Edit the agent configuration file to filter incoming metrics, change log levels, and more.
Using Agent Modes: Edit the agent configuration file to configure agent modes.
Upgrading Sysdig Agent: Upgrade Sysdig agent installed as a service or as a container.
Troubleshooting Sysdig Agent: Troubleshoot common Sysdig agent issues.

1.1.1 -

Agent Installation

Sysdig agents are delivered as either a container or a service and can be deployed with or without an orchestrator such as Kubernetes or Mesos.

A quick install involves just a few lines of code from the Getting Started Wizard copied into a shell. The complete install instructions address checking for and installing kernel headers if needed, any prerequisite permissions settings for particular environments, and tips about updating the configuration files after initial installation.

Plan the Installation

Topic	Description
Host Requirements	Review the platforms, runtimes, Linux distributions, orchestration, browsers, etc. that are supported.
Kernel Header Troubleshooting	Additional tips if your kernel module or header are not automatically compiled by the agent
Access key	An agent access key is provided with a Sysidg trial
Installation flavours	Choose one of the installations: Slim Agent : A lighter version of the Sysdig agent created by splitting the regular agent image into two components responsible for different functionalities. You install and run these modules as containers. Regular Agent: The Sysdig agent that you can run as a container or a service.

Install Options

Wizard-Based

With the Getting Started Wizard, you can copy/paste a simple line of code to deploy agents in a variety of environments.

Node Image Analyzer by default is installed with Sysdig Agent for Sysdig Secure. See Installing the Image Analyzer for more information.

Behind the scenes, the Wizard auto-detects and completes configuration items such as the required access key and port information. The Wizard is launched from the Start a Free Trial button at sysdig.com.

After the first install, Sysdig Secure and Monitor users can access the Wizard at any time from the Rocket icon in the navbar.

Manual Options

There are a variety of ways to install agents without using the Wizard.

	Flavor	Helm	Manual	Slim Agent	Notes
Kubernetes	Open Source	Readme	Kubernetes vanilla	Install Slim Agent	Used for most environments, e.g. Amazon EKS or EC2 on AWS Cloud or AWS Outpost, EC2, Azure AKS, etc.
	OpenShift	Readme	OpenShift	Used for any environment with OpenShift.
	GKE	Readme	GKE		Used for Google Kubernetes Service environment
Non-Orchestrated	Agent as container or agent as service	n/a	Container Service	Install Slim Agent	Used when there is no orchestrator such as Kubernetes.
IKS		n/a	IBM Cloud with Sysdig)	Install Slim Agent	IBM manages and documents Sysdig installs as part of IKS.

Certain environments may need a different option:

Mesos | Marathon| DCOS: Agent Install: Mesos | Marathon | DCOS
Linux: (Rare, used with custom kernel headers, unique use cases) Agent Install: Manual Linux Installation

1.1.1.1 -

Host Requirements for Agent Installation

Sysdig agents can be installed on a wide array of Linux hosts. Check your environment to ensure it meets the minimum supported platform, operating system, runtime, and orchestration requirements and uses the appropriate installation instructions.

Versioning Scheme

We recommend that you use the latest version of the agent. Sysdig supports n-3 versions back based on the minor number. For example, if the latest release is v12.0.0, we will support n-3 versions back, up to v11.2.0.

End of Support

Sysdig agents that are older than version 0.85.1, released October 1, 2018, will no longer connect to the Sysdig US-East SaaS platform with default agent values.

Going forward all the agent releases will have a 3-year deprecation policy. This implies:

Sysdig Support might not be able to help you troubleshoot or address the problems with agents past the deprecation date.
Sysdig will no longer provide prebuilt kernel probe binaries for these agent releases. You need to build the kernel probe binaries on the fly by using the hosts kernel headers.
These changes is effective starting Sysdig agent v12.1.0.

Agent Installation Requirements

Orchestration Platforms

Sysdig agent is supported on the following Orchestration platforms:

Orchastration Platforms	Documentation
Kubernetes v1.9+	Agent Install: Kubernetes
Oracle Kubernetes Engine (OKE)	Steps for OKE
Google Kubernetes Engine (GKE)	Steps for GKE
OpenShift	Steps for OpenShift
IBM Cloud Kubernetes Service (IKS)	Agent Install: IKS (IBM Cloud with Sysdig)
Microsoft Azure Cloud Services	Agent Install: Non-Orchestrated
Microsoft Azure Kubernetes Service (AKS)	Agent Install: Kubernetes
Amazon Elastic Kubernetes Service (Amazon EKS)	Agent Install: Kubernetes
Amazon Elastic Container Service (Amazon ECS)	Agent Install: Non-Orchestrated + AWS Integration Instructions
Rancher	Steps for Rancher
RancherOS	Agent Install: Non-Orchestrated
Mesos/Marathon	Agent Install: Mesos/Marathon
Docker Datacenter (DDC)	Agent Install: Non-Orchestrated

If you are not using an orchestrator in your environment, follow the instructions for Agent Install Non-Orchestrated .

Linux Distributions and Kernels

Sysdig agent is supported on the following Linux distributions:

Platforms	Linux Distribution
Core Set	Debian Ubuntu Ubuntu (Amazon) CentOS Red Hat Enterprise Linux (RHEL) SuSE Linux Enterprise Server RHEL CoreOS (RHCOS) Fedora Fedora CoreOS Linux Mint SUSE Linux Enterprise Server (SLES) Amazon Linux Amazon Linux v2 Amazon Bottlerocket Google Container Optimized OS (COS) Oracle Linux (UEH) Oracle Linux (RHCK)
AWS EC2	Amazon Linux 2 Amazon Bottlerocket Core set (see above)
GCP	Core set (see above) COS
Azure	Core set (see above)

Container Runtimes

Sysdig agent supports the detection of the following:

Docker
LXC
CRI-O
containerd
Podman
Mesos

Prerequisites for Podman Environments

Sysdig agent supports running as a Podman container.

Enable Podman API Service for all the users.
The agent will not able to collect Podman-managed container metadata, such as the container name, if the API service is not enabled.
Secure rules and policies that depend on container metadata other than the container ID will not work.
Pausing and terminating containers will not work because Policy actions for Podman are not supported.
The containers started as a non-root user will have the podman_owner_uid label associated with it if the API service is enabled for that user. The value of podman_owner_uid will be the numeric user ID corresponding to the user that started the container.

Container Registries

Quay.io

For example, to pull the latest agent container from Quay.io:

docker pull quay.io/sysdig/agent

Java Versions and Vendors

Sysdig agent supports the following:

Java versions: v7 and above
Vendors: Oracle, OpenJDK

For Java-based applications (Cassandra, Elasticsearch, Kafka, Tomcat, Zookeeper and etc.), the Sysdig agent requires the Java runtime environment (JRE) to be installed to poll for metrics (beans).

If the Docker-container-based Sysdig agent is installed, the JRE is installed alongside the agent binaries and no further dependencies exist. However, if you are installing the service-based agent (non-container) and you do not see the JVM/JMX metrics reporting, your host may not have the JRE installed or it may not be installed in the expected location: usr/bin/java

Minimum Resource Requirements

The resource requirements of the agent are subjective to the size and load of the host— more activity equates to more resources required. At a minimum, the agent requires 2% of the total CPU and 512MiB of memory.

It is typical to see between 5-20KiB/s of bandwidth consumed—different variables can increase the throughput required such as the number of metrics, events, Kubernetes objects, and which products and features are enabled. When a Sysdig Capture is being collected, you can expect to see a spike in bandwidth while the capture file is being ingested.

We do not recommend placing bandwidth shaping or caps on the agent to ensure data can be sent to our collection service. For more information, see Tuning Sysdig Agent.

Additional Requirements

Access key

The installation of the Sysdig agent requires an access key.

This key and the agent installation instructions are presented to you after activating your account and using a web-based wizard upon initial login.

The same information can also be found in the Settings > Agent Installation menu of the web interface after logging in. See Agent Installation: Overview and Key for details.

Network connection

A Sysdig agent (containerized or native) is installed into each host being monitored and will need to be able to connect to the Sysdig Monitor backend servers to report host metrics. The agent must be able to reach the Sysdig Collector addresses. For example, for US East, it is ‘collector.sysdigcloud.com’ (via multiple IPs) over port tcp/6443 . See Sysdig Collector Ports for supported ports for other regions.

The agent supports the HTTP proxy for communicating with Sysdig backend components. For more information, see Enable HTTP Proxy for Agents.

1.1.1.1.1 -

Kernel Header Troubleshooting

In addition to the information on Host Requirements for Agent Installation, this page describes how the agent uses kernel headers and tips on troubleshooting, if needed.

About Kernel Headers and the Kernel Module

The Sysdig agent requires a kernel module in order to install successfully on a host. This can be obtained in three ways:

Agent compiles the module using kernel headers.
If the hosts in your environment already have kernel header files pre-installed, no special action is needed. Or you can install the kernel headers manually; see below.
Agent auto-downloads precompiled modules from Sysdig’s AWS storage location.
If the headers are not already installed but the agent is able to auto-download, no special action is needed. If there is no internet connectivity, you can use method 3 (download from an internal URL).
Agent downloads precompiled modules from an internal URL.
Use the environment variable SYSDIG_PROBE_URL. See also Understanding the Agent Config Files. Contact Sysdig support for assistance.

TracePoints

All supported distribution released kernels have this support but if creating a custom kernel, it must support the following options:

CONFIG_TRACEPOINTS
CONFIG_HAVE_SYSCALL_TRACEPOINTS

To Install Kernel Headers

In some cases, the host(s) in your environment may use Unix versions that do not match the provided headers, and the agent may fail to install correctly. In those cases, you must install the kernal headers manually.

Debian-Style

For Debian-syle distributions, run the command:

apt-get -y install linux-headers-$(uname -r)

RHEL-Style

For RHEL-style distributions, run the command:

yum -y install kernel-devel-$(uname -r)

RancherOS

For RancherOS distributions, the kernel headers are available in the form of a system service and therefore are enabled using the ros service command:

sudo ros service enable kernel-headers-system-docker
sudo ros service up -d kernel-headers-system-docker

NOTE: Some cloud hosting service providers supply pre-configured Linux instances with customized kernels. You may need to contact your provider’s support desk for instructions on obtaining appropriate header files, or for installing the distribution’s default kernel.

To Correct Kernel Header Errors in AWS AMI

During an agent installation in an Amazon machine image (AMI) you may encounter the following errors while the installer is trying to compile the Sysdig kernel module:

Errors

“Unable to find kernel development files for the current kernel version” or
“FATAL: Module sysdigcloud-probe not found”

This indicates your machine is running a kernel in an older AMI for which the kernel headers are no longer available in the configured repositories. The issue has to do with Amazon purging packages in the yum repository when new Amazon Linux machine images are released.

The solution is either to update your kernel to a version for which header files are readily available (recommended), or perform a one-time installation of the kernel headers for your older AMI.

Option 1: Upgrade Your Host’s Kernel

First install a new kernel and reboot your instance:

sudo yum -y install kernel
sudo reboot

After rebooting, check to see if the host is reporting metrics to your Sysdig account. If not, you may need to issue three more commands to install the required header files:

sudo yum -y install kernel-devel-$(uname -r)
sudo /usr/lib/dkms/dkms_autoinstaller start
sudo service dragent restart

Option 2: Install Older Kernel Headers

Although it is recommended to upgrade to the latest kernel for security and performance reasons, you can alternatively install the older headers for your AMI.

Find the the AMI version string and install the appropriate headers with the commands:

releasever=$(cat /etc/os-release | grep 'VERSION_ID' | grep -Eo "[0-9]{4}\.[0-9]{2}")
sudo yum -y --releasever=${releasever} install kernel-devel-$(uname -r)

Issue the remaining commands to allow the Sydig Agent to start successfully:

sudo /usr/lib/dkms/dkms_autoinstaller start
sudo service dragent restart

Reference: Find Your AWS Instance Image Version

The file /etc/image-id shows information about the original machine image with which your instance was set up:

[ec2-user ~]$ cat /etc/image-id
image_name="amzn-ami-hvm"
image_version="2017.03"
image_arch="x86_64"
image_file="amzn-ami-hvm-2017.03.0.20170401-x86_64.ext4.gpt"
image_stamp="26a3-ed31"
image_date="20170402053945"
recipe_name="amzn ami"
recipe_id="47cfa924-413c-d460-f4f2-2af7-feb6-9e37-7c9f1d2b"

This file will not change as you install updates from the yum repository.

The file /etc/system-release will tell what version of the AWS image is currently installed:

[ec2-user ~]$ cat /etc/system-release
Amazon Linux AMI release 2017.03

1.1.1.1.2 -

Tuning Sysdig Agent

The resource requirements for the Sysdig agent are subjective to the size and load of the host. Increased activity equates to higher resource requirements. At a minimum, the agent requires 2% of the total CPU and 512 MiB of memory.

You might see 5 to 20 KiB/s of bandwidth consumed. Different variables can increase the throughput required. For example:

The number of metrics
The number of events
Kubernetes objects
Products and features enabled

When a Sysdig Capture is being collected, you can expect to see a spike in the bandwidth while the capture file is being ingested.

Sysdig does not recommend placing bandwidth shaping or caps on the agent to ensure that data is sent to the Sysdig Collection service.

In general, in larger clusters, the agent requires more memory, and in servers with a high number of cores, the agent requires more CPU cores to monitor all the system calls. You will use CPU cores on the host and the Kubernetes nodes visible to the agent as proxies for the rate of events processed in the agent.

Similarly, there are different factors that are at play, and considering all the factors, we recommend the following:

Small: CPU core count <= 8. Kubernetes nodes <=10

Medium: 8 < CPU core count <= 32. 10 < Kubernetes nodes <= 100

Large: CPU core count > 32. Kubernetes nodes > 100

While you can expect the behavior with the given numbers to be better than simply using the default values, Sysdig cannot guarantee that resource allocation will be correct for all the cases.

Cluster Size	Small	Medium	Large
Kubernetes CPU Request	1	3	5
Kubernetes CPU Limit	1	3	5
Kubernetes Memory Request	1024 MB	3072 MB	6144 MB
Kubernetes Memory Limit	1024 MB	3072 MB	6144 MB
Dragent Memory Watchdog	512 MB	1024 MB	2048 MB
Cointerface Memory Watchdog	512 MB	2048 MB	4096 MB

Note that the agent has its own memory watchdog to prevent runaway memory consumption on the host in case of memory leaks. The default values of the watchdog are specified in the following agent configuration file.

watchdog:
  max_memory_usage_mb: 1024
  max_memory_usage_subprocesses:
    sdchecks: 128
    sdjagent: 256
    mountedfs_reader: 32
    statsite_forwarder: 32
    cointerface: 512
    promscrape: 640

The recommended value for promscrape depends on the amount of timeseries and label data that required to be scraped on a particular node. The cluster size does not have an effect on promscrape memory usage.

max_memory_usage_mb corresponds to the dragent process in the agent. All the values are given in MiB.

For example, use the following agent configuration to match the agent watchdog settings with large values:

watchdog:
  max_memory_usage_mb: 2048
  max_memory_usage_subprocesses:
    sdchecks: 128
    sdjagent: 256
    mountedfs_reader: 32
    statsite_forwarder: 32
    cointerface: 4096
    promscrape: 640

1.1.1.2 -

Quick Install Sysdig Agent on Kubernetes

Sysdig provides a single-line Sysdig Agent installer to help you get started quickly in Kubernetes environments. The code for using the installer is also presented in the Get Started pages of the Sysdig Monitor and Sysdig Secure UIs, with some of the values auto-completed.

This page documents the single-line installer options in more detail.

You can also access the help by using the following command:

 $ ./install-agent-kubernetes --help

Access from Get Started Pages

To access the quick-install code pre-filled with some of your environment variables:

Log in as admin to Sysdig Monitor or Sysdig Secure.
Select the Get Started page (rocket icon).
Choose Install the Agent, select the appropriate deployment type (e.g. Helm or Kubernetes), and copy the auto-generated code, filling in remaining variable values as required.

Sample Usage

$ install-agent-kubernetes [-a | --access_key <value>] [-t | --tags <value>] [-c | --collector <value>] [-cp | --collector_port <value>] [-s | --secure <value>] [-cc | --check_certificate <value>] [-ns | --namespace | --project <value>] [-ac | --additional_conf <value>] [-op | --openshift] [-as | --agent_slim] [-av | --agent_version <value>] [-ae | --api_endpoint <value> ] [-na | --nodeanalyzer ] [-ia | --imageanalyzer ] [-am | --analysismanager <value>] [-ds | --dockersocket <value>] [-cs | --crisocket <value>] [-cv | --customvolume <value>] [-cn | --cluster_name <value>] [-r | --remove ] [-h | --help]

Options

Option	Description
`-a`	The agent access key. You can retrieve this from `Settings > Agent Installation` in either Sysdig Monitor or Sysdig Secure.
`-t`	The list of tags for the host where the agent is installed. For example: "role:webserver, location:europe", "role:webserver" or "webserver".
`-c` or `collector_url`	The collector URL for Sysdig Monitor or Sysdig Secure. This value is region-dependent in SaaS and is auto-completed on the Get Started page in the UI. It is a custom value in on-prem installations.
`-cp`	The collector port. The default is 6443.
`-s`	Use a secure SSL/TLS connection to send metrics to the collector. This option is enabled by default.
`-cc`	Enable strong SSL certificate check. The default is true.
`-ns`	If a value is provided, the agent will be deployed to the specified namespace/project. The default is `sysdig-agent`.
`-op`	If provided, perform the agent installation using the OpenShift command line.
`-ac`	If a value is provided, the additional configuration will be appended to the agent configuration file.
`-av`	If a version is provided, use the specified agent version. The default is the latest version.
`-r`	If a value is provided, the daemonset, configmap, cluster role binding, service acccount and secret associated with the Sysdig Agent will be removed from the specified namespace.
`-ae`	The `api_endpoint` is the region-dependent domain for the Sysdig product, without the protocol. E.g. `secure.sysdig.com`, `us2.app.sysdig.com`, `eu1.app.sysdig.com`
`-h`	Print this usage and exit.
Sysdig Secure Only
`-na`	If provided, will install the Node Analyzer tools. It is an error to set both -ia and -na.
`-ds`	The docker socket for Image Analyzer.
`-cs`	The CRI socket for Image Analyzer.
`-cv`	The custom volume for Image Analyzer.
`-h`	Print this usage and exit.
Sysdig Secure Only (Legacy) These values apply to the Node Image Analyzer (v1) in Sysdig Secure.
`-am`	The Analysis Manager endpoint for Sysdig Secure.
`-ia`	If provided, will install the Node Image Analyzer (v1). It is an error to set both -ia and -na. The v1 Node Image Analyzer will be deprecated and replaced by the NA tools.

1.1.1.3 -

Agent Install: Kubernetes

This document describes how to install a regular Sysdig agent container in a Kubernetes environment. This document assumes you will run the agent container as a Kubernetes pod, which then enables the Sysdig agent automatically to detect and monitor your Kubernetes environment.

If you want to install a slim agent, see Install Slim Agent.

It is relevant for any platform where Kubernetes is deployed, including Amazon environments (EKS, EC2, ECS), Azure Container Service (AKS), Google Kubernetes Engine (GKE), Red Hat OpenShift, Oracle Kubernetes Engine (OKE), and IBM Cloud Kubernetes Service (IKS).

You use DaemonSets to deploy agents on every node in your Kubernetes environment. Once deployed, Sysdig Monitor automatically begins monitoring all of your hosts, apps, pods, and services and automatically connects to the Kubernetes API server to pull relevant metadata about the environment. If licensed, Sysdig Secure launches with default policies that you can view and configure to suit your needs. You can access the front-end web interfaces for Sysdig Monitor and Sysdig Secure immediately.

Prerequisites

A supported distribution. See Host Requirements for Agent Installation for details.
Kubernetes v 1.9+: The agent installation on Kubernetes requires using v1.9 or higher because the APIs used to fetch kubernetes metadata are only present in v1.9+.
Sysdig account and access key: Request a trial or full account at Sysdig.com and click the Activate Account button. You create a Sysdig user name and password.
The Getting Started Wizard provides an access key.

Runtime Support: CRI-O and Containerd

By default, Sysdig agents deployed in Kubernetes automatically detect metadata from containerd and CRI-O (in addition to Docker), as long as the prerequisites are fulfilled.

After reviewing the information on this page, continue with the Sysdig agent installation steps: Kubernetes Agent Installation Steps.

Containerd Support

As of agent version 0.88.1, the Sysdig agent will automatically detect containerd metadata (as well as any Docker metadata) in your environment, as long as the prerequisites are fulfilled.

Prerequisites

Agent version: Sysdig agent version 0.88.1 or higher
NOTE: If you are upgrading from an earlier version of the agent, you must also download the latest sysdig-agent-daemonset-v2.yamlfrom GitHub.
Configuration parameter: In the agent config file, new_k8s: true must be set.
As of agent 9.6.0, new_k8s is enabled by default.
See Enable Kube State Metrics and Cluster Name below for details on editing the config file.
Kubernetes-only: The containerd API must support CRI (a Kubernetes runtime interface).

Results in the Sysdig Monitor UI

If the Sysdig agent detects containerd metadata, it will be reported in the front end as follows:

Explore/Dashboard views: The icon next to container-specific items (container.name, container.id, etc.) shows whether it’s a Docker or containerd object.
Spotlight: Updated for containerd display.
Events: Containerd events die and oom are enabled by default.
Events create and exit are also supported.

CRI-O Support

The Sysdig agent will automatically detect CRI-O metadata (as well as any Docker and/or containerd metadata) in your environment, as long as the Prerequisites are fulfilled.

Prerequisites

Platform version: Sysdig SaaS March 2019or higher
Agent version: Sysdig agent v 0.89.4 March 27, 2019 or higher.
NOTE: If you are upgrading from an earlier version of the agent, you must also download the latest sysdig-agent-daemonset-v2.yamlfrom GitHub.
Configuration parameter: In the agent config file, new_k8s: true must be set.
See Enable Kube State Metrics and Cluster Name below for details on editing the config file.
Kubernetes-only: The API must support CRI (a Kubernetes runtime interface).

Results in the Sysdig Monitor UI

Events: There are no CRI-O events, so the Events pane remains unchanged.
Explore/Dashboard views: The icon next to container-specific items (container.name, container.id, etc.) shows CRI-O type.
Supported Metrics: By default, the same metrics are supported for CRI-O as for Docker and containerd, except for image id (container.image.id).

Enable Image ID Metrics with cri: extra_queries

As of agent version 0.92.1, this setting is enabled by default.

To enable image id metrics, edit the agent configuration file dragent.yaml to contain the following:

cri:
  extra_queries: true

See Understanding the Agent Config Files for more information on editing dragent.yaml.

Complete the Installation

Choose the appropriate link to complete the installation steps:

Steps for Kubernetes (Vanilla)
All environments except IKS, GKE, or those using OpenShift.
Steps for GKE
Steps for Rancher
Steps for OpenShift
Steps for OKE

1.1.1.3.1 -

Steps for Kubernetes (Vanilla)

Preparation

Kernel Headers

The Sysdig agent requires kernel header files to install successfully on a host.

This setup step is required for some environments and not others, as noted.

If the hosts in your environment match the pre-compiled kernel modules available from Sysdig, no special action is required.

To do so:

For Debian-style distributions, run the command:

apt-get -y install linux-headers-$(uname -r)

For RHEL-style distributions, run the command:

yum -y install kernel-devel-$(uname -r)

Background info: see also About Kernel Headers and the Kernel Module.

Background Info

You can review Agent Install: Kubernetes | GKE | OpenShift | IBM and the Host Requirements for Agent Installation for additional context, if desired.

Installation Steps

To deploy agents using Kubernetes daemonsets, you will download the following configuration files, edit them as required, and deploy them.

sysdig-agent-clusterrole.yaml
sysdig-agent-service.yaml
sysdig-agent-daemonset-v2.yaml
sysdig-agent-configmap.yaml

HELM CHART OPTION Kubernetes also offers a package manager, Helm, which uses charts to simplify this process.

If you are using Helm charts in your K8s environment, we recommend using them to deploy Sysdig agents, as described here.

Deploy the Agents

Download the sample files:
- sysdig-agent-clusterrole.yaml
- sysdig-agent-daemonset-v2.yaml
- sysdig-agent-configmap.yaml
- sysdig-agent-service.yaml
Create a namespace to use for the Sysdig agent.
You can use whatever naming you prefer. In this document, we used sysdig-agent for both the namespace and the service account.
The default service account name was automatically defined in sysdig-agent-daemonset-v2.yaml, at the line: serviceAccount: sysdig-agent.
```
kubectl create ns sysdig-agent
```

Create a secret key:

kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent

Create a cluster role and service account, and define the cluster role bindingthat grants the Sysdig agent rules in the cluster role, using the commands:

kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent

Edit sysdig-agent-configmap.yaml to add the collector address , port , and the SSL/TLS information:
```
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
```
- For SaaS, find the collector address for your region.
- For On-prem, enter the collector endpoint defined in your environment.
- check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Step 5 Set Up SSL Connectivity to the Backend.

(All installs) Apply the sysdig-agent-configmap.yaml file:

kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

(All installs) Apply the sysdig-agent-service.yaml file:
```
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
```
This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

(All installs) Apply the daemonset-v2.yaml file :

kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent

The agents will be deployed. See Getting Started with Sysdig Monitor to view some metrics in the Sysdig Monitor UI. You can make further edits to the configmap as described below.Getting Started with Sysdig Monitor

Enable Kube State Metrics and Cluster Name

These steps are optional but recommended.

Edit sysdig-agent-configmap.yaml to uncomment the line: new_k8s: true
This allows kube state metrics to be automatically detected, monitored, and displayed in Sysdig Monitor.
For more information, see the Kube State Metrics entry in the Sysdig blog.
As of agent 9.6.0, new_k8s is enabled by default.
Edit sysdig-agent-configmap.yaml to uncomment the line: **k8s_cluster_name: **and add your cluster name.
Setting cluster name here allows you to view, scope, and segment metrics in the Sysdig Monitor UI by the Kubernetes cluster.
Note: Alternatively, if you assign a tag with “cluster” in the tag name, Sysdig Monitor will display that as the Kubernetes cluster name.

Apply the configmap changes using the command:

kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Proceed to verify the metrics in the Sysdig Monitor UI.

There are two ways to update the agent configuration

Option 1: Edit the files locally and apply the changes with kubectl apply -f:

kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Option 2: Use kubectl edit to edit files on the fly:

kubectl edit configmap sysdig-agent -n sysdig-agent

Running agents will automatically pick the new configuration after Kubernetes pushes the changes across all the nodes in the cluster.

Additional Options

Connect to the Sysdig Backend via Static IPs (SaaS only)

Sysdig provides a list of static IP addresses that can be whitelisted in a Sysdig environment, allowing users to establish a network connection to the Sysdig backend without opening complete network connectivity. This is done by setting the Collector IP to collector-static.sysdigcloud.com.

The sysdig-agent-configmap.yaml file can be edited either locally or using the edit command in Kubernetes. refer to the section above for more information.

To configure the collector IP in a Kubernetes SaaS instance:

Open sysdig-agent-configmap.yaml in a text editor.
Uncomment the following lines:
- collector:
- collector_port
Set the collector: value to collector-static.sysdigcloud.com
See SaaS Regions and IP Ranges and identify the correct URL associated with your Sysdig collector and region.
Set the collector_port: value to 6443
Save the file.

The example file below shows how the sysdig-agent-configmap.yaml file should look after configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-agent
data:
  dragent.yaml: |
    ### Agent tags
    # tags: linux:ubuntu,dept:dev,local:nyc

    #### Sysdig Software related config ####

    # Sysdig collector address
    collector: collector-static.sysdigcloud.com

    # Collector TCP port
    collector_port: 6443

    # Whether collector accepts ssl/TLS
    ssl: true

    # collector certificate validation
    ssl_verify_certificate: true

    # Sysdig Secure
    security:
                enabled: true

    #######################################
    # new_k8s: true
    # k8s_cluster_name: production

Verify Metrics in Sysdig Monitor UI

The steps below give one way to do the check.

Access Sysdig Monitor:
SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for US East, the URL is https://app.sysdigcloud.com.
For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.
Log in with your Sysdig user name and password.
Select the Explore tab to see if metrics are displayed.
(Once you have enabled new_k8s:true): To verify that kube state metrics and cluster name are working correctly: Select the Explore tab and create a grouping by kubernetes.cluster.name and kubernetes.pod.name.
As of agent 9.6.0, new_k8s is enabled by default.
Select an individual container or pod to see details.

Kubernetes metadata (pods, deployments etc.) appear a minute or two later than the nodes/containers themselves; if pod names do not appear immediately, wait and retry the Explore view.

If agents are disconnecting, there could be an issue with your MAC addresses. See Troubleshooting Agent Installation for tips.

1.1.1.3.2 -

Steps for GKE

Google Kubernetes Engine (GKE) is a managed environment for running Kubernetes in Google Cloud, in order to deploy containerized applications. As of Sysdig agent version 0.88, Sysdig supports all flavors of GKE, including Ubuntu and GKE’s default Container-Optimized OS (COS).

GKE COS environments require eBPF probe to support agent installation.

Preparation

Open Port 6443 for Agent Egress

Because GKE uses stateful firewalls, you must actively open port 6443 for the Sysdig agent outbound traffic.

In earlier versions, the Sysdig Agent connected to port 6666. This behavior has been deprecated, as the Sysdig agent now connects to port 6443.

GKE COS/eBPF-Specific Requirements

Linux kernel version >= 4.14.
When performing the installation steps, you will add one additional parameter to install the eBPF probe. See Step 7, below. Note that the eBPF probe is supported only in GKE COS environments.

Background Info

You can review Agent Install: Kubernetes | GKE | OpenShift | IBM and the Host Requirements for Agent Installation for additional context, if desired.

Installation Steps

To deploy agents using Kubernetes daemonsets, you will download the following configuration files, edit them as required, and deploy them.

sysdig-agent-clusterrole.yaml
sysdig-agent-daemonset-v2.yaml
sysdig-agent-configmap.yaml

HELM CHART OPTION Kubernetes also offers a package manager, Helm, which uses charts to simplify this process.

If you are using Helm charts in your K8s environment, we recommend using them to deploy Sysdig agents, as described here.

Deploy the Agents

Download the sample files:
- sysdig-agent-clusterrole.yaml
- sysdig-agent-daemonset-v2.yaml
- sysdig-agent-configmap.yaml
- sysdig-agent-service.yaml
Create a namespace to use for the Sysdig agent.
You can use whatever name you want. In this document, we used sysdig-agent for both the namespace and the service account.
```
kubectl create ns sysdig-agent
```

Create a secret key:

kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent

If you are running Kubernetes 1.6 or higher, you must
Grant your user the ability to create roles in Kubernetes by running the following command (see Google documentation for more):
```
kubectl create clusterrolebinding your-user-cluster-admin-binding --clusterrole=cluster-admin --user=your.google.cloud.email@example.org
```
Create a service account for the Sysdig agent using the clusterrole.yaml file.
The Sysdig agent must be granted read-only access to certain Kubernetes APIs, which the agent uses to populate metadata and provide component metrics.
Sysdig provides a config file in GitHub. Deploying this file creates a cluster role and service account in Kubernetes, and defines cluster role binding that grants the Sysdig agent rules in the cluster role.
Run the following commands (using whatever namespace you defined in Step 2):
```
kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
```
Edit sysdig-agent-configmap.yaml to add the collector address, port, and the SSL/TLS information :
```
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
```
- For SaaS, find the collector address for your region.
- For On-prem, enter the collector endpoint defined in your environment.
- check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Step 5 Set Up SSL Connectivity to the Backend.
(All installs) Apply the sysdig-agent-configmap.yaml file using the command:
```
kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
```
FOR GKE COS ONLY: To enable the eBPF probe required for COS, uncomment the following parameters in sysdig-agent-daemonset-v2.yaml under the env section:
```
env:
  - name: SYSDIG_BPF_PROBE
    value: ""
```
Apply the sysdig-agent-service.yaml file:
```
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
```
This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

(All installs) Apply the daemonset-v2.yaml file using the command:

kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent

The agents will be deployed and you can see Getting Started with Sysdig Monitor to view some metrics in the Sysdig Monitor UI. You can make further edits to the configmap as described below.Getting Started with Sysdig Monitor

Enable Kube State Metrics and Cluster Name

These steps are optional but recommended.

Edit sysdig-agent-configmap.yaml to uncomment the line: new_k8s: true
This allows kube state metrics to be automatically detected, monitored, and displayed in Sysdig Monitor.
For more information, see the Kube State Metrics entry in the Sysdig blog.
As of agent 9.6.0, new_k8s is enabled by default.
Edit sysdig-agent-configmap.yaml to uncomment the line: **k8s_cluster_name: **and add your cluster name.
Setting cluster name here allows you to view, scope, and segment metrics in the Sysdig Monitor UI by the Kubernetes cluster.
Note: Alternatively, if you assign a tag with “cluster” in the tag name, Sysdig Monitor will display that as the Kubernetes cluster name.

Apply the configmap changes using the command:

kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Proceed to verify the metrics in the Sysdig Monitor UI.

There are two ways to update the agent configuration

Option 1: Edit the files locally and apply the changes with kubectl apply -f:

kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Option 2: Use kubectl edit to edit files on the fly:

kubectl edit configmap sysdig-agent -n sysdig-agent

Running agents will automatically pick the new configuration after Kubernetes pushes the changes across all the nodes in the cluster.

Verify Metrics in Sysdig Monitor UI

The steps below give one way to do the check.

Access Sysdig Monitor:
SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for US East, the URL is https://app.sysdigcloud.com.
For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.
Log in with your Sysdig user name and password.
Select the Explore tab to see if metrics are displayed.
(Once you have enabled new_k8s:true): To verify that kube state metrics and cluster name are working correctly: Select the Explore tab and create a grouping by kubernetes.cluster.name and kubernetes.pod.name.
As of agent 9.6.0, new_k8s is enabled by default.
Select an individual container or pod to see details.

Kubernetes metadata (pods, deployments etc.) appear a minute or two later than the nodes/containers themselves; if pod names do not appear immediately, wait and retry the Explore view.

If agents are disconnecting, there could be an issue with your MAC addresses. See Troubleshooting Agent Installation for tips.

Additional Options

Connect to the Sysdig Backend via Static IPs (SaaS only)

The sysdig-agent-configmap.yaml file can be edited either locally or using the edit command in Kubernetes. refer to the section above for more information.

To configure the collector IP in a Kubernetes SaaS instance:

Open sysdig-agent-configmap.yaml in a text editor.
Uncomment the following lines:
- collector:
- collector_port
Set the collector: value to collector-static.sysdigcloud.com
Set the collector_port: value to 6443
Save the file.

The example file below shows how the sysdig-agent-configmap.yaml file should look after configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-agent
data:
  dragent.yaml: |
    ### Agent tags
    # tags: linux:ubuntu,dept:dev,local:nyc

    #### Sysdig Software related config ####

    # Sysdig collector address
    collector: collector-static.sysdigcloud.com

    # Collector TCP port
    collector_port: 6443

    # Whether collector accepts ssl/TLS
    ssl: true

    # collector certificate validation
    ssl_verify_certificate: true

    # Sysdig Secure
    security:
      enabled: true

    #######################################
    # new_k8s: true
    # k8s_cluster_name: production

1.1.1.3.3 -

Steps for OKE

Oracle Kubernetes Engine (OKE) is a managed environment for running Kubernetes in Oracle Cloud, in order to deploy containerized applications. As of Sysdig agent version 12.0.1, Sysdig supports all flavors of OKE.

OKE environments require eBPF probe to support agent installation.

The instructions below describe a standard OKE agent install and call out the special steps needed to install the eBPF probe.

Preparation

Open Port 6443 for Agent Egress

Because OKE uses stateful firewalls, you must actively open port 6443 for the Sysdig agent outbound traffic.

OKE by default allows network access to the sysdig Agent on 6443, but ensure that firewall rules are open and the agent can connect to the Sysdig backends.

eBPF-Specific Requirements

Linux kernel version >= 4.14.
When performing the installation steps, you will add one additional parameter to install the eBPF probe. See Step 7, below.

Installation Steps

Identify the appropriate endpoint depending on your Sysdig account region. For more information, see SaaS Regions and IP Ranges. More info here https://docs.sysdig.com/en/docs/administration/saas-regions-and-ip-ranges/

After making clear which region your account belongs to, please choose one of the following methods:

Deploy Using Helm Charts

To deploy agent using Helm charts (https://helm.sh/), run the following:

Export the access token and the name of the OKE cluster:

export SDC_ACCESS_TOKEN=xxxx # Get it from the UI (User > Settings > Sysdig Secure API Token).
export SDC_COLLECTOR_URL=collector-static.sysdigcloud.com # us-west by default. Please check the right region.
export SDC_NODEANALYZER_URL=secure.sysdig.com # us-west by default. Please check the right region. 
export OKE_CLUSTER_NAME=my-cluster # OKE cluster name

Create a namespace to use for the Sysdig agent:
```
 kubectl create ns sysdig-agent
```

Set up the helm repo:

 helm repo add sysdig https://charts.sysdig.com
 helm repo update

Install the agent:

 helm install sysdig-agent --namespace sysdig-agent --set sysdig.accessKey=$SDC_ACCESS_TOKEN --set sysdig.settings.collector=$SDC_COLLECTOR_URL --set sysdig.settings.collector_port=6443 --set clusterName=$OKE_CLUSTER_NAME sysdig/sysdig --set nodeAnalyzer.apiEndpoint=$SDC_NODEANALYZER_URL --set ebpf.enabled=true

For more information,charts.

Deploy Using Daemonsets

Download the sample files:
- sysdig-agent-clusterrole.yaml
- sysdig-agent-daemonset-v2.yaml
- sysdig-agent-configmap.yaml
- sysdig-agent-service.yaml
Create a namespace to use for the Sysdig agent.
You can use whatever name you want. In this document, we used sysdig-agent for both the namespace and the service account.
```
kubectl create ns sysdig-agent
```

Create a secret key:

kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent

If you are running Kubernetes 1.6 or higher, you must Create a service account for the Sysdig agent by using the clusterrole.yaml file.
The Sysdig agent must be granted read-only access to certain Kubernetes APIs, which the agent uses to populate metadata and provide component metrics.
Sysdig provides a config file in GitHub. Deploying this file creates a cluster role and service account in Kubernetes, and defines cluster role binding that grants the Sysdig agent rules in the cluster role.
Run the following commands by using the namespace you defined in Step 2:
```
kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent
```
Edit sysdig-agent-configmap.yaml to add the collector address, port, and the SSL/TLS information :
```
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
```
- For SaaS, find the collector address for your region.
- For On-prem, enter the collector endpoint defined in your environment.
- check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Step 5 Set Up SSL Connectivity to the Backend.
(All installs) Apply the sysdig-agent-configmap.yaml file using the command:
```
kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent
```
To enable the eBPF probe uncomment the following parameters in sysdig-agent-daemonset-v2.yaml under the env section:
```
env:
  - name: SYSDIG_BPF_PROBE
    value: ""
```
Apply the sysdig-agent-service.yaml file:
```
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
```
This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

(All installs) Apply the daemonset-v2.yaml file using the command:

kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent

Verify Metrics in Sysdig Monitor UI

The steps below give one way to do the check.

Access Sysdig Monitor:
SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for US East, the URL is https://app.sysdigcloud.com.
For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.
Log in with your Sysdig user name and password.
Select the Explore tab to see if metrics are displayed.
(Once you have enabled new_k8s:true): To verify that kube state metrics and cluster name are working correctly: Select the Explore tab and create a grouping by kubernetes.cluster.name and kubernetes.pod.name.
As of agent 9.6.0, new_k8s is enabled by default.
Select an individual container or pod to see details.

Kubernetes metadata (pods, deployments etc.) appear a minute or two later than the nodes/containers themselves; if pod names do not appear immediately, wait and retry the Explore view.

If agents are disconnecting, there could be an issue with your MAC addresses. See Troubleshooting Agent Installation for tips.

Additional Options

Connect to the Sysdig Backend via Static IPs (SaaS only)

The sysdig-agent-configmap.yaml file can be edited either locally or using the edit command in Kubernetes. refer to the section above for more information.

To configure the collector IP in a Kubernetes SaaS instance:

Open sysdig-agent-configmap.yaml in a text editor.
Uncomment the following lines:
- collector:
- collector_port
Set the collector: value to collector-static.sysdigcloud.com
Set the collector_port: value to 6443
Save the file.

The example file below shows how the sysdig-agent-configmap.yaml file should look after configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-agent
data:
  dragent.yaml: |
    ### Agent tags
    # tags: linux:ubuntu,dept:dev,local:nyc

    #### Sysdig Software related config ####

    # Sysdig collector address
    collector: collector-static.sysdigcloud.com

    # Collector TCP port
    collector_port: 6443

    # Whether collector accepts ssl/TLS
    ssl: true

    # collector certificate validation
    ssl_verify_certificate: true

    # Sysdig Secure
    security:
      enabled: true

    #######################################
    # new_k8s: true
    # k8s_cluster_name: production

1.1.1.3.4 -

Steps for OpenShift

Review the Prerequisites in Agent Install: Kubernetes | GKE | OpenShift | IBM and the Host Requirements for Agent Installation, then proceed with the installation.

Kernel Headers

The Sysdig agent requires kernel header files to install successfully on a host.

This setup step is required for some environments and not others, as noted.

If the hosts in your environment match the pre-compiled kernel modules available from Sysdig, no special action is required.

To do so:

For Debian-style distributions, run the command:

apt-get -y install linux-headers-$(uname -r)

For RHEL-style distributions, run the command:

yum -y install kernel-devel-$(uname -r)

Background info: see also About Kernel Headers and the Kernel Module.

Configure for OpenShift

If you are using Red Hat OpenShift, these steps are required. They describe how to create a project, assign and label the node selector, create a privileged service account, and add it to a cluster role.

Copy/Paste Sample Code Block

In the example code, this document uses sysdig-agent for the PROJECT NAME (-n), the SERVICE ACCOUNT (-z), and the NODE SELECTOR.

You can copy-paste the code as-is, or follow the steps below to customize your naming conventions.

oc adm new-project sysdig-agent --node-selector=''
oc project sysdig-agent
oc create serviceaccount sysdig-agent
oc adm policy add-scc-to-user privileged -n sysdig-agent -z sysdig-agent
oc adm policy add-cluster-role-to-user cluster-reader -n sysdig-agent -z sysdig-agent

Customize the Code

You can use your own Project name and Service Account name if desired.

Note that if you use a different Service Account name, you will need to edit the default service account in the Sysdig Installation Steps, below.

Create a new OpenShift project for the Sysdig agent deployment and use an empty string for the node selector:
```
oc adm new-project PROJECT-NAME --node-selector=""
```
Change to the new OpenShift Project for the Sysdig agent deployment:
```
oc project PROJECT-NAME
```

Create a service account for the project:

oc create serviceaccount SERVICE-ACCOUNT

Add the service account to privileged Security Context Constraints:

oc adm policy add-scc-to-user privileged -n PROJECT-NAME -z SERVICE-ACCOUNT

Add the service account to the cluster-reader Cluster Role:

oc adm policy add-cluster-role-to-user cluster-reader -n PROJECT-NAME -z SERVICE-ACCOUNT

Sysdig Installation Steps

To deploy agents using Kubernetes daemonsets, you download the configuration files, edit them as required, and deploy them.

sysdig-agent-daemonset-v2.yaml
sysdig-agent-clusterrole.yaml
sysdig-agent-configmap.yaml
sysdig-agent-service.yaml

HELM CHART OPTION Kubernetes also offers a package manager, Helm, which uses charts to simplify this process.

If you are using Helm charts in your K8s environment, we recommend using them to deploy Sysdig agents, as described here.

Deploy the Agents

Download the sample files:
- sysdig-agent-daemonset-v2.yaml
- sysdig-agent-clusterrole.yaml
- sysdig-agent-configmap.yaml
- sysdig-agent-service.yaml

Create the sysdig-agent cluster role and assign it to the service account:

 oc apply -f sysdig-agent-clusterrole.yaml
 oc adm policy add-cluster-role-to-user sysdig-agent -n PROJECT-NAME -z SERVICE-ACCOUNT

Create a secret key using the command:

oc create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent

If you created a service account name other than sysdig-agent: Edit sysdig-agent-daemonset-v2.yamlto provide your custom value:``
```
serviceAccount: sysdig-agent
```
Edit sysdig-agent-configmap.yaml to add the collector address, port, and the SSL/TLS information:
```
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
```
- For SaaS, find the collector address for your region.
- For On-prem, enter the collector endpoint defined in your environment.
- check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Step 5 Set Up SSL Connectivity to the Backend.
(All installs) Apply the sysdig-agent-configmap.yaml file using the command:
```
oc apply -f sysdig-agent-configmap.yaml -n sysdig-agent
```
Apply the sysdig-agent-service.yaml file:
```
oc apply -f sysdig-agent-service.yaml -n sysdig-agent
```
This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

(All installs) Apply the daemonset-v2.yaml file:

oc apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent

Enable Kube State Metrics and Cluster Name

These steps are optional but recommended.

Edit sysdig-agent-configmap.yaml to uncomment the line: new_k8s: true
This allows kube state metrics to be automatically detected, monitored, and displayed in Sysdig Monitor.
For more information, see the Kube State Metrics entry in the Sysdig blog.
As of agent 9.6.0, new_k8s is enabled by default.
Edit sysdig-agent-configmap.yaml to uncomment the line: **k8s_cluster_name: **and add your cluster name.
Setting cluster name here allows you to view, scope, and segment metrics in the Sysdig Monitor UI by the Kubernetes cluster.
Note: Alternatively, if you assign a tag with “cluster” in the tag name, Sysdig Monitor will display that as the Kubernetes cluster name.

Apply the configmap changes using the command:

oc apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Proceed to verify the metrics in the Sysdig Monitor UI.

There are two ways to update the agent configuration

Option 1: Edit the files locally and apply the changes with oc apply -f:

oc apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Option 2: Use oc edit to edit files on the fly:

oc edit configmap sysdig-agent -n sysdig-agent

Running agents will automatically pick the new configuration after Kubernetes pushes the changes across all the nodes in the cluster.

Verify Metrics in Sysdig Monitor UI

The steps below give one way to do the check.

Access Sysdig Monitor:
SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for US East, the URL is https://app.sysdigcloud.com.
For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.
Log in with your Sysdig user name and password.
Select the Explore tab to see if metrics are displayed.
(Once you have enabled new_k8s:true): To verify that kube state metrics and cluster name are working correctly: Select the Explore tab and create a grouping by kubernetes.cluster.name and kubernetes.pod.name.
As of agent 9.6.0, new_k8s is enabled by default.
Select an individual container or pod to see details.

Kubernetes metadata (pods, deployments etc.) appear a minute or two later than the nodes/containers themselves; if pod names do not appear immediately, wait and retry the Explore view.

If agents are disconnecting, there could be an issue with your MAC addresses. See Troubleshooting Agent Installation for tips.

Additional Options

Connect to the Sysdig Backend via Static IPs (SaaS only)

The sysdig-agent-configmap.yaml file can be edited either locally or using the edit command in Kubernetes. refer to the section above for more information.

To configure the collector IP in a Kubernetes SaaS instance:

Open sysdig-agent-configmap.yaml in a text editor.
Uncomment the following lines:
- collector:
- collector_port
Set the collector: value to collector-static.sysdigcloud.com
Set the collector_port: value to 6443
Save the file.

The example file below shows how the sysdig-agent-configmap.yaml file should look after configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-agent
data:
  dragent.yaml: |
    ### Agent tags
    # tags: linux:ubuntu,dept:dev,local:nyc

    #### Sysdig Software related config ####

    # Sysdig collector address
    collector: collector-static.sysdigcloud.com

    # Collector TCP port
    collector_port: 6443

    # Whether collector accepts ssl/TLS
    ssl: true

    # collector certificate validation
    ssl_verify_certificate: true

    # Sysdig Secure
    security:
      enabled: true

    #######################################
    # new_k8s: true
    # k8s_cluster_name: production

1.1.1.3.5 -

Steps for Rancher

Preparation

General Requirements

You can review Agent Install: Kubernetes | GKE | OpenShift | IBM and the Host Requirements for Agent Installation for additional context if desired.

Kernel Headers

The Sysdig agent requires a kernel module in order to be installed successfully on a host. On RancherOS distributions, the Unix version does not match the provided headers, and the agent might fail to install correctly. Therefore, you must install the kernel headers manually.

For RancherOS distributions, the kernel headers are available in the form of a system service and therefore are enabled using the ros service command:

$ sudo ros service enable kernel-headers-system-docker
$ sudo ros service up -d kernel-headers-system-docker

Some cloud hosting service providers supply pre-configured Linux instances with customized kernels. You may need to contact your provider’s support desk for instructions on obtaining appropriate header files, or for installing the distribution’s default kernel.

Install Agent

To deploy agents using Kubernetes daemonsets, download the following configuration files, edit them as required, and deploy them.

sysdig-agent-clusterrole.yaml
sysdig-agent-service.yaml
sysdig-agent-daemonset-v2.yaml
sysdig-agent-configmap.yaml

HELM CHART OPTION Kubernetes also offers a package manager, Helm, which uses charts to simplify this process.

If you are using Helm charts in your K8s environment, we recommend using them to deploy Sysdig agents, as described here.

Deploy Agent

Download the sample files:
- sysdig-agent-clusterrole.yaml
- sysdig-agent-daemonset-v2.yaml
- sysdig-agent-configmap.yaml
- sysdig-agent-service.yaml
Create a namespace to use for the Sysdig agent.
You can use whatever naming you prefer. In this document, we used sysdig-agent for both the namespace and the service account.
The default service account name was automatically defined in sysdig-agent-daemonset-v2.yaml, at the line: serviceAccount: sysdig-agent.
```
kubectl create ns sysdig-agent
```

Create a secret key:

kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent

Create a cluster role and service account, and define the cluster role bindingthat grants the Sysdig agent rules in the cluster role, using the commands:

kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
kubectl create serviceaccount sysdig-agent -n sysdig-agent
kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent

Edit sysdig-agent-configmap.yaml to add the collector address , port , and the SSL/TLS information:
```
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
```
- For SaaS, find the collector address for your region.
- For On-prem, enter the collector endpoint defined in your environment.
- check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Step 5 Set Up SSL Connectivity to the Backend.

(All installs) Apply the sysdig-agent-configmap.yaml file:

kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

(All installs) Apply the sysdig-agent-service.yaml file:
```
kubectl apply -f sysdig-agent-service.yaml -n sysdig-agent
```
This allows the agent to receive Kubernetes audit events from the Kubernetes API server. See Kubernetes Audit Logging for information on enabling Kubernetes audit logging.

(All installs) Apply the daemonset-v2.yaml file :

kubectl apply -f sysdig-agent-daemonset-v2.yaml -n sysdig-agent

Enable Kube State Metrics and Cluster Name

These steps are optional but recommended.

Edit sysdig-agent-configmap.yaml to uncomment the line: new_k8s: true
This allows kube state metrics to be automatically detected, monitored, and displayed in Sysdig Monitor.
For more information, see the Kube State Metrics entry in the Sysdig blog.
As of agent 9.6.0, new_k8s is enabled by default.
Edit sysdig-agent-configmap.yaml to uncomment the line: **k8s_cluster_name: **and add your cluster name.
Setting cluster name here allows you to view, scope, and segment metrics in the Sysdig Monitor UI by the Kubernetes cluster.
Note: Alternatively, if you assign a tag with “cluster” in the tag name, Sysdig Monitor will display that as the Kubernetes cluster name.

Apply the configmap changes using the command:

kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Proceed to verify the metrics in the Sysdig Monitor UI.

Connect to the Sysdig Backend via Static IPs (SaaS only)

The sysdig-agent-configmap.yaml file can be edited either locally or using the edit command in Kubernetes. refer to the section above for more information.

To configure the collector IP in a Kubernetes SaaS instance:

Open sysdig-agent-configmap.yaml in a text editor.
Uncomment the following lines:
- collector:
- collector_port
Set the collector: value to collector-static.sysdigcloud.com
See SaaS Regions and IP Ranges and identify the correct URL associated with your Sysdig collector and region.
Set the collector_port: value to 6443
Save the file.

The example file below shows how the sysdig-agent-configmap.yaml file should look after configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-agent
data:
  dragent.yaml: |
    ### Agent tags
    # tags: linux:ubuntu,dept:dev,local:nyc

    #### Sysdig Software related config ####

    # Sysdig collector address
    collector: collector-static.sysdigcloud.com

    # Collector TCP port
    collector_port: 6443

    # Whether collector accepts ssl/TLS
    ssl: true

    # collector certificate validation
    ssl_verify_certificate: true

    # Sysdig Secure
    security:
                enabled: true

    #######################################
    # new_k8s: true
    # k8s_cluster_name: production

Verify Metrics in Sysdig Monitor UI

Kubernetes metadata (pods, deployments etc.) appear a minute or two later than the nodes/containers themselves; if pod names do not appear immediately, wait and retry the Explore view.

If agents are disconnecting, there could be an issue with your MAC addresses. See Troubleshooting Agent Installation for tips.

Access Sysdig Monitor:
SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for US East, the URL is https://app.sysdigcloud.com.
For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.
Log in with your Sysdig user name and password.
Select the Explore tab to see if metrics are displayed.
(Once you have enabled new_k8s:true): To verify that kube state metrics and cluster name are working correctly: Select the Explore tab and create a grouping by kubernetes.cluster.name and kubernetes.pod.name.
As of agent 9.6.0, new_k8s is enabled by default.
Select an individual container or pod to see details.

1.1.1.4 -

Agent Install: Non-Orchestrated

This section describes how to install the Sysdig agent directly on a Linux host, without using an orchestrator, such as Kubernetes or Mesos.

The agent can be installed in two ways:

As a standard container
If you want to install the lighter version of the Sysdig agent, see Install Slim Agent.
As a non-containerized service

The steps for each flavor differ slightly depending on whether you are using the SaaS or on-premises version of the Sysdig platform.

If you are installing the Sysdig agent in an environment that has Kubernetes, use the Agent Install: Kubernetes instructions instead.

Prerequisites

See Host Requirements for Agent Installation. There you can check the requirements concerning:
- Supported Linux distributions
- Network connection
- Sysdig access key
- Cloud service providers (AWS, Google, and Microsoft Azure) and any steps you may need to configure to integrate the Sysdig agent.

On kernel headers: The Sysdig agent requires kernel header files in order to install successfully on a host, and the agent is delivered with precompiled headers. If the hosts in your environment match the kernel versions included with the agent, no special action is needed.

Run any commands as root or with the sudo command.
Have your Sysdig access key on hand.
If you launch an agent install from www.sysdig.com, the welcome wizard will present an access key.

Docker Container Agent Installation

The Sysdig agent can be deployed as a Docker container.

The commands below can also be copy/pasted from the Welcome Wizard or the Agent Installation page in the Sysdig UI.

In that case, your access key will already be included in the command automatically.

SaaS

Run the agent image, providing the access key and (optional) user-defined tags:

docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host\
 -e ACCESS_KEY=[ACCESS_KEY] \
 -e COLLECTOR=[COLLECTOR_ADDRESS] \
-e TAGS=[TAGS] \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro --shm-size=512m sysdig/agent

For the COLLECTOR, find the address for your region.

On-Premises

Provide collector and SSL/TLS information in addition to access key and optional tags:

docker run -d --name sysdig-agent \
--restart always \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
[-e TAGS=[TAGS]]
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro --shm-size=512m sysdig/agent

CHECK_CERTIFICATE should be set to false if a self-signed certificate or private, CA-signed cert is used.

Service Agent Installation on Linux Host

Use these instructions to install the agent on the host itself, not in a container. Install on each host in the environment.

The command lines below can also be copy/pasted from the Welcome wizard or the Settings>Agent Installation page in the Sysdig Monitor interface.

In that case, your access key will already be included in the command automatically.

The Sysdig agent depends on several python modules, some of which might not be installed on the hosts where the agent is running as a service. When the required dependencies are not available, the sdchecks component in the agent will report errors in the log files, such as:

 >> Error, sdchecks[0] ModuleNotFoundError: No module named 'posix_ipc'

To address these errors, install the missing modules using the pip install command.

SaaS

Run the following command:
```
curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- --access_key [ACCESS_KEY] --collector [COLLECTOR_ADDRESS] [--tags [TAGS]]
```
Where [ACCESS_KEY] is your unique agent access key string. For example, 1234-your-key-here-1234. [TAGS] is an optional list of user-defined agent tags. For example, role:webserver,location:europe.
Find the collector endpoint for your region listed here.
Make sure restarting the agent results in starting the service:
```
sudo systemctl enable dragent
```

On-Premises

Run the following command:

curl -s https://download.sysdig.com/stable/install-agent | sudo bash -s -- --access_key [ACCESS_KEY] --collector [COLLECTOR_ADDRESS] --secure true --check_certificate true [--tags [TAGS]]

check_certificate should be set to false if a self-signed certificate, a private, or a CA-signed certificate is used. See information about SSL in on-premises here.

Make sure restarting the agent results in starting the service:
```
sudo systemctl enable dragent
```

Connect to the Sysdig Backend via Static IPs (SaaS only)

user@host:~$ docker run --name sysdig-agent \
--privileged \
--net host \
--pid host \
-e ACCESS_KEY=[ACCESS_KEY] \
-e TAGS=[TAGS] \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
-v /lib/modules:/host/lib/modules:ro \
-v /usr:/host/usr:ro \
-e COLLECTOR=collector-static.sysdigcloud.com \
-e COLLECTOR_PORT=6443 \
-e SECURE=true \
-e CHECK_CERTIFICATE=true \
--shm-size=512m \
sysdig/agent

Note on Manual Agent Installation

In the following cases, it may be preferable to perform a manual installation.

Full control over the deployment process
Integration with configuration management tools
Custom kernel
Unsupported distribution

If desired, see Agent Install: Manual Linux Installation.

1.1.1.5 -

Install Slim Agent

The slim agent is a lighter version of the Sysdig agent that is created by splitting the regular agent image into two components responsible for different functions. The slim agent reduces the surface area of attack for potential vulnerabilities and is, therefore, more secure.

You install the slim agent package as two separate containers:

agent-kmodule: Responsible for downloading and building the kernel module. The image is short-lived. The container exits after the kernel module is loaded. The transient nature of the container reduces the time and opportunities for exploiting any potential vulnerabilities present in the container image.
Prerequisites: The package depends on Dynamic Kernel Module Support (DKMS) and requires the compiler and kernel headers installed if you are using the agent-kmodule to build the kernel probe. Alternatively, you can use it without the kernel headers. In such cases, the agent-kmodule will attempt to download a pre-built kernel probe if it is present in the Sysdig probe repository.
The module contains:
- The driver sources
- A post-install script that builds the module upon installation
agent-slim: Responsible for running the agent module once the kernel module has been loaded. When the slim agent is up and running it functions the same way as the regular agent.

Install Slim Agent in a Non-Orchestrated Environment

The agent is installed by running sysdig/agent-kmodule first, followed by running sysdig/agent-slim.

Every host restart requires subsequent running of agent-kmodule and agent-slim containers.

Build and load the kernel module:

docker run -it --privileged --rm --name sysdig-agent-kmodule -v /usr:/host/usr:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro sysdig/agent-kmodule

Run the agent module:

docker run -d --name sysdig-agent \
--privileged \
--net host \
--pid host\
-e ACCESS_KEY=[ACCESS_KEY] \
-e COLLECTOR=[COLLECTOR_ADDRESS] \
-v /var/run/docker.sock:/host/var/run/docker.sock \
-v /dev:/host/dev \
-v /proc:/host/proc:ro \
-v /boot:/host/boot:ro \
sysdig/agent-slim

Install Slim Agent on Kubernetes

The agent is installed by scheduling both the agent-kmodule and agent-slim containers into a single daemonset. The agent-kmodule container is defined as an init container, which ensures that it runs first and must succeed in order for the other containers to run.

Download sysdig-agent-slim-daemonset-v2.yaml, edit it as required, and deploy.

An example daemonset is given below:

### WARNING: this file is supported from Sysdig Agent 0.80.0
# apiVersion: extensions/v1beta1  # If you are in Kubernetes version 1.8 or less please use this line instead of the following one
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: sysdig-agent
  labels:
    app: sysdig-agent
spec:
  selector:
    matchLabels:
      app: sysdig-agent
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: sysdig-agent
    spec:
      volumes:
      - name: modprobe-d
        hostPath:
          path: /etc/modprobe.d
      - name: dshm
        emptyDir:
          medium: Memory
      - name: dev-vol
        hostPath:
          path: /dev
      - name: proc-vol
        hostPath:
          path: /proc
      - name: boot-vol
        hostPath:
          path: /boot
      - name: modules-vol
        hostPath:
          path: /lib/modules
      - name: usr-vol
        hostPath:
          path: /usr
      - name: run-vol
        hostPath:
          path: /run
      - name: varrun-vol
        hostPath:
          path: /var/run
      - name: sysdig-agent-config
        configMap:
          name: sysdig-agent
          optional: true
      - name: sysdig-agent-secrets
        secret:
          secretName: sysdig-agent
      hostNetwork: true
      hostPID: true
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
      # The following line is necessary for RBAC
      serviceAccount: sysdig-agent
      terminationGracePeriodSeconds: 5
      initContainers:
      - name: sysdig-agent-kmodule
        image: sysdig/agent-kmodule
        imagePullPolicy: Always
        securityContext:
          privileged: true
        resources:
          requests:
            cpu: 1000m
            memory: 384Mi
          limits:
            memory: 512Mi
        volumeMounts:
        - mountPath: /etc/modprobe.d
          name: modprobe-d
          readOnly: true
        - mountPath: /host/boot
          name: boot-vol
          readOnly: true
        - mountPath: /host/lib/modules
          name: modules-vol
          readOnly: true
        - mountPath: /host/usr
          name: usr-vol
          readOnly: true
      containers:
      - name: sysdig-agent
        # WARNING: the agent-slim release is currently dependent on the above
        # initContainer and thus only functions correctly in a kubernetes cluster
        image: sysdig/agent-slim
        imagePullPolicy: Always
        securityContext:
          privileged: true
        resources:
          # Resources needed are subjective to the actual workload.
          # Please refer to Sysdig Support for more info.
          requests:
            cpu: 600m
            memory: 512Mi
          limits:
            cpu: 2000m
            memory: 1536Mi
        readinessProbe:
          exec:
            command: [ "test", "-e", "/opt/draios/logs/running" ]
          initialDelaySeconds: 10
        volumeMounts:
        - mountPath: /host/dev
          name: dev-vol
          readOnly: false
        - mountPath: /host/proc
          name: proc-vol
          readOnly: true
        - mountPath: /host/run
          name: run-vol
        - mountPath: /host/var/run
          name: varrun-vol
        - mountPath: /dev/shm
          name: dshm
        - mountPath: /opt/draios/etc/kubernetes/config
          name: sysdig-agent-config
        - mountPath: /opt/draios/etc/kubernetes/secrets
          name: sysdig-agent-secrets

See Sysdig Cloud Scripts for the latest daemonset.

Create a namespace to use for the Sysdig agent.
```
# kubectl create ns sysdig-agent
```
You can use whatever naming you prefer. In this document, we used sysdig-agent for both the namespace and the service account. The default service account name was automatically defined in sysdig-agent-slim-daemonset-v2.yaml, at the line: serviceAccount: sysdig-agent.

Create a secret key:

# kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent

Create a cluster role and service account, and define the cluster role binding that grants the Sysdig agent rules in the cluster role, using the commands:

# kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
# kubectl create serviceaccount sysdig-agent -n sysdig-agent
# kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent

Edit sysdig-agent-configmap.yaml to add the collector``address``` and portand theSSL/TLS` information :
```
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
```
- For SaaS, find the collector address for your region.
- For On-prem, enter the collector endpoint defined in your environment.
- check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Step 5 Set Up SSL Connectivity to the Backend.

Apply the configuration changes:

# kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Deploy the kernel module and slim agent containers using the daemonset:

# kubectl apply -f sysdig-agent-slim-daemonset-v2.yaml -n sysdig-agent

Install Slim Agent on GKE

Download sysdig-agent-slim-daemonset-v2.yaml.

An example daemonset is given below:

### WARNING: this file is supported from Sysdig Agent 0.80.0
# apiVersion: extensions/v1beta1  # If you are in Kubernetes version 1.8 or less please use this line instead of the following one
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: sysdig-agent
  labels:
    app: sysdig-agent
spec:
  selector:
    matchLabels:
      app: sysdig-agent
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: sysdig-agent
    spec:
      volumes:
      - name: modprobe-d
        hostPath:
          path: /etc/modprobe.d
### uncomment for minikube
#      - name: etc-version
#        hostPath:
#          path: /etc/VERSION
#          type: FileOrCreate
      - name: dshm
        emptyDir:
          medium: Memory
      - name: dev-vol
        hostPath:
          path: /dev
      - name: proc-vol
        hostPath:
          path: /proc
      - name: boot-vol
        hostPath:
          path: /boot
      - name: modules-vol
        hostPath:
          path: /lib/modules
      - name: usr-vol
        hostPath:
          path: /usr
      - name: run-vol
        hostPath:
          path: /run
      - name: varrun-vol
        hostPath:
          path: /var/run
      - name: sysdig-agent-config
        configMap:
          name: sysdig-agent
          optional: true
      - name: sysdig-agent-secrets
        secret:
          secretName: sysdig-agent
      # This section is for eBPF support. Please refer to Sysdig Support before
      # uncommenting, as eBPF is recommended for only a few configurations.
      - name: bpf-probes
        emptyDir: {}
      - name: osrel
        hostPath:
          path: /etc/os-release
          type: FileOrCreate
      hostNetwork: true
      hostPID: true
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
      # The following line is necessary for RBAC
      serviceAccount: sysdig-agent
      terminationGracePeriodSeconds: 5
      initContainers:
      - name: sysdig-agent-kmodule
        image: quay.io/sysdig/agent-kmodule
        imagePullPolicy: Always
        securityContext:
          privileged: true
        resources:
          requests:
            cpu: 1000m
            memory: 384Mi
          limits:
            memory: 512Mi
        # This section is for eBPF support. Please refer to Sysdig Support before
        # uncommenting, as eBPF is recommended for only a few configurations.
        env:
          - name: SYSDIG_BPF_PROBE
            value: ""
        volumeMounts:
        - mountPath: /etc/modprobe.d
          name: modprobe-d
          readOnly: true
### uncomment for minikube
#        - mountPath: /host/etc/VERSION
#          name: etc-version
#          readOnly: true
        - mountPath: /host/boot
          name: boot-vol
          readOnly: true
        - mountPath: /host/lib/modules
          name: modules-vol
          readOnly: true
        - mountPath: /host/usr
          name: usr-vol
          readOnly: true
        # This section is for eBPF support. Please refer to Sysdig Support before
        # uncommenting, as eBPF is recommended for only a few configurations.
        - mountPath: /root/.sysdig
          name: bpf-probes
        - mountPath: /host/etc/os-release
          name: osrel
          readOnly: true
      containers:
      - name: sysdig-agent
        # WARNING: the agent-slim release is currently dependent on the above
        # initContainer and thus only functions correctly in a kubernetes cluster
        image: quay.io/sysdig/agent-slim
        imagePullPolicy: Always
        securityContext:
          privileged: true
        resources:
          # Resources needed are subjective to the actual workload.
          # Please refer to Sysdig Support for more info.
          requests:
            cpu: 600m
            memory: 512Mi
          limits:
            cpu: 2000m
            memory: 1536Mi
        readinessProbe:
          exec:
            command: [ "test", "-e", "/opt/draios/logs/running" ]
          initialDelaySeconds: 10
        # This section is for eBPF support. Please refer to Sysdig Support before
        # uncommenting, as eBPF is recommended for only a few configurations.
        env:
         - name: SYSDIG_BPF_PROBE
            value: ""
        volumeMounts:
        - mountPath: /host/dev
          name: dev-vol
          readOnly: false
        - mountPath: /host/proc
          name: proc-vol
          readOnly: true
        - mountPath: /host/run
          name: run-vol
        - mountPath: /host/var/run
          name: varrun-vol
        - mountPath: /dev/shm
          name: dshm
        - mountPath: /opt/draios/etc/kubernetes/config
          name: sysdig-agent-config
        - mountPath: /opt/draios/etc/kubernetes/secrets
          name: sysdig-agent-secrets
        # This section is for eBPF support. Please refer to Sysdig Support before
        # uncommenting, as eBPF is recommended for only a few configurations.
        - mountPath: /root/.sysdig
          name: bpf-probes

Either use the single-line command from the Getting Started section of the Sysdig application or continue with the step 3 through 7.

$ curl -s https://download.sysdig.com/stable/install-agent-kubernetes | sudo bash -s -- --access_key 84d1d241-cde3-4ecc-9ecf-9a735ed0df45 --collector collector-staging.sysdigcloud.com --collector_port 6443 --nodeanalyzer --api_endpoint secure-staging.sysdig.com

Ensure that you uncomment the following sections:

eBPF Probes under spec volume:

- name: bpf-probes
  emptyDir: {}
- name: osrel
  hostPath:
     path: /etc/os-release
     type: FileOrCreate

Environment variable for eBPF under initContainers:
```
env:
  - name: SYSDIG_BPF_PROBE
    value: ""
```

Mount path for eBPF under initContainers:

- mountPath: /root/.sysdig
  name: bpf-probes
- mountPath: /host/etc/os-release
  name: osrel
  readOnly: true

Environment variable for eBPF under sysdig-agent:
```
env:
 - name: SYSDIG_BPF_PROBE
   value: ""
```

Mount path for eBPF under volume mounts

- mountPath: /root/.sysdig
  name: bpf-probesenv:

Create a namespace to use for the Sysdig agent.
```
$ kubectl create ns sysdig-agent
```
You can use whatever naming you prefer. In this document, we used sysdig-agent for both the namespace and the service account. The default service account name was automatically defined in sysdig-agent-slim-daemonset-v2.yaml, at the line: serviceAccount: sysdig-agent.

Create a secret key:

$ kubectl create secret generic sysdig-agent --from-literal=access-key=<your sysdig access key> -n sysdig-agent

Create a cluster role and service account, and define the cluster role binding that grants the Sysdig agent rules in the cluster role, using the commands:

$ kubectl apply -f sysdig-agent-clusterrole.yaml -n sysdig-agent
$ kubectl create serviceaccount sysdig-agent -n sysdig-agent
$ kubectl create clusterrolebinding sysdig-agent --clusterrole=sysdig-agent --serviceaccount=sysdig-agent:sysdig-agent

Edit sysdig-agent-configmap.yaml to add the collector``address``` and portand theSSL/TLS` information :
```
collector:
collector_port:
ssl: #true or false
check_certificate: #true or false
```
- For SaaS, find the collector address for your region.
- For On-prem, enter the collector endpoint defined in your environment.
- check_certificate should be set to false if a self-signed certificate or private, CA-signed cert is used. See also Step 5 Set Up SSL Connectivity to the Backend.

Apply the configuration changes:

$ kubectl apply -f sysdig-agent-configmap.yaml -n sysdig-agent

Deploy the kernel module and slim agent containers using the daemonset:

# kubectl apply -f sysdig-agent-slim-daemonset-v2.yaml -n sysdig-agent

1.1.1.6 -

Agent Install: Manual Linux Installation

Manual installation of the native Linux agent is recommended in the following cases:

Full control over the deployment process
Integration with configuration management tools
Custom kernel
Unsupported distribution (within Debian/Fedora flavors)

Otherwise, you may want to just follow the standard Installation Guide:

Agent Install: Non-Orchestrated

NOTE: If you are installing the Sysdig agent in an orchestrated infrastructure such as Kubernetes, Mesos/Marathon, use the respective Installation Guides:

Run the commands as root or with sudo.

Installation Options

Review the Host Requirements for Agent Installation. Then follow the steps for the appropriate Linux distribution, below.

Debian, Ubuntu

Trust the Sysdig Monitor GPG key, configure the apt repository, and update the package list:

curl -s https://download.sysdig.com/DRAIOS-GPG-KEY.public | apt-key add -
curl -s -o /etc/apt/sources.list.d/draios.list http://download.sysdig.com/stable/deb/draios.list
apt-get update

Install kernel development files.
The following command might not work with every kernel. Make sure to customize the name of the package properly.
```
apt-get -y install linux-headers-$(uname -r)
```
Install, configure, and restart the Sysdig agent.
```
apt-get -y install draios-agent
echo customerid: ACCESS_KEY >> /opt/draios/etc/dragent.yaml
echo tags: [TAGS] >> /opt/draios/etc/dragent.yaml
service dragent restart
```
Replace ACCESS_KEY with your unique access key string. Inability to retrieve the key indicates that the administrator of your instance might have it turned off for non-admin users. Please contact your Sysdig administrator to receive the key. If you still have issues please contact Sysdig support.
[TAGS] is an optional parameter you can use to list one or more tags for this host (see below).

CentOS, RHEL, Fedora, Amazon AMI, Amazon Linux 2

Trust the Sysdig Monitor GPG key, configure the yum repository.

$ rpm --import https://download.sysdig.com/DRAIOS-GPG-KEY.public
$ curl -s -o /etc/yum.repos.d/draios.repo http://download.sysdig.com/stable/rpm/draios.repo

Install the EPEL repository
The following command is required only if DKMS is not available in the distribution. You can verify if DKMS is available with yum list dkms
The command below contains a sample release number; be sure to update with the correct release.
```
$ rpm -i http://mirror.us.leaseweb.net/epel/6/i386/epel-release-6-8.noarch.rpm
```
Install kernel development files.
The following command might not work with every kernel. Make sure to customize the name of the package properly.
```
$ yum -y install kernel-devel-$(uname -r)
```
Install, configure, and start the Sysdig agent.
```
$ yum -y install draios-agent
$ echo customerid: ACCESS_KEY >> /opt/draios/etc/dragent.yaml
$ echo tags: [TAGS] >> /opt/draios/etc/dragent.yaml
$ sudo systemctl enable dragent
$ sudo systemctl start dragent
```
Replace ACCESS_KEY with your unique access key string. Inability to retrieve the key indicates that the administrator of your instance might have it turned off for non-admin users. Please contact your Sysdig administrator to receive the key. If you still have issues please contact Sysdig support.
[TAGS] is an optional parameter you can use to list one or more tags for this host (see below).
If you using a non-systemd Linux distribution, use the service command to start dragent.
```
$ service dragent restart
```

Other Linux Distributions

The Sysdig Agent is unsupported outside of the Debian, Fedora, and Amazon distributions.

Agent Tags

Tagging your hosts is highly recommended. Agent Tags allow you to sort nodes of your infrastructure into custom groups in Sysdig Monitor.

Replace the [TAGS] parameter above with a comma-separated list of TAG_NAME:TAG_VALUE .

For example: role:webserver,location:europe

1.1.1.7 -

Agent Install: IKS (IBM Cloud with Sysdig)

IBM maintains the documentation for Sysdig agent installation on IBM Cloud Kubernetes Service (IKS).

For more information, see the IBM Cloud Monitoring with Sysdig documentation:

1.1.1.8 -

Agent Install: Mesos | Marathon | DCOS

Marathon is the container orchestration platform for Mesosphere’s Datacenter Operating System (DC/OS) and Apache Mesos.

This guide describes how to install the Sysdig agent container on each underlying host in your Mesos cluster. Once installed, the agent will automatically connect to the Mesos and Marathon APIs to pull relevant metadata about the environment and will begin monitoring all of your hosts, apps, containers, and frameworks.

Standard Installation Instructions

Review the Host Requirements for Agent Installation.

In this three-part installation, you:

Deploy the Sysdig agent on all Mesos Agent (aka “Slave”) nodes, either automatically or by creating and posting a .json file to the leader Marathon API server.
Deploy the Sysdig agent on the Mesos Master nodes.
Special configuration steps: modify the Sysdig agent config file to monitor Marathon instances.

Deploy the Sysdig agent on your Mesos Agent nodes

Preferred Option: Automatic install (DC/OS 1.11+)

If you’re using DC/OS 1.8 or higher, then you can find Sysdig in the Mesosphere Universe marketplace and install it from there.

It will automatically deploy the Sysdig agent container on each of your Mesos Agent nodes as a Marathon app.

Proceed to Deploy the Sysdig Agent.

Alternate Option: Post a .json file

If you are using a version of DC/OS earlier than 1.8 then:

Create a JSON file for Marathon, in the following format.

The COLLECTOR address comes from your own environment in on-prem installations. For SaaS installations, find the collector endpoint for your region listed here.

COLLECTOR_PORT, SECURE, and CHECK_CERT are used in environments with Sysdig’s on-premises backend installed.

{
  "backoffFactor": 1.15,
  "backoffSeconds": 1,
  "constraints": [
    [
      "hostname",
      "UNIQUE"
    ]
  ],
  "container": {
    "docker": {
      "forcePullImage": true,
      "image": "sysdig/agent",
      "parameters": [],
      "privileged": true
    },
    "type": "DOCKER",
    "volumes": [
      {
        "containerPath": "/host/var/run/docker.sock",
        "hostPath": "/var/run/docker.sock",
        "mode": "RW"
      },
      {
        "containerPath": "/host/dev",
        "hostPath": "/dev",
        "mode": "RW"
      },
      {
        "containerPath": "/host/proc",
        "hostPath": "/proc",
        "mode": "RO"
      },
      {
        "containerPath": "/host/boot",
        "hostPath": "/boot",
        "mode": "RO"
      },
      {
        "containerPath": "/host/lib/modules",
        "hostPath": "/lib/modules",
        "mode": "RO"
      },
      {
        "containerPath": "/host/usr",
        "hostPath": "/usr",
        "mode": "RO"
      }
    ]
  },
  "cpus": 1,
  "deployments": [],
  "disk": 0,
  "env": {
    "ACCESS_KEY": "ACCESS_KEY=YOUR-ACCESS-KEY-HERE",
    "CHECK_CERT": "false",
    "SECURE": "true",
    "TAGS": "example_tag:example_value",
    "name": "sdc-agent",
    "pid": "host",
    "role": "monitoring",
    "shm-size": "350m"
  },
  "executor": "",
  "gpus": 0,
  "id": "/sysdig-agent",
  "instances": 1,
  "killSelection": "YOUNGEST_FIRST",
  "labels": {},
  "lastTaskFailure": {
    "appId": "/sysdig-agent",
    "host": "YOUR-HOST",
    "message": "Container exited with status 70",
    "slaveId": "1fa6f2fc-95b0-445f-8b97-7f91c1321250-S2",
    "state": "TASK_FAILED",
    "taskId": "sysdig-agent.3bb0759d-3fa3-11e9-b446-c60a7a2ee871",
    "timestamp": "2019-03-06T00:03:16.234Z",
    "version": "2019-03-06T00:01:57.182Z"
  },
  "maxLaunchDelaySeconds": 3600,
  "mem": 850,
  "networks": [
    {
      "mode": "host"
    }
  ],
  "portDefinitions": [
    {
      "name": "default",
      "port": 10101,
      "protocol": "tcp"
    }
  ],
  "requirePorts": false,
  "tasks": [
    {
      "appId": "/sysdig-agent",
      "healthCheckResults": [],
      "host": "YOUR-HOST-IP",
      "id": "sysdig-agent.0d5436f4-3fa4-11e9-b446-c60a7a2ee871",
      "ipAddresses": [
        {
          "ipAddress": "YOUR-HOST-IP",
          "protocol": "IPv4"
        }
      ],
      "localVolumes": [],
      "ports": [
        4764
      ],
      "servicePorts": [],
      "slaveId": "1fa6f2fc-95b0-445f-8b97-7f91c1321250-S2",
      "stagedAt": "2019-03-06T00:09:04.232Z",
      "startedAt": "2019-03-06T00:09:06.912Z",
      "state": "TASK_RUNNING",
      "version": "2019-03-06T00:09:04.182Z"
    }
  ],
  "tasksHealthy": 0,
  "tasksRunning": 1,
  "tasksStaged": 0,
  "tasksUnhealthy": 0,
  "unreachableStrategy": {
    "expungeAfterSeconds": 0,
    "inactiveAfterSeconds": 0
  },
  "upgradeStrategy": {
    "maximumOverCapacity": 1,
    "minimumHealthCapacity": 1
  },
  "version": "2019-03-06T00:09:04.182Z",
  "versionInfo": {
    "lastConfigChangeAt": "2019-03-06T00:09:04.182Z",
    "lastScalingAt": "2019-03-06T00:09:04.182Z"
  }
}

See Table 1: Environment Variables for Agent Config Filef or the Sysdig name:value definitions.

Complete the “cpus”, “mem” and “labels” (i.e. Marathon labels) entries to fit the capacity and requirements of the cluster environment.

Update the created.json file to the leader Marathon API server:

$ $curl -X POST http://$(hostname -i):8080/v2/apps -d @sysdig.json -H "Content-type: application/json"

Deploy the Sysdig Agent

After deploying the agent to the Mesos Agent nodes, you will install agents on each of the Mesos Master nodes as well.

If any cluster node has both Mesos Master and Mesos Agent roles, do not perform this installation step on that node. It already will have a Sysdig agent installed from the procedure in step A. Running duplicate Sysdig agents on a node will cause errors.

Use the Agent Install: Non-Orchestrated instructions to install the agent directly on each of your Mesos Master nodes.

When the Sysdig agent is successfully installed on the master nodes, it will automatically connect to the local Mesos and Marathon (if available) API servers via http://localhost:5050 and http://localhost:8080 respectively, to collect cluster configuration and current state metadata in addition to host metrics.

Special Configuration Steps

In certains situations, you may need to add additional configurations to the dragent.yaml file:

If the Sysdig agent cannot be run directly on the Mesos API server
If the API server is protected with a username/password.

Descriptions and examples are shown below.

If the Sysdig Agent Cannot Run On the Mesos API Server

Mesos allows multiple masters. If the API server can not be instrumented with a Sysdig agent, simply delegate ONE other node with an agent installed to remotely receive infrastructure information from the API server.

NOTE: If you manually configure the agent to point to a master with a static configuration file entry, then automatic detection/following of leader changes will no longer be enabled.

Add the following Mesos parameter to the delegated agent’s dragent.yaml file to allow it to connect to the remote API server and authenticate, either by:

a. Directly editing dragent.yaml on the host, or

b. Converting the YAML code to a single-line format and adding it as an ADDITIONAL_CONF argument in a Docker command.

See Understanding the Agent Config Files for details.

Specify the API server’s connection method, address, and port. Also specify credentials if necessary.

YAML example:

mesos_state_uri: http://[acct:passwd@][hostname][:port]
marathon_uris:
  - http://[acct:passwd@][hostname][:port]

Although marathon_uris: is an array, currently only a single “root” Marathon framework per cluster is supported. Multiple side-by-side Marathon frameworks should not be configured in order for our agent to function properly. Multiple side-by-side “root” Marathon frameworks on the same cluster are currently not supported. The only supported multiple-Marathon configuration is with one “root” Marathon and other Marathon frameworks as its apps.

If the Mesos API server requires authentication

If the agent is installed on the API server but the API server uses a different port or requires authentication, those parameters must be explicitly specified.

Add the following Mesos parameters to the API server’s dragent.yaml to make it connect to the API server and authenticate with any unique account and password, either by:

a. Directly editing dragent.yaml on the host, or

b. Converting the YAML code to a single-line format and adding it as an ADDITIONAL_CONF argument in a Docker command.

See Understanding the Agent Config Files for details.

Specify the API server’s protocol, user credentials, and port:

mesos_state_uri: http://[username:password@][hostname][:port]
marathon_uris:
  - http://[acct:passwd@][hostname][:port]

*HTTPS protocol is also supported.

Troubleshooting: Turning Off Metadata Reception

In troubleshooting cases where auto-detection and reporting of your Mesos infrastructure needs to be temporarily turned off in a designated agent:

Comment out the Mesos parameter entries in the agent’s dragent.yaml file.
Example parameters to disable: mesos_state_uri, marathon_uris
If the agent is running on the API server (Master node) and auto-detecting a default configuration, you can add the line:
mesos_autodetect: false
either directly in the dragent.yaml file or as an ADDITIONAL_CONF parameter in a Docker command.
Restart the agent.

1.1.1.9 -

Airgapped Agent Installation

Airgapped environments are those that do not have the network access to pull images from the container repository. Agent installation requires sysdigcloud-probe and you cannot download a pre-compiled module in an airgapped environment. Therefore, ensure that you compile your own sysdigcloud-probe before installing the agent.

Prepare the Sysdig Probe Builder Images

On a machine with internet connectivity, build the Sysdig probe container and create a tar file of the image.

Get the probe builder artifacts from the repository:

$ git clone https://github.com/draios/sysdig
$ git checkout probe-builder
$ cd sysdig

Build the container image:

$ docker build -t airgap/sysdig-probe-builder probe-builder/

Create the container and run:

$ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock airgap/sysdig-probe-builder:latest -P -b airgap/

Save the images to a tar archive:
```
$ docker save airgap/sysdig-probe-builder | gzip > builders.tar.gz
```
Ensure that you make this tar available to the airgapped machines where you intend to install the Sysdig agent.

Set Up Kernel Module

Set up a local repository to host the pre-compiled kernel module:

$ kubectl run my-nginx --image=nginx --port=80
$ kubectl expose deployment my-nginx --port=80 --type=NodePort

Copy sysdigcloud-probe to the repository you have created:

$ kubectl cp sysdigcloud-probe-<version> my-nginx-xxxxxxxx-xxxx:/usr/share/nginx

Install Agent in Docker Environment

Install Sysdig agent by pointing SYSDIG_PROBE_URL to the local repository:

For docker-based installations:

$ docker run -d --name sysdig-agent --restart always --privileged --net host --pid host -e ACCESS_KEY=WWWWW-YYYY-XXXX-ZZZZ-123456789 -e SECURE=true -e SYSDIG_PROBE_URL=http://www.mywebserver.net:80/ -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro --shm-size=512m sysdig/agent

Where -e SYSDIG_PROBE_URL=http://www.mywebserver:80/ is the local nginx pod with the loaded module.

To use secure communication with a self-signed or untrusted certificate, apply the -e SYSDIG_PROBE_INSECURE_DOWNLOAD=true environment variable.

Check the agent log. You will see a similar message:

Found custom module URL http://mywebserver:80/, will use it * Trying to download precompiled module from http://mywebserver:80/sysdigcloud-probe-<version>

Continue with the instructions in Agent Install: Non-Orchestrated.

Install Agent in Kubernetes Environment

Open your agent daemonset and update the SYSDIG_PROBE_URL to point to the local repository:
```
- name: SYSDIG_PROBE_URL
  value: http://www.mywebserver:80/
```
If you would like to use secure communication with a self-signed or untrusted certificate, apply the SYSDIG_PROBE_INSECURE_DOWNLOAD environment variable.
```
- name: SYSDIG_PROBE_INSECURE_DOWNLOAD
  value: true
```
Continue with the instructions in Agent Install: Kubernetes.

1.1.1.10 -

Troubleshooting Agent Installation

This section describes methods for troubleshooting two types of issue:

Disconnecting Agents
Can’t See Metrics After Agent Install

Disconnecting Agents

If agents are disconnecting, there could be problems with addresses that need to be resolved in the agent configuration files. See also Understanding the Agent Config Files.

Check for Duplicate MAC addresses

The Sysdig agent will use the eth0 MAC address to identify the different hosts within an infrastructure. In a virtualized environment, you should confirm each of your VM’s eth0 MAC addresses are unique.

If a unique address cannot be configured, you can supply an additional parameter in the Sysdig agent’s dragent.yaml configuration file: machine_id_prefix: prefix

The prefix text can be any string and will be prepended to the MAC address as reported in the Sysdig Monitor web interface’s Explore tables.

Example: (using ADDITIONAL_CONF rather than Kubernetes Configmap)

Here is an example Docker run command installing the parameter via the ADDITIONAL_CONF parameter

docker run --name sysdig-agent --privileged --net host --pid host -e ACCESS_KEY=abc123-1234-abcd-4321-abc123def456 -e TAGS=tag1:value1 -e ADDITIONAL_CONF="machine_id_prefix: MyPrefix123-" -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro sysdig/agent

The resulting /opt/draios/etc/dragent.yaml config file would look like this:

customerid:abc123-1234-abcd-4321-abc123def456
tags: tag1:value1
machine_id_prefix: MyPrefix123-

You will then see all of your hosts, provided that all the prefixes are unique. The prefix will be visible whenever the MAC address is displayed in any view.

Check for Conflicting MAC addresses in GKE environments

In Google Container Engine (GKE) environments, MAC addresses could be repeated across multiple hosts. This would cause some hosts running Sysdig agents not to appear in your web interface.

To address this, add a unique machine ID prefix to each config you use to deploy the agent to a given cluster (i.e. each sysdig-daemonset.yaml file).

Note: This example uses the (v1) ADDITIONAL_CONF, rather than (v2) Configmap method.

- name: ADDITIONAL_CONF value: "machine_id_prefix: mycluster1-prefix-"

Can’t See Metrics After Agent Install

If agents were successfully installed, you could log in to the Sysdig Monitor UI, but no metrics are displayed in the Explore panel, first confirm that the agent license count has not been exceeded. Then check for any proxy, firewall, or host security policies preventing proper agent communication to the Sysdig Monitor backend infrastructure.

Check License Count

If network connectivity is good, the agent will connect to the backend but will be disconnected after a few seconds if the license count has been exceeded.

To check whether you are over-subscribed, go to Settings > Subscription.

See Subscription for details.

Check Network Policy

Agent Connection Port

Check your service provider VPC security groups to verify that network ACLs are set to allow the agent’s outbound traffic over TCP ports. See Sysdig Collector Ports for the supported TCP ports for each region.

Outbound IP Addresses

Due to the distributed nature of the Sysdig Monitor infrastructure, the agent must be open for outbound connections to collector.sysdigcloud.com on all outbound IP addresses.

Check Amazon’s public IP ranges file to see all the potential IP addresses the Sysdig agent can use to communicate with the Sysdig backend databases.

AWS Metadata Endpoint

AWS metadata is used for gathering information about the instance itself, such as instance id, public IP address, etc.

When running on an AWS instance, access to the following AWS metadata endpoint is also needed: 169.254.169.254

Check Local Host Policy

The agent requires access to the following local system resources in order to gather metrics:

Read/Write access to /dev/sysdig devices.
Read access to all the files under /proc file system.
For container support, the Docker API endpoint /var/run/docker.sock

If any settings or firewall modifications are made, you may need to restart the agent service. In a shell on the affected instances issue the following command:

sudo service dragent restart

1.1.1.11 -

Identify Agent Version

Use one of the following methods to determine the version of the agents installed in your environment:

Explore

Segmenting metrics by using agent.version shows the installed versions of agents in your environment. For example, segment the uptime metric across your environment by using agent.version . Hover over the graph to see the list of agent versions.

The image shows the list of agent versions in n/a.

Dashboard

Use the Sysdig Agent Health Dashboard to determine the agent versions:

Log in to the Sysdig Monitor.
Select Dashboards and expand Host Infrastructure Dashboards.
Open the Sysdig Agent Health & Status template or create your own from the template.
The Sysdig Agent and Health & Status Dashboard shows the agent version corresponding to each host in your environment.

1.1.1.12 -

Using Node Leases

The Sysdig agent uses Kubernetes Lease to control how and when connections are made to the Kubernetes API Server. This mechanism prevents overloading the Kubernetes API server with connection requests during agent bootup.

Kubernetes node leases are automatically created for agent version 12.0.0 and above. On versions prior to 12.0.0, you must configure node leases as given in the KB article.

Prerequisites

Sysdig Agent v11.3.0 or above
Kubernetes v1.14 or above

Types of Leases

The agent creates the following leases:

Cold Start

During boot up, the Sysdig agent connects to the Kubernetes API server to retrieve Kubernetes metadata and build a cache. The cold-start leases control the number of agents that build up this cache at any given time. An agent will grab a lease, build its cache, and then release the lease so that another agent can build its cache. This mechanism prevents agents from creating a “boot storm” which can overwhelm the API server in large clusters.

Delegation

In Kubernetes environments, two agents are marked as delegated in each cluster. The delegated agents are the designated agents to request more data from the API server and produce KubeState metrics. The delegation leases will not be released until the agent is terminated.

View Leases

To view the leases, run the following:

$ kubectl get leases -n sysdig-agent

You will see an output similar to the following:

NAME           HOLDER             AGE
cold-start-0                      20m
cold-start-1                      20m
cold-start-2                      21m
cold-start-3   ip-10-20-51-167    21m
cold-start-4                      21m
cold-start-5                      21m
cold-start-6                      20m
cold-start-7                      21m
cold-start-8                      20m
cold-start-9   ip-10-20-51-166   21m
delegation-0   ip-10-20-52-53    21m
delegation-1   ip-10-20-51-98    21m

Troubleshoot Leases

Verify Configuration

When lease-based delegation is working as expected, the agent logs show one of the following:

Getting pods only for node <node>
Getting pods for all nodes.
Both (occasionally on the delegated nodes)

Run the following to confirm that it is working:

$ kubectl logs sysdig-agent-9l2gf -n sysdig-agent | grep -i "getting pods"

The configuration is working as expected if the output on a pod is similar to the following:

2021-05-05 02:48:32.877, 15732.15765, Information, cointerface[15738]: Only getting pods for node ip-10-20-51-166.ec2.internal

Unable to Create Leases

The latest Sysdig ClusterRole is required for the agent to create leases. If you do not have the latest ClusterRole or if you have not configured the ClusterRole correctly, the logs show the following error:

Error, lease_pool_manager[2989554]: Cannot access leases objects: leases.coordination.k8s.io is forbidden: User "system:serviceaccount:sysdig-agent:sysdig-agent" cannot list resource "leases" in API group "coordination.k8s.io" in the namespace "sysdig-agent"

Contact Sysdig Support for help.

Optional Agent Configuration

Several configuration options exist for leases. It is recommended to not change the default settings unless prompted by Sysdig Customer Support.

Configuration	Default	Description
`k8s_coldstart: enabled: <true/false>`	`true` above agent versions 12.0.0	When true, the agent will attempt to create `cold-start` leases to control the number of agents which are allowed to build their cache at one time.
`k8s_coldstart: max_parallel_cold_starts: <int>`	10	The number of `cold-start` leases to be created. This is the number of agents that can connect to the API Server simultaneously during agent initialization.
`k8s_coldstart: namespace: <string>`	`sysdig-agent`	The namespace to be created. This shouldn’t be needed in agent version 12.0.0 because the DownwardAPI in the ClusterRole will provide the appropriate namespace.
`k8s_coldstart: enforce_leader_election: <true/false>`	`false`	When `true`, the agent will not fall back to the previous method if it cannot create leases.This can be useful if the previous method caused API Server problems.
`k8s_delegation_election: <true/false>`	`true` above agent versions 12.0.0	When `true`, the agent will create `delegation` leases to control which set of agents generate global cluster metrics.

1.1.2 -

Agent Configuration

Out of the box, the Sysdig agent will gather and report on a wide variety of pre-defined metrics. It can also accommodate any number of custom parameters for additional metrics collection.

Use this section when you need to change the default or pre-defined settings by editing the agent configuration files, or for other special circumstances.

Understanding the Agent Config Files: Describes the core configuration files, how to access them, and how to add custom environment variables.
Edit the Agent Config File to Filter Data
Optional: Change the Agent Log Level
Optional: Agent Auto-Config: Describes how to use the Sysdig REST APIs and Python client wrappers to auto-orchestrate changes to all agents in an environment, in the (rare) situation in which a standard orchestration tool such as Kubernetes, Chef, or Ansible is not being used.

Integrations for Sysdig Monitor also require editing the agent config files.

By default, the Sysdig agent is configured to collect metric data from a range of platforms and applications. You can edit the agent config files to extend the default behavior, including additional metrics for JMX, StatsD, Prometheus, or a wide range of other applications. You can also monitor log files for targeted text strings.

1.1.2.1 -

Understand the Agent Configuration

Out of the box, the Sysdig agent will gather and report on a wide variety of pre-defined metrics. It can also accommodate any number of custom parameters for additional metrics collection.

The agent relies on a pair of configuration files to define metrics collection parameters:

dragent.default.yaml

The core configuration file. You can look at it to understand more about the default configurations provided.

Location: "/opt/draios/etc/dragent.default.yaml."

CAUTION. This file should never be edited.

dragent.yaml or configmap.yaml (Kubernetes)

The configuration file where parameters can be added, either directly in YAML as name/value pairs, or using environment variables such as 'ADDITIONAL_CONFLocation: "/opt/draios/etc/dragent.yaml."

The “dragent.yaml” file can be accessed and edited in several ways, depending on how the agent was installed. This document describes how to modify dragent.yaml.

One additional file, dragent.auto.yaml is also created and used in special circumstances. See Optional: Agent Auto-Config for more detail.

Access and Edit the Config File

There are various ways to add or edit parameters indragent.yaml.

Option 1: With dragent.yaml (for testing)

It is possible to edit the container’s file directly on the host.

Add parameters directly in YAML.

Access dragent.yamldirectly at"/opt/draios/etc/dragent.yaml."
Edit the file. Use proper YAML syntax.
See the examples at the bottom of the page.
Restart the agent for changes to take effect

Native agent: service dragent restart
Container agent: docker restart sysdig-agent

Option 2: With configmap.yaml(Kubernetes)

Configmap.yaml is the configuration file where parameters can be added, either directly in YAML as name/value pairs, or using environment variables such as ‘ADDTIONAL_CONF."

If you install agents as DaemonSets on a system running Kubernetes, you use configmap.yaml to connect with and manipulate the underlyingdragent.yamlfile.

Add parameters directly in YAML.

Edit the files locally and apply with the changes withkubectl -f.

Access theconfigmap.yaml.
Edit the file as needed.
Apply the changes:
kubectl apply -f sysdig-agent-configmap.yaml

Running agents will automatically pick the new configuration after Kubernetes pushes the changes across all the nodes in the cluster.

Option 3: With Docker Run (Docker)

Add -e ADDITIONAL_CONF=”<VARIABLES>” to a Docker run command, where <VARIABLES> contains all the customized parameters you want to include, in a single-line format.

Convert YAML Parameters to Single-Line Format

To insert ADDITIONAL_CONF parameters in a Docker run command or a daemonset file, you must convert the YAML code into a single-line format.

You can do the conversion manually for short snippets. To convert longer portions of YAML, use echo|sed commands.

In earlier versions, the Sysdig Agent connected to port 6666. This behavior has been deprecated, as the Sysdig agent now connects to port 6443.

The basic procedure:

Write your configuration in YAML, as it would be entered directly in dragent.yaml.
In a bash shell, use echo and sed to convert to a single line.
sed script: echo "" | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/\\n/g'
Insert the resulting line into a Docker run command or add it to the daemonset file as an ADDITIONAL_CONF.

Example: simple

Insert parameters to turn off StatsD collection and blacklist port 6443.

YAML format

statsd:
    enabled: false
    blacklisted_ports:
    - 6443

Single-line format (manual)

Use spaces, hyphens, and \n correctly when manually converting to a single line:

ADDITIONAL_CONF="statsd:\n enabled: false\n blacklisted_ports:\n - 6443"

Here the single line is incorporated into a full agent startup Docker command.

docker run
  --name sysdig-agent \
  --privileged \
  --net host \
  --pid host \
  -e ACCESS_KEY=1234-your-key-here-1234 \
  -e TAGS=dept:sales,local:NYC \
  -e ADDITIONAL_CONF="statsd:\n    enabled: false\n    blacklisted_ports:\n    - 6443" \
  -v /var/run/docker.sock:/host/var/run/docker.sock \
  -v /dev:/host/dev \
  -v /proc:/host/proc:ro \
  -v /boot:/host/boot:ro \
  -v /lib/modules:/host/lib/modules:ro \
  -v /usr:/host/usr:ro \
sysdig/agent

Example: complex

Insert parameters to override the default configuration for a RabbitMQ app check.

YAML format

app_checks:
  - name: rabbitmq
    pattern:
      port: 15672
    conf:
      rabbitmq_api_url: "http://localhost:15672/api/"
      rabbitmq_user: myuser
      rabbitmq_pass: mypassword
      queues:
        - MyQueue1
        - MyQueue2

Single-line format (echo |sed)

From a bash shell, issue the echo command and sed script.

echo "app_checks:
  - name: rabbitmq
    pattern:
      port: 15672
    conf:
      rabbitmq_api_url: "http://localhost:15672/api/"
      rabbitmq_user: myuser
      rabbitmq_pass: mypassword
      queues:
        - MyQueue1
        - MyQueue2
" | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/\\n/g'

This results in the single-line format to be used with ADDITIONAL_CONF in a Docker command or daemonset file.

"app_checks:\n - name: rabbitmq\n  pattern:\n    port: 15672\n  conf:\n    rabbitmq_api_url: http://localhost:15672/api/\n    rabbitmq_user: myuser\n    rabbitmq_pass: mypassword\n    queues:\n      - MyQueue1\n      - MyQueue2\n"

Option 4: With HELM Format

If you installed the Sysdig agent in Kubernetes using a Helm chart, then no configmap.yaml file was downloaded. You edit dragent.yaml using Helm syntax:

Example

$ helm install 
    --name sysdig-agent
    --set sysdig.settings.tags='linux:ubuntu,dept:dev,local:nyc' 
    --set sysdig.settings.k8s_cluster_name='my_cluster' 
  sysdig/sysdig

Will be transformed into

data:
 dragent.yaml: |
  tags: linux:ubuntu,dept:dev,local:nyc
  k8s_cluster_name: my_cluster

Table 1: Environment Variables for Agent Config File

Name	Value	Description
`ACCESS_KEY`	`<your Sysdig access key>`	Required
`TAGS`	`<meaningful tags you want applied to your instances>`	Optional. These are displayed in Sysdig Monitor for ease of use. For example: `tags: linux:ubuntu,dept:dev,local:nyc` See sysdig-agent-configmap.yaml.
`COLLECTOR`	`<collector-hostname.com> or 111.222.333.400`	Enter the host name or IP address of the Sysdig collector service. Note that when used within `dragent.yaml`, must be lowercase `collector`. For SaaS regions, see: SaaS Regions and IP Ranges.
`COLLECTOR_PORT`	`6443`	On-prem only. The port used by the Sysdig collector service; default 6443.
`SECURE`	`"true"`	On-prem only. If using SSL/TLS to connect to collector service value = "true" otherwise "false."
`CHECK_CERTIFICATE`	`"false"`	On-prem only. Set to "true" when using SSL/TLS to connect to the collector service and should check for valid SSL/TLS certificate.
`ADDITIONAL_CONF`		Optional. A place to provide custom configuration values to the agent as environment variables .
`SYSDIG_PROBE_URL`		Optional. An alternative URL to download precompiled kernel module.

Sample Docker Command Using Variables

docker run \
  --name sysdig-agent \
  --privileged \
  --net host \
  --pid host \
  -e ACCESS_KEY=3e762f9a-3936-4c60-9cf4-c67e7ce5793b \
  -e COLLECTOR=mycollector.elb.us-west-1.amazonaws.com \
  -e COLLECTOR_PORT=6443 \
  -e CHECK_CERTIFICATE=false \
  -e TAGS=my_tag:some_value \
  -e ADDITIONAL_CONF="log:\n file_priority: debug\n console_priority: error" \
  -v /var/run/docker.sock:/host/var/run/docker.sock \
  -v /dev:/host/dev \
  -v /proc:/host/proc:ro \
  -v /boot:/host/boot:ro \
  -v /lib/modules:/host/lib/modules:ro \
  -v /usr:/host/usr:ro \
  --shm-size=350m \
sysdig/agent

1.1.2.2 -

Configure Agent Modes

Agent modes provide the ability to control metric collection to fit your scale and specific requirement. You can choose one of the following modes to do so:

Monitor
Monitor Light
Troubleshooting
Secure

Using a stripped-down mode limits collection of unneeded metrics, which in turn prevents the consumption of excess resources and helps reduce expenses.

Monitor

The Monitor mode offers an extensive collection of metrics. We recommend this mode to monitor enterprise environments.

monitor is the default mode if you are running the Enterprise tier. To switch back to the Monitor mode from a different mode, do one of the following:

Add the following to the dragent.yaml file and restart the agent:
```
feature:
  mode: monitor
```
Remove the parameter related to the existing mode from the dragent.yaml file and restart the agent. For example, to switch from troubleshooting mode to monitor, delete the following lines:
```
feature:
  mode: troubleshooting
```

Monitor Light

Monitor Light caters to the users that run agents in a resource-restrictive environment, or to those who are interested only in a limited set of metrics.

Monitor Light provides CPU, Memory, File, File system, and Network metrics. For more information, see Metrics Available in Monitor Light.

Enable Monitor Light Mode

To switch to the Monitor Light mode, edit the dragent.yaml file:

Open the dragent.yaml file.
Add the following configuration parameter:
```
feature:
  mode: monitor_light
```
Restart the agent.

Troubleshooting

Troubleshooting mode offers sophisticated metrics with detailed diagnostic capabilities. Some of these metrics are heuristic in nature.

In addition to the extensive metrics available in the Monitor mode, Troubleshooting mode provides additional metrics such as net.sql and additional segmentation for file and network metrics. For more information, see Additional Metrics Values Available in Troubleshooting.

Enable Troubleshooting Mode

To switch to the Troubleshooting mode, edit the dragent.yaml file:

Open the dragent.yaml file.
Add the following configuration parameter:
```
feature:
  mode: troubleshooting
```
Restart the agent.

Secure Mode

The secure mode supports only Sysdig Secure features.

Sysdig agent collects no metrics in the secure mode, which, in turn, minimizes network consumption and storage requirement in the Sysdig backend. Lower resource usage can help reduce costs and improve performance.

In the Secure mode, the Monitor UI shows no data because no metrics are sent to the collector.

This feature requires agent v10.5.0 or above.

Enabling Secure Mode

Open the dragent``.yaml file.
Add the following:
```
feature:
  mode: secure
```
Restart the agent.

1.1.2.2.1 -

Metrics Available in Monitor Light

Monitor Light provides cpu, memory, file, file system, and network metrics.

Metrics	Description
`cpu.cores.used`	See System.System
`cpu.cores.used.percent`
`cpu.idle.percent`
`cpu.iowait.percent`
`cpu.nice.percent`
`cpu.stolen.percent`
`cpu.system.percent`
`cpu.used.percent`
`cpu.user.percent`
`load.average.percpu.1m`
`load.average.percpu.5m`
`load.average.percpu.15m`
`memory.bytes.available`
`memory.bytes.total`
`memory.bytes.used`
`memory.bytes.used`
`memory.bytes.virtual`
`memory.pageFault.major`
`memory.pageFault.minor`
`memory.swap.bytes.available`
`memory.swap.bytes.total`
`memory.swap.bytes.used`
`memory.swap.used.percent`
`memory.used.percent`
`file.bytes.in`
`file.bytes.out`
`file.bytes.total`
`file.iops.in`
`file.iops.out`
`file.iops.total`
`file.open.count`
`file.time.in`
`file.time.out`
`file.time.total`
`fs.bytes.free`
`fs.bytes.total`
`fs.bytes.used`
`fs.free.percent`
`fs.inodes.total.count`
`fs.inodes.used.count`
`fs.inodes.used.percent`
`fs.largest.used.percent`
`fs.root.used.percent`
`fs.used.percent`
`net.bytes.in`
`net.bytes.out`
`net.bytes.total`
`proc.count`
`thread.count`
`container.count`
`system.uptime`
`uptime`

1.1.2.2.2 -

Additional Metrics Values Available in Troubleshooting

In addition to the extensive set of metrics available in the monitor mode, additional metrics, such as net.sql and net.mongodb, as well as additional segmentations for file and network metrics are available.

Metrics	Additional Metrics Values Available When Segmented by	Supported Agent Versions
`file.error.total.count`	`file.name` and `file.mount` labels	Version 10.1.0 or above
`file.bytes.total`
`file.bytes.in`
`file.bytes.out`
`file.open.count`
`file.time.total`
`host.count`
`host.error.count`
`proc.count`
`proc.start.count`
`net.mongodb.collection`	all	Version 10.2.0 or above
`net.mongodb.error.count`
`net.mongodb.operation`
`net.mongodb.request.count`
`net.mongodb.request.time`
`net.sql.query`	all
`net.sql.error.count`
`net.sql.query.type`
`net.sql.request.count`
`net.sql.request.time`
`net.sql.table`
`net.http.error.count`	`net.http.url`	Version 10.3.0 or above
`net.http.method`
`net.http.request.count`
`net.http.request.time`
`net.bytes.in`
`net.bytes.out`
`net.request.time.worst.out`
`net.request.count`
`net.request.time`
`net.bytes.total`
`net.http.request.time.worst`	all

1.1.2.2.3 -

Metrics Not Available in Essentials Mode

The following metrics will not be reported in the essentials mode when compared with monitor mode:

Metrics	Segmented By
`net.bytes.in`	`net.connection.server`, `net.connection.direction`, `net.connection.l4proto` , and `net.connection.client` labels
`net.bytes.out`
`net.connection.count.total`
`net.connection.count.in`
`net.connection.count.out`
`net.request.count`
`net.request.count.in`
`net.request.count.out`
`net.request.time`
`net.request.time.in`
`net.request.time.out`
`net.bytes.total`
`net.mongodb.collection`	all
`net.mongodb.error.count`
`net.mongodb.operation`
`net.mongodb.request.count`
`net.mongodb.request.time`
`net.sql.query`	all
`net.sql.error.count`
`net.sql.query.type`
`net.sql.request.count`
`net.sql.request.time`
`net.sql.table`
`net.sql.query`	all
`net.sql.error.count`
`net.sql.query.type`
`net.sql.request.count`
`net.sql.request.time`
`net.sql.table`
`net.http.method`
`net.http.request.count`
`net.http.request.time`
`net.http.statusCode`
`net.http.url`

1.1.2.3 -

Enable HTTP Proxy for Agents

You can configure the agent to allow it to communicate with the Sysdig collector through an HTTP proxy. HTTP proxy is usually configured to offer greater visibility and better management of the network.

Agent Behaviour

The agent can connect to the collector through an HTTP proxy by sending an HTTP CONNECT message and receiving a response. The proxy then initiates a TCP connection to the collector. These two connections form a tunnel that acts like one logical connection.

By default, the agent will encrypt all messages sent through this tunnel. This means that after the initial CONNECT message and response, all the communication on that tunnel is encrypted by SSL end-to-end. This encryption is controlled by the top-level ssl parameter in the agent configuration.

Optionally, the agent can add a second layer of encryption, securing the CONNECT message and response. This second layer of encryption may be desired in the case of HTTP authentication if there is a concern that network packet sniffing could be used to determine the user’s credentials. This second layer of encryption is enabled by setting the ssl parameter to true in the http_proxy section of the agent configuration. See Examples for details.

Configuration

You specify the following parameters at the same level as http_proxy in the dragent.yaml file. These existing configuration options affect the communication between the agent and collector (both with and without a proxy.

ssl: If set to true, the metrics sent from the agent to the collector are encrypted.
ssl_verify_certificate: Determines whether the agent verifies the SSL certificate sent from the collector.

The following configuration options affect the behavior of the HTTP Proxy setting. You specify them under the http_proxy heading in the dragent.yaml file.

proxy_host: Indicates the hostname of the proxy server. The default is an empty string, which implies communication through an HTTP proxy is disabled.
proxy_port: Specifies the port on the proxy server the agent should connect to. The default is 0, which indicates that the HTTP proxy is disabled.
proxy_user : Required if HTTP authentication is configured. This option specifies the username for the HTTP authentication. The default is an empty string, which indicates that authentication is not configured.
proxy_password : Required if HTTP authentication is configured. This option specifies the password for the HTTP authentication. The default is an empty string. Specifying proxy_user with no proxy_password is allowed.
ssl: If set to true, the connection between the agent and the proxy server is encrypted.
Note that this parameter requires the top-level ssl parameter to be enabled, as the agent does not support SSL to the proxy but unencrypted traffic to the collector. This additional security prevents you from misconfiguring the agent assuming the metrics are as well encrypted end-to-end when they are not.
ssl_verify_certificate: Determines whether the agent will verify the certificate presented by the proxy.
This option is configured independently of the top-level ssl_verify_certificate parameter. This option is enabled by default. If the provided certificate is not correct, this option can cause the connection to the proxy server to fail.
ca_certificate: The path to the CA certificate for the proxy server. If ssl_verify_certificate is enabled, the CA certificate must be signed appropriately.

Examples

No SSL

The following example shows no SSL connection between the agent and the proxy server as well as between the proxy server and the collector.

collector_port: 6667
ssl: false
http_proxy:
        proxy_host: squid.yourdomain.com
        proxy_port: 3128
        ssl: false

SSL Between Proxy and Collector

In this example, SSL is enabled only between the proxy server and the collector.

collector_port: 6443
ssl: true
ssl_verify_certificate: true
http_proxy:
        proxy_host: squid.yourdomain.com
        proxy_port: 3128

SSL

The following example shows SSL is enabled between the agent and the proxy server as well as between the proxy server and the collector.

collector_port: 6443
ssl: true
http_proxy:
        proxy_host: squid.yourdomain.com
        proxy_port: 3129
        ssl: true
        ssl_verify_certificate: true
        ca_certificate: /usr/proxy/proxy.crt

SSL with Username and Password

The following configuration instructs the agent to connect to a proxy server located at squid.yourdomain.com on port 3128. The agent will request the proxy server to establish an HTTP tunnel to the Sysdig collector at collector-your.sysdigcloud.com on port 6443. The agent will authenticate with the proxy server using the given user and password combination.

collector: collector-your.sysdigcloud.com
collector_port: 6443
http_proxy:
    proxy_host: squid.yourdomain.com
    proxy_port: 3128
    proxy_user: sysdig_customer
    proxy_password: 12345
        ssl: true
    ssl_verify_certificate: true
    ca_certificate: /usr/proxy/proxy_cert.crt

1.1.2.4 -

Filter Data

The dragent.yaml file elements are wide-reaching. This section describes the parameters to edit in dragent.yaml to perform a range of activities:

1.1.2.4.1 -

Blacklist Ports

Use the blacklisted_ports parameter in the agent configuration file to block network traffic and metrics from unnecessary network ports.

Note: Port 53 (DNS) is always blacklisted.

Access the agent configuration file, using one of the options listed.
Add blacklisted_ports with desired port numbers.
Example (YAML):
blacklisted_ports:
- 6443
- 6379
Restart the agent (if editing dragent.yaml file directly), using either the service dragent restart or docker restart sysdig-agent command as appropriate.

1.1.2.4.2 -

Enable/Disable Event Data

Sysdig Monitor supports event integrations with certain applications by default. The Sysdig agent will automatically discover these services and begin collecting event data from them.

The following applications are currently supported:

Docker
Kubernetes

Other methods of ingesting custom events into Sysdig Monitor are touched upon in Custom Events.

By default, only a limited set of events is collected for a supported application, and are listed in the agent’s default settings configuration file (/opt/draios/etc/dragent.default.yaml).

To enable collecting other supported events, add an events entry to dragent.yaml.

You can also change log entry in dragent.yaml to filter events by severity.

Learn more about it in the following sections.

Supported Application Events

Events marked with * are enabled by default; see the dragent.default.yaml file.

Docker Events

The following Docker events are supported.

  docker:
    container:
      - attach       # Container Attached      (information)
      - commit       # Container Committed     (information)
      - copy         # Container Copied        (information)
      - create       # Container Created       (information)
      - destroy      # Container Destroyed     (warning)
      - die          # Container Died          (warning)
      - exec_create  # Container Exec Created  (information)
      - exec_start   # Container Exec Started  (information)
      - export       # Container Exported      (information)
      - kill         # Container Killed        (warning)*
      - oom          # Container Out of Memory (warning)*
      - pause        # Container Paused        (information)
      - rename       # Container Renamed       (information)
      - resize       # Container Resized       (information)
      - restart      # Container Restarted     (warning)
      - start        # Container Started       (information)
      - stop         # Container Stopped       (information)
      - top          # Container Top           (information)
      - unpause      # Container Unpaused      (information)
      - update       # Container Updated       (information)
    image:
      - delete # Image Deleted  (information)
      - import # Image Imported (information)
      - pull   # Image Pulled   (information)
      - push   # Image Pushed   (information)
      - tag    # Image Tagged   (information)
      - untag  # Image Untaged  (information)
    volume:
      - create  # Volume Created    (information)
      - mount   # Volume Mounted    (information)
      - unmount # Volume Unmounted  (information)
      - destroy # Volume Destroyed  (information)
    network:
      - create     # Network Created       (information)
      - connect    # Network Connected     (information)
      - disconnect # Network Disconnected  (information)
      - destroy    # Network Destroyed     (information)

Kubernetes Events

The following Kubernetes events are supported.

  kubernetes:
    node:
      - TerminatedAllPods       # Terminated All Pods      (information)
      - RegisteredNode          # Node Registered          (information)*
      - RemovingNode            # Removing Node            (information)*
      - DeletingNode            # Deleting Node            (information)*
      - DeletingAllPods         # Deleting All Pods        (information)
      - TerminatingEvictedPod   # Terminating Evicted Pod  (information)*
      - NodeReady               # Node Ready               (information)*
      - NodeNotReady            # Node not Ready           (information)*
      - NodeSchedulable         # Node is Schedulable      (information)*
      - NodeNotSchedulable      # Node is not Schedulable  (information)*
      - CIDRNotAvailable        # CIDR not Available       (information)*
      - CIDRAssignmentFailed    # CIDR Assignment Failed   (information)*
      - Starting                # Starting Kubelet         (information)*
      - KubeletSetupFailed      # Kubelet Setup Failed     (warning)*
      - FailedMount             # Volume Mount Failed      (warning)*
      - NodeSelectorMismatching # Node Selector Mismatch   (warning)*
      - InsufficientFreeCPU     # Insufficient Free CPU    (warning)*
      - InsufficientFreeMemory  # Insufficient Free Mem    (warning)*
      - OutOfDisk               # Out of Disk              (information)*
      - HostNetworkNotSupported # Host Ntw not Supported   (warning)*
      - NilShaper               # Undefined Shaper         (warning)*
      - Rebooted                # Node Rebooted            (warning)*
      - NodeHasSufficientDisk   # Node Has Sufficient Disk (information)*
      - NodeOutOfDisk           # Node Out of Disk Space   (information)*
      - InvalidDiskCapacity     # Invalid Disk Capacity    (warning)*
      - FreeDiskSpaceFailed     # Free Disk Space Failed   (warning)*
    pod:
      - Pulling           # Pulling Container Image          (information)
      - Pulled            # Ctr Img Pulled                   (information)
      - Failed            # Ctr Img Pull/Create/Start Fail   (warning)*
      - InspectFailed     # Ctr Img Inspect Failed           (warning)*
      - ErrImageNeverPull # Ctr Img NeverPull Policy Violate (warning)*
      - BackOff           # Back Off Ctr Start, Image Pull   (warning)
      - Created           # Container Created                (information)
      - Started           # Container Started                (information)
      - Killing           # Killing Container                (information)*
      - Unhealthy         # Container Unhealthy              (warning)
      - FailedSync        # Pod Sync Failed                  (warning)
      - FailedValidation  # Failed Pod Config Validation     (warning)
      - OutOfDisk         # Out of Disk                      (information)*
      - HostPortConflict  # Host/Port Conflict               (warning)*
    replicationController:
      - SuccessfulCreate    # Pod Created        (information)*
      - FailedCreate        # Pod Create Failed  (warning)*
      - SuccessfulDelete    # Pod Deleted        (information)*
      - FailedDelete        # Pod Delete Failed  (warning)*

Enable/Disable Events Collection with events Parameter

To customize the default events collected for a specific application (by either enabling or disabling events), add an events entry to dragent.yaml as described in the examples below.

An entry in a section in dragent.yaml overrides the entire section in the default configuration.

For example, the Pulling entry below will permit only kubernetes pod Pulling events to be collected and all other kubernetes pod events settings in dragent.default.yaml will be ignored.

However, other kubernetes sections - node and replicationController- remain intact and will be used as specified in dragent.default.yaml.

Example 1: Collect Only Certain Events

Collect only ‘Pulling’ events from Kubernetes for pods:

events:
  kubernetes:
    pod:
       - Pulling

Example 2: Disable All Events in a Section

To disable all events in a section, set the event section to none:

events:
  kubernetes: none
  docker: none

Example 3: Combine Methods

These methods can be combined. For example, disable all kubernetes node and docker image events and limit docker container events to [attach, commit, copy] (components events in other sections will be collected as specified by default):

events:
  kubernetes:
    node: none
  docker:
    image: none
    container:
      - attach
      - commit
      - copy

Note: Format Sequences as List or Single Line

In addition to bulleted lists, sequences can also be specified in a bracketed single line, eg.:

events:
  kubernetes:
    pod: [Pulling, Pulled, Failed]

So, the following two settings are equivalent, permitting only Pulling, Pulled, Failed events for pods to be emitted:

events:
  kubernetes:
    pod: [Pulling, Pulled, Failed]

events:
  kubernetes:
    pod:
      - Pulling
      - Pulled
      - Failed

Change Event Collection by Severity with log Parameter

Events are limited globally at the agent level based on severity, using the log settings in dragent.yaml.

The default setting for the events severity filter is information (only warning and higher severity events are transmitted).

Valid severity levels are: none, emergency, alert, critical, error, warning, notice, information, debug.

Example 1: Block Low-Severity Messages

Block all low-severity messages (notice, information, debug):

log:
  event_priority: warning

Example 2: Block All Event Collection

Block all event collection:

log:
  event_priority: none

For other uses of the log settings see Optional: Change the Agent Log Level.

1.1.2.4.3 -

Include/Exclude Custom Metrics

For more information, see Integrate Applications (Default App Checks).

It is possible to filter custom metrics in the following ways:

Ability to include/exclude custom metrics using configurable patterns,
Ability to log which custom metrics are exceeding limits

After you identify those key custom metrics that must be received, use the new ‘include’ and ‘exclude’ filtering parameters to make sure you receive them before the metrics limit is hit.

Filter Metrics Example

Here is an example configuration entry that would be put into the agent config file: (/opt/draios/etc/dragent.yaml)

metrics_filter:
  - include: test.*
  - exclude: test.*
  - include: haproxy.backend.*
  - exclude: haproxy.*
  - exclude: redis.*

Given the config entry above, this is the action for these metrics:

test.* → send

haproxy.backend.request → send

haproxy.frontend.bytes → drop

redis.keys → drop

The semantic is: whenever the agent is reading metrics, they are filtered according to configured filters and the filtering rule order - the first rule that matches will be applied. Thus since the inclusion item for test.* was listed first it will be followed and that second ‘exclude’ rule for the same exact metric entry will be ignored.

Logging Accepted/Dropped Metrics

Logging is disabled by default. You can enable logging to see which metrics are accepted or dropped by adding the following configuration entry into the dragent.yaml config file:

metrics_excess_log: true

When logging of excess metrics is enabled, logging occurs at INFO-level, every 30 seconds and lasts for 10 seconds. The entries that can be seen in /opt/draios/logs/draios.log will be formatted like this:

+/-[type] [metric included/excluded]: metric.name (filter: +/-[metric.filter])

The first ‘+’ or ‘-’, followed by ‘type’ provides an easy way to quickly scan the list of metrics and spot which are included or excluded ('+' means “included”, ‘-’ means “excluded”).

The second entry specifies metric type (“statsd”, “app_check”, “service_check”, or “jmx”).

A third entry spells out whether “included” or “excluded”, followed by the metric name. Finally, inside the last entry (in parentheses), there is information about filter applied and its effect ('+' or ‘-’, meaning “include” or “exclude”).

With this example filter rule set:

metrics_filter:
  - include: mongo.statsd.net*
  - exclude: mongo.statsd.*

We might see the following INFO-level log entries (timestamps stripped):

-[statsd] metric excluded: mongo.statsd.vsize (filter: -[mongo.statsd.*])
+[statsd] metric included: mongo.statsd.netIn (filter: +[mongo.statsd.net*])

1.1.2.4.4 -

Prioritize Designated Containers

To get the most out of Sysdig Monitor, you may want to customize the way in which container data is prioritized and reported. Use this page to understand the default behavior and sorting rules, and to implement custom behavior when and where you need it. This can help reduce agent and backend load by not monitoring unnecessary containers, or– if encountering backend limits for containers– you can filter to ensure that the important containers are always reported.

Overview

By default, a Sysdig agent will collect metrics from all containers it detects in an environment. When reporting to the Monitor interface, it uses default sorting behavior to prioritize what container information to display first.

Understand Default Behavior

Out of the box, it chooses the containers with the highest

CPU
Memory
File IO
Net IO

and allocates approximately 1/4 of the total limit to each stat type.

Understand Simple Container Filtering

As of agent version 0.86, it is possible set a use_container_filter parameter in the agent config file, tag/label specific containers, and set include/exclude rules to push those containers to the top of the reporting hierarchy.

This is an effective sorting tool when:

You can manually mark each container with an include or exclude tag, AND
The number of includes is small (say, less than 100)

In this case, the containers that explicitly match the include rules will take top priority.

Understand Smart Container Reporting

In some enterprises, the number of containers is too high to tag with simple filtering rules, and/or the include_all group is too large to ensure that the most-desired containers are consistently reported. As of Sysdig agent version 0.91, you can append another parameter to the agent config file, smart_container_reporting.

This is an effective sorting tool when:

The number of containers is large and you can’t or won’t mark each one with include/exclude tags, AND
There are certain containers you would like to always prioritize

This helps ensure that even when there are thousands of containers in an environment, the most-desired containers are consistently reported.

Container filtering and smart container reporting affect the monitoring of all the processes/metrics within a container, including StatsD, JMX, app-checks, and built-in metrics.

Prometheus metrics are attached to processes, rather than containers, and are therefore handled differently.

The container limit is set in dragent.yaml under containers:limit:

Understand Sysdig Aggregated Container

The sydig_aggregated parameter is automatically activated when smart container reporting is enabled, to capture the most-desired metrics from the containers that were excluded by smart filtering and report them under a single entity. It appears like any other container in the Sysdig Monitor UI, with the name “sysdig_aggregated.”

Sysdig_aggregated can report on a wide array of metrics; see Sysdig_aggregated Container Metrics. However, because this is not a regular container, certain limitations apply:

container_id and container_image do not exist.
The aggregated container cannot be segmented by certain metrics that are excluded, such as process.
Some default dashboards associated with the aggregated container may have some empty graphs.

Use Simple Container Filtering

By default, the filtering feature is turned off. It can be enabled by adding the following line to the agent configuration:

use_container_filter: true

When enabled, the agent will follow include/exclude filtering rules based on:

container image
container name
container label
Kubernetes annotation or label

The default behavior in default.dragent.yaml excludes based on a container label (com.sysdig.report) and/or a Kubernetes pod annotation (.sysdig.com/report ).

Container Condition Parameters and Rules

Parameters

The condition parameters are described in the following table:

Pattern name	Description	Example
`container.image`	Matches if the process is running inside a container running the specified image	`- include:` `container.image: luca3m/prometheus-java-app`
`container.name`	Matches if the process is running inside a container with the specified name	`- include:` `container.name: my-java-app`
`container.label.*`	Matches if the process is running in a container that has a Label matching the given value	`- include:` `container.label.class: exporter`
`kubernetes.<object>.annotation.* kubernetes.<object>.label.*`	Matches if the process is attached to a Kubernetes object (Pod, Namespace, etc.) that is marked with the Annotation/Label matching the given value.	`- include:` `kubernetes.pod.annotation.prometheus.io/scrape: true`
all	Matches all. Use as last rule to determine default behavior.	`- include:` `all`

Rules

Once enabled (when use_container_filter: true is set), the agent will follow filtering rules from the container_filter section.

Each rule is an include or exclude rule which can contain one or more conditions.
The first matching rule in the list will determine if the container is included or excluded.
The conditions consist of a key name and a value. If the given key for a container matches the value, the rule will be matched.
If a rule contains multiple conditions they all need to match for the rule to be considered a match.

Default Configuraton

The dragent.default.yaml contains the following default configuration for container filters:

use_container_filter: false

container_filter:
  - include:
      container.label.com.sysdig.report: true
  - exclude:
      container.label.com.sysdig.report: false
  - include:
      kubernetes.pod.annotation.sysdig.com/report: true
  - exclude:
      kubernetes.pod.annotation.sysdig.com/report: false
  - include:
        all

Note that it excludes via a container.label and by a kubernetes.pod.annotation.

The examples on this page show how to edit in the dragent.yaml file directly. Convert the examples to Docker or Helm commands, if applicable for your situation.

Enable Container Filtering in the Agent Config File

Option 1: Use the Default Configuration

To enable container filtering using the default configuration in default.dragent.yaml (above), follow the steps below.

1. Apply Labels and/or Annotations to Designated Containers

To set up, decide which containers should be excluded from automatic monitoring.

Apply the container label .com.sysdig.report and/or the Kubernetes pod annotation sysdig.com/report to the designated containers.

2. Edit the Agent Configuration

Add the following line to dragent.yaml to turn on the default functionality:

use_container_filter: true

Option 2: Define Your Own Rules

You can also edit dragent.yaml to apply your own container filtering rules.

1. Designate Containers

To set up, decide which containers should be excluded from automatic monitoring.

Note the image, name, label, or Kubernetes pod information as appropriate, and build your rule set accordingly.

2. Edit the Agent Configuration

For example:

use_container_filter: true

container_filter:
  - include:
      container.name: my-app
  - include:
      container.label.com.sysdig.report: true
  - exclude:
      kubernetes.namespace.name: kube-system
      container.image: "gcr.io*"
  - include:
      all

The above example shows a container_filter with 3 include rules and 1 exclude rule.

If the container name is “my-app” it will be included.
Likewise, if the container has a label with the key “com.sysdig.report” and with the value “true”.
If neither of those rules is true, and the container is part of a Kubernetes hierarchy within the “kube-system” namespace and the container image starts with “gcr.io”, it will be excluded.
The last rule includes all, so any containers not matching an earlier rule will be monitored and metrics for them will be sent to the backend.

Use Smart Container Reporting

As of Sysdig agent version 0.91, you can add another parameter to the config file: smart_container_reporting = true

This enables several new prioritization checks:

container_filter (you would enable and set include/exclude rules, as described above)
container age
high stats
legacy patterns

The sort is modified with the following rules in priority order:

User-specified containers come before others
Containers reported previously should be reported before those which have never been reported
Containers with higher usage by each of the 4 default stats should come before those with lower usage

Enable Smart Container Reporting and sysdig_aggregated

Set up any simple container filtering rules you need, following either Option 1 or Option 2, above.
Edit the agent configuration:
```
smart_container_reporting: true
```
This turns on both smart_container_reporting and sysdig_aggregated. The changes will be visible in the Sysdig Monitor UI.
See also Sysdig_aggregated Container Metrics..

Logging

When the log level is set to DEBUG, the following messages may be found in the logs:

message	meaning
container <id>, no filter configured	container filtering is not enabled
container <id>, include in report	container is included
container <id>, exclude in report	container is excluded
Not reporting thread <thread-id> in container <id>	Process thread is excluded

1.1.2.4.4.1 -

Sysdig Aggregated Container Metrics

Sysdig_aggregated containers can report on the following metrics:

tcounters
- other
  - time_ns
  - time_percentage
  - count
- io_file
  - time_ns_in
  - time_ns_out
  - time_ns_other
  - time_percentage_in
  - time_percentage_out
  - time_percentage_other
  - count_in
  - count_out
  - count_other
  - bytes_in
  - bytes_out
  - bytes_other
- io_net
  - time_ns_in
  - time_ns_out
  - time_ns_other
  - time_percentage_in
  - time_percentage_out
  - time_percentage_other
  - count_in
  - count_out
  - count_other
  - bytes_in
  - bytes_out
  - bytes_other
- processing
  - time_ns
  - time_percentage
  - count
reqcounters
- other
  - time_ns
  - time_percentage
  - count
- io_file
  - time_ns_in
  - time_ns_out
  - time_ns_other
  - time_percentage_in
  - time_percentage_out
  - time_percentage_other
  - count_in
  - count_out
  - count_other
  - bytes_in
  - bytes_out
  - bytes_other
- io_net
  - time_ns_in
  - time_ns_out
  - time_ns_other
  - time_percentage_in
  - time_percentage_out
  - time_percentage_other
  - count_in
  - count_out
  - count_other
  - bytes_in
  - bytes_out
  - bytes_other
- processing
  - time_ns
  - time_percentage
  - count
max_transaction_counters
- time_ns_in
- time_ns_out
- count_in
- count_out
resource_counters
- connection_queue_usage_pct
- fd_usage_pct
- cpu_pct
- resident_memory_usage_kb
- swap_memory_usage_kb
- major_pagefaults
- minor_pagefaults
- fd_count
- cpu_shares
- memory_limit_kb
- swap_limit_kb
- count_processes
- proc_start_count
- threads_count
syscall_errors
- count
- count_file
- count_file_opened
- count_net
protos
- http
  - server_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
  - client_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
- mysql
  - server_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
  - client_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
- postgres
  - server_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
  - client_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
- mongodb
  - server_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
  - client_totals
    - ncalls
    - time_tot
    - time_max
    - bytes_in
    - bytes_out
    - nerrors
names
transaction_counters
- time_ns_in
- time_ns_out
- count_in
- count_out

1.1.2.4.5 -

Include/Exclude Processes

In addition to filtering data by container, it is also possible to filter independently by process. Broadly speaking, this refinement helps ensure that relevant data is reported while noise is reduced. More specifically, use cases for process filtering may include:

Wanting to alert reliably whenever a given process goes down. The total number of processes can exceed the reporting limit; when that happens, some processes are not reported. In this case, an unreported process could be misinterpreted as being “down.” Specify a filter for 30-40 processes to guarantee that they will always be reported.
Wanting to limit the number of noisy but inessential processes being reported, for example: sed, awk, grep, and similar tools that may be used infrequently.
Wanting to prioritize workload-specific processes, perhaps from integrated applications such as NGINX, Supervisord or PHP-FPM.

Note that you can report on processes and containers independently; the including/excluding of one does not affect the including/excluding of the other.

Prerequisites_Processes

This feature requires the following Sysdig component versions:

Sysdig agent version 0.91 or higher
For on-premises installations: version 3.2.0.2540 or higher

Understand Process Filtering Behavior

By default, processes are reported according to internal criteria such as resource usage (CPU/memory/file and net IO) and container count.

If you choose to enable process filtering, processes in the include list will be given preference over other internal criteria.

Processes are filtered based on a standard priority filter description already used in Sysdig yaml files. It is comprised of -include and -exclude statements which are matched in order, with evaluation ceasing with the first matched statement. Statements are considered matched if EACH of the conditions in the statement is met.

Use Process Filtering

Edit dragent.yaml per the following patterns to implement the filtering you need.

Process Condition Parameters and Rules

The process: condition parameters and rules are described below.

Name	Value	Description
`app_checks_always_send:`	true/false	Legacy config that causes the agent to emit any process with app check. With process filtering, this translates to an extra “include” clause at the head of the process filter which matches a process with any app check, thereby overriding any exclusions. Still subject to limit.
`flush_filter:`		Definition of process filter to be used if flush_filter_enabled == true. Defaults to -include all
`flush_filter_enabled:`	true/false	Defaults to false (default process reporting behavior). Set to true to use the rest of the process filtering options.
`limit:`	N (chosen number)	Defines the approximate limit of processes to emit to the backend, within 10 processes or so. Default is 250 processes.
`top_n_per_container:`	N (chosen number)	Defines how many of the top processes per resource category per emitted container to report after included processes. Still subject to limit. Defaults to 1.
`top_n_per_host:`	N (chosen number)	Defines how many of the top processes per resource category per host are reported before included processes. Still subject to limit. Defaults to 1.

The process: Condition Parameters

Rules

container.image: my_container_image Validates whether the container image associated with the process is a wild card match of the provided image name
container.name: my_container_name Validates whether the container name associated with the process is a wild card match of the provided image name
container.label.XYZ: value Validates whether the label XYZ of the container associated with the process is a wildcard match of the provided value
process.name: my_process_name Validates whether the name of the process is a wild card match of the provided value
process.cmdline: value Checks whether the executable name of a process contains the specified value, or any argument to the process is a wildcard match of the provided value
appcheck.match: value Checks whether the process has any appcheck which is a wildcard match of the given value
all Matches all processes, but does not whitelist them, nor does it blacklist them. If no filter is provided, the default is -include all. However, if a filter is provided and no match is made otherwise, then all unmatched processes will be blacklisted. In most cases, the definition of a process filter should end with -include: all.

Examples

Block All Processes from a Container

Block all processes from a given container. No processes from some_container_name will be reported.

process:
  flush_filter_enabled: true
  flush_filter:
  - exclude:
      container.name: some_container_name
  - include:
      allprocess:   flush_filter: - exclude: container.name: some_container_name - include: all

Prioritize Processes from a Container

Send all processes from a given container at high priority.

process:
  flush_filter_enabled: true
  flush_filter:
    - include:
        container.name: some_container_name
    - include:
        all

Prioritize “java” Processes

Send all processes that contain ‘java" in the name at high priority.

process:
  flush_filter_enabled: true
  flush_filter:
    - include:
        process.name: java
    - include:
        all

Prioritize “java” Processes from a Particular Container

Send processes containing “java” from a given container at high priority.

process:
  flush_filter_enabled: true
  flush_filter:
    - include:
        container.name: some_container_name
        process.name: java
    - include:
        all

Prioritize “java” Processes not in a Particular Container

Send all processes that contain “java” in the name that are not in container some_container_nane.

process:
  flush_filter_enabled: true
  flush_filter:
    - exclude:
        container.name: some_container_name
    - include:
        process.name: java
    - include:
        all

Prioritize “java” Processes even from an Excluded Container

Send all processes containing “java” in the name. If a process does not contain “java” in the name and if the container within which the process runs is named some_container_name, then exclude it.

Note that each include/exclude rule is handled sequentially and hierarchically so that even if the container is excluded, it can still report “java” processes.

flush_filter:
   - flush_filter_enabled: true
   - include:
       process.name: java
   - exclude:
      container.name: some_container_name
   - include:
      all

Prioritize “java” Processes and “sql” Processes from Different Containers

Send Java processes from one container and SQL processes from another at high priority.

process:
  flush_filter:
    - flush_filter_enabled: true
    - include:
       container.name: java_container_name
       process.name: java
    - include
       container.name: sql_container_name
       process.name: sql
    - include
        all

Report ONLY Processes in a Particular Container

Only send processes running in a container with a given label.

process:
  flush_filter:
     - flush_filter_enabled: true
     - include:
 =container.label.report_processes_from_this_container_example_label: true
     - exclude:
         all

1.1.2.4.6 -

Collect Metrics from Remote File Systems

Sysdig agent does not automatically discover and collect metrics from external file systems, such as NFS, by default. To enable collecting these metrics, add the following entry to the dragent.yaml file:

remotefs = true

In addition to the remote file systems, the following mount types are also excluded because they cause high load.

mounts_filter:
  - exclude: "*|autofs|*"
  - exclude: "*|proc|*"
  - exclude: "*|cgroup|*"
  - exclude: "*|subfs|*"
  - exclude: "*|debugfs|*"
  - exclude: "*|devpts|*"
  - exclude: "*|fusectl|*"
  - exclude: "*|mqueue|*"
  - exclude: "*|rpc_pipefs|*"
  - exclude: "*|sysfs|*"
  - exclude: "*|devfs|*"
  - exclude: "*|devtmpfs|*"
  - exclude: "*|kernfs|*"
  - exclude: "*|ignore|*"
  - exclude: "*|rootfs|*"
  - exclude: "*|none|*"
  - exclude: "*|tmpfs|*"
  - exclude: "*|pstore|*"
  - exclude: "*|hugetlbfs|*"
  - exclude: "*|*|/etc/resolv.conf"
  - exclude: "*|*|/etc/hostname"
  - exclude: "*|*|/etc/hosts"
  - exclude: "*|*|/var/lib/rkt/pods/*"
  - exclude: "overlay|*|/opt/stage2/*"
  - exclude: "/dev/mapper/cl-root*|*|/opt/stage2/*"
  - exclude: "*|*|/dev/termination-log*"
  - include: "*|*|/var/lib/docker"
  - exclude: "*|*|/var/lib/docker/*"
  - exclude: "*|*|/var/lib/kubelet/pods/*"
  - exclude: "*|*|/run/secrets"
  - exclude: "*|*|/run/containerd/*"
  - include: "*|*|*"

To include a mount type:

Open the dragent.yaml file.
Remove the corresponding line from the exclude list in the mount_filter.

Add the file mount to the include list under mount_filter .

The format is:

# format of a mount filter is:
# ```
# mounts_filter:
#   - exclude: "device|filesystem|mount_directory"
#   - include: "pattern1|pattern2|pattern3"

For example:

mounts_filter:
  - include: "*|autofs|*"mounts_filter:
  - include: "overlay|*|/opt/stage2/*"
  - include: "/dev/mapper/cl-root*|*|/opt/stage2/*"

Save the configuration changes and restart the agent.

1.1.2.4.7 -

Disable Captures

Sometimes, security requirements dictate that capture functionality should NOT be triggered at all (for example, PCI compliance for payment information).

To disable Captures altogether:

Access using one of the options listed.
This example accesses dragent.yaml directly. ``
Set the parameter:
```
sysdig_capture_enabled: false
```
Restart the agent, using the command: ``
```
service dragent restart
```

See Captures for more information on the feature

1.1.2.5 -

Reduce Memory Consumption in Agent

Sysdig provides a configuration option called thin cointerface to reduce the memory footprint in the agent. When the agent is installed as a Kubernetes daemonset, you can optionally enable the thin cointerface in the sysdig-agent configmap.

Pros

Reduces memory consumption
Particularly useful on very large Kubernetes clusters (>10,000 pods)

Cons

Less frequently used option which is therefore less battle-tested
If a watch is dropped and re-list is required (e.g. in case of a network issue, and apiserver update, etc.), there is no cache to maintain the resources. In this case, the agent must process many additional events.

How It Works

In a typical Kubernetes cluster, two instances of agent daemonset are installed to retrieve the data. They are automatically connected to the Kubernetes API server to retrieve the metadata associated with the entities running on the cluster and sends the global Kubernetes state to the Sysdig backend. Sysdig uses this data to generate kube state metrics.

A delegated agent will not have a higher CPU or memory footprint than a non-delegated agent.

On very large Kubernetes clusters (in the range of 10,000 pods) or clusters with several replication controllers, the agent’s data ingestion can have a significant memory footprint on itself and on the Kubernetes API server. Thin cointerface is provided to reduce this impact.

Enabling this option changes the way the agent communicates with the API server and reduces the need to cache data, which in turn reduces the overall memory usage. Thin cointerface does this by moving some processing from the agent’s cointerface process to the dragent process. This change does not alter the data which is ultimately sent to the backend nor will it impact any Sysdig feature.

The thin cointerface feature is disabled by default.

To Enable:

Add the following in either the sysdig-agent’s configmap or via the dragent.yaml file:

thin_cointerface_enabled: true

1.1.2.6 -

Enable Kube State Metrics

Updated versions of the Sysdig agent can collect HPA, PVS, and other kube state metrics with the parameter k8s_extra_resources.

First, you must edit the agent config file, dragent.yaml, as follows:

k8s_extra_resources:
    include:
      - services
      - resourcequotas
      - persistentvolumes
      - persistentvolumeclaims
      - horizontalpodautoscalers

By default, services and resourcequotas are enabled. Consider that enabling any of the elements will overwrite the default list of elements.

For example, enabling horizontalpodautoscalers will remove metrics about services. This might cause misfunctioning of some Sysdig Monitor and Secure features.

1.1.2.7 -

Process Kubernetes Events

Use Go to Process Kubernetes Events

Required: Sysdig agent version 92.1 or higher.

As of agent version 9.5.0, go_k8s_user_events:true is the default setting. Set to false to use the older, C++-based version.

To streamline Sysdig agent processing times and reduce CPU load, you can use an updated processing engine written in Go.

To do so, edit the following code in dragent.yaml:

go_k8s_user_events: true

Kubernetes Audit Events

The agent listens on /k8s-audit for Kubernetes audit events. Configure the path using the following configuration option:

security:{k8s_audit_server_path_uris: [path1, path2]}

For more information, see Kubernetes Audit Logging.

Working with containerd in K3S

If you have containerd using a custom socket, you can specify this parameter in the agent configuration to correctly capture the containers' metadata:

cri:
  socket_path: /run/k3s/containerd/containerd.sock

1.1.2.8 -

Manage Agent Log Levels

Sysdig allows you to configure file log levels for agents globally and granularly.

1.1.2.8.1 -

Change Agent Log Level Globally

The Sysdig agent generates log entries in /opt/draios/logs/draios.log. The agent will rotate the log file when it reaches 10MB in size, keeping the 10 most recent log files archived with a date-stamp appended to the filename.

The default level (info) creates an entry for each aggregated metrics transmission to the backend servers, once per second, in addition to entries for any warnings and errors.

Setting the value lower than info may prohibit troubleshooting agent-related issues.

The type and amount of logging can be changed by adding parameters and log level arguments shown below to the agent’s user settings configuration file here:

/opt/draios/etc/dragent.yaml

After editing the dragent.yaml file, restart the agent at the shell with: service dragent restart to affect changes.

Note that dragent.yaml code can be written in both YAML and JSON. The examples below use YAML.

File Log Level

When troubleshooting agent behavior, increase the logging to debug for full detail:

log:
  file_priority: debug

log:
  file_priority: error

Container Console Logging

log:
  console_priority: warning

Note that troubleshooting a host with less than the default ‘info’ level will be more difficult or not possible. You should revert to ‘info’ when you are done troubleshooting the agent.

A level of ‘error’ will generate the fewest log entries, a level of ‘trace’ will give the most, ‘info’ is the default if no entry exists.

Example in dragent.yaml

customerid: 831f3-Your-Access-Key-9401
tags: local:sf,acct:eng,svc:websvr
log:
 file_priority: warning
 console_priority: info

customerid: 831f3-Your-Access-Key-9401
tags: local:sf,acct:eng,svc:websvr
log: { file_priority: debug, console_priority: debug }

Docker run command

If you are using the “ADDITIONAL_CONF” parameter to start a Docker containerized agent, you would specify this entry in the Docker run command:

-e ADDITIONAL_CONF=“log:  { file_priority: error, console_priority: none }”
-e ADDITIONAL_CONF="log:\n  file_priority: error\n  console_priority: none"

Kubernetes Infrastructure

When running in a Kubernetes infrastructure (installed using the v1 method, comment in the “ADDITIONAL_CONF” line in the agent sysdig-daemonset.yaml manifest file, and modify as needed:

- name: ADDITIONAL_CONF #OPTIONAL pass additional parameters to the agent
  value: "log:\n file_priority: debug\n console_priority: error"

1.1.2.8.2 -

Manage File Logging for Agent Components

Sysdig Agent provides the ability to set component-wise log levels that override the global file logging level controlled by the file_priority configuration option. The components represent internal software modules and can be found in /opt/draios/logs/draios.log.

By controlling logging at the fine-grained component level, you can avoid excessive logging from certain components in draios.log or enable extra logging from specific components for troubleshooting.

To set component-level logging:

Determine the agent component you want to set the log level:

To do so,

Open the /opt/draios/logs/draios.log file.

Copy the component name.

The format of the log entry is:

<timestamp>, <<pid>.<tid>>, <log level>, <component>[pid]:[line]: <message>

For example, the given snippet from a sample log file shows log messages from sdjagent, mountedfs_reader, watchdog_runnable, protobuf_file_emitter, connection_manager, and dragent.

2020-09-07 17:56:01.173, 27979.28018, Information, sdjagent[27980]: Java classpath: /opt/draios/share/sdjagent.jar
2020-09-07 17:56:01.173, 27979.28018, Information, mountedfs_reader: Starting mounted_fs_reader with pid 27984
2020-09-07 17:56:01.174, 27979.28019, Information, watchdog_runnable:105: connection_manager starting
2020-09-07 17:56:01.174, 27979.28019, Information, protobuf_file_emitter:64: Will save protobufs for all message types
2020-09-07 17:56:01.174, 27979.28019, Information, connection_manager:282: Initiating connection to collector
2020-09-07 17:56:01.175, 27979.27979, Information, dragent:1243: Created Sysdig inspector

Open /opt/draios/etc/dragent.yaml.
Edit the dragent.yaml file and add the desired components:
In this example, you are setting the global level to notice and component log levels for sdjagent, watchdog_runnable, protobuf_file_emitter, and connection_manager.
```
log:
  file_priority: notice
  file_priority_by_component:
    - "connection_manager: debug"
    - "protobuf_file_emitter: notice"
    - "watchdog_runnable: warning"
    - "sdjagent: error"
```
The log levels specified for components override global settings.
Restart the agent.
For example, if you have installed the agent as a service, then run:
```
$ service dragent restart
```

1.1.2.8.3 -

Manage Console Logging for Agent Components

Sysdig Agent provides the ability to set component-wise log levels that override the global console logging level controlled by the console_priority configuration option. The components represent internal software modules and can be found in /opt/draios/logs/draios.log.

To set component-level logging:

Determine the agent component you want to set the log level:

To do so,

Look at the console output.
If you’re using an orchestrator like Kubernetes, the log viewer facility, such as the kubectl log command, shows the console log output.

Copy the component name.

The format of the log entry is:

<timestamp>, <<pid>.<tid>>, <log level>, <component>[pid]:[line]: <message>

For example, the given snippet from a sample log file shows log messages from sdjagent, mountedfs_reader, watchdog_runnable, protobuf_file_emitter, connection_manager, and dragent.

2020-09-07 17:56:01.173, 27979.28018, Information, sdjagent[27980]: Java classpath: /opt/draios/share/sdjagent.jar
2020-09-07 17:56:01.173, 27979.28018, Information, mountedfs_reader: Starting mounted_fs_reader with pid 27984
2020-09-07 17:56:01.174, 27979.28019, Information, watchdog_runnable:105: connection_manager starting
2020-09-07 17:56:01.174, 27979.28019, Information, protobuf_file_emitter:64: Will save protobufs for all message types
2020-09-07 17:56:01.174, 27979.28019, Information, connection_manager:282: Initiating connection to collector
2020-09-07 17:56:01.175, 27979.27979, Information, dragent:1243: Created Sysdig inspector

Open /opt/draios/etc/dragent.yaml.
Edit the dragent.yaml file and add the desired components:
In this example, you are setting the global level to notice and component log levels for sdjagent, watchdog_runnable, protobuf_file_emitter, and connection_manager.
```
log:
  console_priority: notice
  console_priority_by_component:
    - "connection_manager: debug"
    - "protobuf_file_emitter: notice"
    - "watchdog_runnable: warning"
    - "sdjagent: error"
```
The log levels specified for components override global settings.
Restart the agent.
For example, if you have installed the agent as a service, then run:
```
$ service dragent restart
```

1.1.2.9 -

Agent Auto-Config

Introduction

If you want to maintain centralized control over the configuration of your Sysdig agents, one of the following approaches is typically ideal:

Via an orchestration system, such as using Kubernetes or Mesos/Marathon.
Using a configuration management system, such as Chef or Ansible.

However, if these approaches are not viable for your environment, or to further augment your Agent configurations via central control, Sysdig Monitor provides an Auto-Config option for agents. The feature allows you to upload fragments of YAML configuration to Sysdig Monitor that will be automatically pushed and applied to some/all of your Agents based on your requirements.

Enable Agent Auto-Config

Independent of the Auto-Config feature, typical Agent configuration lives in /opt/draios/etc and is derived from a combination of base config in the dragent.default.yaml file and any overrides that may be present in dragent.yaml. See also Understanding the Agent Config Files.

Agent Auto-Config adds a middle layer of possible overrides in an additional file dragent.auto.yaml.When present, the the order of config application from highest precedence to lowest now becomes:

dragent.yaml
dragent.auto.yaml
dragent.default.yaml

While all Agents are by default prepared to receive and make use of Auto-Config data, the file dragent.auto.yaml will not be present on an Agent until you’ve pushed central Auto-Config data to be applied to that Agent.

Auto-Config settings are performed via Sysdig Monitor’s REST API. Simplified examples are available that use the Python client library to get or set current Auto-Config settings. Detailed examples using the REST API are shown below.

The REST endpoint for Auto-Config is /api/agents/config. Use the GET method to review the current configuration. The following example shows the initial empty settings that result in no dragent.auto.yaml files being present on your Agents.

curl -X GET \
       --header "Authorization: Bearer xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" \
       https://app.sysdigcloud.com/api/agents/config


Output:
{
    "files": []
}

Use the PUT method to centrally push YAML that will be distributed and applied to your Agents as dragent.auto.yaml files. The content parameter must contain syntactically-correct YAML. The filter option is used to specify if the config should be sent to one agent or all of them, such as in this example to globally enable Debug logging on all Agents:

curl -X PUT \
       --header "Content-Type: application/json" \
       --header "Authorization: Bearer xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" \
       https://app.sysdigcloud.com/api/agents/config -d '
{
  "files": [
    {
      "filter": "*",
      "content": "log:\n  console_priority: debug"
    }
  ]
}'

Alternatively, the filter can specify a hardware MAC address for a single Agent that should receive a certain YAML config. All MAC-specific configs should appear at the top of the JSON object and are not additive to any global Auto-Config specified with “filter”: “*" at the bottom. For example, when the following config is applied, the one Agent that has the MySQL app check configured would not have Debug logging enabled, but all others would.

curl -X PUT \
       --header "Content-Type: application/json" \
       --header "Authorization: Bearer xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" \
       https://app.sysdigcloud.com/api/agents/config -d '
{
  "files": [
    {
      "filter": "host.mac = \"08:00:27:de:5b:b9\"",
      "content": "app_checks:\n  - name: mysql\n    pattern:\n      comm: mysqld\n    conf:\n      server: 127.0.0.1\n      user: sysdig-cloud\n      pass: sysdig-cloud-password"
    },
    {
      "filter": "*",
      "content": "log:\n  console_priority: debug"
    }
  ]
}'

To update the active central Auto-Config settings, simply PUT a complete replacement JSON object.

All connected Agents will receive centrally-pushed Auto-Config updates that apply to them based on the filter settings. Any Agent whose Auto-Config is enabled/disabled/changed based on the centrally-pushed settings will immediately restart, putting the new configuration into effect. Any central Auto-Config settings that would result in a particular Agent’s Auto-Config remaining the same will not trigger a restart.

Disable Agent Auto-Config

To clear all Agent Auto-Configs, use the PUTmethod to upload the original blank config setting of '{ “files”: [] }'.

It is also possible to override active Auto-Config on an individual Agent. To do so, follow these steps for your Agent:

Add the following config directly to the dragent.yaml file: auto_config: false.
Delete the file /opt/draios/etc/dragent.auto.yaml.
Restart the Agent.

For such an Agent to opt-in to Auto-Config again, remove auto_config: false from the dragent.yaml and restart the Agent.

Restrictions

To prevent the possibility of pushing Auto-Config that would damage an Agent’s ability to connect, the following keys will not be accepted in the centrally-pushed YAML.

auto_config
customerid
collector
collector_port
ssl
ssl_verify_certificate
ca_certificate
compression

1.1.2.10 -

Using the Agent Console

Sysdig provides an Agent Console to interact with the Sysdig agent. This is a troubleshooting tool to help you view configuration files and investigate agent configuration problems quickly.

Access Agent Console

From Explore click the Groupings drop-down.
Select Hosts & Container or Nodes.
Click the desired host to investigate the corresponding agent configuration.
Click Options (three dots) on the right upper corner of the Explore tab.
Click Agent Console.

Agent Console Commands

View Help

The ? command displays the commands to manage Prometheus configuration and targets monitored by the Sysdig agent.

$ prometheus ?
$ prometheus config ?
$ prometheus config show ?

Command Syntax

The syntax of the Agent Console commands is as follows:

directory command
directory sub-directory command
directory sub-directory sub-sub-directory command

View Version

Run the following to find the version of the agent running in your environment:

$ version

An example output:

12.0.0

Troubleshoot Prometheus Metrics Collection

These commands help troubleshoot Prometheus targets configured in your environment.

For example, the following commands display and scrape the Prometheus endpoints respectively.

$ prometheus target show
$ prometheus target scrape

Sub-Directory Commands

The Promscrape CLI consists of the following sections.

config: Manages Sysdig agent-specific Prometheus configuration.
metadata: Manages metadata associated with the Prometheus targets monitored by the Sysdig agent.
stats: Helps view the global- and job-specific Prometheus statistics.
target: Manages Prometheus endpoints monitored by Sysdig agent.

Prometheus Commands

Show

The show command displays the information about the subsection. For example, the following example displays the configuration of the Prometheus server.

$ prometheus config show

5  Configuration      Value
6  Enabled            True
7  Target discovery   Prometheus service discovery
8  Scraper            Promscrape v2
9  Ingest raw         True
10  Ingest calculated  True
11  Metric limit       2000

Scrape

The scrape command scrapes a Prometheus target and displays the information. The syntax is:

$ prometheus target scrape -url <URL>

For example:

$ prometheus target scrape -url http://99.99.99.3:10055/metrics

# HELP go_gc_duration_seconds A summary of the GC invocation durations.
7  # TYPE go_gc_duration_seconds summary
8  go_gc_duration_seconds{quantile="0"} 7.5018e-05
9  go_gc_duration_seconds{quantile="0.25"} 0.000118155
10  go_gc_duration_seconds{quantile="0.5"} 0.000141586
11  go_gc_duration_seconds{quantile="0.75"} 0.000171626
12  go_gc_duration_seconds{quantile="1"} 0.00945638
13  go_gc_duration_seconds_sum 0.114420898
14  go_gc_duration_seconds_count 607

View Agent Configuration

The Agent configuration commands have a different syntax.

Run the following to view the configuration of the agent running in your environment:

$ configuration show-dragent-yaml
$ configuration show-configmap-yaml
$ configuration show-default-yaml
$ configuration show-backend-yaml

The output displays the configuration file. Sensitive data, such as the credentials, are obfuscated.

customerid: "********"
watchdog:
  max_memory_usage_mb: 2048

Security Considerations

User-sensitive configuration is obfuscated and not visible through the CLI.
All the information is read-only. You cannot currently change any configuration by using the Agent console.
Runs completely inside the agent. It does not use bash or any other Linux terminals to prevent the risk of command injection.
Runs only via a TLS connection with the Sysdig backend.

Disable Agent Console

This is currently turned on by default. To turn off Agent Console for a particular team:

Navigate to Settings > Teams.
Select the team that you want to disable Agent Console for.
From Additional Permissions, Deselect Agent CLI.
Click Save.

To turn it off in your environment, edit the following in the dragent.yaml file:

command_line:
  enabled: false

1.1.3 -

Agent Upgrade

The steps to upgrade an agent differ depending on whether the agent was originally installed as a Docker container or as a service.

This section describes how to check the current version of the installed agents, and then how to upgrade them.

It is highly recommended to follow upgrade best practices:

Keep upgrades current
Upgrade progressively without skipping versions, and
Test upgrades in a non-mission-critical or staging environment before rolling in to production.

Agent Version Check

Container/Docker Installation

If the agent is installed as container, run a command similar to:

docker exec sysdig-agent /opt/draios/bin/dragent --version

Service Installation

If the agent is installed as a service, run:

/opt/draios/bin/dragent --version

The agent version can also be found in the agent log file: /opt/draios/logs/draios.log.

Look for the “Agent starting” message, which is logged whenever the agent restarts.

Update Agent

Update the containerized agent version as you normally update any container; the basic steps are below.

Use the full run command as shown in the Settings > Agent Installation tab of your account. CoreOS users can use the fleet script also shown on the Agent Installation tab.

Containerized Agent

To see which agent versions are available see this link.

Docker

Basic Steps: stop the agent, remove it, pull the new agent, and install it.

The exact Docker command can also be found in the Sysdig Settings > Agent Installation menu.

docker stop sysdig-agent
docker rm sysdig-agent
docker pull sysdig/agent
docker run . . .

Kubernetes

Check whether .yaml files must be updated:

Updating the agent image does not overwrite the daemonset.yaml and sysdig-agent-configmap.yaml on your local system. Check the Sysdig Agent Release Notes to see if you need to download the latest .yaml files from GitHub.

Perform update:

kubectl set image ds/sysdig-agent sysdig-agent=sysdig/agent:<TAG>

Watch update status:

kubectl rollout status ds/sysdig-agent

Service Agent

For service (non-containerized) agent installations, updates are installed as part of the normal system upgrade available with `apt-get` or `yum`.

Debian, Ubuntu

apt-get update
apt-get -y install draios-agent

CentOS, RHEL, Fedora, Amazon AMI, Amazon Linux 2

yum clean expire-cache
yum -y install draios-agent

1.1.4 -

Uninstall the Agent

This section describes uninstalling the Sysdig agent when it was installed as a service.

If the agent was installed as a container, remove it using standard container commands.

If the agent was installed by an orchestrator, such as Kubernetes, remove it by using the standard orchestrator commands.

Debian/Ubuntu Distributions

To uninstall the agent from Debian Linux distributions, including Ubuntu:

As the sudo user, run the following command in a terminal on each host:

sudo apt-get remove draios-agent

Fedora/CentOS/RHEL/Amazon AMI/ Amazon Linux 2 Distributions

To uninstall the agent from Fedora Linux distributions, including CentOS, Red Hat Enterprise Linux, as well as Amazon AMI and Amazon Linux 2:

As the sudo user, run the following command in a terminal on each host:

sudo yum erase draios-agent

1.2 -

Serverless Agents

Overview

The serverless environment: As cloud platforms have evolved, both the convenience and the abstraction levels have increased simultaneously and new agent models are required.

For example, with Amazon’s ECS and EKS, users remain in charge of managing the underlying virtual host machines. In environments like Fargate, however, the hosts are implicitly allocated by the cloud provider and users simply run their containers without allocating, configuring, or having any knowledge of the underlying compute infrastructure.

While this “container as a service” model is convenient, it can introduce risk, as many users leave the containers unattended and don’t monitor for security events inside them that can exfiltrate secrets, compromise business data, and increase their AWS/cloud provider costs. In addition, it is not possible to install a standard agent in an environment where you do not have access to a host.

For these reasons, Sysdig has introduced a new “serverless agent” that can be deployed in such container-based cloud environments.

Available Platforms

AWS Fargate ECS
Additional platforms coming soon

1.2.1 -

AWS Fargate Serverless Agents

Check the Overview for an explanation of when and why to use serverless agents in “container-as-a-service” cloud environments.

Architecture

The Sysdig serverless agent provides runtime detection through policy enforcement with Falco. At this time, the serverless agent is available for AWS Fargate on ECS. It is comprised of an orchestrator agent and (potentially multiple) workload agents.

The Sysdig serverless orchestrator agent is a collection point installed on each VPC to collect data from the serverless workload agent(s) and to forward them to the Sysdig backend. It also syncs the Falco runtime policies and rules to the workload agent(s) from the Sysdig backend.
The Sysdig serverless workload agent is installed in each task and requires network access to communicate with the orchestrator agent.

Installation: For Fargate ECS

For Fargate ECS, the two components of the serverless agent are installed separately.

For the orchestrator agent, Sysdig provides a yaml to use as a CloudFormation Template which you can deploy through the AWS Console. You need one orchestrator deployment per VPC in your environment which your organization wants to secure.
For the workload agents, you need one workload agent per Fargate task definition. (If you have ten services and ten task definitions, each needs to be instrumented.)
We assume your services use an existing CFT and you will install the workload agent using an automated process which will instrument all the task definitions in your CFT.

Prerequisites

On the AWS side:

AWS CLI configured and permissions to create and use an S3 bucket.
Permissions to upload images to repos, deploy CloudFormation Templates (CFTs), and create task definitions for Fargate
The Fargate tasks you want to instrument with the Sysdig serverless agent
Two subnets that can connect with the internet. (Your service on Fargate must reach the orchestrator agent, and the orchestrator agent must reach the internet to communicate with Sysdig’s back end.)
A NAT gateway, or, if AWS Internet Gateway is used, you will need to uncomment the line AssignPublicIp: ENABLED in the orchestrator.yaml after installing the orchestrator agent.

On the Sysdig side:

Sysdig Secure
Sysdig agent key
Endpoint for the Sysdig collector for your region; check SaaS Regions and IP Ranges for the endpoint to use.

Install the Orchestrator Agent

Obtain the Sysdig Orchestrator Agent yaml to be used as the CloudFormation Template source.
For more information on CloudFormation (CFN), see AWS documentation.
Deploy the orchestrator agent for each desired VPC, using CloudFormation.
The steps below are an outline of the important Sysdig-related parts.
1. Log in to the AWS Console. Select CloudFormation and Create Stack with new resources and specify the orchestrator-agent.yaml as the Template source.
2. Specify the stack details to deploy the orchestrator agent on the same VPC where your service is running.
  Stack name: self-defined
  Sysdig Settings
  - Sysdig Access Key: Use the agent key for your Sysdig platform.
  - Sysdig Collector Host: collector.sysdigcloud.com (default); region-dependent in Sysdig SaaS; custom in Sysdig on-prem.
  - Sysdig Collector Port: 6443 (default), or could be custom for on-prem installations.
  Network Settings
  - VPC Id Choose your VPC.
  - Subnet A & B: These depend on the VPC you choose; select from the drop-down menu
  Advanced Settings
  - Sysdig Agent Tags: Enter a comma-separated list of tags (eg. role:webserver,location:europe)Note: tags will also be created automatically from your infrastructure’s metadata, including AWS, Docker, etc.
  - Sysdig Orchestrator Agent Image:
    quay-io/orchestrator-agent:latest (default)
  - Check Collector SSL Certificate: Default: true. False means no validation will be done on the SSL certificate received from the collector, used for dev purposes only.
3. Click Next, complete the stack creation, and wait for the deployment to complete (usually less than 10 minutes.)
4. In Output, take note of the OrchestratorHost and OrchestratorPort values.
  As of Serverless Agent Version 2.2.0, the orchestrator port can be configured via either SYSDIG_ORCHESTRATOR_PORT (default 6667, which is hardcoded into other parts of the CFT) or the SysdigOrchestratorAgentPort (alternate) parameter in the CloudFormation template.

If AWS Internet Gateway is used (as opposed to a NAT Gateway), uncomment the line AssignPublicIp: ENABLED in the orchestrator.yaml.

Install the Workload Agents

Option 1: Automated Process (Kilt)

Prerequisite: Have the orchestrator agent deployed in the appropriate VPC and have the Orchestrator Host and Port information handy.
Download the appropriate installer for your OS.
These set up Kilt, an open-source library mechanism for injection into Fargate containers.
Create a macro for the serverless worker agents, using the installer. Any service tagged with this macro will have the serverless worker agent(s) added and Fargate data will be collected.
1. Log in to AWS CLI.
2. Create a CFN macro that applies instrumentation. You will need the outputs from previous task. Example:
```
./installer-linux-amd64 cfn-macro install -r us-east-1 MySysdigMacro $OrchestratorHost $OrchestratorPort
```
Add the macro you created to the CFT that you use for your own service at the root.
Use: Transform: MySysdigMacro.
All new deployments of that template will be instrumented.
Defining Entrypoint and Command in CFT:
In this step, the macro will go over the original CloudFormation Template looking for ContainerDefinitions to instrument. This includes replacing the original entry point for the Sysdig entry point, so ideally the CFT should explicitly describe Entrypoint and Command. Otherwise, the macro will try to pull the image to retrieve a default Entrypoint and Command , which only works if the image is publicly available. Otherwise the pull will fail and the instrumentation will not be completed.
Complete!
When instrumentation is complete, Fargate events should be visible in the Sysdig Secure Events feed.

Option 2: Automated Process Terraform

Alternatively, you can use the Sysdig Provider on the Terraform registry to deploy the workload agents, as described below.

Prerequisite: Have the orchestrator agent deployed in the appropriate VPC and have the Orchestrator Host and Port information handy.

Set up the Sysdig Terraform provider:

terraform {
  required_providers {
    sysdig = {
      source = "sysdiglabs/sysdig"
      version = ">= 0.4.0"
    }
  }
}
provider "sysdig" {
  sysdig_secure_api_token = var.sysdig_access_key
}

Pass the container-definition, orchestrator host, and orchestrator port to the sysdig_fargate_workload_agent data source.
Notes:
- The input container definitions must be in JSON format, following CloudFormation naming conventions.
- When deploying the orchestrator agent with CFT, the port is always 6440.
- When deploying the orchestrator agent with CFT, the host is the collector endpoint for your Sysdig region; e.g. "collector-static.sysdigcloud.com" for US East. For other regions, see SaaS Regions and IP Ranges.
```
data "sysdig_fargate_workload_agent" "instrumented" {
  container_definitions = file("container-definitions/hello.json")
     
  sysdig_access_key = var.sysdig_access_key
     
  workload_agent_image = "quay.io/sysdig/workload-agent:latest"
     
  orchestrator_host = module.sysdig_orchestrator_agent.orchestrator_host
  orchestrator_port = module.sysdig_orchestrator_agent.orchestrator_port
}
```

Include the instrumented JSON in your Fargate task definition:

resource "aws_ecs_task_definition" "fargate_task" {
  ...
   
  network_mode = "awsvpc"
  requires_compatibilities = ["FARGATE"]
   
  container_definitions = "${data.sysdig_fargate_workload_agent.instrumented.output_container_definitions}"
}

Complete!
When instrumentation is complete, Fargate events should be visible in the Sysdig Secure Events feed.

Option 3: Manual Install

Review the installation overview and prerequisites.
Install the Orchestrator Agent, as described above. Take note of the OrchestratorHost and OrchestratorPort values.
Instrument the CloudFormation task definition to deploy the workload agent manually.

Instrument Task Definition to Deploy Workload Agents

Add a new container to your existing CloudFormation task definition.
- For the example, we will use SysdigInstrumentation for the image name.
- The entrypoint/command fields can be left empty
- Obtain the image from quay.io/sysdig/workload-agent:latest
Edit the containers you want to instrument.
- Mount volumes from SysdigInstrumentation
- Add SYS_PTRACE capability to your container. See AWS Documentation for detail if needed.
- Set /opt/draios/bin/instrument as the entry point of your container.
  Note: if you are using the entry point in your image, then prepend it: /opt/draios/bin/instrument,your,original,entry,point
Configure the Sysdig instrumentation through environment variables.

Example Task Definitions

CloudFormation Task Before Instrumentation `

TestApp:
  Type: AWS::ECS::TaskDefinition
  Properties:
    NetworkMode: awsvpc
    RequiresCompatibilities:
      - FARGATE
    Cpu: 2048
    Memory: 8GB
    ExecutionRoleArn: !Ref TestAppExecutionRole
    TaskRoleArn: !Ref TestAppTaskRole
    ContainerDefinitions:
      - Name: App
        Image: !Ref TestAppImage
        EntryPoint:
        - /bin/ping
        - google.com
    Tags:
      - Key: application
        Value: TestApp

Task After Manual Instrumentation

TestApp:
  Type: AWS::ECS::TaskDefinition
  Properties:
    NetworkMode: awsvpc
    RequiresCompatibilities:
      - FARGATE
    Cpu: 2048
    Memory: 8GB
    ExecutionRoleArn: !Ref TestAppExecutionRole
    TaskRoleArn: !Ref TestAppTaskRole
    ContainerDefinitions:
      - Name: App
        Image: !Ref TestAppImage
        EntryPoint:
### Added for manual instrumentation
        - /opt/draios/bin/instrument
### End added propertis
        - /bin/ping
        - google.com
### Added for manual instrumentation
        LinuxParameters:
          Capabilities:
            Add:
            - SYS_PTRACE
        VolumesFrom:
        - SourceContainer: SysdigInstrumentation
          ReadOnly: true
        Environment:
        - Name: SYSDIG_ORCHESTRATOR # Required
          Value: <host orchestrator output, region dependent>
        - Name: SYSDIG_ORCHESTRATOR_PORT # Required
          Value: "6667"
        - Name: SYSDIG_ACCESS_KEY
          Value: ""
        - Name: SYSDIG_COLLECTOR
          Value: ""
        - Name: SYSDIG_COLLECTOR_PORT
          Value: ""
        - Name: SYSDIG_LOGGING
          Value: ""
      - Name: SysdigInstrumentation
        Image: !Ref WorkloadAgentImage
### End of added properties
    Tags:
      - Key: application
        Value: TestApp

Environment Variables for Workload Agents

The most commonly used variables include:

SYSDIG_ORCHESTRATOR: (required) use the output OrchestratorHost value
SYSDIG_ORCHESTRATOR_PORT: (required) use the output from OrchestratorPort
SYSDIG_EXTRA_ARGS: use to pass any extra arguments needed to the instrumentation
SYSDIG_EXTRA_CONF: use to add any extra configuration needed to the instrumentation
SYSDIG_LOGGING: add this variable and set to debug if needed. Any other value will print minimal logs at startup. See also: Manage Serverless Agent Logs

Upgrade: For Fargate ECS

To upgrade the serverless agents, you install a second version of both components then kill all the running tasks and restart with the new version. You then delete and clean up the old CloudFormation and Kilt residuals.

Obtain the Sysdig Orchestrator Agent yaml to be used as the CloudFormation Template source.
At this time, the yaml metadata does not specify the agent version, but this link always downloads the latest file.
Perform the steps to Install the Orchestrator Agent.
Note that in step 2.4, the OrchestratorHost and OrchestratorPort values will be unique.
Perform the steps to Install the Workload Agents (if using Kilt. Workload agents installed with Terraform are automatically updated to _latest.)
In step 4, assign a unique name to your macro (Transform: MyV2SysdigMacro).
You now have two versions of the serverless agent components. When you are ready to switch from the earlier version, proceed with the next step.
Stop all running tasks and use CloudFormataion to delete the earlier stack. Redeploy the new stack with the updated CFT. (Transform: MyV2SysdigMacro)
Clean up macros: Delete the previous kilt macro using the installer:
```
./installer-linux-amd64 cfn-macro delete MySysdigMacro
```

1.2.1.1 -

Manage Serverless Agent Logs

As of serverless agent version 2.2.0, task logs and instrumentation logs are handled separately.

Task logs continue to be with whatever log setup you have on your task container.
Instrumentation logs go to a separate log group created by the macro installer:
<macro_name>-SysdigServerlessLogGroup-<uid>.
At this time, the log group name cannot be edited. By default, the logging level is set to info.

Logging Environment Variables

Set Logging Level

SYSDIG_LOGGING is used for instrumentation log level. Default = info. The full list of options are: none | critical| error | warning |notice | info | debug | trace

Log Forwarding

SYSDIG_ENABLE_LOG_FORWARD Default = true
SYSDIG_LOG_LISTEN_PORT Default = 32000
SYSDIG_LOG_FORWARD_ADDR Default = localhost:32000. Where the logwriter of the workload agent listens. Can be set to any other TCP endpoint.

Advanced Configurations

SYSDIG_BUFFER_SIZE: Receiving buffer size in bytes for the TCP listener, which can be increased. Default 1024.
SYSDIG_DEADLINE_SECONDS: Connection deadline, which can be increased if clients are taking longer to connect. Default 3.

See other serverless agent environment variables here.

1.3 -

Prometheus Remote Write

You can collect Prometheus metrics from environments where the Sysdig agent is not available. Sysdig uses the remote_write capabilities to help you do so.

In Sysdig terminology, the remote endpoints that can read Prometheus metrics are known as Prometheus Remote Write. Prometheus Remote Write does not require the Sysdig agent to be installed in the Prometheus environment. This facility expands your monitoring capabilities beyond Kubernetes and regular Linux kernels to environments where the Sysdig agent cannot be installed.

Prometheus Remote Write can collect metrics from:

An existing Prometheus server
Additional environments:
- Windows
- Managed Cloud Environments, such as AWS and IBM
- Fargate
- IoT

Use Sysdig agent in environments where an agent can be installed. However, use the Prometheus Remote Write to collect metrics from ephemeral or batch jobs that may not exist long enough to be scraped by the agent.

With the Prometheus Remote Write, you can either monitor metrics through the Monitor UI or you can use PromQL to query the data by using the standard Prometheus query language.

Enable Prometheus Remote Write

Contact your Sysdig representative to enable Prometheus Remote Write in your environment.

Endpoints and Regions

Prometheus Remote Write resides in the ingest endpoints for each region under /prometheus/remote/write. The public Prometheus Remote Write endpoints for each region are listed below:

Region	Endpoints
US East	https://api.sysdigcloud.com/prometheus/remote/write
US West	https://us2.app.sysdig.com/prometheus/remote/write
European Union	https://eu1.app.sysdig.com/prometheus/remote/write
Asia Pacific (Sydney)	https://app.au1.sysdig.com/prometheus/remote/write

Configure Remote Write in Prometheus Server

You need to configure remote_write in your Prometheus server in order to send metrics to Sysdig Prometheus Remote Write.

The configuration of your Prometheus server depends on your installation. In general, you configure the remote_write section in the prometheus.yml configuration file:

global:
  external_labels:
    [ <labelname>: <labelvalue> ... ]
remote_write:
    - url: "https://<region-url>/prometheus/remote/write"
      bearer_token: "<your API Token>"
      tls_config:
        insecure_skip_verify: true

The communication between your Prometheus server and Prometheus Remote Write should use the authorization header with the Sysdig API key (not the agent access key) as the bearer token.

Alternatively, you can also use the bearer_token_file entry to refer to a file instead of directly including the API token.

Prometheus does not reveal the bearer_token value on the UI.

Customize Metrics

To enable customization, Sysdig provides additional options to control which metrics you want to send to Prometheus Remote Write.

Manage Metrics

Prometheus Remote Write by default sends all the metrics to Sysdig Prometheus Remote Write. These metrics are sent with a remote_write: true label appended to it so you can easily identify them.

Label Metrics

You can specify custom label-value pairs and send them with each time series to the Prometheus Remote Write. Use the external_labels block in the global section in the Prometheus configuration file. This is similar to setting an agent tag and allowing you to filter or scope the metrics in Sysdig Monitor.

For example, if you have two Prometheus servers configured to remote write to Prometheus Remote Write, you can include an external label to identify them easily:

Prometheus 1

global:
  external_labels:
    provider: prometheus1
remote_write:
- url: ...

Prometheus 2

global:
  external_labels:
    provider: prometheus2
remote_write:
- url: ...

Filter Metrics

With the general configuration, all the metrics are by default remotely written to Prometheus Remote Write. You can control the metrics that you collect and send to Sysdig. To select which series and labels to collect, drop, or replace, and reduce the number of active series that are sent to Sysdig, you can set up relabel configurations by using the write_relabel_configs block within your remote_write section.

For example, you can send metrics from one specific namespace called myapp-ns as give below:

remote_write:
- url: https://<region-url>/prometheus/remote/write
  bearer_token_file: /etc/secrets/sysdig-api-token
  write_relabel_configs:
  - source_labels: [__meta_kubernetes_namespace]
    regex: ‘myapp-ns’
    action: keep

Rate Limit

The default limits are configured set for each user and can be raised as required. The defaults are good for most users, and the limits help protect against any misconfigurations.

Feature	Limit
Parallel writes	100 concurrent requests. This doesn’t necessarily mean 100 Prometheus servers because the time at which the data is written is distributed.
Data points per minute	One million. The number of data points sent depends on how often metrics are submitted to Sysdig. A scrape interval of 10s will submit more DPM than an interval of 60s.
Number of writes per minute	10,000

Parallel writes

100 concurrent requests.

This doesn’t necessarily mean 100 Prometheus servers because the time at which the data is written is distributed.

Data points per minute

One million.

The number of data points sent depends on how often metrics are submitted to Sysdig. A scrape interval of 10s will submit more DPM than an interval of 60s.

Number of writes per minute

10,000

Team Scoping

It is possible to scope a Sysdig Team to only access metrics matching certain labels sent via Prometheus remote write. See Manage Teams and Roles

Limitations

Metrics sent to Prometheus Remote Write can be accessed in Explore, but they are not compatible with the scope tree.
Label enrichment is unavailable at this point. Only labels collected at the source can be used. You should add additional labels to perform further scoping or pivoting in Sysdig.
Currently, Sysdig Dashboards do not support mixing metrics with different sampling. For example, 10 seconds and 1-minute samples. For optimal experience, configure the scrape interval to be 10s to combine remote write metrics with agent metrics.
Remote write functionality does not support sending metric metadata. Upstream Prometheus recently added support for propagation of metadata (metric type, unit, description, info) and this functionality will be supported in a future update to Prometheus Remote Write.
- Suffix the metric name with _total, _sum , or _count to store them as a counter. Otherwise, the metrics will be handled as a gauge.
- Units can be set in Dashboards manually.

Learn More

1.4 -

Sysdig Secure for cloud

Sysdig Secure for cloud is the software that connects Sysdig Secure features to your cloud environments to provide unified threat detection, compliance, forensics, and analysis.

Because modern cloud applications are no longer just virtualized compute resources, but a superset of cloud services on which businesses depend, controlling the security of your cloud accounts is essential. Errors can expose an organization to risks that could bring resources down, infiltrate workloads, exfiltrate secrets, create unseen assets, or otherwise compromise the business or reputation. As the number of cloud services and configurations available grows exponentially, using a cloud security platform protects against having an unseen misconfiguration turn into a serious security issue.

Check Sysdig Secure - Secure for cloud to review the features provided per cloud.

Multiple Installation Options

Sysdig Secure for cloud is available on a variety of cloud providers, with simplified installation instructions available from the Get Started screen.

Terraform based

Supported cloud providers at this time are:

CloudFormationTemplate based

Helm Chart based (feature limited)

This installation type is limited to threat detection and image scaning features ONLY for AWS, GCP, and Azure providers

Both Terraform and Cloudformation based deployments use an underlying component of Sysdig Secure for Cloud called “Cloud Connector”. This component is also provided vía Sysdig Helm Charts - Cloud-Connector

Cloud Account Limits

For trial users, the default config Sysdig Secure for cloud can audit a maximum of 50 cloud accounts. If this limit needs to be increased, please contact your account team. If you exceed the license purchased, Sysdig will not block cloud connection or stop the service and the account team will reach out to you.

1.4.1 -

Deploy Sysdig Secure for cloud on AWS

If needed, review the offering description on Sysdig Secure for cloud - AWS

Deployment Options

All following options provide all four cloud features: threat detection, CSPM benchmarks, and image and container registry scanning

Terraform-based for two types of AWS account
- Organizational/management account: This is the account that you use to create the organization in AWS. Organizational accounts create and contain member accounts.
- Single/member account: Each of these is a stand-alone account. A member account that is part of an organization is supported too.
CloudFormation Template (CFT)-based: This option requires explicit creation of an AWS role, which is prompted by the onboarding wizard.

Onboarding Using Terraform

Terraform-based install instructions differ depending on what type of AWS account you are using.

At this time, the options include:

Install for a single AWS account
Install for an organizational/management account

For Single/Member Account

The default code provided in the Get Started page of Sysdig Secure is pre-populated with your Secure API token and will automatically install threat detection with CloudTrail, AWS benchmarks, and container registry and image scanning.

Prerequisites

Have Terraform installed on the local machine.
A Sysdig Secure SaaS account

Permissions

Sysdig Secure administrator permissions
AWS profile credentials configuration
For AWS, you must have administrator permissions, or permissions to create each of the resources specified in the resources list. Sysdig provides an IAM policy containing the required permissions.

Steps

Log in to Sysdig Secure as Admin and select Get Started > Connect your Cloud account. Choose the AWS tab.
Copy the code snippet under Single Account and paste it in the terminal of your local machine. It should be pre-configured with your Sysdig API token.
Then run:
```
$ terraform init
```
When complete, run:
```
$ terraform apply
```
which will present the changes to be made, ask you to confirm them, then make the changes.
Confirm the Services are Working
Check Troubleshooting in case of permissions or account conflict errors.

For Organizational/Management Account

For organizational accounts, the default code provided in the Get Started page of Sysdig Secure is pre-populated with your Secure API token and will automatically install threat detection with CloudTrail (only).

Prerequisites

Have Terraform installed on the local machine.
A Sysdig Secure SaaS account
A Sysdig Secure for Cloud organizational member account ID.
We recommend creating a unique member account for Sysdig Secure for cloud.

Permissions

Sysdig Secure administrator permissions
AWS permissions:
- An existing AWS account as the organization master account with the Organizational CloudTrail service enabled.
- AWS profile credentials configuration of the master account of the organization
  You must also have sufficient permissions for the IAM user or role in the management account to successfully create an organization trail.
  Sysdig provides an IAM policy containing the required permissions.

Steps

Log in to Sysdig Secure as Admin and select Get Started > Connect your Cloud account. Choose the AWS tab.
Copy the code snippet under Organizational Account and paste it in the terminal of your local machine. It should be pre-configured with your Sysdig API token.
Then run:
```
$ terraform init
```
When complete, run:
```
$ terraform apply
```
which will present the changes to be made, ask you to confirm them, then make the changes.
Confirm the Services are Working
Check Troubleshooting in case of permissions or account conflict errors.

Soon, this option will be expanded to include all the features currently in the single account option, as well as the ability to easily add multiple member accounts.

Customizing the Install

Both the Single Account and Organizational Account code examples are configured with sensible defaults for the underlying inputs. But if desired, you can edit the region, module enablement, and other Inputs. See details for:

Resources Created by Each Module

Cloud-bench
- aws_iam_role
- aws_iam_role_policy_attachment
- sysdig_secure_benchmark_task
- sysdig_secure_cloud_account
Cloud-connector
- aws_cloudwatch_log_stream
- aws_ecs_service
- aws_ecs_task_definition
- aws_iam_role
- aws_iam_role_policy
- aws_s3_bucket
- aws_s3_bucket_object
- aws_s3_bucket_public_access_block
- aws_security_group
- aws_sns_topic_subscription
- aws_sqs_queue
- aws_sqs_queue_policy
Cloud-scanning
- aws_cloudwatch_log_group
- aws_cloudwatch_log_stream
- aws_ecs_service
- aws_ecs_task_definition
- aws_iam_role
- aws_iam_role_policy
- aws_security_group
- aws_sns_topic_subscription
- aws_sqs_queue
- aws_sqs_queue_policy

If cloud-connector or cloud-scanning is installed, these additional modules will be installed:

resource-group
- aws_resourcegroups_group
cloudtrail
- aws_cloudtrail
- aws_kms_alias
- aws_kms_key
- aws_s3_bucket
- aws_s3_bucket_policy
- aws_s3_bucket_public_access_block
- aws_sns_topic
- aws_sns_topic_policy
ssm
- aws_ssm_parameter
ecs-fargate-cluster
- aws_ecs_cluster

If cloud-scanning is installed, these additional modules will be installed:

codebuild
- aws_cloudwatch_log_group
- aws_codebuild_project
- aws_iam_role
- aws_iam_role_policy

Onboarding a Single Account using a CFT

Each of the features can be enabled from a single CloudFormation Template (CFT) from the AWS Console.

Deploying the CFT will add the default cloud policies and rules to any existing Sysdig Secure installations.

Prerequisites

A Sysdig Secure SaaS account
An AWS account and AWS services you would like to connect to Sysdig, with appropriate permissions to deploy a CFT.

Steps

Log in to your AWS Console and confirm that you are in the account and AWS region that you want to secure using Sysdig Secure for cloud.
Log in to Sysdig Secure as Admin and select Get Started > Connect your Cloud account. Choose the AWS(CloudFormation) tab.
Click Launch Stack.
The AWS Console opens, at the CloudFormation > Stacks > Quick Create page. The Sysdig CloudFormation template is pre-loaded.
Confirm that you are logged in the AWS account and region where you want to deploy the Sysdig Template.
Provide a Stack name or accept the default.
Fill in the Parameters:
Sysdig Settings
- Sysdig Secure Endpoint:
  Default (US-East): https://secure.sysdig.com. If your Sysdig Secure platform is installed in another region, use that endpoint.
  US West: https://us2.app.sysdig.com
  European Union: https://eu1.app.sysdig.com
- Sysdig Secure API Token: See Retrieve the Sysdig API Token to find yours.
Modules to Deploy: Choose any or all.
- CSPM/Compliance: Deploys the CIS AWS Benchmarks in Sysdig’s Compliance module.
- Threat detection using CloudTrail: Deploys everything needed to detect threats based on CloudTrail events.
- ECR Image Registry Scanning: Integrates container registry scanning for AWS ECR.
- Fargate Image Scanning: Integrates image scanning on any any container image deployed on a serverless Fargate task (in ECS).
Existing Infrastructure: Leave all three entries blank to have a cluster, VPC, and subnet created automatically. Otherwise, you can provide existing:
- ECS Cluster Name
- VPC ID
- Private subnet ID(s)
Confirm the Capabilities required to deploy:
- Check “I acknowledge that AWS CloudFormation might create IAM resources with custom names."
- Check “I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND”
Click Create Stack.
In the AWS Console, the main stack and associated substacks will show “CREATE_IN_PROGRESS”. Refresh the status to see “CREATE_COMPLETE” for all. There is a delay of 5-10 minutes for events to be sent from CloudTrail, but no event is lost.
A success message also appears in the Sysdig Secure Get Started page.

Troubleshooting

1. Resolve 409 Conflict Error

This error may occur if the specified cloud account has already been onboarded to Sysdig.

Solution:

The cloud account can be imported into Terraform by running:

terraform import module.cloud_bench.sysdig_secure_cloud_account.cloud_account CLOUD_ACCOUNT_ID

2. Resolve Permissions Error/Access Denied

This error may occur if your current AWS authentication session does not have the required permissions to create certain resources.

Solution:

Ensure you are authenticated to AWS using a user or role with the required permissions.Onboarding a Single Account using a CFT

Onboarding a Single Account using a CFT

Each of the features can be enabled from a single CloudFormation Template (CFT) from the AWS Console.

Deploying the CFT will add the default cloud policies and rules to any existing Sysdig Secure installations.

Prerequisites

A Sysdig Secure SaaS account
An AWS account and AWS services you would like to connect to Sysdig, with appropriate permissions to deploy a CFT.

Steps

Log in to your AWS Console and confirm that you are in the account and AWS region that you want to secure using Sysdig Secure for cloud.
Log in to Sysdig Secure as Admin and select Get Started > Connect your Cloud account. Choose the AWS tab.
Select the Install via CloudFormation Template link at the bottom of the screen.
The Connect Account dialog is displayed.
Enter:
- The AWS account number with which you want to connect
- An IAM Role name to be created for Sysdig Secure for cloud in AWS. This role name must not yet exist in your account.
  The role provides read-only access to your resources to allow Sysdig to monitor and secure your cloud account. Access is scoped to the managed SecurityAudit policy.
Click Launch Stack.
The AWS Console opens, at the CloudFormation > Stacks > Quick Create page. The Sysdig CloudFormation template is pre-loaded.
Confirm that you are logged in the AWS account and region where you want to deploy the Sysdig Template.
Provide a Stack name or accept the default.
Fill in the Parameters:
Sysdig Settings
- Sysdig Secure Endpoint:
  Default (US-East): https://secure.sysdig.com. If your Sysdig Secure platform is installed in another region, use that endpoint.
  US West: https://us2.app.sysdig.com
  European Union: https://eu1.app.sysdig.com
- Sysdig Secure API Token: See Retrieve the Sysdig API Token to find yours.
Modules to Deploy: Choose any or all.
- CSPM/Compliance: Deploys the CIS AWS Benchmarks in Sysdig’s Compliance module.
- Threat detection using CloudTrail: Deploys everything needed to detect threats based on CloudTrail events.
- ECR Image Registry Scanning: Integrates container registry scanning for AWS ECR.
- Fargate Image Scanning: Integrates image scanning on any any container image deployed on a serverless Fargate task (in ECS).
Existing Infrastructure: Leave all three entries blank to have a cluster, VPC, and subnet created automatically. Otherwise, you can provide existing:
- ECS Cluster Name
- VPC ID
- Private subnet ID(s)
Confirm the Capabilities required to deploy:
- Check “I acknowledge that AWS CloudFormation might create IAM resources with custom names."
- Check “I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND”
Click Create Stack.
In the AWS Console, the main stack and associated substacks will show “CREATE_IN_PROGRESS”. Refresh the status to see “CREATE_COMPLETE” for all. There is a delay of 5-10 minutes for events to be sent from CloudTrail, but no event is lost.
A success message also appears in the Sysdig Secure Get Started page.

Confirm the Services are Working

Log in to Sysdig Secure and check that each module you deployed is functioning. It may take 10 minutes or so for events to be collected and displayed.

Check Overall Connection Status

Data Sources: Select Data Sources from the User menu to see all connected cloud accounts.
Subscription: Select Settings > Subscription to see an overview of your account activity, including cloud accounts.
Insights: Check that Insights have been added to your navigation bar. View activity on the Cloud Account, Cloud User, or Composite insight views.

Check Threat Detection

Policies: Check Policies > Runtime Policies and confirm that the AWS Best Practices policy is enabled. This consists of the most-frequently-recommended rules for AWS and CloudTrail. You can customize it by creating a new policy of the AWS CloudTrail type.
Events: In the Events feed, search ‘cloud’ to show events from AWS CloudTrail.
Force an event: In case you want to manually create an event, choose one of the rules contained in the AWS Best Practices policy and execute it in your AWS account.
ex.: Create a S3 Bucket with Public Access Blocked. Make it public to prompt the event.
Remember that in case you add new rules to the policy you need to give it time to propagate the changes.

Check CSPM/AWS Benchmarks

Compliance: Select Compliance and see that AWS Foundations Benchmark is installed.
Review the benchmark results and confirm the account, region and date added.

Check Scanning for ECR and Fargate

Scan Results: CheckImage Scanning > Scan Resultsand choose the Origins drop-down.
Confirm that AWS Registry and/or AWS Fargate are listed.
Filter by the desired origin and review scan results.
Force an event: Upload an image to a repository on your Elastic Container Registry (ECR). Follow View push commands button guide provided by AWS.

Check Permissions and Entitlements (AWS)

Select Posture > Permissions and Entitlements and check if the following are showing up in the Unused Permissions graphs:

Unused permissions
Inactive users
Policies with the greatest number of granted vs used permissions

Follow the instructions to remediate overly permissive entitlements and reduce security risks.

1.4.2 -

Deploy Sysdig Secure for cloud on GCP

If needed, review the offering description on Sysdig Secure for cloud - GCP.

Deployments on GCP use a Terraform file.

Onboarding Using Terraform

Terraform-based install instructions differ depending on what type of account you are using.

At this time, the options include:

Install for a single project
Install for an organization

The default code provided in the Get Started page of Sysdig Secure is pre-populated with your Secure API token and will automatically install threat detection, benchmarks, and container registry and image scanning.

Prerequisites

A Sysdig Secure SaaS account
A Google Cloud Platform (GCP) account you would like to connect to Sysdig, with appropriate permissions to deploy
Have Terraform installed on the machine from which you will deploy the installation code.
Google’s Cloud SDK deployed in the environment where you will deploy the installation code
For more background information, see the Google Cloud Platform Provider documentation.
Enable the required GCP APIs as described in the Terraform module documentation

Permissions

Sysdig Secure administrator permissions
For GCP, in order to create each of the resources specified in the resources list below, you must have - Owner role if using single-project mode.
- For organizational Organization Admin role is required too.

Steps

Log in to Sysdig Secure as Admin and select Get Started > Connect your Cloud account. Choose the GCP tab.
Copy the code snippet under Single Account or Organizational Account and paste it in the terminal of your local machine. It should be pre-configured with your Sysdig API token.
Then run:
```
$ terraform init
```
When complete, run:
```
$ terraform apply
```
which will present the changes to be made, ask you to confirm them, then make the changes.
Confirm the Services are Working
Check Troubleshooting in case of permissions or account conflict errors.

Resources Created by Each Module

Cloud-bench
- google_iam_workload_identity_pool
- google_iam_workload_identity_pool_provider
- google_project_iam_custom_role
- google_project_iam_member
- google_service_account
- google_service_account_iam_binding
- sysdig_secure_benchmark_task
- sysdig_secure_cloud_account
Cloud-connector
- google_cloud_run_service
- google_eventarc_trigger
- google_project_iam_member
- google_storage_bucket
- google_storage_bucket_iam_member
- google_storage_bucket_object

Cloud-scanning
- google_cloud_run_service
- google_cloud_run_service_iam_member
- google_eventarc_trigger
- google_project_iam_member
- google_pubsub_topic

If Cloud-connector is installed in organizational mode, this additional module will be installed:

Organization-sink
- google_logging_organization_sink
- google_pubsub_topic
- google_pubsub_topic_iam_member

If Cloud-connector is installed in single-project mode, this additional module will be installed:

Project-sink
- google_logging_project_sink
- google_pubsub_topic
- google_pubsub_topic_iam_member

If Cloud-scanning is installed, this additional module will be installed:

Secrets
- google_secret_manager_secret
- google_secret_manager_secret_iam_member
- google_secret_manager_secret_version`

Troubleshooting

1. Insufficient Permissions on Project

This error may occur if your current GCP authentication session does not have the required permissions to access the specified project.

Solution: Ensure you are authenticated to GCP using a user or service account with the required permissions.

2. Insufficient Permissions to Create Resource

This error may occur if your current GCP authentication session does not have the required permissions to create certain resources.

Solution: Ensure you are authenticated to GCP using a user or service account with the required permissions.

If you have sufficient permissions but still get this kind of error, try to authenticate gcloudusing:

$ gcloud auth application-default login

$ gcloud auth application-default login set-quota-project your_project_id

3. Conflicting Resources

This error may occur if the specified GCP project has already been onboarded to Sysdig.

Solution: The cloud account can be imported into terraform by running

terraform import module.single account.module.cloud_bench.sysdig_secure_cloud_account.cloud_account PROJECT_ID , where PROJECT_ID is the numerical ID of the project (not the project name).

4. Workload Identity Federation pool already exists

This error may occur if a Workload Identity Federation Pool or Pool Provider has previously been created, and then deleted, either via the GCP console or with terraform destroy. When a delete request for these resources is sent to GCP, they are not completely deleted, but marked as “deleted”, and remain for 30 days. These “deleted” resources will block creation of a new resource with the same name.

Solution: The “deleted” pools must be restored using the GCP console, and then imported into terraform, using terraform import module.single-account.module.cloud_bench.google_iam_workload_identity_pool.pool POOL_ID and module.single-account.module.cloud_bench.google_iam_workload_identity_pool_provider.pool_provider POOL_ID/PROVIDER_ID

5. Received an email from Google Cloud Platform citing a Configuration Error

Email contains error codes such as:

Error Code: topic_not_found
or
Error Code: topic_permission_denied

Cause: The resources Sysdig deployed with Terraform will eventually be consistent, but it could happen that some pre-required resources are created but not ready yet.

Solution: This is a known issue that will only take place within first minutes of the deployment. Eventually, resource health checks will pass and modules will work as expected.

Confirm the Services are Working

Log in to Sysdig Secure and check that each module you deployed is functioning. It may take 10 minutes or so for events to be collected and displayed.

Check Overall Connection Status

Data Sources: Select Data Sources from the User menu to see all connected cloud accounts.
Subscription: Select Settings > Subscription to see an overview of your account activity, including cloud accounts.
Insights: Check that Insights have been added to your navigation bar. View activity on the Cloud Account, Cloud User, or Composite insight views.

Check Threat Detection

Policies: Check Policies > Runtime Policies and confirm that the GCP Best Practices policy is enabled. This consists of the most-frequently-recommended rules for GCP.
Events: In the Events feed, search ‘cloud’ to show events from GCP.

Check GCP Benchmarks

Tasks: Select Compliance > Benchmarks > Tasks and confirm a task with the name Sysdig Secure for Cloud (GCP) exists.
Results: After a few minutes, check results of the benchmark are available by clicking on the Sysdig Secure for Cloud (GCP) task. Note that results may take up to 15 minutes to appear.

Check GCP Scanning

Scan Results: CheckImage Scanning > Scan Resultsand choose the Origins drop-down.
Confirm that GCP is listed.
Filter by the desired origin and review scan results.
Force an event: Upload an image to a new Repository in a Container Registry. Follow repository Setup Instructions provided by GCP.

1.4.3 -

Deploy Sysdig Secure for cloud on Azure

If needed, review the offering description on Sysdig Secure for cloud - Azure.

Deployments on Azure use a Terraform file.

Onboarding Using Terraform

Terraform-based install instructions differ depending on what type of account you are using.

At this time, the options include:

Install for a single subscription
Install for tenant subscriptions

Prerequisites

A Sysdig Secure SaaS account
An Azure account you would like to connect to Sysdig, with appropriate permissions to install.
Have Terraform installed on the machine from which you will deploy the installation code.

Permissions

Sysdig Secure administrator permissions
On Azure, the installing user must have the roles of Security Administrator and Owner or Contributor.

Steps

Log in to Sysdig Secure as Admin and select Get Started > Connect your Cloud account. Choose the Azure tab.
Copy the code snippet under Single Subscription or Tenant Subscriptions and paste it in the terminal of your local machine. It should be pre-configured with your Sysdig API token.
Then run:
```
$ terraform init
```
When complete, run:
```
$ terraform apply
```
which will present the changes to be made, ask you to confirm them, then make the changes.
Confirm the Services are Working
Check Troubleshooting in case of permissions or account conflict errors.

Resources Created by Each Module

Cloud-bench
- azurerm_lighthouse_assignment
- azurerm_lighthouse_definition
- azurerm_role_definition
- azurerm_subscription
- sysdig_secure_cloud_account
- sysdig_secure_benchmark_task
Cloud-connector
- azurerm_container_group
- azurerm_network_profile
- azurerm_storage_account
- azurerm_storage_blob
- azurerm_storage_container
- azurerm_subnet
- azurerm_virtual_network

If Cloud-connector is installed, these additional modules will also be installed:

Container-registry
- azurerm_container_registry
- azurerm_eventgrid_event_subscription
Enterprise-application
- azuread_application
- azuread_application_password
- azuread_service_principal
- azuread_service_principal_password
- azurerm_role_assignment
- azurerm_role_definition
Eventhub
- azurerm_eventhub
- azurerm_eventhub_authorization_rule
- azurerm_eventhub_namespace
- azurerm_eventhub_namespace_authorization_rule
- azurerm_monitor_diagnostic_setting
- azurerm_resource_group

Troubleshooting

1. Insufficient Permissions on Subscription

This error may occur if your current Azure authentication session does not have the required permissions to create resources in the specified subscription.

Solution: Ensure you are authenticated to Azure using a Non-Guest user with the Contributor or Owner role on the target subscription.

Error: Error Creating/Updating Lighthouse Definition "dd9be15b-0ee9-7daf-b942-5e173dae13fb" (Scope "/subscriptions/***"): managedservices.RegistrationDefinitionsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="InsufficientPrivilegesForManagedServiceResource" Message="The requested user doesn't have sufficient privileges to perform the operation."
       
     with module.cloudvision_example_existing_resource_group.module.cloud_bench.module.trust_relationship["***"].azurerm_lighthouse_definition.lighthouse_definition,
         on ../../../modules/services/cloud-bench/trust_relationship/main.tf line 28, in resource "azurerm_lighthouse_definition" "lighthouse_definition":
         28: resource "azurerm_lighthouse_definition" "lighthouse_definition" {

2. Conflicting Resources

This error may occur if the specified Azure Subscription has already been onboarded to Sysdig

Solution: The resource can be imported into Terraform by using the terraform import command. This will bring the resource under management in the current Terraform workspace.

Error: A resource with the ID "/subscriptions/***/resourceGroups/sfc-resourcegroup" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_resource_group" for more information.
       
         with module.cloudvision_example_existing_resource_group.module.infrastructure_eventhub.azurerm_resource_group.rg[0],
         on ../../../modules/infrastructure/eventhub/main.tf line 6, in resource "azurerm_resource_group" "rg":
          6: resource "azurerm_resource_group" "rg" {

Confirm the Services are Working

Log in to Sysdig Secure and check that each module you deployed is functioning. It may take 10 minutes or so for events to be collected and displayed.

Check Overall Connection Status

Data Sources: Select Data Sources from the User menu to see all connected cloud accounts.
Subscription: Select Settings > Subscription to see an overview of your account activity, including cloud accounts.
Insights: Check that Insights have been added to your navigation bar. View activity on the Cloud Account, Cloud User, or Composite insight views.

Check Threat Detection

Policies: Check Policies > Runtime Policies and confirm that the Azure Best Practices policy is enabled. This consists of the most-frequently-recommended rules for Azure DevOps.
Events: In the Events feed, search ‘cloud’ to show events from Azure.

Check Benchmarks

Tasks: Select Compliance > Benchmarks > Tasks and confirm a task with the name Sysdig Secure for Cloud (Azure) exists.
Results: After a few minutes, check results of the benchmark are available by clicking on the Sysdig Secure for Cloud (Azure) task. Note that results may take up to 15 minutes to appear.

Check Scanning

Scan Results: CheckImage Scanning > Scan Resultsand choose the Origins drop-down.
Confirm that Azure is listed.
Filter by the desired origin and review scan results.

1.5 -

Rapid Response: Installation

With Rapid Response, Sysdig has introduced a way to grant designated Advanced Users in Sysdig Secure the ability to remote connect into a host directly from the Event stream and execute desired commands there.

Rapid Response team members have access to a full shell from within the Sysdig Secure UI. Responsibility for the security of this powerful feature rests with you: your enterprise and your designated employees.

Install and Configure Rapid Response

Prerequisites

Sysdig Secure On-Premises 4.0+
SaaS enablement available on a per-case basis at this time.
Have on hand:
- Your Sysdig agent access key
- Your Sysdig API endpoint (custom, depending on your on-prem installation)
- A passphrase used to encrypt all traffic between the user and host.
  NOTE: Sysdig cannot recover this passphrase. If lost, a user will not be able to start a session, nor will any session logs be recoverable.
Optionally, these can be added to the environment variables:
```
export API_ENDPOINT=https://secure-staging.mycompany.com
export ACCESS_KEY=$YOUR_SYSDIG_API_KEY
export PASSPHRASE=$ENCRYPTION_PASSPHRASE
export API_TLS_SKIP_CHECK=false
```

Install or Upgrade Sysdig Platform

This feature is available as of Sysdig Platform v.4.0, on-premises. Be sure your system has been upgraded appropriately. SaaS enablement is on a per-case basis at this time; please discuss your situation with Sysdig Support.

Install Host Component

The Rapid Response agent can be installed as a Docker container or as a Kubernetes DaemonSet.

As Docker Container

Mount the host directories and binaries to gain access to the host.

docker run --hostname $HOST_NAME -d quay.io/sysdig/rapid-response-host-component:latest --endpoint $API_ENDPOINT --access-key $ACCESS_KEY --password $PASSPHRASE

Customize the Docker image.
The container is simply bash shell. To add custom scripts without needing to mount the underlying host filesystem, you can bake this into the Docker container, e.g. by installing kubectl, gcloud, netstat, or another command-line utility.
```
FROM quay.io/sysdig/rapid-response-host-component:latest AS base-image

FROM alpine:3.13
COPY --from=base-image /usr/bin/host /usr/bin/host

# add custom scripts and other directives

ENTRYPOINT ["host"]
```

As Kubernetes DaemonSet

Create a namespace and secrets for the Rapid Response agent:

kubectl create ns rapid-response
kubectl create secret generic sysdig-rapid-response-host-component-access-key --from-literal=access-key=$ACCESS_KEY -n rapid-response
kubectl create secret generic sysdig-rapid-response-host-component-passphrase --from-literal=passphrase=$PASSPHRASE -n rapid-response

Create the configmap and change the API_ENDPOINT parameter:

echo "apiVersion: v1
kind: ConfigMap
metadata:
  name: sysdig-rapid-response-host-component
data:
  api-endpoint: ${API_ENDPOINT}
  api-tls-skip-check: 'false'" | kubectl apply -n rapid-response -f -

Deploy the DaemonSet.

Note: The agent does not automatically have access to the host filesystem; there are several mounts commented-out in the manifest that must be uncommented to investigate the host.

echo "# apiVersion: extensions/v1beta1  # If you are in Kubernetes version 1.8 or less please use this line instead of the following one
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: sysdig-rapid-response-host-component
  labels:
    app: sysdig-rapid-response-host-component
spec:
  selector:
    matchLabels:
      app: sysdig-rapid-response-host-component
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: sysdig-rapid-response-host-component
    spec:
      hostNetwork: true
      volumes:
        # Add custom volume here
        # Uncomment these lines if you'd like to map /root/ from the
        # host into the container.
        #- hostPath:
        #    path: /
        #  name: host-root-vol
        - name: sysdig-rapid-response-host-component-config
          configMap:
            name: sysdig-rapid-response-host-component
            optional: true
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
      containers:
        - name: sysdig-rapid-response-host-component
          image: quay.io/sysdig/rapid-response-host-component
          #securityContext:
            # The privileged flag is necessary for OCP 4.x and other Kubernetes setups that deny host filesystem access to
            # running containers by default regardless of volume mounts. In those cases, access to the CRI socket would fail.
          #  privileged: true
          imagePullPolicy: Always
          resources:
            limits:
              cpu: 500m
              memory: 500Mi
            requests:
              cpu: 250m
              memory: 250Mi
          # Add custom volume mount here
          # Uncomment these lines if you'd like to map /root/ from the
          # host into the container.
          #volumeMounts:
          #- mountPath: /host
          #  name: host-root-vol
          env:
            - name: API_ENDPOINT
              valueFrom:
                configMapKeyRef:
                  name: sysdig-rapid-response-host-component
                  key: api-endpoint
            - name: API_TLS_SKIP_CHECK
              valueFrom:
                configMapKeyRef:
                  name: sysdig-rapid-response-host-component
                  key: api-tls-skip-check
            - name: ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: sysdig-rapid-response-host-component-access-key
                  key: access-key
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: sysdig-rapid-response-host-component-passphrase
                  key: passphrase" | kubectl apply -n rapid-response -f -

Complete the Configuration

After installation/upgrade, complete the following steps:

Request enablement of the feature from Sysdig Support.
Configure an S3 bucket for Rapid Response logs: If you are using the default Cassandra storage for Capture files, you will need to configure an AWS or custom S3 bucket to store Rapid Response log files after a session. If you have already configured an S3 bucket for Captures, then Rapid Response logs will be routed there automatically, into their own folder.
Manage the following port/firewall considerations:
- Ensure the host component is able to reach the endpoint defined in API_ENDPOINT
- Ensure there are no intermediate proxies that could enforce maximum time to live (since sessions could potentially have long durations)
- Ensure that the host component can reach the object storage (S3 bucket) when configured.
Configure and use Rapid Response in the Sysdig Secure UI: See Rapid Response.

1.6 -

Node Analyzer: Multi-Feature Installation

What Is the Node Analyzer?

The Node Analyzer (NA) provides a method for deploying the components for three different Sysdig Secure features:

(Node) Image Analyzer: an existing tool that can now be installed and/or upgraded in a new way, alongside the other two components.
Benchmarks: Installs a new component (called a benchmark runner) which is required to use Benchmarks, including an updated interface and new improved features. The legacy Benchmark tool can still be accessed.
Host Scanning: a new tool for scanning not just the images/containers on a host, but the host itself.

Installation Options

All the Node Analyzer components, along with the Sysdig agent, are deployed per node or host. You can deploy them using various methods:

Fresh Install: Agent + Node Analyzer

If you are installing Sysdig Secure for the first time and have not yet deployed any agents, you can use a single-line install to deploy both the Sysdig agent and the Node Analyzer (NA) tools. The script will make changes to each node or host within a cluster.

curl -s  https://download.sysdig.com/stable/install-agent-kubernetes | sudo bash -s
-- --access_key ACCESS_KEY --collector COLLECTOR_URL --collector_port 6443 --nodeanalyzer --api_endpoint API_ENDPOINT

For SaaS, see also the Get Started page in Sysdig Secure. Under “Connect Your Data Sources,” the script is generated with your endpoints automatically inserted.

On-Premises with Self-Signed Cert:

If you want the Node Analyzer to report to an On-Prem Sysdig backend that uses a self-signed certificate, then: Add -cc false to the command line so the node analyzer will accept it.

To find the values yourself:

access_key: This is the agent access key. You can retrieve this from Settings > Agent Installation in the Sysdig Secure UI.
collector_url: This value is region-dependent in SaaS and is auto-completed on the Get Started page in the UI. (It is a custom value in on-prem installations.)
api_endpoint: This is the base URL ( region-dependent) for Sysdig Secure and is auto-completed on the Get Started page. E.g. secure.sysdig.com, us2.app.sysdig.com, eu1.app.sysdig.com.

When finished, you can Access the Node Analyzer Features.

Upgrade/Install Node Analyzer Tools Only

Use this script in the following conditions:

Agent is already installed, you just want the NA tools
Node Image Analyzer already installed; you want to upgrade it to v2
You want to add Benchmarks v2 and Host Scanning features to your existing Sysdig Secure environment, as well as upgrade or install the Image Analyzer.

Note that if you already have the Node Image Analyzer (v1) installed, this script will upgrade that component automatically. An agent MUST already be installed. The script will make changes to every node in the cluster.

curl -s https://download.sysdig.com/stable/install-node-analyzer | sudo bash -s -- --api_endpoint API_ENDPOINT

When finished, you can Access the Node Analyzer Features.

Daemonset Install

To deploy the Node Analyzer using Kubernetes daemonsets, download the following configuration files, edit them as annotated within the files, and deploy them.

To deploy the Node Analyzer concurrently with the Sysdig agent, you would also download the sysdig-agent-clusterrole.yaml, sysdig-agent-daemonset-v2.yaml, and sysdig-agent-configmap.yaml and deploy them as described in Agent Install: Kubernetes.

You need to deploy these YAMLs after installing the Sysdig agent in the same nodes, and also in the same namespace (sysdig-agent by default).

When finished, you can Access the Node Analyzer Features.

Install with Helm

Use the “Sysdig” Helm chart, which installs the Sysdig agent and the Node Analyzer, with the following commands:

helm repo add sysdig https://charts.sysdig.com
helm repo update
helm install sysdig-agent --set sysdig.accessKey=ACCESS_KEY --set sysdig.settings.collector=COLLECTOR_URL --set sysdig.settings.collector_port=6443 sysdig/sysdig --set nodeAnalyzer.collectorEndpoint=API_ENDPOINT

To find the values:

access_key: This is the agent access key. You can retrieve this from Settings > Agent Installation in the Sysdig Secure UI.
collector_url: This value is region-dependent in SaaS and is auto-completed on the Get Started page in the UI. (It is a custom value in on-prem installations.)
api_endpoint: This is the base URL ( region-dependent) for Sysdig Secure and is auto-completed on the Get Started page. E.g. secure.sysdig.com, us2.app.sysdig.com, eu1.app.sysdig.com.

Access the Node Analyzer Features

Confirm Image Analyzer

Select Scanning > Image Results.
Check for scanned container image results that originate with the Sysdig Node Image Analyzer.

Use Benchmarks

Select Benchmarks |Tasks.
Either configure a new task or review your upgraded tasks. Click a line item to see the associated benchmark report.
Your active team scope is applied when loading benchmarks results. Log in with the broadest team and user credentials to see the full report.

Use Host Scanning

Check vulnerabilities in hosts or nodes, both for operation system packages (e.g. rpm, dpkg) and non-operating system packages (e.g. Java packages, Ruby gems).

Select Scanning > Hosts.
Review the Host vulnerabilities listed.
Your active team scope is applied when loading host scanning results. Log in with the broadest team and user credentials to see the full report.

Alternate Install Cases

The installation options above should be sufficient for the majority of users; the options below allow for customizations and special cases.

Running Node Analyzer Behind a Proxy

Depending on your organization’s network design, you may require the HTTP requests from Node Analyzer features to pass through a proxy in order to reach the Sysdig Secure backend. To do so, you must edit all three configmaps:

These are in the sysdig-agent namespace by default.

Configure the following variables:

http_proxy/https_proxy Use with the relevant proxy URL, e.g. http://my_proxy_address:8080.
In most cases, it is enough to specify http_proxy. as it applies to HTTPS connections as well.
no_proxy Use this parameter to exclude certain subnets from using the proxy, adding a comma-separated exclusion list, e.g. 127.0.0.1,localhost,192.168.0.0/16,172.16.0.0/12,10.0.0.0/8

If the proxy server requires authentication it is possible to specify credentials in the URL, e.g. http://username:password@my_proxy:8080.

Running in a Non-Kubernetes Environment

This is handled per-component.

Benchmarks (Non-Kubernetes)

It is possible to deploy the benchmark runner as a single Docker container:

docker run -d -v /:/host:ro -v /tmp:/host/tmp --privileged --network host --pid host -e BACKEND_ENDPOINT=https://<sysdig_backend_endpoint> -e ACCESS_KEY=<Sysdig agent access key> -e BACKEND_VERIFY_TLS=false -e TAGS=<custom_tags> quay.io/sysdig/compliance-benchmark-runner:latest

Note: If you don’t want to pass the access key directly via the command line, consider using an alternative method of passing environment variables, such as docker-compose.
The BACKEND_ENDPOINT is only required if for Sysdig on-prem or when using a Sysdig SaaS region other than US-EAST.
For example, for the EU SaaS endpoint would be: https://eu1.app.sysdig.com.
See also: SaaS Regions and IP Ranges.
BACKEND_VERIFY_TLS=false is only needed if you are using an on-prem backend with a self-signed certificate.
TAGS: The list of tags for the host where the agent is installed. For example: “role:webserver, location:europe”, “role:webserver” or “webserver”.

Image Analyzer (Non-Kubernetes)

It is also possible to run the image analyzer as a single Docker container:

docker run -d -v /var/run:/var/run --privileged --network host -e AM_COLLECTOR_ENDPOINT=https://<sysdig_backend_endpoint>/internal/scanning/scanning-analysis-collector -e ACCESS_KEY=<Sysdig agent access key> -e VERIFY_CERTIFICATE=false quay.io/sysdig/node-image-analyzer:latest

Note: If you don’t want to pass the access key directly via the command line, consider using an alternative method of passing environment variables, such as docker-compose.
The AM_COLLECTOR_ENDPOINT is only required if for Sysdig on-prem or when using a Sysdig SaaS region other than US-EAST.
For example, for the EU SaaS endpoint would be: https://eu1.app.sysdig.com/internal/scanning/scanning-analysis-collector .
See also: SaaS Regions and IP Ranges.
VERIFY_CERTIFICATE=false is only needed if you are using an on-prem backend with a self-signed certificate.

Host Scanning (Non-Kubernetes)

To install the Host Scanning component in a non-Kubernetes environment, you can use:

docker run -d -v /:/host:ro --privileged -e HOST_BASE=/host -e AM_COLLECTOR_ENDPOINT=https://<sysdig_backend_endpoint>/internal/scanning/scanning-analysis-collector -e ACCESS_KEY=<Sysdig agent access key> -e VERIFY_CERTIFICATE=false -e SCHEDULE=@dailydefault quay.io/sysdig/host-analyzer:latest

Note: If you don’t want to pass the access key directly via the command line, consider using an alternative method of passing environment variables, such as docker-compose.
The BACKEND_ENDPOINT is only required if for Sysdig on-prem or when using a Sysdig SaaS region other than US-EAST.
For example, for the EU SaaS endpoint would be: https://eu1.app.sysdig.com.
See also: SaaS Regions and IP Ranges.
BACKEND_VERIFY_TLS=false is only needed if you are using an on-prem backend with a self-signed certificate.
TAGS: The list of tags for the host where the agent is installed. For example: “role:webserver, location:europe”, “role:webserver” or “webserver”.

For Image Analyzer Component Only

These cases affect only the Image Analyzer component of the Node Analyzer installation.

Installing Image Analyzer Component Alone

It is still possible to install the image analyzer component without benchmarks or host scanning. This option normally would apply only to previous users of the former node image analyzer who want to upgrade just that component, for whatever reason.

This can be done by downloading the sysdig-image-analyzer-daemonset.yaml and sysdig-image-analyzer-configmap.yaml and deploying.

You need to deploy these YAMLs after installing the Sysdig agent in the same nodes, and also in the same namespace (sysdig-agent by default).

Kubernetes Requiring Custom Socket Path

By default, the image analyzer will automatically detect the socket to mount from:

Docker socket from /var/run/docker/docker.sock
CRI-O socket from/var/run/crio/crio.sock
CRI-containerd socket from/var/run/containerd/containerd.sock

Some setups require the analyzer to use custom socket paths.

If the socket is located outside /var/run, the corresponding volume must be mounted as well. You can configure it via the single line installer script or by manually editing the daemonset and configmap variables.

When using the installer, use the-cv option to mount an additional volume and add -ds -cs or -cd to specify a Docker, CRI, or CRI-containerd socket respectively.

See the script -help command for additional information.

Examples:

For K3S, which uses containerd, add:

-cd unix:///run/k3s/containerd/containerd.sock -cv /run/k3s/containerd

For Pivotal, which uses a custom path for the Docker socket, use:

-ds unix:///var/vcap/data/sys/run/docker/docker.sock -cv /var/vcap/data/sys/run/docker

Daemonset Resource Limit Considerations

During its regular operation, the Image Analyzer uses much less memory than the limit specified in the daemonset configuration. However, in some cases, processing an image may require more memory, depending, for example, on image size, content or package types.

This issue can be detected by looking for abnormal spikes in the memory usage of the Image Analyzer pods which are also showing analysis errors. In such cases we recommend trying to increase the analyzer memory usage up to three times the size of the unprocessed images, if the cluster available memory allows.

Component Configurations

Image Analyzer Configmap Options

For special cases, the image analyzer can be configured by editing the sysdig-image-analyzer configmap in the sysdig-agent namespace with the following options:

Option	Description
`docker_socket_path`	The Docker socket path, defaulting to `unix:///var/run/docker/docker.sock` If a custom path is specified, ensure it is correctly mounted from the host inside the container.
`cri_socket_path`	The socket path to a CRI compatible runtime, such as CRI-O, defaulting to `unix:///var/run/crio/crio.sock.` If a custom path is specified, ensure it is correctly mounted from the host inside the container.
`containerd_socket_path`	The socket path to a CRI-Containerd daemon, defaulting to `unix:///var/run/containerd/containerd.sock` If a custom path is specified, ensure it is correctly mounted from the host inside the container.
`collector_endpoint`	The endpoint to the Scanning Analysis collector, specified in the following format: `https://<API_ENDPOINT>/internal/scanning/scanning-analysis-collector`
`ssl_verify_certificate`	Can be set to `"false"` to allow insecure connections to the Sysdig backend, such as for on-premise installs that use self-signed certificates. By default, certificates are always verified.
`debug`	Can be set to `"true"` to show debug logging, useful for troubleshooting.
`http_proxy`	Proxy configuration variables.
`https_proxy`
`no_proxy`

Host Scanning Configuration Options

The analyzer component of the Host Scanning feature can be configured by editing the sysdig-host-analyzer configmap in thesysdig-agentnamespace with the following options:

Option	Description
`schedule`	The scanning schedule specification for the host analyzer expressed as a crontab string such as “5 4 * * *” (more examples). The default value of @dailydefault instructs the analyzer to automatically pick a schedule that will start shortly after it is deployed and will perform a scan every 24 hours.
`dirs_to_scan`	The list of directories to inspect during the scan, expressed as a comma separated list such as /etc,/var/lib/dpkg,/usr/local,/usr/lib/sysimage/rpm,/var/lib/rpm,/lib/apk/db
`collector_endpoint`	The endpoint to the Scanning Analysis collector, specified in the following format: `https://<API_ENDPOINT>/internal/scanning/scanning-analysis-collector`
`max_send_attempts`	The number of times the analysis collector is allowed to retry sending results if backend communication fails
`ssl_verify_certificate`	Can be set to `"false"` to allow insecure connections to the Sysdig backend, such as for on-premise installs that use self-signed certificates. By default, certificates are always verified.
`debug`	Can be set to `"true"` to show debug logging, useful for troubleshooting.
`http_proxy`	Proxy configuration variables.
`https_proxy`
`no_proxy`

Benchmark Runner Configuration Options

The benchmark runner component can be configured by editing the sysdig-benchmark-runner configmap in the sysdig-agent namespace with the following options:

Option	Description
`collector_endpoint`	The Secure API endpoint, specified in the following format: `https://<API_ENDPOINT>`
`ssl_verify_certificate`	Can be set to `"false"` to allow insecure connections to the Sysdig backend, such as for on-premise installs that use self-signed certificates. By default, certificates are always verified.
`debug`	Can be set to `"true"` to show debug logging, useful for troubleshooting.
`http_proxy`	Proxy configuration variables.
`https_proxy`
`no_proxy`

1.7 -

Install Admission Controller

If you have installed the CLI-based version of the Admission Controller, the UI-based version is not backwards-compatible. You will need to uninstall the old version and install the UI-based version instead.

To understand and use the Admission Controller after installing it, see Admission Controller.

Prerequisites

Helm 3
Kubernetes 1.16 or higher

Install the Admission Controller

The component must be installed on each cluster where you want to use it.

Make sure kubectl is pointing to the target cluster where the Admission Controller will be installed.

Add and synchronize the Helm repository:

helm repo add sysdig https://charts.sysdig.com
helm repo update

Install the Admission Controller on the target cluster, e.g.:

helm install sysdig-admission-controller \
--create-namespace \
--namespace sysdig-admission-controller \
--set sysdig.secureAPIToken=$SYSDIG_API_TOKEN \
--set clusterName=$CLUSTER_NAME \
--set sysdig.url=https://$SYSDIG_SECURE_ENDPOINT \
--set features.k8sAuditDetections=true \
sysdig/admission-controller

Check that installation was successful in the Sysdig UI. Log in to Sysdig Secure and select Image Scanning>Admission Controller|Policy Assignments.
By default, the cluster shows Connected (healthy), but Disabled (grey dot right of the name). Admission Controllers are disabled by default to avoid accidentally blocking deployment.

Installation Parameters

--create-namespace: If supplied, will create a namespace
--namespace: Desired namespace where the Admission Controller will be installed
--set sysdig.secureAPIToken: Sysdig Secure API token as found in the Sysdig UI under Settings/User Profile. Note that this user must have administrator rights
--set clusterName: User-defined name for this cluster that will appear in the admission controller interface in Sysdig’s backend. The cluster name needs to match the agent cluster name.
--set features.k8sAuditDetections: (true/false) Set true to enable Kubernetes audit logging via the Admission Controller. See also: Kubernetes Audit Logging (legacy installation) and Select the Policy Type (Kubernetes Audit Policies)
--set sysdig.url: Sysdig endpoint. Default https://secure.sysdig.com is for the us-east region.
For us-west use https://us2.app.sysdig.com
For European Union, use https://eu1.app.sysdig.com
For APAC, use https://app.au1.sysdig.com
For GCP, use https://app.us4.sysdig.com/
For on-prem, your own enpoints.
- See also SaaS Regions and IP Ranges.
verifySSL: (true/false) Sets the verification of the Sysdig Secure API; default: true (we recommend only changing this to false when doing initial testing / evaluation of an on-premises installation)
scanner.verifyRegistryTLS: (true/false) Verify TLS from registries on image pull; default: true (we recommend only changing this to false when doing initial testing / evaluation)
scanner.psp.create: (true/false) Whether to create a psp policy and role / role-binding; default: false

Enable in Sysdig Labs (for Image Scanning)

Log in to Sysdig Secure as administrator and select Settings|User Profile.
Under Sysdig Labs, enable the Admission Controller feature and click Save.
The links to the Admission Controller pages will appear under Image Scanning in the left-hand navigation.

Upgrades

Upgrading from Scanning-Only Admission Controller

If you already have the Sysdig Admission Controller installed and want to upgrade:

helm upgrade \
--namespace sysdig-admission-controller \
--set features.k8sAuditDetections=true \
--reuse-values \
sysdig-admission-controller sysdig/admission-controller

For those customers who already have the Admission Controller AND already enabled Kubernetes audit logging via the legacy method, you can still install/upgrade to the new Admission Controller. Just be sure to set features.k8sAuditDetections=falseto avoid collecting and displaying duplicate events.

Uninstall the CLI-based Version

Deploy the following:

$ helm uninstall -n sysdig-admission-controller sysdig-admission-controller

2 -

Administration

The topics in this section are designed for platform administrators who may install Sysdig Monitor and Secure on-premises, deploy and configure the agents, and configure, administer, and troubleshoot the environment.

Administering Sysdig Platform

This section helps you navigate to the topics of administering the Sysdig user and notification management.

Feature Documentation	Description
Super Admin Management	Locate the super admin, super user, and login tokens.
User and Team Administration	Understand Sysdig’s users, teams, and role permissions.
Notifications Management	Add, edit, or delete a variety of notification channel types, and disable or delete notifications when they are not needed.
Find Your Customer ID and Name	Find customer ID for SaaS deployments.
Authentication and Authorization (SaaS)	Set up secure access control for SaaS deployments.

License Management

This section helps you navigate to the topics on License Management.

Feature Documentation	Description
Subscription	Review your account status regarding payment tier and licensed numbers of agents, serverless agents, cloud accounts.

Troubleshooting Sysdig Platform

This section helps you troubleshoot on-premises and agent installation.

Feature Documentation	Description
Troubleshoot On-Prem Installations	Review general issues and troubleshooting tips for on-prem installations. Troubleshoot On-Prem Installations.
Troubleshoot Sysdig Agent	Browse troubleshooting tips for Sysdig agents.
Contact Sysdig Support	Get help from Sysdig Support.

On-Premises Deployments

This section provides guidelines for deploying a Sysdig Platform on-premises.

Feature Documentation	Description
System Architecture	Understand the Sysdig Platform components and their relationships to each other and the environment.
System Requirements	Review the hardware components and software resources required to host the Sysdig Platform.

Installing Sysdig Platform On-Premises

When installing Sysdig Platform on-premises, follow the instructions specific to your environment. Where available, the Installer tool is the recommended option.

For release 3.6.0 and higher, this material has moved to version-specific folders in GitHub.

For legacy installation instructions, see On-Premises Installation.

For information on upgrading environments up to version 3.5.1, see On-Premises Upgrades. For upgrading beyond 3.6.0, there will be version-specific folders in GitHub.

Administration for Sysdig Platform On-Premises

This section helps you navigate to the topics on securing the Sysdig Platform components.

Feature Documentation	Description
Manage User Profile and Password	Access the current user’s login credentials, team, and role, and retrieve the API token to use with custom scripts or applications.
Securing User Passwords	Secure user credentials for Sysdig Platform components.
Authentication and Authorization (On-Prem Options)	Set up secure access control for the on-prem deployments.

2.1 -

Administration Settings

The Settings panel can be accessed from both Sysdig Monitor and Sysdig Secure UIs, and by both administrator and non-admin users.

Access Settings Panel

Access the panel from the lower left corner of the Sysdig Monitor or Sysdig Secure navigation bar.

Settings Panel: Admin vs User

The administrative tasks for the Sysdig platform are accessed through the Settings panel. Non-admin users have access to some of the pages; administrators have additional rights to manage Users, Teams, and to add licenses.

Logging in as Administrator reveals additional links in the Settings panel:


Administrator Settings	Non-Admin User Settings

2.1.1 -

User Profile and Password

Access the User Profile page to review and perform necessary actions:

Access the User Profile page:

Log in to Sysdig Monitor or Sysdig Secure and select Settings.
Select User Profile.
Review settings and perform actions below.

Review User’s Email, Role, and Current Team

The current user’s login email address, current team, and role on that team are listed in the User Profile section.

Change Admin Settings (visible to admins only)

If logged on as Administrator, you can access Admin Settings on this page which apply globally.

Hide Agent Install: Toggle this slider to hide the Agent Installation link in the Settings menu from non-admin users.
See also Navigate the Settings Panel: Admin vs User and Agent Installation: Overview and Key.

Retrieve the Sysdig API Token

When using the Sysdig API with custom scripts or applications, an API security token (specific to each team) must be supplied.

Log in to Sysdig Monitor or Sysdig Secure and select Settings.
Select User Profile.The Sysdig Monitor or Sysdig Secure API token is displayed (depending on which interface and team you logged in to).
You can Copy the token for use, or click the Reset Token button to generate a new one.
When reset, the previous token issued will immediately become invalid and you will need to make appropriate changes to your programs or scripts.

Change Your Password

Use the Password Management fields to change this user’s password.

Required: Minimum 8 characters, not the last password used.

Recommended: We advise following NIST’s most up-to-date recommendations, with an emphasis on length and uniqueness.

Enable Beta Functions from Sysdig Labs

Toggle the feature settings listed under Sysdig Labs to enable/disable specific beta functionalities to your installation. Data that has already been stored will not be affected by beta toggles.

(If there are no beta features, Sysdig Labs will not be displayed.)

2.1.1.1 -

Retrieve the Sysdig API Token

When using the Sysdig API with custom scripts or applications, an API security token (specific to each user+team) must be supplied.

Log in to Sysdig Monitor or Sysdig Secure and select Settings.
Select User Profile.The Sysdig Monitor or Sysdig Secure API token is displayed (depending on which interface and team you logged in to).
You can Copy the token for use, or click the Reset Token button to generate a new one.
When reset, the previous token issued will immediately become invalid and you will need to make appropriate changes to your programs or scripts.

2.1.2 -

User and Team Administration

This page describes the concepts behind Sysdig’s users, teams, and role permissions.

Understanding Sysdig Users

Users in Sysdig are identified by user name, email address, and password or third-party authentication option.

Users are either:

Invited manually by an Administrator via the Sysdig UI, or
Authenticated through a third-party system, or
Entered directly in the Sysdig database through the Admin API, which can bypass the invitation process if needed.

When invited, the new user is created in the Sysdig database upon the user’s first successful login to the Sysdig UI. Before the user accepts the invitation, enters a password, and logs in, they have a “pending” status.

System-Based Privileges

From the outset, users in the Sysdig environment have one of three types of system privileges

**(Super) Admin:**This is the administrator whose email address is associated with the Sysdig billing account. This user has administrator access to everything. Most relevant in on-prem installations.
Administrator: Any administrator can grant Admin system privileges to any user. Administrators are automatically members of all teams.
Administrators can create/delete users; create/configure/delete teams; create/delete notification channels; manage licenses; and configure Agents from links in the Settings menu that are hidden from non-admins.
User (non-admin): By default, new users have read/write privileges to create, delete, and edit content in the Sysdig interface. They do not see options in the Settings menu that are restricted to Administrators.
User rights are further refined based on team and team role assignments, as described below.

When a user is created, it is automatically assigned to a default team (described below).

Notice that this default workflow grants all new users Edit access.

Understanding Sysdig Teams

Teams can be thought of as service-based access control. Teams are created and assigned separately in Sysdig Monitor and Sysdig Secure.

Purpose of Teams

Organizing users into teams enables enforcing data-access security policies while improving users' workflows. There are different team roles, each of which has read/write access to different aspects of the app.

This limits the exposure of data to those who actually need it, and also makes users more productive by focusing them on data that is relevant to them.

The following are some potential use cases for Teams.

“Dev” vs “Prod”: Many organizations prefer to limit access to production data. Permits isolating physical infrastructure and the applications on top.
Microservices: Scoping data for individual dev teams to see their own dashboards and field their own alerts. Permits team creation based on logical isolation using orchestration or config management metadata in Sysdig Monitor.
Platform as a Service: Where Ops teams need to see the entire platform. Enabling certain people to see all data for all services as well as the underlying hardware. This is perfect for managed service providers who are managing a multi-tenant environment, or devops teams using a similar model within their own organization.
Restricted environments: Limiting data access for security and compliance. Certain services, such as authentication and billing, may have a very specific set of individuals authorized to access them.
Organizations that need to segment monitoring for efficiency: Wide-ranging use case from very large organizations forming teams to simplify access, to smaller orgs creating ephemeral troubleshooting teams, to teams formed to optimize QA and Support access to system data.

Operations Teams and Default Teams

Out of the box, the Sysdig Platform has one immutable team for each product. Depending on licensing, an organization may use one or both:

Monitor Operations team
Secure Operations team

Key traits of the immutable Operations teams:

The teams cannot be deleted
Users in Operations teams have full visibility to all resources in that product
Administrators must switch to the Operations team before changing configuration settings for any team

Administrators create additional teams and can designate any team to become the default team for that product. The number of teams allowed in an environment is determined by licensing.

Users entered in the Sysdig Monitor UI are auto-assigned to the Monitor default team; users entered in the Sysdig Secure UI are auto-assigned to the Secure default team.

If the Essentials tier is licensed, only the default teams and roles are enabled. See Subscription for more details.

If upgrading from Essentials to Enterprise, Capture functionality will become available. Users must go to Settings>Teams><Your Team> and check the Enable Captures box. They must then log out and log in again.

Team-Based Roles and Privileges

Users can be assigned roles that expand or limit their basic system privileges on a per-team basis.

System Role	Team Role
Admin	Member of every team, with full permissions regardless of team assignment. Can create/delete/configure all users. Can create/delete/configure all teams.
	Team Manager (Monitor)	Advanced User (Monitor)		Standard User (Monitor)
Non-Admin (Sysdig Monitor)	Can create/edit/delete dashboards, alerts, or other content + ability to add/delete team members or change team member permissions. NOTE: Team Managers only have user administration rights within the specific team(s) for which they are designated Managers.	Can create/edit/delete dashboards, alerts, or other content.		Equivalent to an Advanced User withno access to the Explore page (e.g. for developers who are not interested in Monitoring information).
	Team Manager (Secure)	Advanced User (Secure)	Service Manager (Secure)	Standard User (Secure)
Non-Admin (Sysdig Secure)	Same permissions as the Advanced User + ability to add/delete team members or change team member permissions.. NOTE: Team Managers only have user administration rights within the specific team(s) for which they are designated Managers.	Can access every Secure feature within the team scope in read and write mode. Advanced Users can create, delete, or update runtime policies, image scanning policies or any other content. The Advanced User cannot manage users. Free Tier users are automatically assigned to Advanced User role.	Same as Standard User, plus ability to invite existing users to the team and manage the notifications channels assigned to the team.	Can push container images to the scanning queue, view image scanning results, and display the runtime security events within the team scope. Standard Users cannot access Benchmarks, Activity Audit, Policy definitions, or certain write functions within other Secure features.
	See How Team Membership Affects Users' Experience of the UI for more detail.

How Team Membership Affects Users' Experience of the UI

Team membership affects user experience of the Sysdig Monitor or Sysdig Secure UIs in various ways.

At the highest level, the dashboards, alerts, and policy events you see are limited by the settings of the team you are switched to.

In more detail, team settings affect the:

Default landing page: The UI entry point is set on a per-team basis.
Explore tab and dashboards: These are set per-team, per-user, and can be shared with the team.
On first login, all team members see the same Dashboards Assigned to Me view. If a user changes those dashboards, only that user will see the changes.
Dashboards created while part of a team are only visible to the user when logged in to that team, and if shared, are only visible to other team members.
Visible data: A team’s scope settings limit the data visible to team members while they are switched to that team, even if a user belongs to other teams with different settings that reveal additional data. In Sysdig Secure, for example, only the policy events that fired within your scope will be visible.
Alert and Event: These settings are team-wide*.*** Any member of a team can change the team’s alert settings, and any additions or edits are visible to all members of the team.
Captures: Can only be taken on hosts/containers visible to team members, and members see only the list of captures initiated by other members who were switched to the current team.
API Token: Note that the Sysdig Monitor API Token found under Settings > User Profile is unique per-user, per-team. (See User Profile and Password . This is necessary to enable the generation of Custom Events via the API to target a specific team.

Switching Teams in the UI

Users can switch between all teams to which they’ve been assigned, and Administrators can switch between all teams that have been created.

To do so:

Click the Selector button in the lower-left corner of the navigation bar.
The assigned teams for this user are listed under Switch Teams.
NOTE: With version 3.6.0, you can also search for teams in the switcher.
Click another team name.
A popup window gives an overview of the new team-based view of the environment. The UI changes according to the team settings.

Onboarding Best Practices:

Plan teams and roles strategically to isolate access to data, customize interfaces, and streamline workflows.

In general, administrators should:

Create teams, invite users, and set roles in a planned manner
Start with some dashboards and alerts for given teams to get started with

Note: When a user logs in to a team for first time, they will see a wizard introducing dashboards, alerts, etc. specific to that team.

Restricting New User Rights by Default

By default, new users (added manually or through a third-party authenticator) are assigned Advanced User rights. If a administrator wants to limit new users' rights further, there are several ways to do so.

Between sending the invitation and the user’s first log in, change the user’s Role in the default Monitor team to Read User.
Note that there could theoretically be a lag in which the user would briefly have had Edit status.
Integrate users into Sysdig via the Admin API and define read-only permissions upon import.
Create a default team, in either Sysdig Monitor or Sysdig Secure, with very limited scope and visibility. Manually assign users to additional teams with broader permissions as needed.

Integrating Users and Teams via API

If you are working with Sysdig Support Engineers to provision users and teams via the Sysdig API, note how the user and team role names within the UI map to the API ROLE names.

User roles

Regular (non-admin) = ROLE_USER

Admin = ROLE_CUSTOMER

Team roles

Advanced user = ROLE_TEAM_EDIT

Standard user = ROLE_TEAM_STANDARD

View-only user = ROLE_TEAM_READ

Team manager = ROLE_TEAM_MANAGER

Service manager (Sysdig Secure only) = ROLE_TEAM_SERVICE_MANAGER

2.1.2.1 -

Manage Users

This page describes how to add, delete, and configure user information from within the Sysdig Monitor or Sysdig Secure UI.

Only Admin users can configure user account information.

Users added in Sysdig Monitor will appear in the full list of users for both Sysdig Monitor and Sysdig Secure, if both products are in use. However, users will not have log in access to Sysdig Secure until they are added to a Sysdig Secure team.

Create a User

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Users.
Click the Add User link.
Enter the user’s email address, first name and last name:
Click Save to send the user invite, or click Cancel to discard the user.
For on-premises environments, you may need to have pre-configured your SMTP parameters in your Replicated or Kubernetes installation configmap.

The new user will be added to the User Management table. Their status will be listed as Pending until the invitation is accepted.

Admin privileges cannot be assigned until the invitation has been accepted, and the user has logged into the interface for the first time. They can, however, be added to additional teams or have team-based roles assigned. For more information on configuring teams roles, refer to the Manage Teams and Roles documentation.

Edit User Information

To edit an existing user:

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Users.
Select the user from the User Management table.
Optional: Edit the first name / last name.
Optional: Toggle the Admin switch to enable/disable administrator privileges.
Click Save to save the changes, or Cancel to revert the unsaved changes.
User emails are read-only, and cannot be changed.

Delete a User

To delete an existing user:

Deleting a user cannot be undone. Any dashboards or explore groupings that the user created for any team will be permanently deleted.

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Users.
Select the user from the User Management table.
Click Delete User.
Click Yes, delete to confirm the change.

2.1.2.2 -

Manage Teams and Roles

The use of teams provides a strategic way to organize groups, streamline workflows, or protect data, as needed by an organization. Administrators who design and implement teams should have in-depth knowledge of organizational infrastructure and goals.

Only Advanced users can configure team permissions. Teams and roles must be assigned separately in Sysdig Monitor and Sysdig Secure.

For more information, including foundational concepts, see User and Team Administration.

Create a Team

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Teams.
Click Add Team.
Configure the team options and click Save.

For more information on each configuration option, refer to Table 1: Team Settings.

Ensure that the team names are unique in both Monitor and Secure products. For example, if you attempt at creating a team in Secure with the same name as one created in Monitor, you will see an error message stating that a team with the same name already exists and you will be prevented from creating the team.

Table 1: Team Settings

Setting	Req'd	Description
Color	Yes	Assigns a color to the team to make them easier to identify quickly in a list.
Name	Yes	The name of the team as it will appear in the “Switch to” drop-down selector and other menus.
Description	No	Longer description for the team.
Default Team	No	If users are not assigned to any team, they will automatically be a part of this team if it's turned on.
Default User Role	No	If no specific choice is made, Advanced User will be automatically selected. Choose a different role from the drop-down menu to set a different default user role for this team.
Default Entry Point	Yes	Defaults to the Explore page; choose an alternate entry if needed.
Team Scope	Yes	Determines the highest level the data to which team members will have visibility. Agent Metrics: If set to Host, Team members can see all Host-level and Container-level information. If set for Container, Team members can see only Container-level information. Prometheus Remote Write Metrics: Visible if Prometheus Remote Write is enabled for your account. Use this option to determine what level of Prometheus Remote Write data that your Team members can view. You can further limit what data team members can see by specifying tag/value expressions for metrics for each data source. The drop-down menu defaults to “is”, but can be changed to “is not”, “in”, "contains", and etc. Complex policies can be created by clicking Add another to create AND chains of several expressions. Note that making changes to the Team Scope settings can have a dramatic impact on what’s visualized in the Team’s Dashboards that are already configured, so you may want to carefully review these before/after your change.
Additional Permissions		Sysdig Capture: Enable this option to allow this team to take Sysdig Captures. Captures will only be visible to members of this team. WARNING: Captures will include detailed information from every container on a host, regardless of the team’s Scope. Infrastructure Events: Enable this option to allow this team to view ALL Infrastructure and Custom Events from every user and agent. Otherwise, this team will only see infrastructure events sent specifically to this team. AWS Data: Enable this option to give this team access to AWS metrics and tags. All AWS data is made available, regardless of the team’s Scope. Agent CLI: Enable this option to give this team access to Agent Console. Infrastructure Event: Enable this option to give this team access to infrastructure events.
Team Users	No	Click to select any non-Admin users to be immediately added to this Team. Admins are filtered out by default, since they are members of every team automatically.

Configure an Entry Page or Dashboard for a Team

Some Sysdig Monitor teams benefit from using a default entry point other than the usual Explore page, as users who don’t need in-depth monitoring information can onboard and navigate Sysdig Monitor more efficiently.

Use the Default Entry Point setting on the Team page, as shown in Create a Team.

Note: If selecting a dashboard, open the secondary Dashboard drop-down menu, or type the name of the dashboard to select it.

(The dropdown is only populated with shared dashboards accessible by anyone on the team.)

Add and Configure Team Members

Users can be assigned to multiple teams. Team assignment is made from the Team page (not the User page), and must be done by an Administrator or Team Manager.

Assign a User to a Team

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
SelectTeams.
Select the relevant team from the list, or search for it with the search box, and then select the relevant team.
In the Team Users section, click the Assign User button.
Select the user from the drop-down list, or search for it and then select it.
Click the Role drop-down menu to select the user role:
Optional: Repeat steps 3 to 5 for each additional user.
Click Save.

Assign Users a Team-Based Role

Review Team-Based Roles and Privileges for an overview.

Note that the Advanced User permission can be further refined into either a View-only user or a Team Manager.

Managers can add or delete members from a team, or toggle members' rights between Edit, Read, or Manager.

Note that Admins have universal rights and are not designated as Team Managers, Advanced Users, View-Only users, or Standard users.

Manager or Advanced User permissions can be assigned even to Pending users; administrators do not have to wait for the user’s first login to set these roles.

To assign a role to a user on a team:

Log in to Sysdig Monitor or Sysdig Secure as Administrator and either create a team or select a team to edit.
Add a user or select a user from the list of team members.
Select the appropriate role from the drop-down menu.
Reminder of the role privileges:
Admin: Member of every team with full permissions. Can create/delete/configure all users and teams.
Team Manager: Advanced User privileges + ability to add/delete team members or change team member permissions.
Advanced User:
In Sysdig Monitor: Read/write access to the components of the application available to the team. Can create/edit/delete dashboards, alerts, or other content.
In Sysdig Secure: Read/write access to the components of the application available to the team. Can create, delete, or update runtime policies, image scanning policies or any other content.
View-Only:
In Sysdig Monitor: Read access to the environment within team scope, but cannot create, edit, or delete dashboards, alerts, or other content.
In Sysdig Secure: Read access to every Secure feature in the team scope, but cannot modify runtime policies, image scanning policies or any other content.
Standard User:
In Sysdig Monitor: An Advanced User withno access to the Explore page (e.g. for developers who are not interested in Monitoring information).
In Sysdig Secure: Can send container images to the scanning queue, view image scanning results, and display the runtime security events within the team scope. Standard Users cannot access Benchmarks, Activity Audit, Policy definitions, or certain write functions within other Secure features.
Service Manager: Sysdig Secure only. Same as Standard User, plus ability to invite existing users to the team and manage the notifications channels assigned to the team.
Save edits.

Edit Team Configuration

To configure an existing team:

Log in to Sysdig Monitor or Sysdig Secure as administrator and selectSettings.
Select Teams.
Select the relevant team from the list, or search for it with the search box, and then select the relevant team.
Edit as needed, and click Save. For more information regarding the configuration options, refer to Table 1: Team Settings.

Delete a Team

When a team is deleted, some users may become “orphans”, as they are no longer a part of any team. These users will be moved to the default team.

The default team cannot be deleted. A new default team must be selected before the old default team can be deleted.

To delete a created team:

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings. ``
Select Teams.
Select the relevant team from the list, or search for it with the search box, and then select the relevant team.
Click Delete team, then Yes, delete to confirm the change.

2.1.3 -

Notifications Management

Alerts are used in Sysdig Monitor when Event thresholds have been crossed, and in Sysdig Secure when Policy violations have occurred. Alerts can be sent over a variety of supported notification channels.

Notification Management describes how to add, edit, or delete a variety of notification channel types, and how to disable or delete notifications when they are not needed, for example, during scheduled downtime.

2.1.3.1 -

Set Up Notification Channels

In the Settings panel of either Sysdig Monitor or Sysdig Secure, set up the notification channels to be used for alerting.

Notification channel management can be finessed by role-based access as follows:

Notification channels can now be “global” or limited to a particular team
Global channels can be managed by admins and can be viewed/used by other roles, while team-limited channels are available only to team members
Team Manager , Advanced User, and Service Manager (Secure) roles can create/update/delete team-scoped notification channels, they can also read and use the global ones
Standard and View Only roles can read team-limited and global notification channels
Admins will be able to create global notification channels and migrate channels from “global” to “team-limited”, and also from one team to another.

Add a Notification Channel

To add a new notification channel:

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Notification Channels.
The Notifications main page is displayed:
Click Add Notification Channel +, and select the desired notification channel.
Follow the channel-specific steps to complete the configuration process:

After you have set up a notification channel, it will appear as an available option to be assigned when you Add an Alert .

Edit a Notification Channel

To edit a notification channel:

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Notification Channels.
Locate the target channel and click the Edit button.
Make the edits and click Done Editing to save the changes.

Test a Notification Channel

To test a notification channel:

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Notification Channels.
Select the three dots next to a created Notification Channel and click Test Channel.

If a notification is not received within 10 minutes, the notification channel is not working, and the configuration should be reviewed.

2.1.3.1.1 -

Sysdig Monitor integrates easily with AWS Simple Notification Service (SNS).

On the AWS side:

To automatically push Sysdig Monitor alerts to the SNS topic of your choice:

From the AWS console, open the SNS management console
In the Create topic section, Create a new topic (if needed).
The topic’s Name, ARN, (optional) Display name, and Topic owner’s AWS account ID are displayed in the Details section.
Select the topic on the list.
Expand Access policy - optional.
Select Basic (By default).
Under Define who can publish messages to the topic, select Only the specified AWS accounts and enter the Sysdig Monitor account ID: 273107874544 (US-East Only).
For account IDs corresponding to other regions, see SaaS Regions and IP Ranges.
Click Create topic.
Ensure that you subscribe to the created topic.
1. On the navigation panel, choose Subscriptions.
2. On the Create subscription page, enter the Topic ARN of the topic you created earlier.
3. Specify other details and click Create subscription.

For further information about AWS SNS, refer to the AWS documentation.

For SNS notification, you can click the ‘help’ button for tips on setting up your SNS topic.

You will need to allow publishing rights to the Sysdig Monitor account ID corresponding to your region.

This can be done by creating a new policy on your SNS topic in AWS Console as shown in the below images:

Select “Edit topic policy” as shown below from “Other topic actions.”
In the “Basic view” tab of the “Edit topic policy” dialog, select “Only these AWS users” from the publisher’s list and enter the Sysdig ID.

In the Sysdig Monitor UI:

Complete steps 1-3 in Set Up a Notification Channel to log in to the Sysdig UI and select Amazon SNS Topic.
Enter the Topic created on the AWS side, along with a Channel Name, Enablement, and Notification toggles as appropriate.
From Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Click Save.

2.1.3.1.2 -

Email Notifications

To send an alert notification via email, you must first set up the email notification channel.

To do so, complete steps 1-3 in Set Up a Notification Channel, then:

Select Email.
Enter the relevant details for the email notification:
From Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Click Save.
If you enabled Test notification, a test email will be sent.

You can now configure an alert to use email notifications.

For on-premises environments, you may need to have pre-configured your SMTP parameters in your Replicated or Kubernetes installation configmap.

2.1.3.1.3 -

Team Email Notifications

You can notify all the users of a team when an alert is triggered. Sysdig allows you to create a new notification channel where you can select a team from a list of existing ones as the target of the channel.

To send an alert notification via email to a team, you must first set up the team email notification channel. To do so, complete steps 1-3 in Set Up a Notification Channel, then:

Select Team Email.
Enter the relevant details for the email notification:
Select the Team Name and specify a Channel Name to identify the channel you are creating.
From Shared With drop-down, choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Click Save.
If you enabled Test notification, a test email will be sent.

You can now configure an alert to use email notifications.

2.1.3.1.4 -

PagerDuty Notifications

To send an alert notification via PagerDuty, you must first set up the PagerDuty notification channel.

Prerequisites

Have an account configured at PagerDuty.com.
Have your PagerDuty credentials available (account, password, and service).
Have the base user role of Manager. With a PagerDuty base user role of Manager, you can auto-fetch the service information during the Sysdig/PagerDuty integration process.
Check your team and base user permissions. If your PagerDuty team permissions are Manager but base user permissions are Responder or lower, you can enter the necessary data in the Sysdig UI manually.
Base user roles in the PagerDuty UI.

Configure PagerDuty

To launch the process from the Sysdig UI, complete steps 1-3 in Set Up Notification Channels and select PagerDuty.
Do one of the following:
- Select Auto-fetch when prompted. Ensure that you have the base user role of Manager or higher in PagerDuty.
- Select Manual and enter the necessary configuration parameter. Skip to Step 5 for details.
Once you complete the pre-configuration, the PagerDuty Integration screen is displayed.
Do one of the following:
- Enter the email and password associated with your PagerDuty account, and click Sign In.
- Enter the appropriate PagerDuty subdomain for single sign-on and click Sign In Using Your Identity Provider.
A PagerDuty service selection screen is displayed.
- Option 1: If you have never integrated before, you are prompted to choose a PagerDuty Servicename.
- Option 2: If at least one service has already been integrated, you can select that one.
Click Connect.
Once integration is authorized, the Sysdig page for a new PagerDuty notification channel is displayed, with the information auto-filled.
From Shared With, choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Do one of the following:
- Confirm the auto-populated information and click Save.
- If you chose Manual entry in Step 2, then type the information and click Save.

You can now Add an Alert to use PagerDuty notifications.

Known Issues

There is a known issue whereby changing a notification from “Acknowledged” to “Unacknowledged” does not update correctly in PagerDuty.

What occurs:

Event has triggered Notification, Notification is sent to PagerDuty.
Open Event and click on “Acknowledge” button in Sysdig.
Notification is sent to PagerDuty, and status is changed to “Acknowledged.”
Open Event and click on “UnAcknowledge” button in Sysdig.
Status is not changed in PagerDuty. It remains “Acknowledged” rather than changing to “Triggered” in PagerDuty.

Unknown issues & support

Sysdig support section.

2.1.3.1.5 -

Slack Notifications

To send an alert notification via Slack you must first set up the Slack notification channel.

To do so:

Prerequisite:

Have a Slack account configured at Slack.com and know which notification channel to use for notifications.

To launch the process from the Sysdig UI, complete steps 1-3 in Set Up Notification Channels and select Slack.
You will be prompted to log in to your Slack account.
Select a Slack channel from the drop-down list to be used for notifications and click Authorize.
From Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Complete configuration as desired and click Done.
Click Test to check the new functionality.

You can now configure an alert to use Slack notifications.

2.1.3.1.6 -

VictorOps Notifications

To integrate with your VictorOps

Log in to VictorOps.
Go to Settings > Alert Behavior > Integrations in the VictorOps interface.
Select REST from the list of Featured Integrations.
Complete steps 1-3 in Set Up a Notification Channel to log in to the Sysdig UI and select VictorOps.
Enter the VictorOps parameters in the Sysdig Notification Channel fields, as follows:
API Key: everything between "/alert/" and “/$routing_key” in the REST URL
Routing Key: A VictoOps way of routing alerts to appropriate teams. See their Routing Keys documentation for details, if needed.
Channel Name: Choose a meaningful name like “VictorOps”.
Enable the channel and desired notification types.
From Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Click Save.

2.1.3.1.7 -

OpsGenie Notifications

Go directly to the OpsGenie Integrations Page to configure the integration on the OpsGenie side.
OpsGenie maintains documentation on how to integrate with Sysdig products (formerly called Sysdig Cloud) here.
Complete steps 1-3 in Set Up a Notification Channel to log in to the Sysdig UI and select OpsGenie.
Copy/paste your OpsGenie integration API key and add a Channel Name, Enablement, and Notification toggles as appropriate.
From Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Click Save.

2.1.3.1.8 -

Configure a Microsoft Teams Channel

Sysdig Monitor supports sending an alert notification to Microsoft teams. Microsoft Teams has different types of integrations for third-party applications, of which Sysdig supports Incoming Webhooks.

About Incoming Webhooks

Incoming Webhooks are a type of Connector in Teams that provide a simple way for an external app to share content in team channels. They are often used as tracking and notification tools. Microsoft Teams provides a unique URL to which you can send a JSON payload with the message that you want to POST, typically in a card format. Cards are UI containers that contain content and actions related to a single topic and are a way to present message data in a consistent way.

You will need to enter the URL that you copied from the Connector. Sysdig will format a message by using a custom card template and send it to the channel. The message will show up as a new notification in the Microsoft application.

Prerequisites

Have the destination URL handy. You can copy it from the Connectors > Incoming Webhook window on the Microsoft Teams UI. For more information, see Add an incoming webhook to a Teams channel.
Webhooks via HTTPS work only if a signed or valid certificate is in use.

Enable Microsoft Teams

Complete steps 1-3 in Set Up a Notification Channel and choose Microsoft Teams.
Enter the configuration options:
- URL: The destination URL you have copied from Microsoft Teams UI.
- Channel Name: Add a meaningful name for your Microsoft Teams channel.
- Enabled: Toggle on or off.
- Notification options: Toggle for notifications when alerts are resolved or acknowledged.
- Test notification: Toggle to be notified that the configured URL is working.
- Shared With: Choose whether to apply this channel globally. All Teams or to a specific team from the drop-down.
Click Save.

2.1.3.1.9 -

Configure a Webhook Channel

Sysdig Monitor and Sysdig Secure support sending an alert notification to a destination, such as a website, custom application, and so on for which Sysdig does not have a native integration. Do this using a custom Webhook channel.

Prerequisites

Webhooks via HTTPS only work if a signed/valid certificate is in use.
Have your desired destination URL on hand.

Enable Webhook

Complete steps 1-3 in Set Up a Notification Channel and choose Webhook.
Enter the webhook channel configuration options:
- URL: The destination URL to which notifications will be sent.
- Channel Name: Add a meaningful name, such as Ansible, PagerDuty, OpsGenie, and so on.
- Enabled: Toggle on and off notifications.
- Notification options: Toggle for notifications when alerts are resolved or acknowledged.
- Test notification: Toggle to be notified that the configured URL is working.
- Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
- Allow insecure connections: Enable if you want to skip the TLS verification.
- Custom headers: Add custom headers to your alert notification.
  If your webhook integrations require additional headers you specify them by using a custom header.
  For example, Ansible uses token-based authentication, which requires an entry for the bearer token. This entry is not included in the default header, but you can add it using a custom header.
  Alternatively, you can choose to add custom headers programmatically as described in Configure Custom Headers and Custom Data Programmatically.
- Custom Data: Specify the custom data you want to attach to the alert notification. The data must be a valid JSON document. This information will be included in the request body of the HTTP call. Systems that receive these webhook alerts can parse the data and take action based on the contents.
Click Save.

When the channel is created, you can use it on any alerts you create.

Then, when the alert fires, the notification will be sent as a POST in JSON format to your webhook endpoint. (See Alert Output, below.)

For testing purposes, you can use a third-party site to create a temporary endpoint to see exactly what a Sysdig alert will send in any specific notification.

Configure Custom Headers and Custom Data Programmatically

By default, alert notifications follow a standard format (see Description of POST Data, below).

However, some integrations require additional headers and/or data, which you can append to the alert format using a custom header or custom data entry.

For example, Ansible uses token-based authentication, which requires an entry for the bearer token. This entry is not included in the default alert template built into Sysdig, but you can add it using a custom header.

In addition to the Webhook UI option, you can do this from the command line, as described below.

additionalHeaders is usually used for authentication
customData is used to add values to the alert

After it has been created via the API, any manipulation will mangle the notification channel. Use with care.

Sample Use Case

This example adds two custom headers and defines additional custom data, as well as the format for that data.

Use the curl command to retrieve all configured notification channels:

curl -X GET https://app.sysdigcloud.com/api/notificationChannels -H 'Authorization: Bearer API-KEY'

Add the custom headers and execute the request:

curl -X PUT https://app.sysdigcloud.com/api/notificationChannels/1 -H 'Authorization: Bearer API-KEY' -H 'Content-Type: application/json' -d '{
  "notificationChannel": {
    "id": 1,
    "version": 1,
    "type": "WEBHOOK",
    "enabled": true,
    "name": "Test-Sysdig",
    "options": {
      "notifyOnOk": true,
      "url": "https://hookb.in/v95r78No",
      "notifyOnResolve": true,
      "customData": {
        "String-key": "String-value",
        "Double-key": 2.3,
        "Int-key": 23,
        "Null-key": null,
        "Boolean-key": true
      },
      "additionalHeaders": {
        "Header-1": "Header-Value-1",
        "Header-2": "Header-Value-2"
      }
    }
  }
}'

Standard Alert Output

Alerts that use a custom webhook for notification send a JSON-format with the following data.

Description of POST Data

{
  "timestamp": 1620222000000000, // Time when the alert triggered in microseconds
  "timespan": 60000000, // duration of the alert in microseconds (how long the value should be true before triggering)
  "alert": {
    "severity": 2, // severity from 0 to 7, use severityLabel for a human readable version
    "editUrl": "https://app-staging.sysdigcloud.com/#/alerts/21998727", // alert edit URL
    "severityLabel": "Medium", // human readable version of severity
    "subject": "CPU temp is High on homebridge:9100 is Triggered", // Alert subject
    "scope": null, // scope of the alert if set from the UI
    "name": "CPU temp is High", // name of the alert
    "description": null, // description, not used ATM
    "id": 21998727, // alert id
    "body": "CPU temp is High on homebridge:9100 is Triggered\n\n\nEvent Generated:\n\nSeverity:         Medium\n    Metric:\n    node_hwmon_temp_celsius = 65.8121\nSegment:\n    instance = 'homebridge:9100'\nScope:\n    Everywhere\n\nTime:             05/05/2021 01:40 PM UTC\nState:            Triggered\nNotification URL: https://app-staging.sysdigcloud.com/#/events/notifications/l:2419200/14918845/details\n\n------\n\nTriggered by Alert:\n\nName:         CPU temp is High\nTeam:         Monitor Operations\nScope:\n    Everywhere\nSegment by:   instance\nWhen:         avg(avg(node_hwmon_temp_celsius)) > 40\nFor at least: 1 m\nAlert URL:    https://app-staging.sysdigcloud.com/#/alerts/21998727\n\n\n"
  },
  "event": {
    "id": 14918845, // id of the generated event
    "url": "https://app-staging.sysdigcloud.com/#/events/notifications/l:604800/14918845/details" // url of the event in the feed
  },
  "state": "ACTIVE", // status of the alert, can be ACTIVE or OK
  "resolved": true,
  "entities": [ // list of entities that triggered the alert, at the moment we send a notification per entity, so this array will always contain a single object
    {
      "entity": "instance = 'homebridge:9100'", // segment that triggered
      "metricValues": [ // value of the metric at the time of triggering
        {
          "metric": "node_hwmon_temp_celsius",
          "aggregation": "avg",
          "groupAggregation": "avg",
          "value": 65.812167
        }
      ]
    }
  ],
  "endEntities": [ // list of entities when the alert was resolved (same as "entities")
    {
      "entity": "instance = 'homebridge:9100'",
      "metricValues": [
        {
          "metric": "node_hwmon_temp_celsius",
          "aggregation": "avg",
          "groupAggregation": "avg",
          "value": 39.812167
        }
      ]
    }
  ],
  "condition": "avg(avg(node_hwmon_temp_celsius)) > 40", // alert condition in string form
  "source": "Sysdig Cloud", // source of the event
  "labels": { // list of labels associated to this event (they strongly depend on the segmentation and scope of the alert)
    "instance": "homebridge:9100"
  }
}

Example of Failure

$ curl -X GET https://app.sysdigcloud.com/api/notificationChannels -H 'authorization: Bearer dc1a42cc-2a5a-4661-b4d9-4ba835fxxxxx’'

{"timestamp":1543419336542,"status":401,"error":"Unauthorized","message":"Bad credentials","path":"/api/notificationChannels"}

Example of Success

$ curl -X GET https://app.sysdigcloud.com/api/notificationChannels -H 'Authorization: Bearer dc1a42cc-2a5a-4661-b4d9-4ba835fxxxxx'
{"notificationChannels":[{"id":18968,"version":2,"createdOn":1543418691000,"modifiedOn":1543419020000,"type":"WEBHOOK","enabled":true,"sendTestNotification":false,"name":"robin-webhook-test","options":{"notifyOnOk":true,"url":"https://postb.in/6dtwzz7l","notifyOnResolve":true}}]}
$

The webhook feature is used to integrate the following channels:

Configure ServiceNow

2.1.3.1.10 -

Configure ServiceNow

Sysdig can be integrated with ServiceNow using a custom webhook.

ServiceNowSetup

Prerequisites

Have a ServiceNow account set up and working.
If necessary, refer to ServiceNow developer documentation here.

Create Scripted Rest API Details in ServiceNow GUI

Login to ServiceNow (developer entry) and create a Scripted REST API:
Click New and submit the form with the following:
Name: SysdigAlert API ID: sysdigalert
Return to the Scripted REST APIs and open the resource just created.
Scroll down to the related list area, select Resources, and click New. This will create a new Scripted REST API resource.
Fill in the Name field e.g. Demo.
Scroll down to Security and clear the checkbox that requires authentication.
Change the HTTP method from GET to POST.

The resource is created.

Add Code to the New Scripted API

Now give the resource the code to execute.

The default objects to work with in a Scripted REST API Resource are response and request.

For more details on request and response see Scripted_REST_Request_API and Scripted_REST_Response_API

The created resource will already have some example code:

(function process(/*RESTAPIRequest*/ request, /*RESTAPIResponse*/ response) {

    // implement resource here

})(request, response);

Change this default code to:

(function process(/*RESTAPIRequest*/ request, /*RESTAPIResponse*/ response) {

 gs.info(request.body.dataString);

})(request, response);

Note the following resource path to this newly created resource is now visible: /api/snc/sysdigalert.
The url to this resource would be https://yourInstance.service-now.com/<resource_Path or https://yourInstance.service-now.com/api/snc/sysdigalert ``
Click Submit/Update on this resource.

Sysdig Webhook Setup

Now that the custom API endpoint in ServiceNow is created, you can configure Sysdig alerts to use a custom webhook to trigger the ServiceNow integration.

API URL: your instance name URL

Name: ServiceNow (or whatever name you’d like for this Sysdig alert webhook)

Notify when OK: Optional

Notify when Resolved: Optional

Test Notification: Use this toggle and/or set up a test alert as described in the following section.

Test Integration

To test if this ServiceNow integration is set up and working correctly, you can set up a test alert to trigger.

For example, you could create an alert for CPU usage:

In ServiceNow, navigate to System Log > All to see a sample triggered webhook.

2.1.3.1.11 -

Configure IBM Cloud Functions Channel

Sysdig supports automatically sending alert notifications to an IBM Cloud Functions Channel. You generally use it for the following use cases.

Configure an IBM Functions as a new notification channel in Sysdig Monitor.
Pass extra parameters to IBM Functions.
Modify an IBM Functions.
Delete an IBM Functions.

The following notification channel types are supported:

Public (with or without X-Require-Whisk-Auth header)
Private (using IAM token)

To configure IBM Cloud Functions Channel:

Log in to the Sysdig UI and select IBM Cloud Functions Channel by completing steps 1-3 as described in Set Up a Notification Channel.
Specify the channel URL.
You can specify one of the following channel types.
- Public: The public channels are of type IBM Web Action. They require no authentication and can be used to implement HTTP handlers that respond with headers, status code, and body content of different types.
  URL: An example URL is https://eu-gb.functions.cloud.ibm.com/api/v1/namespaces/13eeeeee-623b-4776-ba35-4065bcbfee7b/actions/hello-world/myaction
- Private: The private channels are of type IAM Secured action. These actions require IAM token-based authentication.
  URL: An example URL is https://eu-gb.functions.cloud.ibm.com/api/v1/web/namespaces/13eeeeee-623b-4776-ba35-4065bcbfee7b/actions/hello-world/helloworld?param=true
Continue with one of the following:
- Configure a Private Channel
- Configure a Public Channel

Configure a Private Channel

Specify the following:

IAM API Key:
Channel Name: A unique name to identify the channel.
Enable the channel and desired notification types:
- Enabled: The toggle button to enable or disable the IBM channel.
- Notify when Resolved: Send a new notification when the alert condition is no longer triggered. Enable or disable the notification toggle as appropriate.
- Notify when Acknowledged: Send a new notification when the alert is manually acknowledged by a user. Enable or disable the notification toggle as appropriate.
- Test notification: Send a notification when the changes are saved. Enable or disable the notification toggle as appropriate.
Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Additional Parameters: Specify optional parameters to pass to the function. For example, name: jane is passed to the action as {name: "Jane"}.

Configure a Public Channel

Specify the following:

Whisk Auth Token (optional): Optionally provide the Whisk authentication token. If you specify one, the public channel (web action) can only be invoked by requests that provide appropriate authentication credentials. See Securing web actions for more details.
Channel Name: A unique name to identify the channel.
Enable the channel and desired notification types:
- Enabled: The toggle button to enable or disable the IBM channel.
- Notify when Resolved: Send a new notification when the alert condition is no longer triggered. Enable or disable the notification toggle as appropriate.
- Notify when Acknowledged: Send a new notification when the alert is manually acknowledged by a user. Enable or disable the notification toggle as appropriate.
- Test notification: Send a notification when the changes are saved. Enable or disable the notification toggle as appropriate.
Shared With: Choose whether to apply this channel globally (All Teams) or to a specific team from the drop-down.
Additional Parameters: Specify optional parameters to pass to the function. For example, hostname: BLR is passed to the action as {hostname: "BLR"}. The URL would be /demo/hello.http?hostname=BLR.

2.1.3.2 -

Disable or Delete a Notification Channel

Temporarily Disable a Notification Channel

To temporarily disable a notification channel:

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Notification Channels.
Toggle the Enabled slider to off.

Mute Notifications During Downtime

Administrators can choose to turn off all alert events and notifications if desired, for example, during a scheduled system downtime.

Muting notifications affects all channels globally. When muting is switched on, no notifications will be sent through any configured channel. You can choose whether to notify specific channels that notifications are temporarily disabled. Muting and re-enabling notifications is a MANUAL process.

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Notification Channels.
Select the Downtime toggle.
Optional: check the **Yes**box to Notify Channels when prompted, and select the desired channels.
At this time, only Email and Slack channels can be notified when downtime is started/stopped.

Delete a Notification Channel

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Notification Channels.
Select the three dots next to a created channel and click Delete Channel.

Configure an Alert Start-Up Delay (On-Premises Only)

Sysdig alert jobs begin immediately at start-up. However, in instances where Sysdig goes down unexpectedly, or without proper shutdown/startup procedures implemented, data can be missing, triggering alert notifications.

A start-up delay in alert jobs can be configured in on-premises environments, by setting the draios.alerts.startupDelay parameter. The parameter requires a duration value; the example below shows a duration of 10 minutes:

draios.alerts.startupDelay=10m

This parameter can be configured for either Replicated or Kubernetes environments:

For Replicated environments, add the parameter to the Sysdig application JVM options list. For more information, refer to the Sysdig Install with Replicated documentation.
For Kubernetes environments, add the parameter to the **sysdigcloud.jvm.worker.options **parameter in the configmap. For more information on editing the configmap refer to the On-Premises Installation documentation.

2.1.4 -

AWS: Integrate AWS Account and CloudWatch Metrics (Optional)

When the Sysdig agent is installed in an AWS environment, the Sysdig Platform can collect both general metadata and various types of CloudWatch metrics.

There are three ways to integrate an AWS account into Sysdig:

By manually entering an AWS access key and secret key, and manually managing/rotating them as needed
By passing a parameter that allows Sysdig to autodetect an AWS ECS role and its permissions, passing an “implicit key” (On-Prem only).
The implicit option requires no manual key rotation as AWS handles those permissions behind the scenes.
Using AWS Role delegation. Role delegation is an alternative to the existing integration methods using the access keys. This method is considered secure as sharing developer access keys with third-parties is not recommended by Amazon.

The Sysdig Monitor UI includes links to help easily integrate CloudWatch metrics into Sysdig Monitor, as described below.

After integrating with an AWS account, data will become visible in the Sysdig UI after a 10-15 minute delay.

Entry Point in the Sysdig UI

The Sysdig interface prompts you to perform this integration from the administrator’s Settings menu.

Once an agent has been installed, log in to Sysdig Monitor or Sysdig Secure as administrator to perform integration steps or review/modify existing AWS settings.

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Choose AWS Accounts.
A page showing manual key integration, with access key and secret key fields displayed.
NOTE: If there is no AWS integration yet then click on ADD and provide the access key and secret key.

Integrate AWS Account Manually

Have your AWS EC2 account details available. Integration begins on the AWS side and is completed in the Sysdig Monitor UI.

In AWS

Create an IAM Policy for Sysdig Access

You could use the existing IAMReadOnly policy instead, but creating a Sysdig-specific policy provides more granular access control, the activity can be easily distinguished in CloudTrail, and it is considered best practice.

In AWS, select IAM and create a policy to be used for Sysdig. (Sample policy name: SysdigMonitorPolicy.)
Using the JSON editor view, copy/paste the Sysdig-specific policy code into the new policy and save it.
You can review the policy in the Visual Editor.

When reviewing the completed policy in the Visual editor, you should see something like:

Create an IAM User and Grant Programmatic Access

Use an existing IAM user, or (best practice) create a specific IAM user for the Sysdig Backend to programmatically access CloudWatch and use its data.

In the IAM Console, add a User.
Select AWS Access Type: Programmatic Access.
Select ‘Attach existing policies directly’, search for and then select the newly created policy (Sample policy name: SysdigMonitorPolicy.)
Select ‘Create User’ option.
Copy and save the resulting access key and secret key (Note: the Secret is only displayed once, so make sure to download the credentials file or store the key securely that you can reference again.)

In the Sysdig Monitor UI

Enter the Access and Secret Key

Log in to Sysdig Monitor or Sysdig Secure as the administrator and select Settings.
Select AWS.
Add an account by entering the **User Access Key**and **Secret Key**and clicking Save.
The Credentials will be listed with a Status of **OK**checked.

Should an Error occur, double-check the credentials entered. Mis-typing is the most common cause of errors.

Enable CloudWatch Integration

Navigate to the AWS page in the Sysdig Monitor UI, if you are not already there.
Toggle the **CloudWatch Integration Status**to Enabled.
Sysdig Monitor will poll the CloudWatch API every five minutes. Note that this incurs additional charges from AWS.

After integrating with an AWS account, data will become visible in the Sysdig UI after a 10-15 minute delay.

Refetch Credentials

If the integrated AWS account changes on the AWS side, an Error will be listed in the Credentials Status on the Settings > AWS page.

Use the Refetch Now button to re-establish the integration.

Integrate AWS Account Using the Implicit Key (On-Prem Only)

If Sysdig is installed in an EC2 instance, you can take advantage of the existing EC2 IAM role of that instance. This can simplify administration, as you do not have to manually rotate public and private keys provided to the Sysdig backend.

Use Implicit Key

Prerequisites

Have your on-premises Sysdig platform installed in an AWS EC2 instance that has a proper IAM role.

For this option, you cannot use the AWS Integration step in the Welcome Wizard.

To enable implicit key, you must set the following parameter:

-Ddraios.providers.aws.implicitProvider=true

Use the parameter either during initial installation, or, if you already entered keys manually, to switch to an implicit key.

If switching, you must then restart the api, worker, and collector components in the backend.

In the Settings > AWS page, the former credentials will be overwritten it will show implicit key.

Enablement steps depend on whether you are using Kubernetes or Replicated as your orchestrator.

Kubernetes

Edit the config.yaml to add to the following entries (in the Data section of config.yaml):

sysdigcloud.jvm.api.options:
sysdigcloud.jvm.worker.options:
sysdigcloud.jvm.collector.options:

If you are switching from manual to implicit keys, you must also restart the API, worker, and collector components.
See To Make Configuration Changes for details.
Enable Cloudwatch integration in the Sysdig UI.

Changing the AWS Services that are Polled

Sysdig is designed to collect metadata for particular AWS services, which are reflected in the IAM policy code.

The services are:

DynamoDB
EC2 hosts
ECS
Elasticache
RDS
SQS

When you implement the code and integration steps as described above, it will trigger two types of collection: first the metadata for each service is collected, and then Sysdig will poll for the metrics about the metadata returned. So, if the service is not enabled in your environment, no metadata (and no metrics) are collected about it. If it is enabled, but you do not want to poll metrics, then delete the lines of code related to that service from the IAM policy. This will avoid potential unwanted AWS API requests and potential AWS charges.

Security Groups

If you have an on-premises Sysdig Backend, and have restricted outbound security groups, you may need to allow HTTPS & DNS access in order for the Sysdig Backend components to make connection to the Amazon APIs. As Amazon API endpoints are referenced by name and have a large number of IP’s, this may need to be full 0.0.0.0/0 outbound access for HTTPS & DNS.

If you need to filter just to Amazon IP ranges, you can use the following as a guide: https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

Retrieving CloudWatch Data for Particular AWS Regions

To enable metrics collection from only certain AWS regions in your environment, it is necessary to open a ticket with Sysdig Support. See Contact Support for details.

For information on the resulting AWS services visible in Sysdig Monitor, see the AWS-related information in the Metrics Dictionary (also available from within the Sysdig Monitor UI).

For information on how licensing affects AWS service views, see About AWS Cloudwatch Licensing.

2.1.4.1 -

IAM Policy Code to Use

Best Practice: Create a Sysdig-specific IAM policy to be used for granting programmatic access to Sysdig. Copy/paste the code snippet below into this policy. It enables Sysdig to collect metadata and CloudWatch metrics from the following services, as applicable to your environment:

Dynamodb
EC2 hosts
ECS
Elasticache
RDS
SQS

If you want to use your own AWS S3 bucket to store Sysdig capture files, you can append those code snippets to this IAM Policy as well. See Storage: Configure AWS Capture File Storage (Optional) for details.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "autoscaling:Describe*",
                "cloudwatch:Describe*",
                "cloudwatch:Get*",
                "cloudwatch:List*",
                "dynamodb:ListTables",
                                "dynamodb:Describe*",
                "ec2:Describe*",
                "ecs:Describe*",
                "ecs:List*",
                "elasticache:DescribeCacheClusters",
                "elasticache:ListTagsForResource",
                "elasticloadbalancing:Describe*",
                "rds:Describe*",
                "rds:ListTagsForResource",
                "sqs:ListQueues",
                "sqs:GetQueueAttributes",
                "sqs:ReceiveMessage"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

See Changing the AWS Services that are Polled for more detail.

2.1.4.2 -

Integrate with AWS Role Delegation

This section describes how to configure Sysdig Monitor to utilize the Amazon Web Service (AWS) AssumeRole functionality and authorize Sysdig Monitor to discover cloud assets, grab CloudWatch metrics from your AWS account, and utilize custom S3 bucket for storing captures. Upon integrating with an AWS role, you can delegate access to AWS resources that are not associated with your Sysdig AWS account.

Setting up cross-account access through roles eliminates the need to create individual IAM users in each account. In addition, users don’t have to sign out of one account and sign in to another in order to access resources in different AWS accounts.

Role delegation is an alternative to the existing integration method using the access keys. This method is considered secure as sharing developer access keys with third-parties is not recommended by Amazon.

Prerequisites and Guidelines

This topic assumes that you have the following ready and you are familiar with AWS.

Sysdig Monitor API Token
External ID
API endpoint. In this topic, it is referred to as {{host}}
- SaaS: See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, in US East, the endpoints are:
  - Monitor: https://app.sysdigcloud.com
  - Secure: https://secure.sysdig.com
  For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.
- On-Prem: Depends on the on-prem deployment.
Administrator privileges to configure AWS integration
API client. Examples in this topic use curl
AWS account ID
- SaaS: The default AWS account ID is 273107874544 (US East region). For other regions, check AWS account IDs .
- On-Prem: Customer-specific.

Enable AWS Role Delegation with API

This section describes how to enable AWS role delegation using an API.

Instructions for SaaS

Instructions for On-Prem

Get Your External ID

Retrieve your external ID as follows:

curl -k --request GET \

--url host/api/users/me \

--header 'authorization: Bearer e71d7c0f-501e-47d4-a159-39da8b716f44' | jq '.[] | .customer | .externalId'

An example of External ID from the response will be 04acdd59-4c98-4d11-8ee5-424326248161.

Configure Role Delegation

Integrating the Sysdig Platform with Amazon Web Services requires configuring role delegation using AWS IAM.

Create a new role in the AWS IAM Console:
1. For the role type, select Another AWS account.
2. (SaaS) Enter the Sysdig account ID for Account ID.
  This means that you are granting read-only access to your AWS data.
3. Select Require external ID and enter the one you retrieved in the previous step. Leave MFA disabled.
Click Next: Permissions.
Create the following policies:
- sysdig_cloudwatch: Gives access to the list and describe supported AWS resources and get CloudWatch metrics for them.
- sysdig_s3: Defines the bucket name where we wish to store the captures
  For more information on policies, see IAM Policy Code to Use.
For detailed instructions on how to create a policy, see Integrate AWS Account Manually.
1. If a policy has already been created, search for it on this page and select it, then skip to step. Otherwise, click Create Policy, which opens in a new window.
2. Click Review policy.
3. Name the policy and provide an apt description. For example, sysdig_cloudwatch.
4. Click Create Policy.
  You can now close this window.
In the Create role window, refresh the list of policies and select the policies you just created.
Click Next: Review.
Give the role a name and an apt description. For example, sysdig_role.
Click Create Role.

Get Role ARN

Select Roles > sysdig-role.
Copy Role ARN.

Add the AWS Account

Using the role that you have created, add an AWS account on the Sysdig Monitor side. Use the following API call:

curl --request POST \
  --url {{host}}/api/providers \
  --header 'authorization: Bearer e71d7c0f-501e-47d4-a159-39da8b716f44' \
  --header 'content-type: application/json' \
  --data '{"name": "aws","credentials": {"role": "<Role_ARN>"},"alias": "role_delegation"}'

Replace <Role_ARN> with the one that you have copied in the previous section.

The response lists all the providers. An example response is given below:

{

  "provider": {

    "id": 7,
    "name": "aws",
    "credentials": {

      "id": "role_delegation",
      "role": "arn:aws:iam::485365068658:role/sysdig-access3"
    },
    "tags": [],
    "status": {

      "status": "configured",
      "lastUpdate": null,
      "percentage": 0,
      "lastProviderMessages": []
    },
    "alias": "role_delegation"
  }
}

Verify the role delegation has been created.

Log in to Sysdig Monitor or Sysdig Secure as administrator.
Select Settings > AWS.
The role that you have been created will be added to the list of AWS Accounts.
Proceed to enable CloudWatch and AWS S3 bucket.
See AWS: Integrate AWS Account and CloudWatch Metrics (Optional) for more information.

Additional Configuration for On-Prem

Create an AWS user that will be used to fetch temporary credentials.

Assign a policy to the user to allow AssumeRole. For example:

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": "arn:aws:iam::{ACCOUNT-ID}:role/{ROLE_NAME}*"
  }
}

Make the access keys available to users from one of the sources:
- Environment variables
- Java system properties
- Instance profile credentials delivered through the Amazon EC2 metadata service.
  EC2 metadata service is recommended if the installation is on AWS.

Example: Set Environment Variables on a Kubernetes Installation

Create Secret:

apiVersion: v1
kind: Secret
metadata:
  name: aws-credentials
type: Opaque
data:
  aws.accessKey: {{BASE64_ENCODED_ACCESS_KEY_ID}}
  aws.secretKey: {{BASE64_ENCODED_ACCESS_KEY_SECRET}}

Expose variables in deployment descriptors (sysdigcloud-collector, sysdigcloud-worker, sysdigcloud-api) and reference values in the newly created secret:
```
- name: AWS_ACCESS_KEY_ID
    valueFrom:
    secretKeyRef:
        key: aws.accessKey
        name: aws-credentials
- name: AWS_SECRET_ACCESS_KEY
    valueFrom:
    secretKeyRef:
        key: aws.secretKey
        name: aws-credentials
```
Add variables to descriptors on each platform update until new variables are part of the installer.

Set Up Resource Discovery

The supported AWS are EC2, RDS, Elastic Load Balancer (ELB), ElastiCache, SQS, DynamoDB, and Application Load Balancer (ALB).

By default, all the resources are fetched for all regions supported by AWS. You can avoid this by whitelisting regions when creating a provider key via the API. Example body of the provider key request when whitelisting regions:

{
    "name": "aws",
    "credentials": {
        "role": "arn:aws:iam::676966947806:role/test-assume-role"
    },
    "additionalOptions": "{\"regions\":[\"US_EAST_1\",\"US_EAST_2\"]}"
}

Enable AWS Role Delegation with UI

Use the AWS option in the Settings menu to configure AWS role delegation.

Log in to the Sysdig Monitor as an administrator and select Settings.
Click AWS.
The AWS Account page is displayed.
Click Add Accounts.
The Identity Authentication page opens to the Role Delegation tab.
Specify the following:
- Role ARN: The Role ARN associated with the role you have created for role delegation. The ID is available on the summary page of the role on the AWS console. For more information, see Integrate with AWS Role Delegation.
- AWS External ID: Ensure that AWS External ID is displayed on the page.
  - SaaS: For account IDs corresponding to your region, see SaaS Regions and IP Ranges
  - On-Prem: Customer-specific.
Click Save.

2.1.5 -

Storage: Configure Options for Capture Files

The Sysdig Capture feature allows you to record detailed system trace data via remote connection from any of your agent-installed hosts. In SaaS installations, by default, this data will be stored on Sysdig’s secure Amazon S3 storage location. This location will have a separate partition for your account. In on-premises installations, by default, the data will be stored in the Cassandra database.

This page describes two custom alternatives: using an AWS S3 bucket (available for SaaS and on-prem) and using custom S3 storage.

Storage Options	SaaS	On-Prem
Sysdig Provided Storage	Sysdig provided	Installation provided Cassandra AWS S3 S3 Compatible
AWS S3	Configure using the API or the UI	Installation provided Configure using the API or the UI
S3 Compatible	Using the API	Installation provided

Sysdig Provided Storage

Sysdig provided

Installation provided

AWS S3

Configure using the API or the UI

Installation provided
Configure using the API or the UI

S3 Compatible

Using the API

Installation provided

if Google Cloud Storage is used as the S3 compatible storage, you will not be able to bulk delete captures due to compatibility issues with Google’s S3 API implementation. You can delete captures one by one or delete them directly from the Google console.

Configure AWS S3 Storage

To configure this option, use the fields provided by Sysdig Settings UI and then append some code to the IAM Policy you created in AWS for Sysdig integration.

Prerequisites

Your AWS account must be integrated with Sysdig, but the CloudWatch feature is not required to be enabled.
See AWS: Integrate AWS Account and CloudWatch Metrics (Optional)
Ensure that your S3 bucket name is available.

To use your own AWS S3 bucket to store Sysdig capture files, append the following code snippets to the **AWS Identity and Access Management (IAM) **page.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:Put*",
                "s3:List*",
                "s3:Delete*",
                "s3:Get*"
            ],
            "Effect": "Allow",
            "Resource": [
            "arn:aws:s3:::BUCKET_NAME",
            "arn:aws:s3:::BUCKET_NAME/*"
            ]
        }
    ]
}

If you are using AWS KMS for AWS S3 encryption, ensure that necessary privileges are given to the Sysdig Account or Role to use the custom key.
Use the Key users option to do so:

In the Sysdig UI

Log in as Administrator to Sysdig Monitor or Sysdig Secure.
From the Selector button in the lower-left navigation, select Settings >Sysdig Storage.
Enable the **Use a custom S3 bucket**toggle and enter your AWS S3 bucket name.

To Test: Capture a Trace File in Sysdig Monitor UI

When enabled, you will have the option to select between “Sysdig Monitor Storage” or your own storage bucket when configuring a file capture. See Create a Sysdig Capture File.

(SaaS) Configure Custom S3 Storage Endpoint

You can set up a custom Amazon-S3-compatible storage, such as Minio or IBM Cloud Object Storage, for storing Captures in a Sysdig SaaS deployment. The capture storage location can be used for both Sysdig Monitor and Sysdig Secure. This is an API-only functionality and currently, no UI support is available.

The following APIs are supported for this functionality:

List existing AWS integrations
Create a new AWS integration
Update an existing AWS integration
Configure storage configuration

Prerequisites

Ensure that the feature is enabled for your account.
Use the access key and secret as the credentials.
Configure a new AWS integration. Set the skipFetch field to true. This will cause the AWS integration to ignore fetching data from this account. Therefore, both the AWS metadata and AWS CloudWatch will not be fetched and you can use this storage exclusively for Sysdig Capture.
Configure the storage interface with the new account, by specifying the AWS integration ID to use to authenticate the endpoint, bucket name, and the path specified in the bucket.

Limitation

The AWS account ID is currently shown as null on the UI.

List Existing AWS Integration

The API returns the list of configured AWS integrations.

REST Resource: Providers

GET {{host}}/api/providers
Authorization: Bearer {{API_Token}}

Response Parameters

Field	Response
`id`	String The unique identifier of the integration.
`name`	String The name of the integration and by default is set to `aws`.
`credentials`	String The dictionary containing the information about how Sysdig authenticates to AWS: `id`: The AWS access key ID. `role` : The AWS Role if role-based authentication is used.
`skipFetch`	Boolean `true` : The account will be used only for configuring storage. In this case, it will not fetch AWS metadata and CloudWatch metrics. `false`: The account will be used only for fetching AWS metadata and CloudWatch metrics.
`status`	String Status denotes the status of the integration.
`accountId`	String The unique identifier of the AWS account. The value will be `null` if you are using AWS-compatible storage.
`tags`	Ignore this deprecated field.

Sample Response

{
  "providers": [
    {
      "id": 2398,
      "name": "aws",
      "credentials": {
        "id": "AKIA4JRXW5ZVZU6MHNPE",
        "role": null
      },
      "skipFetch" : false,
      "status": {
        "status": "done",
        "lastUpdate": 1617274193293,
        "percentage": 100,
        "lastProviderMessages": []
      },
      "alias": null,
      "accountId": "845151661675"
    }
  ]
}

Create a New AWS Integration

REST Resource: Providers

POST {{host}}/api/providers
content-type: application/json
Authorization: Bearer {{API_Token}}

{
    "name":"aws",
    "skipFetch": false,
    "credentials": {
        "id":"<AWS_Access_Key_ID>",
        "role":null,
        "key":"<AWS_SecretKey>"
    }
}

Request Parameters

Field	Description
`name`	String The name of the integration and by default is set to `aws`.
`skipFetch`	Boolean `false`: The account will be used only for fetching AWS metadata and CloudWatch metrics. `true` : The account will be used only for configuring storage. In this case, it will not fetch AWS metadata and CloudWatch metrics.
`credentials`	The dictionary containing the information about how Sysdig authenticates to AWS: `id`: The AWS access key ID. `role`: The AWS Role if role-based authentication is used. `key`: The secret associated with the account.

Field

Description

name

String

The name of the integration and by default is set to aws.

skipFetch

Boolean

false: The account will be used only for fetching AWS metadata and CloudWatch metrics.

true : The account will be used only for configuring storage. In this case, it will not fetch AWS metadata and CloudWatch metrics.

credentials

The dictionary containing the information about how Sysdig authenticates to AWS:

id: The AWS access key ID.
role: The AWS Role if role-based authentication is used.
key: The secret associated with the account.

Update Custom Storage Settings

To update existing storage settings, perform a PUT HTTP call to the endpoint as follows:

REST Resource: Settings

PUT {{host}}/api/sysdig/settings
content-type: application/json
Authorization: Bearer {{API_Token}}

{
    "enabled":true,
    "buckets":[
        {
            "folder":"/folder1/folder2",
            "name":"bucketName",
            "providerKeyId": 3,
            "endpoint": "http://127.0.0.1:9009"
        }
    ]
}

Request Parameters

Field	Description
`enabled`	Boolean Indicates whether the custom storage is configured. If the value is `false`, Sysdig-provided storage is used.
`buckets`	Returns the list of buckets that you can set. Currently, only one bucket is supported. `folder`: The bucket path where the captures will be stored. `name`: The unique bucket name. `providerKeyId`: The unique ID of the AWS or compatible provider to be used to configure storage. Currently, only one bucket can be configured at a time. `endpoint`: The endpoint of the provider. It is configured only if you wish to set AWS compatible and is automatically set for AWS. For example, https://storage.googleapis.com/ for Google Cloud https://s3.us-south.cloud-object-storage.appdomain.cloud for IBM. For a full list of endpoints, please refer to the relevant provider documentation.

Field

Description

enabled

Boolean

Indicates whether the custom storage is configured.

If the value is false, Sysdig-provided storage is used.

buckets

Returns the list of buckets that you can set. Currently, only one bucket is supported.

folder: The bucket path where the captures will be stored.
name: The unique bucket name.
providerKeyId: The unique ID of the AWS or compatible provider to be used to configure storage. Currently, only one bucket can be configured at a time.
endpoint: The endpoint of the provider. It is configured only if you wish to set AWS compatible and is automatically set for AWS.
For example, https://storage.googleapis.com/ for Google Cloud
https://s3.us-south.cloud-object-storage.appdomain.cloud for IBM.
For a full list of endpoints, please refer to the relevant provider documentation.

(On-Prem) Configure Custom S3 Endpoint

You can set up a custom Amazon-S3-compatible storage, such as Minio or IBM Cloud Object Storage, for storing Captures in a Sysdig on-premises deployment. The capture storage location can be used for both Sysdig Monitor and Sysdig Secure. This is an API-only functionality and currently, no UI support is available.

You must configure values.yaml corresponding to your Sysdig installation in order for this configuration to work.

Prerequisites

Your on-premise installation is Installer-based. If you have installed Sysdig Platform manually and you want to configure custom S3 buckets to store your capture files, contact your Sysdig representative.
Ensure that AWS-client compatible credentials used for authentication are present in the environment.
Ensure that the list, get, and put operations are functional on the S3 bucket that you wish to use. Confirm this by using the S3 native tools, for example, as described in AWS CLI for IBM Cloud.

Configure Installer

Configure the following parameters in the values.yaml file so that collectors, workers, and the API server are aware of the custom endpoint configuration.

sysdig.s3.enabled

Required: true
Description: Specifies if storing Sysdig Captures in S3 or S3-compatible storage is enabled or not.
Options:true|false
Default:false

For example:

sysdig:
  s3:
    enabled: true

sysdig.s3.endpoint

Required: true
Description: The S3 or S3-compatible endpoint for the bucket. This option is ignored if sysdig.s3.enabled is not configured.

For example:

sysdig:
  s3:
    endpoint: <your S3-Compatible custom bucket>

sysdig.s3.capturesFolder
```
 Required: false
 Description: Name of the folder in S3 bucket to be used for storing captures, this option is ignored if sysdig.s3.enabled is not configured.
```
For example:
```
sysdig:
  s3:
    capturesFolder: my_captures_folder
```
The path to the capture folder in the S3 bucket will be {customerId}/{my_captures_folder}. For on-prem deployments, the customerID is 1. If finance is your capture folder, the path to the folder in the S3 bucket will be 1/finance.

sysdig.s3.bucketName

Required: true
Description: The name of the S3 or S3-compatible bucket to be used for captures. This option is ignored if sysdig.s3.enabled is not configured

For example:

sysdig:
  s3:
    bucketName: <Name of the S3-compatible bucket to be used for captures>

sysdig.accessKey

Required: true
Description: The AWS or AWS-compatible access key to be used by Sysdig components to write captures in the S3 bucket.

For example:

sysdig:
  accessKey: <AWS-compatible access key>

sysdig.secretKey

Required: true
Description: The AWS or AWS-compatible secret key to be used by Sysdig components to write captures in the s3 bucket.

For example:

sysdig:
  secretKey: <AWS-compatible secret key>

For example, the following AWS CLI command uploads a Sysdig Capture file to a Minio bucket:

aws --profile minio --endpoint http://10.101.140.1:9000 s3 cp <Sysdig Capture filename> s3://test/

In this example, the endpoint is http://10.101.140.1:9000/ and the name of the bucket is test.

When you finish the S3 configuration, continue with the instructions on on-premise installation by using the Installer.

2.1.6 -

Find Your Customer ID and Name

SaaS customers of Sysdig can be identified by a unique customer number and name, provided by email when the Sysdig environment is first provisioned. While it is generally unnecessary to know the customer number and it is not prominently displayed in the user interface, some configuration settings may require it.

For on-premises environments, the customer ID will typically be 1.

On the UI

To retrieve the customer ID:

Log into the Sysdig interface.
Open Settings > Authentication.
The Authentication section lists the Customer ID and Customer Name associated with your account.

Using the API

To retrieve the customer ID:

Log into the Sysdig interface.
Navigate to the URL endpoint /api/user/me?_product=SDC.
E.g. https://app.sysdigcloud.com/api/user/me?_product=SDC
The JSON file contents are displayed in the browser.
Find the customer:id portion of the JSON to determine the customer ID and name:

2.1.7 -

Agent Installation: Overview and Key

The Agent Installation page provides a shortcut for copy/pasting the necessary code lines for different flavors of agent installation.

You can also retrieve the agent access key (copy/paste).

This page can be hidden from non-admins if administrators choose. See also Change Admin Settings in the User Profile page.

Retrieve the Agent Access Key

To retrieve the key or use the agent install code snippets:

Log in to Sysdig Monitor or Sysdig Secure (maybe as administrator) and select Settings.
Choose Agent Installation.
Optional: Use the Copy button to copy the access key at the top of the page.
Optional: Review and use the sample code to install an agent, as listed.

2.1.8 -

Subscription

Try all of Sysdig’s features for free; when you are ready to upgrade, contact Sysdig sales.

See also: Getting Started with Sysdig Monitor and Getting Started with Sysdig Secure.Getting Started with Sysdig Monitor

Subscription Types

Free Tier

With Free Tier, use *Sysdig Secure for cloud *functions free forever:

For one single account in one cloud region (AWS for v1)
Manage cloud posture with a daily run of CIS Benchmarks
Detect threats with out-of-the-box CloudTrail detection rules based on Falco
Scan containers (ECR/Fargate) automatically and within your cloud environment for up to 250 images a month

30-Day Trial

Test all the features of Sysdig Monitor and/or Sysdig Secure with the free 30-day trial.

After 30 days, your account will be disabled and you can contact Sysdig sales to upgrade to an Enterprise license.

Enterprise Tier

You can license Sysdig Secure, Sysdig Monitor, or both (Sysdig Platform). For details, see https://sysdig.com/pricing/.

Review Current Subscription

Log on as Administrator to Sysdig Monitor or Sysdig Secure.
From the Selector button in the left-hand navigation, choose Settings > Subscription.
Your current plan details are displayed.

You can license each of these elements independently:

Sysdig agents (host agents)
Cloud accounts (Secure only, see also: Data Sources)
Fargate tasks/serverless agents (Secure only, see also Serverless Agents)

The number of licenses purchased has the following effects on how Sysdig is used:

The agent count defines the maximum number of connected host agents you can deploy. E.g if you purchase 100 licenses, you can install 100 agents.
- In AWS Service Monitoring powered by Cloudwatch, it also determines the number of AWS objects that can be viewed in the Sysdig Monitor Dashboards (unrelated to the number of agents actually installed). In other words, if you have 100 licenses purchased, you can only see 100 AWS objects per region, per service type.
**Fargate Tasks using Sysdig Serverless Agents: ** Defines the number of serverless agents connected to Sysdig backend.
Cloud accounts- licensed number: Number of cloud accounts you can connect to Sysdig backend.

About Host Agent Licenses

Reserved vs On-Demand Agents

The distinction between reserved and on-demand agents is financial, not technical; when on-demand agents are used they perform exactly like reserved agents.

Reserved Agents

Reserved agents are dedicated agents that are provisioned for a user regardless of usage. You can purchase reserved agents on a monthly or annual basis. As a Sysdig SaaS account administrator, you can increase your reserved agents at any time from within the Sysdig application.

On-Demand Agents

On-demand agents are for short-term use and you pay only for what you use at an hourly rate. You have the ability to add and control on-demand agents. For example, an organization might schedule scale testing for two days and license an extra 500 on-demand agents for that time frame.

In the Sysdig application, use the Customize Your Plan > Enable On-demand Agents option on the Subscription page to add or remove agents. There is a hard limit of 500 agents for any account. If the total of reserved and on-demand exceeds this limit, you will not be able to purchase additional agents. On-demand agents are available only in Sysdig SaaS applications.

Connect Agents to the Backend

The Sysdig platform uses a concurrent licensing model in determining when to allow an installed agent to connect to the back-end servers and report on host metrics. This means you can install Sysdig agents onto any number of instances. However, only the licensed number of agents will be allowed to connect and send metrics for recording and reporting.

Agents connect on a “first-come, first-served” basis and in the event of an over-subscription (more agents wanting to communicate than are licensed) they will attempt to reconnect on a periodic basis. Once an existing communicating instance goes down and disconnects, the next agent attempting to connect will be allowed in.

To avoid having agents refused connection due to over-subscription, monitor the number of established and allowed connections. To see how many licenses are in use, see the Settings > Subscription page. Use this information to either purchase additional license capacity from the UI, or to shut down lower-priority agents via normal orchestration and system administration means.

Technical Details

Multiple Installs: An agent is essentially an “install” of the software. If your system changes external IP addresses, or if you shut down a VM image and bring it back up elsewhere, this will remain the same agent connection. However, identical installs that are simultaneously sending data (usually an accident) will be considered two connections. A MAC address is used to identify a host for licensing purposes.

Time Lag for License Release: When shutting down a host for any reason, the agent’s license will not be immediately released. This permits the agent to retain its licensing slot for short outages or a reboot. The time-out interval can take up to 20 minutes, and if the connection has not been re-established within the interval the license will be released for use by the next host waiting to connect.

2.1.8.1 -

Time Series Billing

Sysdig Monitor allows you to consume custom metrics through a flexible and cost-effective Time Series Billing model aligned with your usage. With the enhanced billing experience, you can view your time-series consumption at a glance, analyze trends, and change subscription plans if need be.

Use the Sysdig Subscription page to control your licensing, and thereby usage and spending. Based on your current subscription tier, time-series usage, and the number of active agents, you can estimate the bill and take further actions.

Time Series Billing works only in SaaS environments and is not currently available in on-prem environments.

Benefits of Time Series Billing

You consume more than the per-agent limit because Time Series Billing accounts for the following:
- Time series through Monitoring Integrations
  - Instrumented natively
    For example, Kubernetes Control plane, Persistent Volume Claim, and Istio.
  - Using Exporter and AppCheck
    For example, Nginx and Redis.
- Time series through Prometheus Remote Write
- Time series through optional Pay as you Go (PAYG) and metric packs.
See Use Cases for more details.
Previously, the technical limit was 10K, no PAYG and metrics packs mechanisms, no system in place to bill metrics collected outside agents.
Validate what you are being charged on, understand and control metric usage, and drop the data that is not required, either by metric or by the scope of the metric. See Control Time Series Ingestion.
Pay as you go and metric pack.

Consume Time Series

Time series consumption is calculated by using the reserved time series included in the subscription. The basic plan includes 2000 time series per agent, and you can purchase more by adding on-demand agents or metric packs.

Sysdig meters and bills only custom metrics.

Prometheus
JMX
StatsD
App checks

Reserved Time Series

The number of time series included with the subscription. The value is calculated as (the number of reserved agents + the number of connected on-demand agents) * the number of time series per agent. Time series consumed beyond your subscription limit will be charged and is aggregated across all agents running in your environment. What it means is that you can consume 3000 metrics on an agent and 1000 on another without incurring additional charges.

Contact Sales to purchase beyond your subscription limit.

Time Series Billing limit of 2000 is applicable only to custom metrics, while Sysdig and Sysdig KSM are included at no additional charges.

Metric Pack

A metric pack includes 1000 time series and is charged per month.

View Your Subscription

Time Series Visualization

To help you so, Sysdig provides an at-a-glance visualization of the following:

Time Series Usage
- Reserved: See Reserved Time Series.
- Overage: Time series ingested beyond Reserved time series is Overage.
- Ingested: The time series that are collected, analyzed, processed for storage.
Time Series Usage Dashboard
Reserved and On-Demand Agents
Agent Usage Dashboard
Usage history in CSV format

Edit Subscription

On the Subscription page, under Subscription Details , click the three dots.
Click Edit Subscription.
The Subscription Plan page gives you the directions to change the subscription plan.

Monitor Time Series Usage

Time Series Metrics

To help you identify the usage trends that are important to you, Sysdig provides the following metrics:

sysdig_ts_usage: The metric reports the number of time series ingested for a user in a 20-minutes interval. The dashboard reports the 1-hour usage, which is the sum of the maximum of three 20-minute sysdig_ts_usage measurements taken in an hour. This metric can be segmented on metric categories as well.
sysdig_ts_usage_10s: The metric reports the number of time series ingested for a user in every 10-seconds window, per host (agent), and per metric category.

Download Usage

You can download the usage report in a CSV file. On the Subscription page, under Subscription Details, click Download Usage to download a copy of the usage report. You can view the following:

User ID
Time
Number of Reserved Agents
Number of Connected On-Demand Agents
Time Series included per agent
Total used time series
The ratio of used and reserved time series

Time Series Usage Dashboard

Sysdig provides a Time Series Usage Dashboard with insight into the usage data. You can view time series ingestion at a glance and discover and analyze trends. The dashboard shows the average number of active time series per host; current ingestion rate; churn percentage; and so on.

Access the Time Series Dashboard

On the Subscription page, under Usage, click Time Series Dashboard. You can view the following:

Current 1 Hour Ingestion
Current Ingestion from Agents
Churn Percentage
The average number of time series per host
The number of time series ingested per category
Host-level ingestion

Calculate Time Series Usage

Time series usage is calculated by using the sysdig_ts_usage metric. The metric reports the number of time series ingested for a user in an hour (sum of the maximum of three 20-minutes). For each hour, the number of time series ingested is calculated, and then the value is deducted from the number of reserved time series. This value is stored as the usage record.

An hour period is considered in order to take the churn into account. Sysdig uses the sysdig_ts_usage_10s metric to calculate the spikes caused by churns and provides you the churn percentage in the dashboard.

Sysdig uses the 95th percentile to calculate the exceeding cost of usage. At the end of the month, the 95th percentile of the total number of active series ingested per hour is calculated. Calculating the 95th percentile reduces the chances of billing you for unexpected spikes causes by churns.

Churn Rate

When a time series stops receiving new data points, it becomes inactive. This event is called time series churn. It occurs more often during an upgrade in a Kubernetes cluster or when containers are replaced by new ones. Restarts, redeploys, dynamic workloads also cause churn and generate additional time series.

In such cases, the container_id label in a container metric changes, and subsequently, all the existing time series are replaced by new time series (with the new container_id value).

The churn rate is the number of time series that churn over time. Sysdig uses the sysdig_ts_usage_10s metric to analyze these scenarios.

The Time Series Usage Dashboard provides a ratio of time series detected at 1-hour period and 10-seconds period. This ratio is known as the churn percentage and it is expressed as this PromQL query:

(sum(sysdig_ts_usage{metric_category!='PROMETHEUS_REMOTE_WRITE'}) - sum(sysdig_ts_usage_10s)) / sum(sysdig_ts_usage{metric_category!='PROMETHEUS_REMOTE_WRITE'}) * 100

The time series collected by Prometheus Remote Write are excluded from this ratio because they are not collected by the Sysdig agent.

Example

The billing is calculated per month. A basic subscription will provide you 2000 time series per agent. This limit is applicable only to custom metrics, while you can continue consuming Sysdig and KSM metrics without incurring additional costs. Time series consumed beyond your subscription limit will be charged and is aggregated across all agents running in your environment.

For example, if you have three agents running with the following consumption:

Agent 1 collecting 2000 time series per hour
Agent 2 collecting 1000 time series per hour
Agent 3 collecting 4000 time series per hour

Time series billing is calculated as follows:

Total consumption = 7000

Allowed number of time series per hour: 3 * 2000 = 6000

Effectively, you are paying only for (7000 - 6000) = 1000 because the cost is calculated on the aggregated time series consumed across all the agents running in your environment.

Control Time Series Ingestion

For more information on controlling metric usage, see the following:

Use Cases

Agent and Remote Write Plan

See the following example with the following configuration:

Two Prometheus Servers
- Prometheus Server 1 generates 50,000 time series
- Prometheus Server 2 generates 150,000 time series
A Sysdig agent that collects 1000 time series
A subscription capacity of 2000 time series

The billing for the month is calculated as follows:

Time series usage: Total usage - Subscription capacity

(50,000 + 150,00 + 1000) - 2000 = 199,000

If the base price is $7.5 for up to a unit of 1K time series per month, the total base cost is calculated as follows:

The number of units consumed = (199,000 / 1000) = 199

The total cost = $7.5 * 199 = $1592.50

Agents, Remote Write, and Metric Pack Plan

See the following example with the following configuration:

Two Prometheus Servers
- Prometheus Server 1 generates 50,000 time series
- Prometheus Server 2 generates 150,000 time series
A Sysdig agent that collects 1000 time series
100 metric pack, which is equivalent to 100000 time series

The billing for the month is calculated as follows:

Total subscription capacity: Total usage - (subscription capacity + time series from metric pack)

201,000 - (100,000 + 2000) = 99,000

If the base price is $7.5 for up to a unit of 1K time series per month and $5 for a metric pack of 1K time series, the total base cost is calculated as follows:

The number of units consumed = (199,000 / 1000) = 199

The total cost = Cost of metric pack + cost of total usage

($7.5 * 199) + (100 * $5) = $2092.50

2.1.8.2 -

About AWS Cloudwatch Licensing

In the Explore tab or Dashboards of Sysdig Monitor, the number of metrics displayed for each AWS service is limited by the number of agent licenses purchased and/or used, by region.

The license count:

Includes Reserved agents plus On-Demand agents (even if not in use).
Is used to determine how many AWS resources are displayed for each service in each region.
Is not transferable between different AWS services.

AWS Service Type Priorities and Limits

For each AWS service type, services are displayed in the following priority:

EC2: Pick instances with agents installed, then instances belonging to ECS, instance is launched before another, alphabetically by instance ID, up to license count.
RDS: Pick by creation time, oldest instances first, up to license count.
ELB: Pick by number of balanced instances (larger ELBs 1st), then by creation time, oldest first, up to license count.
ElastiCache: Sort by name and display up to license count items.
SQS: Sort queues by name and pick up to license count number of queues to fetch. Data is shown only for queues that are reporting metrics.
DynamoDB: Sort by name and display up to license count items.
ALB: Sort by name and display up to license count items.

For more information on AWS metrics, see AWS in the Metrics Dictionary.

Sample Use Case

Suppose you have 200 AWS instances, have purchased 100 Sysdig agent licenses, and have actually installed 50 agents.

The following limits would apply to your views of AWS services, per region:

EC2: The 50 instances with agents installed would be shown first, then 50 more instances, first from EC2, then from ECS, then per uptime.
RDS: 100 RDS listings would be shown, oldest first.
ELB: 100 ELBs would be shown (largest first), then by creation time, oldest first.
ElastiCache: 100 ElastiCache objects would be shown, alphabetically by name.
SQS: 100 SQS queues that are reporting metrics would be shown.
DynamoDB: 100 DynamoDBs would be shown, alphabetically by name.
ALB: 100 ALBs would be shown, alphabetically by name.

To increase the limit of items in the AWS Services views, contact Sysdig Sales to enable additional resources depending on your license agreement.

2.1.9 -

Authentication and Authorization (SaaS)

Sysdig Monitor and Sysdig Secure are designed to work with several user authentication/authorization methods:

Type	Enabled by Default	Integration Steps Required
User email/password	Yes	No
Google OAuth	No	No
SAML	No	Yes
OpenID Connect	No	Yes

The user’s view:

The pages in this section describe the integration and enablement steps required for SAML or OpenID Connect, and the Identity Provider (IdP) services that support these protocols, such as Okta, OneLogin, Keycloak.

In the SaaS environment, Google login can be enabled with a simple drop-down selection; the integration has already been performed.

See SaaS Regions and IP Ranges before proceeding to configure authentication.

To integrate SAML or OpenID Connect with both Sysdig Monitor and Sysdig Secure, you must go through the integration steps twice, once for each Sysdig product.

Workflow

With the new Authorization UI, the basic process of enabling a Single Sign-On (SSO) option is:

Determine which SSO option (GoogleOAuth, SAML, OpenID) your enterprise uses, and which IdP service (Okta, OneLogin, etc.) is used, if any.
Enter the required connection settings for the chosen SSO on the appropriate Authentication tab. (Note: for Google, the settings are already entered.)
Configure any associated IdP settings on the IdP side.
Select the SSO option from the Enabled Single Sign-On drop-down and click Save Authentication.
If enabling for both Sysdig Monitor and Sysdig Secure, repeat the process on the second application.

View of the Authentication page for Google OAuth in the SaaS environment.

2.1.9.1 -

Google OAuth (SaaS)

This guide is specific to cloud-based (SaaS) Sysdig environments. If you are configuring an On-Premises Sysdig environment, refer to Google OAuth (On-Prem) instead.

In the SaaS environment, Google users have the option to log in via Google OAuth.

As the SaaS platform is preconfigured to permit such logins, environments that already use Google services (such as G Suite) may find this the most convenient approach for simplified login.

Enable Google OAuth

Since Google OAuth is pre-configured by Sysdig, the administrator needs only select it as the chosen Authentication option to enable it.

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Authentication.
(Select the Google OAuth tab if you want to see the preconfigured (un-editable) settings. )
Select Google OAuth from the Enabled Single Sign-On dropdown and click Save Authentication.
Repeat for Sysdig Monitor or Sysdig Secure, if you want to enable on both applications.

User Experience

Note the following requirements for successful Google OAuth login:

The user must have already logged in successfully at least once to your environment (such as via email-based Invitation and having set an initial password)
The user’s login username in the Sysdig platform must precisely match the user’s Google email address (that is, it cannot be a shortened/altered Google email alias)

For such a user to log in via Google OAuth, click the Log in with Google button.

If the user’s browser has not already successfully authenticated via Google and/or has multiple Google profiles known by their browser, they will be presented a Google page to select a profile and enter a password (if necessary) before being redirected back to your Sysdig environment.

See also User and Team Administration for information on creating users.

2.1.9.2 -

SAML (SaaS)

This guide is specific to cloud-based (SaaS) Sysdig environments. If you are configuring an On-Premises Sysdig environment, refer to SAML (On-Prem) instead.

SAML support in the Sysdig platform allows authentication via your choice of Identity Provider (IdP).

The Sysdig platform ordinarily maintains its own user database to hold a username and password hash. SAML instead allows for redirection to your organization’s IdP to validate username/password and other policies necessary to grant access to Sysdig application(s). Upon successful authentication via SAML, a corresponding user record in the Sysdig platform’s user database is automatically created, though the password that was sent to the IdP is never seen nor stored by the Sysdig platform.

This section describes how to integrate and enable SAML with both Sysdig Monitor and Sysdig Secure.

For specific IdP integration information, refer to:

Basic Enablement Workflow

Step	Options	Notes
1. Know which IdP your company uses and will be configuring.	Okta (SAML) OneLogin (SAML) ADFS (SAML) Azure Active Directory (SAML)	These are the IdPs for which Sysdig has performed detailed interoperability testing and confirmed how to integrate using their standard docs. If your IDP is not listed, it may still work with the Sysdig platform. Contact Sysdig Support for help.
2.Decide the login flow you want users to experience (choose from three options):	Click SAML button and enter a company name	Open the domain URL corresponding to your Sysdig application and region and enter your company name. For example, domain URLs of Monitor and Secure for US East are app.sysdigcloud.com and secure.sysdig.com respectively. Contact the Sysdig Support to set your company name on the account. This is applicable to all supported IdPs.
	Type/bookmark a URL in browser	For example, the URLs for the US East are: Monitor: https://app.sysdigcloud.com/api/saml/COMPANY_NAME?redirectRoute=%2F&companyName=COMPANY_NAME Secure: https://secure.sysdig.com/api/saml/COMPANY_NAME?product=SDS&redirectRoute=%2F&companyName=COMPANY_NAME For example, for the EU region: Monitor: https://eu1.app.sysdig.com/api/saml/COMPANY_NAME?redirectRoute=%2F&companyName=COMPANY_NAME> Secure: https://eu1.app.sysdig.com/api/saml/COMPANY_NAME?product=SDS&redirectRoute=%2F&companyName=COMPANY_NAME For URLs corresponding to other regions, see SaaS Regions and IP Ranges.
	Log in from an IdP interface	The individual IdP integration pages describe how to add Sysdig to the IdP interface. You will need your Sysdig customer number on hand.
3. Perform the configuration steps in your IdP interface and collect the resulting config attributes.	Okta (SAML) OneLogin (SAML) ADFS (SAML) Azure Active Directory (SAML)	Collect metadata URL (or XML) and test it. If you intend to configure IDP-initiated login flow, have your Sysdig customer number on hand. It will be referenced in later configuration steps as `CUSTOMER_ID_NUMBER`.
4 a. Log in to Sysdig Monitor or Sysdig Secure `Settings` (as admin) and enter the necessary configuration information in the UI. Enable SAML as your SSO. 4 b. Repeat process for the other Sysdig product, if you are using both Monitor and Secure.		You will enter a separate redirect URL in your IdP for each product; otherwise the integration processes are the same.

Administrator Steps

Configure IdP

Select the appropriate IdP from the list below, and follow the instructions:

Enable SAML in Settings

To enable baseline SAML functionality:

Enter SAML Connection Settings

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings from the User Profile button in the left navigation.
Select Authentication.
Select the SAML tab.
Enter the relevant parameters (see table below) and click Save.
It is strongly recommended that “Signed Assertion” and “Validate Signature” are enabled to ensure that the SAML SSO process is as secure as possible.

Connection Setting	Options	Description	Sample Entry
Metadata	URL	The URL provided at the end of the IdP configuration steps.
	XML	An option that can be used for an IdP that doesn’t support extracting metadata XML via URL.
Signed Assertion	off/on	Should Sysdig check for assertions signed in responses (to assist in validating correct IdP).	ON
Email Parameter	email	Name of parameter in the SAML response for user email ID. Sysdig uses this to extract the user’s email from the response.	email
Validate Signature	off/on	Sysdig backend should verify that the response is signed.	ON
Verify Destination	off/on	Flag to control whether Sysdig should check the “destination” field in the SAMLResponse. Recommend ON, as a security measure. May be OFF in special cases, such as a proxy in front of the Sysdig back end.	ON
Create user on login	off/on	Flag to control whether a user record should be created in the Sysdig database after first successful SAML log in.
Disable username and password login	off/on	Switch “on” to disallow user name and password log in. (Useful with SAML OpenID.)

Select SAML for SSO

Select SAML from the Enabled Single Sign-On dropdown
Click Save Authentication.
Repeat entire enablement process for Sysdig Monitor or Sysdig Secure, if you want to enable on both applications.

Configure SAML Single Logout

Sysdig supports SAML Single Logout (SLO).

SLO is a feature in federated authentication where Sysdig users can sign out of both their Sysdig session (Service Provider) and associated IdP (Identity Provider) simultaneously. SLO allows you to terminate all sessions established via SAML SSO by initiating a single logout process. Closing all user sessions prevents unauthorized users from gaining access to Sysdig resources.

SLO Process

When a user initiates a logout, Sysdig sends a digitally-signed logout request to the IdP. The IdP validates the request and terminates the current login session, then redirects the user back to the Sysdig login page.

Caveats

SLO is currently supported only in US-West and EU-Central regions.
Sysdig does not support HTTP Post binding for single logout, and therefore, SLO with Okta is not functional at this point.

Configure IdP

Configure logout URLs:
- Monitor: <base_URL>/api/saml/slo/logout
- Secure: <base_URL>/api/saml/slo/secureLogout
Choose HTTP Redirect as the binding method.
This option is an alternative to the HTTP POST method, which Sysdig does not support currently.
If your IdP mandates, upload the public key for Sysdig.
Contact Sysdig Support to retrieve the public key associated with your deployment.
Certain IDPs, such as Azure, don’t require uploading the public key.

Configure Sysdig

Log in to Sysdig Monitor or Sysdig Secure as an administrator and select Settings.
For on-prem deployments, log in as the super admin.
Navigate to Settings > Authentication, and select SAML under Connection Settings.
Enter the SAML configuration.
Ensure that Enable SAML single logout is toggled on.
Click Save.
Ensure that you select SAML from the Enable Single Sign On drop-down.

User Experience

As noted in the Basic Enablement Workflow above, you can offer users three ways to log in with a SAML configuration:

They can begin at the Sysdig SaaS URL and click the SAML button.
See SaaS Regions and IP Ranges and identify the correct Sysdig SaaS URL associated with your Sysdig application and region. For example, URLs of Monitor and Secure for US East are:
Monitor: app.sysdigcloud.com
Secure: secure.sysdig.com
They will be prompted to enter a Company Name, so the Sysdig platform can redirect the browser to your IdP for authentication.
Contact Sysdig Support to set your company name on the account.
You can provide an alternative URL to avoid the user having to enter a company name, in the format:
Sysdig Monitor: https://app.sysdigcloud.com/api/saml/ COMPANY_NAME
Sysdig Secure: https://secure.sysdig.com/api/saml/ COMPANY_NAME?product=SDS
For other regions, the format is https://<region>.app.sysdig.com/api/saml/auth. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Secure in the EU, you use https://eu1.app.sysdig.com/api/saml/secureAuth.
You can configure an IdP-initiated login flow when configuring your IdP. The users then select the Sysdig application from your IDP’s app directory and do not browse directly to a Sysdig application URL at all.

Users that complete their first successful SAML login to Sysdig Secure may receive the error message “User doesn’t have permission to login in Sysdig Secure”. This is because only members of the Secure Operations team are permitted access to Sysdig Secure, and newly-created logins are not in this team by default. Such a user should contact an Administrator for the Sysdig environment to be added to the Secure Operations team.

Environments that wish to have all users access Secure by default could use this sample Python script to frequently “sync” the team memberships.

See Developer Documentation for tips on using the sample Python scripts provided by Sysdig.

See also User and Team Administration for information on creating users.

Caveats

SAML Assertion Encryption/Decryption is not currently supported.

2.1.9.2.1 -

Okta (SAML)

Review SAML (SaaS) before you begin.

Configure Sysdig Monitor and/or Sysdig Secure as a SAML application using Okta’s documentation for Setting Up a SAML Application in Okta. The notes below call out specific steps that require additional action.

Sysdig-Specific Steps for Okta Configuration

IDP-initiated Login Flow

If you don’t intend to configure IDP-initiated login flow, check the boxes for “Do not display application icon to users” and “Do not display application icon in the Okta Mobile app”.

URL, URI and RelayState Values

Enter the values shown in the table below. If you wish to configure IDP-initiated login flow, replace CUSTOMER-ID-NUMBER with the number retrieved as described in Find Your Customer Number.

See SaaS Regions and IP Ranges and identify the correct URLs associated with your Sysdig application and region. For example, in US East, the endpoints are:

Setting	Value for Sysdig Monitor	Value for Sysdig Secure
Single sign on URL	`https://app.sysdigcloud.com/api/saml/auth`	`https://secure.sysdig.com/api/saml/secureAuth`
Audience URI (SP Entity ID)	`https://app.sysdigcloud.com/api/saml/metadata`	`https://app.sysdigcloud.com/api/saml/metadata`
Default RelayState (optional - only configure if you intend to use IDP-initiated login flow)	#/&customer=`CUSTOMER-ID-NUMBER`	#/&customer=`CUSTOMER-ID-NUMBER`

Setting

Value for Sysdig Monitor

Value for Sysdig Secure

Single sign on URL

https://app.sysdigcloud.com/api/saml/auth

https://secure.sysdig.com/api/saml/secureAuth

Audience URI (SP Entity ID)

https://app.sysdigcloud.com/api/saml/metadata

Default RelayState

(optional - only configure if you intend to use IDP-initiated login flow)

#/&customer=CUSTOMER-ID-NUMBER

For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com/api/saml/auth.

Email and Name Values

Instead of the values shown in the Okta example, add the values:

Name	Value
`email`	`user.email`
`first name`	`user.firstName`
`last name`	`user.lastName`

Note that the attributes are case sensitive, so use caution when entering them.

Only email is required. However, including first/last name is recommended, since these values will now be included in the records created in the Sysdig platform’s database when new users successfully login via SAML for the first time.

SAML Configuration Metadata Value

Copy the URL and paste in the Metadata entry on the SAML Configuration page in the SAML connection settings.

Test Metadata (Optional)

To ensure the metadata URL you copy at the end of the IDP configuration procedure is correct, you can test it by directly accessing it via your browser.

When accessing the URL, your browser should immediately download an XML file that begins similarly to the example shown below. No entry of credentials or other security measures should be required to successfully download it. If this is not the case, revisit the IDP configuration steps.


`<?xml version=` `"1.0"` `?>` `<EntityDescriptor xmlns=` `"urn:oasis:names:tc:SAML:2.0:metadata"` `entityID=` `"https://app.onelogin.com/saml/metadata/680358"` `>` `<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">` `...`

<?xml version= "1.0" ?> <EntityDescriptor xmlns= "urn:oasis:names:tc:SAML:2.0:metadata" entityID= "https://app.onelogin.com/saml/metadata/680358" >

`<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">`

...

2.1.9.2.2 -

OneLogin (SAML)

Review SAML (SaaS) before you begin.

Configure Sysdig Monitor and/or Sysdig Secure as a SAML application using OneLogin’s article titled Use the OneLogin SAML Test Connector. The notes below call out specific steps that require additional action.

Sysdig-Specific Steps for OneLogin Configuration

Adding the SAML Test Connector

At the step for “Adding the SAML Test Connector”, select SAML Test Connector (IdP w/ attr w/ sign response). If you don’t intend to configure IDP-initiated login flow, uncheck the slider so it will no longer be “Visible in portal”.

Test Connector Configuration Page Settings

At the “Test Connector Configuration Page”, enter the values shown in the table below. If you wish to configure IDP-initiated login flow, replace CUSTOMER-ID-NUMBER with the number retrieved as described in the Find Your Customer Number article.

See SaaS Regions and IP Ranges and identify the correct URLs associated with your Sysdig application and region. For example, given below are the URLs for the US East region.

Field	Value for Sysdig Monitor	Value for Sysdig Secure
RelayState (optional - only configure if you intend to use IDP-initiated login flow)	#/&customer=`CUSTOMER-ID-NUMBER`	#/&customer=`CUSTOMER-ID-NUMBER`
Recipient	`https://app.sysdigcloud.com/api/saml/auth`	`https://secure.sysdig.com/api/saml/secureAuth`
ACS (Consumer) URL Validator	`https://app.sysdigcloud.com`	`https://secure.sysdig.com`
ACS (Consumer) URL	`https://app.sysdigcloud.com/api/saml/auth`	`https://secure.sysdig.com/api/saml/secureAuth`

(Optional) If you want the user’s First Name and Last Name to be included in the records created in the Sysdig platform’s database when new users successfully login via SAML for the first time, click to the Parameters tab. Click Add parameter and create each of two New Fields, checking the box each time to Include in SAML assertion. Then click to Edit each field and select the Value shown from the drop-down menu before clicking Save.

Field Name	Value
`first name`	`First Name`
`last name`	`Last Name`

Note that the Field Names are case sensitive, so be careful to enter them as all lowercase.

The following shows an example of a correctly-configured field for First Name:

Issuer URL

Click to the SSO tab, copy the Issuer URL, and paste in the Metadata entry on the SAML Configuration page in the SAML connection settings.

Test Metadata (Optional)

To ensure the metadata URL you copy at the end of the IDP configuration procedure is correct, you can test it by directly accessing it via your browser.


`<?xml version=` `"1.0"` `?>` `<EntityDescriptor xmlns=` `"urn:oasis:names:tc:SAML:2.0:metadata"` `entityID=` `"https://app.onelogin.com/saml/metadata/680358"` `>` `<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">` `...`

<?xml version= "1.0" ?> <EntityDescriptor xmlns= "urn:oasis:names:tc:SAML:2.0:metadata" entityID= "https://app.onelogin.com/saml/metadata/680358" >

`<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">`

...

2.1.9.2.3 -

ADFS (SAML)

Review SAML (SaaS) before you begin.

These instructions assume you already have a working, Internet-accessible ADFS ( Active Directory Federation Service) server. Interoperability testing has been performed specifically with ADFS on Windows Server 2012 R2.

Follow the instructions below to configure ADFS with the ADFS Management tool in the Windows Server Manager.

Right-click to Service > Edit Federation Service Properties. Note the hostname in the Federation Service Identifier, as this will be used in the metadata URL that you paste in the Metadata entry on the SAML Configuration page in the Sysdig authentication settings. Specifically, the metadata URL will be of the format https://HOSTNAME/FederationMetadata/2007-06/FederationMetadata.xml. Also, so that the Sysdig platform can access this URL directly, this host must resolve in DNS and have a valid (not self-signed) SSL/TLS certificate.
Add a Relying Party Trust configuration for the Sysdig application.
1. Right-click to Relying Party Trusts > Add Relying Party Trust and click Start to begin the wizard.
2. In the Select Data Source step, click the button to Enter data about the relying party manually, then click Next
3. Enter a Display name of your choosing (e.g. “Sysdig Monitor” or “Sysdig Secure”), then click Next
4. Click Next to accept the default option to use AD FS profile
5. Click Next to skip the selection of an optional token encryption certificate (Sysdig does not support this option)
6. Check the box to Enable support for the SAML 2.0 Web SSO protocol, then enter one of the following values for Relying party SAML 2.0 SSO service URL:
  If configuring Sysdig Monitor, enter: https://app.sysdigcloud.com/api/saml/auth
  If configuring Sysdig Secure, enter: https://secure.sysdig.com/api/saml/secureAuth
  Then click Next.
7. For the Relying party trust identifier, enter one of the following values:
  If configuring Sysdig Monitor, enter: https://app.sysdigcloud.com
  If configuring Sysdig Secure, enter: https://secure.sysdig.com
  Then click Add, then click Next
8. Click Next to skip configuration of multi-factor authentication
9. Choose a policy for whether users will be permitted to login to the Sysdig application. The default to Permit all users to access the relying party will typically be acceptable. Click Next.
10. Review the summary and click Next to complete the configuration of the Relying Party Trust
11. The next step will involve adding Claim Rules, so you can leave the box checked to Open the Edit Claim Rules dialog and click the Close button to be brought immediately into the Claim Rules editor
Ensure that the SamlResponseSignature option matches the Sysdig authentication configuration.
1. Use the Set-AdfsRelyingPartyTrust/Get-AdfsRelyingPartyTrust cmdlets via PowerShell to configure SamlResponseSignature .
```
-SamlResponseSignature
Specifies the response signatures that the relying party expects. The acceptable values for this parameter are:

AssertionOnly
MessageAndAssertion
MessageOnly
```
  For more information, see Set-AdfsRelyingPartyTrust.
2. Navigate to Settings > Authentication on the Sysdig app and check the Sysdig authentication setting maps to the SamlResponseSignature :
  For MessageAndAssertion, enable both the options.
Next, use the Claim Rules to ensure that login data is sent as needed to the Sysdig platform. A user’s login to the Sysdig platform is based on an email address, and a default ADFS configuration would not send the email address as required. The following configuration ensures the correct field from Active Directory is delivered in the claim.
1. If not already in the Claim Rules editor from the previous step, navigate to it by right-clicking on the Relying Party Trust that was just created and selecting Edit Claim Rules
2. Click Add Rule. At the following screen, accept the default rule template to Send LDAP Attributes as Claims and click Next.
3. Enter a name for the rule, select Active Directory as the Attribute store, then use the pull-down selectors to pick E-Mail Address as both the LDAP Attribute and Outgoing Claim Type, then similarly make pull-down selections for Given Name and Surname. Once these selections are made, click Finish.
4. Now click Add Rule again, this time selecting the template for Transform an incoming claim
5. Enter a name for the rule, then use the pull-downs to select an Incoming claim type of E-Mail Address, an Outgoing claim type of Name ID, and an Outgoing name ID format of Email, then click Finish.
6. (Optional) If you want the user’s First Name and Last Name to be included in the records created in the Sysdig platform database when new users successfully login via SAML for the first time, additional Transform rules must also be created. Only the email-based username is strictly required and we already created a rule for this, so this step is optional.
  If you wish to do this, click Add Rule and once again select the template for Transform an incoming claim. Enter a name for the rule, then use the pull-down to select an Incoming claim type of Given Name, and for the Outgoing claim type, directly type first name into the field. After clicking Finish, click Add Rule and create a similar rule to transform the Incoming claim type of Surname to the Outgoing claim type of last name.
7. Having clicked Finish after creating your last rule, you will see all rules now in the editor. Click Ok, and your ADFS configuration for your Sysdig application is complete.

(Optional) The steps above represent a Service-Provider-Initiated SAML configuration. If you would prefer an IdP-initiated SAML configuration, this is also possible with ADFS, but requires the additional steps described below.

The Sysdig platform requires a specific setting of RelayState in order to accept IdP-initiated login flows. On the ADFS versions tested, we’ve found this use of RelayState is disabled by default, and a Microsoft article describes the topic in detail. To enable it, as described in a Microsoft forum thread, on your ADFS host, edit %systemroot%\ADFS\Microsoft.IdentityServer.Servicehost.exe.config and add <useRelayStateForIdpInitiatedSignOn enabled="true" /> to the <microsoft.identityserver.web> section. Once the modification is saved, restart ADFS services for the change to take effect.
You will need to retrieve your Sysdig customer number as described in the Find Your Customer Number article.
You will then need to generate an IdP-initiated login URL.
In addition to having the correct settings, it must be properly URL encoded. To ease this configuration, use this ADFS RelayState Generator tool. When launched, enter the values below, then hit the Generate URL button.
- For the IDP URL String, enter https://YOUR_ADFS_SERVER/adfs/ls/idpinitiatedsignon.aspx
- For the Relying Party Identifier, enter one of the following values:
  - If configuring Sysdig Monitor, enter https://app.sysdigcloud.com
  - If configuring Sysdig Secure, enter https://secure.sysdig.com
  For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com/. See SaaS Regions and IP Ranges for more information.
- For the Relay State/Target App, enter #/&customer=CUSTOMER-ID-NUMBER, substituting the CUSTOMER-ID-NUMBER you retrieved in the previous step
  This Results URL will be used in the metadata URL that you paste in the Metadata entry in the SAML connection settings .
Use the Results URL from the tool to test your IdP-initiated login. Note that per this Microsoft forum thread, it is apparently not possible to configure ADFS to use such a URL when your users select the application from the pull-down menu at https://YOUR_ADFS_SERVER/adfs/ls/idpinitiatedsignon.aspx. However, you may embed the URL into a custom portal or bookmarks list.
Now you can test login using an Active Directory user that has an Email address configured.

Test Metadata (Optional)

To ensure the metadata URL you copy at the end of the IDP configuration procedure is correct, you can test it by directly accessing it via your browser.


`<?xml version=` `"1.0"` `?>` `<EntityDescriptor xmlns=` `"urn:oasis:names:tc:SAML:2.0:metadata"` `entityID=` `"https://app.onelogin.com/saml/metadata/680358"` `>` `<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">` `...`

<?xml version= "1.0" ?> <EntityDescriptor xmlns= "urn:oasis:names:tc:SAML:2.0:metadata" entityID= "https://app.onelogin.com/saml/metadata/680358" >

`<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">`

...

2.1.9.2.4 -

Azure Active Directory (SAML)

This topic explains how to configure SAML Single Sign On (SSO) with Azure Active Directory (AD) and helps you configure Sysdig to allow users to access Sysdig application by using SSO.

Prerequisites

Administrator privileges on Sysdig and Azure.

Configure the Sysdig Application in Azure AD

Log in to the Azure AD portal.
Select Azure Active Directory, then click Enterprise Applications.
The Enterprise applications - All application screen is displayed.
Click New Application.
On the Add an Application screen, select Non-gallery application.
Give your application a name, and click Add at the bottom of the page.
On the menu, select Single sign-on.
Choose SAML as the sign-on method.
Edit the Basic SAML Configuration as follows:
1. In the configuration page, click the edit icon.
2. Specify the following:
  - Identifier (Entity ID): Uniquely identifies the Sysdig application. Azure AD sends the identifier to the Sysdig application as the audience parameter of the SAML token. Sysdig validates this as part of the SSO process.
    For example, the identifier for Sysdig Monitor for the EU region is https://eu1.app.sysdig.com.
    See SaaS Regions and IP Ranges for the complete list of entity IDs for different regions.
  - Reply URL: Specifies where Sysdig expects to receive the SAML token.
    For example, the identifier for Sysdig Monitor for the EU region is https://eu1.app.sysdig.com/api/saml/auth.
  - See SaaS Regions and IP Ranges for the complete list of reply URLs for different regions.
  - Relay State: Specifies to the application where to redirect the user after authentication is completed. Typically the value is a valid URL for Sysdig. If you are configuring SSO for SaaS, change the relay state to reflect the correct customer number associated with your Sysdig application. For on-prem installations, the customer number is always 1.
    The format is:
```
#/&customer=1234
```
  - Sign on URL: It is the sign-in page for the Sysdig application that will perform the service provider-initiated SSO. Leave it blank if you want to perform identity-provider-initiated SSO.
  For more information on configuration parameters, see Configure SAML-based single sign-on to non-gallery applications.

Sysdig-Specific Steps for Active Directory Configuration

Under SAML Signing Certificate, copy the App Federation Metadata URL.
Log in to your Sysdig instance as an admin.
For on-prem deployments, log in as the super admin.
Navigate to Settings > Authentication, and select SAML under Connection Settings.
Enter the following:
- Metadata: Enter the App Federation Metadata URL you copied.
- Email Parameter: Set the value to emailaddress.
  Azure AD claims are:
```
saml = AD
givenname = user.givenname
surname = user.surname
emailaddress = user.mail
name = user.userprincipalname
Unique User Identifier = user.userprincipalname
```
  In the Sysdig application, you need to set the email to emailaddress which is what Azure AD sends to Sysdig in the SAML assertion. Alternatively, Azure AD can be modified to send another attribute.
Click Save.
Select SAML from the Enable Single Sign On drop-down.

Create a User in Azure Active Directory Domain

Log in to the Azure AD portal.
Click Azure Active Directory, and note down the domain name.
Select Azure Active Directory, then Users.
The Users - All Users screen is displayed.
Select New Users .
You can either create a new user or invite an existing AD.
Enter name, username, and other details, then click Create.
In the Profile page, add the Email and Alternate Email parameters. The values can match

Assign the User to the Sysdig Application

Navigate to the Sysdig application.
Click Users and Group, then click the Add user button.
Select the Users and Groups checkbox, then choose the newly created user to add to the application.
Click Select, then Assign at the bottom of the screen.

Enable Authentication Settings in the Sysdig Instance

Ensure that Flag to enable/disable create user on login is enabled. Typically this setting is enabled by default.

If you are using both Sysdig Monitor and Secure, ensure that the user accounts are created on both the products. A user that is created only on one Sysdig application will not be able to log in to another by using SAML SSO.

if you are on Sysdig Platform versions 2.4.1 or prior, contact Sysdig Support to help with user creation.

(Optional) Configure Sysdig as a New Application

If Azure Active Directory does not allow you to create Sysdig as a Non- Gallery application, perform the following:

In Azure AD, click Enterprise Applications > New Application.
Select Application you’re developing.
You will be taken to the app registration page:
Select New Registration:
Provide a name for the application you are registering.
Enter the redirect URI.
For example, the redirect URI for Sysdig Monitor for the EU region is https://eu1.app.sysdig.com/api/saml/auth. See SaaS Regions and IP Ranges for the redirect URLs for other regions.
Click Register to complete the registration.
In the Overview tab click Add an Application ID URI:
Click Add a scope.
Add the application ID URI as follows:
```
https://<your_sysdig_url>:443
```
Replace <*your_sysdig_*url> with the URL appropriate to your application and region. See SaaS Regions and IP Ranges for more information.
In the Overview tab, click Endpoints, and copy the Federation Metadata URL.
Log in to Sysdig, navigate to SAML Authentication screen, and enter the Federation Metadata URL.
You will still need to ensure that the user creation on the login option is enabled.
Save the settings.

2.1.9.3 -

OpenID Connect (SaaS)

This guide is specific to cloud-based (SaaS) Sysdig environments. If you are configuring an On-Premises Sysdig environment, refer to OpenID Connect (On-Prem) instead.

OpenID support in the Sysdig platform allows authentication via your choice of Identity Provider (IdP). This section describes how to integrate and enable OpenID Connect with both Sysdig Monitor and Sysdig Secure.

Overview

Summary of OpenID Functionality in Sysdig

The Sysdig platform ordinarily maintains its own user database to hold a username and password hash. OpenID instead allows for redirection to your organization’s IdP to validate username/password and other policies necessary to grant access to Sysdig application(s). Upon successful authentication via OpenID, a corresponding user record in the Sysdig platform’s user database is automatically created, though the password that was sent to the IdP is never seen nor stored by the Sysdig platform.

Basic Enablement Workflow

Step	Options	Notes
1. Know which IdP your company uses and will be configuring.	Okta (OpenID) OneLogin (OpenID) Keycloak (OpenID) Azure (OpenID)	These are the OpenID Providers for which Sysdig has performed detailed interoperability testing and confirmed how to integrate using their standard docs. If your OpenID Provider is not listed (including ones that do not support OpenID Connect Discovery), it may still work with the Sysdig platform. Contact Sysdig Support for help.
2. Decide the login flow you want users to experience: 3 options	Click OpenID button and enter a company name	From app.sysdigcloud.com or secure.sysdig.com > page to enter company name.
	Type/bookmark a URL in a browser	Monitor: https://app.sysdigcloud.com/api/oauth/openid/`COMPANY_NAME` Secure: https://secure.sysdig.com/api/oauth/openid/ `COMPANY_NAME`?product=SDS Contact Sysdig for the Company Name associated with your account.
	Log in from an IdP interface	The individual IdP integration pages describe how to add Sysdig to the IdP interface. You will need your Company Name on hand.
3. Perform the configuration steps in your IdP interface and collect the resulting config attributes.	Okta (OpenID) OneLogin (OpenID) Keycloak (OpenID) Azure (OpenID)	Collect metadata URL (or XML) and test it. If you intend to configure IDP-initiated login flow, you need the following: Redirect URLs See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, domain URLs of Monitor and Secure for US East are: Monitor: https://app.sysdigcloud.com/api/oauth/openid/auth Secure: https://secure.sysdig.com/api/oauth/openid/secureAuth For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region your where Sysidig application is hosted. For example, for Sysdig Monitor you use https://eu1.app.sysdig.com/api/oauth/openid/auth.
4 a. Log in to Sysdig Monitor or Sysdig Secure and configure authentication. 4 b. Repeat process for the other Sysdig product, if you are using both Monitor and Secure.		Log in to Sysdig Monitor Settings (as super admin) and enter the necessary configuration information in the UI. Save and Enable OpenID as your SSO. Log in to Sysdig Monitor Settings (as super admin) and enter the necessary configuration information in the UI. Save and Enable OpenID as your SSO. You will enter a separate redirect URL in your IdP for each product; otherwise the integration processes are the same.

Administrator Steps

Configure IdP

Select the appropriate IdP link below, and follow the instructions:

Enable OpenID in Settings

To enable baseline OpenID functionality:

Enter OpenID Basic Connection Settings

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Authentication.
Select the OpenID tab.
Enter the relevant parameters (see table below) and click Save.

Connection Setting	Description
Client ID	ID provided by your IdP
Client Secret	Secret provided by your IdP
Issuer URL	URL provided by your IdP. Example:`https://YOUR-ONELOGIN-DOMAIN.onelogin.com/oidc`

Okta, OneLogin, and Keycloak support metadata auto-discovery, so these settings should be sufficient for those IdPs.

Enter OpenID Additional Settings (if needed)

In some cases, an OpenID IdP may not support metadata auto-discovery, and additional configuration settings must be entered manually.

In this case:

On the OpenID tab, toggle the Metadata Discovery button to OFF to display additional entries on the page.
Enter the relevant parameters derived from your IdP (see table below) and click Save.

Connection Setting	Description
Base Issuer	Required. Often the same Issuer URL, but can be different for providers that have a separate general domain and user-specific domain (for example, general domain: https://openid-connect.onelogin.com/oidc, user-specific domain: https://sysdig-phil-dev.onelogin.com/oidc)f
Authorization Endpoint	Required. Authorization request endpoint
Token Endpoint	Required. Token exchange endpoint
JSON Web Key Set Endpoint	Required. Endpoint that contains key credentials for token signature verification
Token Auth Method	Authentication method. Supported values: `client_secret_basic` , `client_secret_post` . (case insensitive)

Select OpenID for SSO

Select OpenIDfrom the Enabled Single Sign-On dropdown.
Click Save Authentication.
Repeat entire enablement process for Sysdig Monitor or Sysdig Secure, if you want to enable on both applications.

User Experience

As noted in the Basic Enablement Workflow above, you can offer users three ways to log in with an OpenID configuration:

They can begin at the Sysdig SaaS URL and click the OpenID button.
See SaaS Regions and IP Ranges and identify the correct SaaS URL associated with your Sysdig application and region. For example, URLs of Monitor and Secure for US East are:
Monitor: app.sysdigcloud.com
Secure: secure.sysdig.com
For other regions, the format is https://<region>.app.sysdig.com. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com.
They will be prompted to enter a Company Name, so the Sysdig platform can redirect the browser to your IdP for authentication.
=
You can provide an alternative URL to avoid the user having to enter a company name, in the format:
Monitor: https://app.sysdigcloud.com/api/oauth/openid/ CompanyName Secure: https://secure.sysdig.com/api/oauth/openid/ CompanyName?product=SDS
You can configure an IdP-initiated login flow when configuring your IdP. The users then select the Sysdig application from your IDP’s app directory and do not browse directly to a Sysdig application URL at all.

See also User and Team Administration for information on creating users.

2.1.9.3.1 -

Okta (OpenID)

OpenID Provider Configuration for Okta

Review OpenID Connect (SaaS) before you begin.

The notes below describe minimal steps to be taken in Okta. You may need to adjust the steps based on the specifics of your environment.

Log in to your Okta organization as a user with administrative privileges and click to the Admin dashboard
Click on the Add Applications shortcut, then click the Create New App button
Select Web as the Platform type, then click OpenID Connect as the Sign-on method, then click Create
Create a new application:
- Enter your choice of General Settings.
- For Login redirect URIs, enter one of the following values:
  See SaaS Regions and IP Ranges and identify the correct domain URL (redirect URL) associated with your Sysdig application and region. For example, domain URLs of Monitor and Secure for US East are:
  - Sysdig Monitor: https://app.sysdigcloud.com/api/oauth/openid/auth
  - Sysdig Secure: https://secure.sysdig.com/api/oauth/openid/secureAuth
  For other regions, the format is https://<region>.app.sysdig.com.
  Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com/api/oauth/openid/auth.
- Click Save.
You should next be placed in a General tab. Take note of the Client ID and Client secret that are shown.
You will enter them on the OpenID Configuration page in the Sysdig authentication settings.
Click to the Sign On tab. Take note of the Issuer URL that is shown, as it will need to be sent to Sysdig Support.
You will enter it in the OpenID Configuration page in the OpenID settings.

2.1.9.3.2 -

OneLogin (OpenID)

OpenID Provider Configuration for OneLogin

Review OpenID Connect (SaaS) before you begin.

The notes below describe minimal steps to be taken in OneLogin. You may need to adjust the steps based on the specifics of your environment.

Log in to your OneLogin organization as a user with administrative privileges and click to Apps > Custom Connectors, then click the New Connector button.
Create a new Connector:
- Enter your choice of connector name.
- Select a Sign on Method of OpenID Connect.
- For Redirect URI, enter one of the following values:
  See SaaS Regions and IP Ranges and identify the correct domain URL (redirect URL) associated with your Sysdig application and region. For example, domain URLs of Monitor and Secure for US East are:
  - Sysdig Monitor: https://app.sysdigcloud.com/api/oauth/openid/auth
  - Sysdig Secure: https://secure.sysdig.com/api/oauth/openid/secureAuth
  For other regions, the format is https://<region>.app.sysdig.com.
  Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Secure you use https://eu1.sysdig.com/api/oauth/openid/secureAuth.
- Click Save.
From the More Actions pull-down menu, select Add App to Connector
Click Save to add the app to your catalog. Once clicked, additional tabs will appear.
Click to the SSO tab. Change the setting in the Token Endpoint drop-down to POST, then click Save.
While still on the SSO tab, take note of the Client ID and Client Secret that are shown (click Show client secret to reveal it).
You will enter them in the OpenID settings.
Note that the Issuer URL will consist of https://YOUR-ONELOGIN-DOMAIN.onelogin.com/oidc
You will enter them in the OpenID settings.

During testing, we’ve found OneLogin sometimes does not persist changes that are made in the OpenID Provider configuration. If you make changes to your OneLogin configuration and experience issues such as HTTP 400 Bad Request when attempting logins to your Sysdig application, you may need to delete your Custom Connector and App config in OneLogin and recreate it from scratch.

2.1.9.3.3 -

Keycloak (OpenID)

Configure OpenID Provider for Keycloak

Review OpenID Connect (SaaS) before you begin.

The notes below describe minimal steps to be taken in Keycloak. You may need to adjust the steps based on the specifics of your environment.

Log in to your Keycloak server’s Administrative Console.
Select a realm or create a new one.
Click Clients, then click the **Create**button.
Enter the Client ID of your choosing (e.g. “SysdigMonitor”) and take note of it.
You will enter it in the OpenID Configuration page in the Sysdig Authentication Settings.
Make sure the Client Protocol drop-down has openid-connect selected. Click the Save button.
Configure OpenID Connect client:
- Click the toggle for Authorization Enabled to ON
- For Valid Redirect URI, enter one of the following values:
  See SaaS Regions and IP Ranges and identify the correct domain URL (Redirect URI) associated with your Sysdig application and region. For example, domain URLs of Monitor and Secure for US East are:
  - Sysdig Monitor: https://app.sysdigcloud.com/api/oauth/openid/auth
  - Sysdig Secure: https://secure.sysdig.com/api/oauth/openid/secureAuth
  For other regions, the format is https://<region>.app.sysdig.com.
  Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor you use https://eu1.app.sysdig.com/api/oauth/openid/auth.
- Click Save .
Click to the Credentials tab. Take note of the Secret that is shown.
You will enter it in the OpenID settings
Note that the Issuer URL will consist of https://KEYCLOAK_SERVER_ADDRESS/auth/realms/REALM_NAME, where KEYCLOAK_SERVER_ADDRESS and REALM_NAME are derived from your environment where you just created the configuration. You will enter it in the OpenID settings.

2.1.9.3.4 -

Azure (OpenID)

OpenID Connect is a security-token based extension of the OAuth 2.0 authorization protocol to do single sign-on. Azure Active Directory provides an implementation of OpenID Connect (OIDC) protocol and Sysdig supports it for single sign-on and API access to Sysdig application.

Enabling Azure OpenID Connect for single sign-on to Sysdig applications include configuration on the Microsoft Active Directory as well as on the Sysdig application.

Prerequisites

Administrator privileges on Sysdig and Azure Active Directory (AD).

Configuring Sysdig Application in Azure AD

Log in to the Azure AD portal.
Search for Azure Active Directory and do one of the following:
- Select your Active Directory service
- Create a new one.
Click App registration > New registration.
In the Register an application page, specify the following:
- Name: Display name to identify your Sysdig application. For example, Sysdig Secure.
- Supported account types: For Sysdig SaaS, choose Accounts in this organizational directory only (Default Directory only - Single tenant). All user and guest accounts created in your active directory can use Sysdig application and API.
- Redirect URI: Authenticated Sysdig users are redirected to this URI.
  See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, domain URLs of Monitor and Secure for US East are:
  - For Sysdig Secure: https://secure.sysdig.com/api/oauth/openid/secureAuth
  - For Sysdig Monitor: https://app.sysdigcloud.com/api/oauth/openid/auth
  For other regions, the format is:
  https://<region>.app.sysdig.com
  Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor you use https://eu1.app.sysdig.com/api/oauth/openid/auth.
  For on-prem installations, the redirect URI will be deployment-specific.
  You can add only a single redirect URI on this page. Use the Authentication page associated with your application to add additional redirect URIs.
Click Register.
Add additional redirect URIs.
1. Select your application from App registration.
2. Click Authentication from the left navigation.
3. Add the redirect URIs corresponding to Monitor and Secure.
Create a Secret for the Sysdig application.
It is a string that the Sysdig application uses to prove its identity when requesting a token.
1. Click Certificates & secrets.
2. Under Client Secrets, click New client secret.
3. Enter a description that identifies the secret and choose an expiration period.
4. Click Add.
5. Copy the client secret. You will need the client secret while configuring OpenID Connect SSO on the Sysdig application.
Copy the Client ID and OpenID Connect endpoints corresponding to the application that you have created.
1. Select your application from App registration.
2. Copy the Application (client) ID.
  You will need the client ID while configuring OpenID Connect SSO on the Sysdig application.
3. Click Endpoints.
4. Copy the OpenID Connect metadata document and open it in a browser.
5. Copy the OpenID Connect URI (Issuer URI).
  For example, https://login.microsoftonline.com/5a4b56fc-dceb-4a64-94ff-21e08e5892f5/v2.0

Configure Sysdig Settings

To enable Azure OpenID functionality on the Sysdig application, you need the following:

Client ID
Client Secret
Issuer URL.

See Enable OpenID in Settings to learn how to complete your configuration.

2.1.9.4 -

Disable Password Authentication (SaaS)

Sysdig Platform supports disabling password-based authentication on both SaaS and on-prem deployments. As an administrator (super administrator for on-prem), you can use an API to achieve it. This configuration is applicable to those who use single sign-on.

For On-Prem environments, see Disable Password Authentication.

SaaS Deployments

As an administrator, perform the following:

Get the Sysdig Platform settings:
See SaaS Regions and IP Ranges and identify the correct domain URL associated with your Sysdig application and region. For example, for Sysdig Monitor on US East is:
```
GET https://app.sysdigcloud.com/api/auth/settings/
```
For other regions, the format is https://<region>.app.sysdig.com/api/auth/settings. Replace <region> with the region where your Sysidig application is hosted. For example, for Sysdig Monitor in the EU, you use https://eu1.app.sysdig.com/api/auth/settings.

Find the ID of the active SSO setup:

GET https://app.sysdigcloud.com/api/auth/settings/active

Retrieve the specific settings associated with the SSO setup:
```
GET https://app.sysdigcloud.com/api/auth/settings/{id}
```
The setting is displayed in a JSON file.
In the JSON file, change the following from false to true:
```
settings/forbidPasswordLogin: True
```
Update the setting with a request to the same URL with the same JSON, with the changed parameter. URL depends on the type of deployment.
```
PUT https://app.sysdigcloud.com/api/auth/settings/{id}
```

Migrating from the ConfigMap Method

Previously, the sysdigcloud.restrict.password.login parameter in the Kubernetes ConfigMap has been used to disable password authentication. After installing 3.2.0, deployments utilizing the sysdigcloud.restrict.password.login settings will be automatically migrated to use the new settings.

2.1.9.5 -

Configure Customized Session Expiration

(For SaaS) When you want inactive sessions to deactivate after a time-out period, you can configure it on the Sysdig application. You can determine how long a user’s browser can be idle after which they will be automatically logged out from the session.

To do so

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Authentication.
Scroll down and locate the Session Expiration settings.
Specify the Session Expiration setting:
1. Enable session expiration by using the Terminate session after inactivity period (in minutes) of slider.
2. Specify the time-out period in minutes.
3. Click Save.

2.1.10 -

Configure Theme Preference

A theme specifies the visual appearance of the Sysdig applications. Theme Preferences allows you to change the look and feel of the Sysdig applications to match your visual and accessibility requirements. The list of available themes includes Light, Dark, Match OS Preferences.

To configure a theme:

Log in to Sysdig Secure or Sysdig Monitor.
Navigate to Settings > User Profile.
Select a desirable theme from the Theme Preferences drop-down.
Match OS Preferences: The theme will be aligned with that of your operating system. For example, if your Desktop theme is Dark, the app theme will also be set to Dark.
Your OS theme will override the application theme preferences. For example, changing the OS theme to Dark while your application theme preference is Light will automatically switch the application theme to Dark.

2.2 -

Data Sources (cloud)

If you connect a cloud account using Sysdig Secure for cloud, you can review the details on the Data Sources page.

Review Data Sources

Access the Page

Log in to Sysdig Secure and select Data Sources from the User menu at the bottom of the navigation bar.
The Cloud Accounts overview is displayed.

NOTE: If no cloud accounts have been connected yet, you will be linked to the Get Started page to connect them now.

Review Cloud Accounts

Use the Cloud Accounts overview to:

Confirm that the incoming data sources you expected are present
Get an overview of the status
Check whether managed clusters in the accounts were detected and whether an agent was installed with them.
Click +Add Account to return to the Get Started page and connect an AWS, GCP, or Azure cloud account.

The page lists:

Platform: AWS, GCP, Azure
Account ID: The AWS Account ID, GCP Project ID, or Azure Subscription ID
Alias: As defined when connected
Region(#): Each account may be deployed in multiple regions; click on a numbered entry to expand and view all the regions.
Date Added: Date the account was added to Sysdig Secure
Date Last Seen: Date of last observed activity on the account/region.
Clusters Connected (x/y): This displays the number of managed clusters detected in the account/region (y) and the number of clusters with at least one agent installed (x).
For example:
- 0/0 = no clusters contain an agent, no clusters detected
- 1/17 = 1 cluster contains an agent, 17 total clusters detected

Review Managed Clusters

From the Clusters Connected column you can drill down to the Kubernetes Cluster details :

From here, you could gather the information needed to connect agents to the detected managed clusters.

2.3 -

Data Retention

As of July 2020, the data retention policies for Sysdig Monitor and Sysdig Secure are as described below.

When a host or instance is no longer monitored (i.e. when an agent is uninstalled), the historical data continues to be retained for the stated times.

If required, it is possible to change the standard data retention settings using Sysdig’s REST API. A Sysdig support team or professional services member should assist, as there are a variety of storage and timeline implications to consider before making such a change.

Sysdig Monitor Retention Limits

	Essentials	Enterprise
Metrics data	10s : 4 hours 1min : 1 week 10 min : 2 weeks 1 hr: 2 months 1 day: 15 months	same as Essentials
Events	Alert events: 30 days Custom events: 14 days OR 10M (per customer)	same as Essentials

Metrics data

10s : 4 hours

1min : 1 week

10 min : 2 weeks

1 hr: 2 months

1 day: 15 months

same as Essentials

Events

Alert events: 30 days

Custom events: 14 days OR 10M (per customer)

same as Essentials

Sysdig Secure Retention Limits

	Essentials	Enterprise
Policy events	1M events or 90 days	same as Essentials
Activity audit	5 days	90 days
Benchmarks	30 days	90 days
Scan results	Image data is kept for a maximum of 7 days. Sysdig Secure will retain a maximum of 3 tags per repository and a maximum of 3 different images per tag * Images used by a container that is monitored by a Sysdig agent (Runtime images) will always be kept, regardless of the limits above.	Image data is kept for a maximum of 90 days. Sysdig Secure will retain a maximum of 5 tags per repository and a maximum of 5 different images per tag * Images used by a container that is monitored by a Sysdig agent (Runtime images) will always be kept, regardless of the limits above.
* The image eviction conditions above are applied simultaneously; the retention policy will trigger for the first one that matches. For example, consider an Essentials account with a combination of `registry/repo:tag` - (i.e. `docker.io/alpine:3.12.1`) that already contains three different image digests. If a new image digest is pushed to the same name, the oldest image will be evicted, regardless of its age.

2.4 -

SaaS Regions and IP Ranges

Sysdig SaaS applications are deployed in five data center regions—US East (Virginia), US West AWS (Oregon), US West GCP (Dallas), AP Australia (Sydney) and the European Union (Frankfurt). At the data centers, Sysdig ensures the best security and compliance standards for your data. This page lists the current Sysdig SaaS domains and IP ranges for each region.

For code-based access: Note: The endpoints for Sysdig Monitor and Sysdig Secure are the same in the US West (AWS and GCP), AP Australia, and EU regions. When configuring code-based access to Sysdig Secure, use the endpoint rather than the website URL.

For Single Sign-On: Sysdig SaaS users require the website address to reach the Sysdig applications. Use the appropriate website URL while configuring a single sign-on.

Collector: Additionally, Sysdig agents in a SaaS-based deployment need to be able to reach the Sysdig collector. Depending on your network configuration, you might need to modify your firewall configuration to permit outbound connections from agents to the collector.

Americas

US East (North Virginia)

Sysdig Application	Domain	IP Range
Sysdig Monitor	https://app.sysdigcloud.com	For US East, IPs are assigned dynamically as the system scales. Therefore, we cannot provide the source IP range of the originating traffic.
Sysdig Secure	https://secure.sysdig.com	Splunk Event Forwarding - 18.209.200.129
Sysdig Collector	collector.sysdigcloud.com
Node Image Analyzer	https://collector-static.sysdigcloud.com/internal/scanning/scanning-analysis-collector

US West (Oregon)

Sysdig Application	Domain	IP Range
Sysdig Monitor	https://us2.app.sysdig.com	All the traffic originating from the US West datacenter will have one of the following source IP addresses: 54.218.164.215 54.244.190.180 44.232.85.27 The inbound IP addresses are: 54.190.202.108 54.203.169.53 54.70.9.188
Sysdig Secure	Endpoint: https://us2.app.sysdig.com Website URL: https://us2.app.sysdig.com/secure/
Sysdig Collector	ingest-us2.app.sysdig.com
Node Image Analyzer	https://us2.app.sysdig.com/internal/scanning/scanning-analysis-collector

European Union

Sysdig Application	Domain	IP Range
Sysdig Monitor	https://eu1.app.sysdig.com	All traffic originating from the European Union (EU) datacenter will have one of the following source IP addresses: 3.127.3.205 3.127.111.42 18.157.104.82 The inbound IP addresses are: 18.156.190.126 18.157.62.50 3.126.167.54
Sysdig Secure	Endpoint: https://eu1.app.sysdig.com Website URL: https://eu1.app.sysdig.com/secure/
Sysdig Collector	ingest-eu1.app.sysdig.com
Node Image Analyzer	https://eu1.app.sysdig.com/internal/scanning/scanning-analysis-collector

Asia Pacific (Sydney)

Sysdig Application	Domain	IP Range
Sysdig Monitor	https://app.au1.sysdig.com	Outbound IPs: 13.236.248.84 13.236.151.38 13.54.145.96 The inbound IPs: 13.238.59.195 52.62.57.59 52.64.82.29
Sysdig Secure	Endpoint: https://app.au1.sysdig.com/ Website URL: https://app.au1.sysdig.com/secure/
Sysdig Collector	ingest.au1.sysdig.com
Node Image Analyzer	https://app.au1.sysdig.com/internal/scanning/scanning-analysis-collector

US West (GCP)

Sysdig Application	Domain	IP Range
Sysdig Monitor	https://app.us4.sysdig.com	Outbound IPs: 34.145.19.124 The inbound IPs: 34.145.19.124 34.145.123.253
Sysdig Secure	Endpoint: https://app.us4.sysdig.com/ Website URL: https://app.us4.sysdig.com/secure/
Sysdig Collector	ingest.us4.sysdig.com
Node Image Analyzer	https://app.us4.sysdig.com/internal/scanning/scanning-analysis-collector

Sysdig Collector Ports

Sysdig Agent uses the following ports to communicate with the Sysdig Collector.

Regions	Port
US East	Collector: SSL/TLS 6443 Collector: TCP 6666 UI/API: HTTPS 443
US West	Collector: SSL/TLS 6443 UI/API: HTTPS 443
EU	Collector: SSL/TLS 6443 UI/API: HTTPS 443
Asia Pacific (Sydney)	Collector: SSL/TLS 6443 UI/API: HTTPS 443
US West (GCP)	Collector: SSL/TLS 6443 UI/API: HTTPS 443

AWS Account IDs

Regions	AWS Account IDs
US East	`273107874544`
US West	`263844535661`
EU	`263844535661`

Redirect URLs for Authentication

SAML

Sysdig Monitor

ACS URL / Sign on URL: https://app.sysdigcloud.com/api/saml/auth

Entity ID: https://app.sysdigcloud.com

Sysdig Secure

ACS URL / Sign on URL: https://secure.sysdig.com/api/saml/secureAuth

Entity ID: https://secure.sysdig.com/

Sysdig Monitor

ACS URL / Sign on URL: https://eu1.app.sysdig.com/api/saml/auth
Entity ID: https://eu1.app.sysdig.com
ACS URL / Sign on URL: https://us2.app.sysdig.com/api/saml/auth
Entity ID: https://us2.app.sysdig.com
ACS URL / Sign on URL: https://app.au1.sysdig.com/api/saml/auth
Entity ID: https://app.au1.sysdig.com
ACS URL / Sign on URL: https://app.us4.sysdig.com/api/saml/auth
Entity ID: https://app.us4.sysdig.com

Sysdig Secure:

ACS URL / Sign on URL: https://eu1.app.sysdig.com/api/saml/secureAuth
Entity ID: https://eu1.app.sysdig.com/secure/
ACS URL / Sign on URL: https://us2.app.sysdig.com/api/saml/secureAuth
Entity ID: https://us2.app.sysdig.com/secure/
ACS URL / Sign on URL: https://app.au1.sysdig.com/api/saml/secureAuth
Entity ID: https://app.au1.sysdig.com/secure/
ACS URL / Sign on URL: https://app.us4.sysdig.com/api/saml/secureAuth
Entity ID: https://app.us4.sysdig.com/secure/

OpenID

Sysdig Monitor

Redirect URL:

https://app.sysdigcloud.com/api/oauth/openid/auth

Sysdig Secure

Redirect URL:

https://secure.sysdig.com/api/oauth/openid/secureAuth

Sysdig Monitor

Redirect URL:

Sysdig Secure

Redirect URL:

Google OAuth

Sysdig Monitor

https://app.sysdigcloud.com/api/oauth/google/auth

Sysdig Secure

https://secure.sysdig.com/api/oauth/google/secureAuth

Sysdig Monitor

Sysdig Secure

Prometheus Endpoints

Use the following Prometheus endpoints for Grafana integrations.

Region	Endpoint
US East	https://app.sysdigcloud.com/prometheus
US West	https://us2.app.sysdig.com/prometheus
US West (GCP)	https://app.us4.sysdig.com/prometheus
EU Central	https://eu1.app.sysdig.com/prometheus
Asia Pacific (Sydney)	https://app.au1.sysdig.com/prometheus

2.5 -

On-Premises Deployments

The term “on-premises” (or “on-prem”) is both industry-standard and evolving, so it means different things to different people.

In the context of Sysdig, on-prem customers install and manage the Sysdig backend components as they see fit. This could be in a data center, or in an enterprise’s cloud-provider space, such as AWS or GKE.

Install and Upgrade information is now on GitHub.

With version 3.6.0, the Sysdig Platform can no longer be installed using Replicated.

Oversight Services Now Offered for All Installs and Upgrades

As part of our continued focus on our customers, we are now offering oversight services for all on-premise installs and upgrades. Your Technical Account Manager (TAM), in conjunction with our support organization and Professional Services [where applicable], will work with you to:

Assess your environment to ensure it is configured correctly
Review your infrastructure to validate the appropriate storage capacities are available
Review and provide recommendations for backing up your Sysdig data
Work with you to ensure our teams are ready to assist you during the install and upgrade process
Provide the software for the install
Be available during the process to ensure a successful deployment

You can always review the process in the documentation on GitHub (v. 3.6.0+) or the standard docs site (for older versions).

If you are a new customer looking to explore Sysdig, please head over here to sign up for a trial on our SaaS Platform. Alternatively, you can contact us here.

2.5.1 -

Architecture & System Requirements

Before installing an on-premises solution, review the Sysdig architecture, sizing tips, configuration options, and installation options.

The actual installation instructions can be found in GitHub.

2.5.1.1 -

Architecture

Review the diagram and component descriptions. When installing on-premises, you can decide where to deploy various components.

Sysdig Agent

Sysdig will collect monitoring and security information from all the target entities. To achieve this, one Sysdig agent should be deployed in each host. These hosts can be:

The nodes that make up a Kubernetes or OpenShift cluster
Virtual machines or bare metal
Living in a cloud environment (i.e. AWS, Google Cloud, IBM Cloud, Azure, etc.) or on the customer’s premises

The Sysdig agent can be installed as a container itself using a Helm chart, Kubernetes operator, etc.

Once the agent is installed in the host it will automatically start collecting information from the running containers, container runtime, the orchestration API (Kubernetes, OpenShift, etc), metrics from defined Prometheus endpoints, auto-detected JMX sources, StatsD, and integrations via app checks, as well as the host itself.

The Sysdig agent maintains a permanent communication channel with the Sysdig backend which is used to encapsulate messages containing the monitoring metrics, infrastructure metadata, and security events. The channel is protected using standard TLS encryption and transports data using binary messages. Using this channel, the agent can transmit data, but also receive additional configuration from the backend, such as security runtime policies or benchmarks.

Sysdig Backend

The Sysdig backend is used directly in its SaaS version, thus being managed transparently by Sysdig Inc., or it can also be installed on the customer’s premises. This distinction does not affect the operation of the platform described below.

Once the agent messages are received in the backend, they are processed and extracted into data available to the platform - time series, infrastructure and security events, and infrastructure metadata.

The main components of the backend/platform include:

Extraction and post-processing of the metric data from the agent, so that full time-series, with all the necessary infrastructure metadata, is available to the user
Maintenance of the infrastructure metadata (most notably Kubernetes state), so that all events and time series can be enriched and correctly grouped
Storage of time-series and event data
Processing of time-series data to calculate alert triggers
Queuing the security events triggered by the agents to be shown on the event feed, notifying by the configured notification channels and alerts and forwarding via the Event Forwarder to external platforms like Splunk, Syslog or IBM MCM / Qradar
Aggregating and post-processing other security data such as container fingerprints that will be used to generate container profiles, or security benchmark results.

The Sysdig platform then stores this post-processed data in a set of internal databases that will be combined by the API service to create the data views, such as dashboards, event feeds, vulnerability reports, or security benchmarks.

Sysdig APIs

The Sysdig platform provides several ways to consume and present its internal data. All APIs are RESTful, HTTP JSON-based, and secured using TLS. The same APIs are used to power the Sysdig front end, as well as any API clients (such as sdc-cli).

Monitor API
- User and Team management API
- Dashboard API
- Events API
- Alerts API
- Data API (proprietary Sysdig API for querying time-series data)
Secure API
- Image Scanning API
- Security Events API
- Activity Audit API
- Secure Overview API
PromQL API: Prometheus compatible HTTP API for querying time -series data

These enable different use cases:

User access to the platform via the Sysdig user interface
Programmatic input and extraction of data, i.e.
- Automatic user creation
- Terraform scripts to save or recover configuration state
- Inline scanning to push scanning results from the CI/CD pipeline
- Instrumentation using the sdc-cli.
PromQL API interface that can be used to connect any PromQL-compatible solutions, such as Grafana.

2.5.1.2 -

System Requirements

Supported Distributions

Linux Distributions

A 64-bit Linux distribution with a minimum kernel version of 3.10, and support of docker-engine 1.7.1 or later, is required for each server instance.

Recommended Linux distributions: RedHat, Ubuntu, Amazon AMI, Amazon Linux 2.

Docker Requirements

For the Docker installation, running devicemapper in ‘loopback mode’ is not supported. It has known performance problems and a different storage driver should be used.

Please see this note from our Replicated infrastructure partner: devicemapper-installation-warning.

Installing the latest version of Docker is recommended.

Cassandra

Cassandra is used as the metrics store for Sysdig agents. It is the most dynamic component of the system, and requires additional attention to ensure that your system is performing well and highly responsive.

This component is stateful, and should be treated more carefully than stateless components. Cassandra sizing is based on a minimum replication factor as well as the number of agents writing data.

A minimum replication factor of 3 is recommended for the Sysdig application, which allows the cluster to survive the failure of 1 Cassandra instance.

Each agent consumes anywhere from 500MB to 2GB of Cassandra storage, with average sizing at 1.5GB/agent. Because of Sysdig’s data aggregation model, this storage should comfortably handle multi-year history. This needs to then be multiplied by the replication factor to determine the total disk space required. A rough calculation might be:

100 agents = 150GB raw, X replication factor of 3, = 450GB total

To be safe we recommend that you size some additional disk space as buffer (say 25-50%) on top of that.

Network Configuration

The following firewall/security configurations are required for inbound and outbound traffic for the Sysdig platform:

Ports

Port	State	Direction	Description
6666	Open (optional)	Inbound	Agent communication (unencrypted)
6443	Open	Inbound	Agent Communication (TLS/encrypted)
443	Open	Inbound	Sysdig Monitor user-interface access inbound
443*	Open	Outbound	*Optional, used if collecting AWS CloudWatch metrics. See also AWS: Integrate AWS Account and CloudWatch Metrics (Optional).
443*	Open	Outbound	*Optional, needed if using Sysdig Secure Image Scanning to download vulnerability definitions. Must be open to Cloudflare IP ranges: https://www.cloudflare.com/ips/.
8800	Open	Inbound	Replicated Management Console access (for on-premises installations that don't use Kubernetes)

Warning: Port 6666 should only be opened if agents will be communicating with the collectors without encryption.

Additional ports may need to be configured for the Replicated infrastructure manager. Refer to the Replicated port requirements documentation for more information.

HTTP/HTTPS and Proxy Support

All non-airgapped hosts require outbound HTTP/S internet access for:

License validation
Pulling Sysdig/Agent containers from the Docker hub repository
Release update checks

Note: Sysdig does not support HTTP/S proxies for Sysdig platform components.

Summary: Plan Proxy Support for Notification Channels, CloudWatch Metrics, Capture Storage

In release #760 and newer of the Sysdig platform back-end, an option is available to configure outgoing HTTP/HTTPS connections to be made via proxy. This has been tested and supports outgoing web connections that are necessary to support the following features:

Notification Channels
- PagerDuty
- Slack
- Amazon SNS
- VictorOps
- OpsGenie
- WebHook
Gathering of AWS CloudWatch data
Capture storage to an AWS S3 bucket

Proxied web connectivity to support authentication mechanisms (SAML. OpenID Connect, OAuth) are not supported at this time.

Configure Proxy Using JVM Options

The proxy settings are configured via the JVM options passed to the Sysdig software components. JVM options can be added/appended at any time (with a required restart).

In a Replicated on-premises install, use the Advanced Settings panel to enter JVM options in the Sysdig application JVM options field. (See “Define Advanced Settings” on Install Components (Replicated).)
If JVM settings have already been set, log in to the Replicated Management console and choose the Settings tab. At the bottom of the screen, check the box to Show Advanced Settings to reveal the configuration option.

In a Kubernetes-based on-premises install, set the sysdigcloud.jvm.options in the config.yaml used to set the ConfigMap:

# Optional: Sysdig Cloud application JVM options. For heavy load environments you'll need to tweak
# the memory or garbage collection settings
  sysdigcloud.jvm.api.options: ""
  sysdigcloud.jvm.worker.options: ""
  sysdigcloud.jvm.collector.options: ""

Enter the proxy parameters, as in the example below.

This JVM options string will forward all HTTP and HTTPS traffic via outgoing port 8888 on a proxy at hostname proxy.example.com. The IP address may be specified instead of hostname.

-Dhttp.proxyHost=proxy.example.com -Dhttp.proxyPort=8888 -Dhttps.proxyPort=8888 -Dhttps.proxyHost=proxy.example.com
# Optional: Sysdig Cloud application JVM options. For heavy load environments you'll need to tweak
# the memory or garbage collection settings
sysdigcloud.jvm.api.options: -Xms2048m -Xmx2048m -Dhttp.proxyHost=xxx.xxx.sysdig.com -Dhttp.proxyPort=80 -Dhttps.proxyHost=xxx.xxx.sysdig.com -Dhttps.proxyPort=80

Exclusions

Do not use local host or 127.0.0.1. By default, HTTP/HTTPS requests to localhost or 127.0.0.1 will not be directed by the back-end toward any configured proxy, which is necessary for the functioning of some web components internal to the Sysdig platform containers.
If you deploy the Sysdig platform in AWS, add an additional proxy parameter
-Dhttp.nonProxyHosts=169.254.169.254
Rational: This provides a work-around for the backend occasionally making HTTP requests to a special instance metadata address 169.254.169.254, which is undesirable when using a proxy.
This IP address will be excluded from proxying by default in a future release.
If you have additional proxy exclusions you wish to specify that are unique to your environment, these can also be added using the pipe separator.
For example, assume your deployment was in AWS and you also had a webhook target 192.168.1.2 that was not reachable via your proxy.To exclude both:
Replicated: your complete string to enter into the console for Sysdig application JVM options would be:
```
-Dhttp.proxyHost=proxy.example.com -Dhttp.proxyPort=8888 -Dhttps.proxyPort=8888 -Dhttps.proxyHost=proxy.example.com -Dhttp.nonProxyHosts=169.254.169.254|192.168.1.2
```
Kubernetes: when setting the sysdigcloud.jvm.api.options and sysdigcloud.jvm.worker.options in the config.yaml for the ConfigMap, the pipe separator must be double-escaped, such as:
```
-Dhttps.proxyPort=80 -Dhttps.proxyHost=xx.xx.sysdig.com -Dhttp.nonProxyHosts=169.123.169.123\\|127.0.0.1\\|localhost\\|.sysdig.com"
```

Time Synchronization

The Sysdig platform requires the system clocks to be closely synchronized between hosts. When provisioning hosts for installation, ensure the system clocks are synchronized.

Recommended: Install NTP to ensure all host clocks stay synchronized.

2.5.1.3 -

Securing User Passwords

For MySQL, Redis, and the initial “super admin” user, a strong password is recommended, 16-20 characters, alphanumeric.
For Cassandra and MySQL, it is also possible to set up third-party authentication
For Redis, users can set up an SSH tunnel and Sysdig can connect over this tunnel.

2.5.2 -

On-Premises Installation

When planning to install Sysdig products on-premises, enterprises should:

Organize resources for a test environment and the production environment
Understand the architecture, component requirements, and installation options in Architecture & System Requirements
Decide whether to install using Replicated orchestrator or Kubernetes
Consider the SSO options and plan accordingly (see Authentication and Authorization (On-Prem Options)).

Oversight Services Now Offered for All Installs and Upgrades

Assess your environment to ensure it is configured correctly
Review your infrastructure to validate the appropriate storage capacities are available
Review and provide recommendations for backing up your Sysdig data
Work with you to ensure our teams are ready to assist you during the install and upgrade process
Provide the software for the install
Be available during the process to ensure a successful deployment

You can always review the process in the documentation on GitHub (v. 3.6.0+) or the standard docs site (for older versions).

If you are a new customer looking to explore Sysdig, please head over here to sign up for a trial on our SaaS Platform. Alternatively, you can contact us here.

2.5.2.1 -

Installer (Kubernetes | OpenShift)

For v 3.6.0+, go to the GitHub repo. On-prem installation documentation is transitioning to GitHub.

All on-premises installations and upgrades are now scheduled with and guided by Sysdig technical account managers and professional services division. See Oversight Services Now Offered for All Installs and Upgrades .

For customers, the instructions in this section are for review purposes only.

The Sysdig Installer tool is a binary containing a collection of scripts that help automate the on-premises deployment of the Sysdig platform (Sysdig Monitor and/or Sysdig Secure), for environments using Kubernetes or OpenShift. Use the Installer to install or upgrade your Sysdig platform. It is recommended as a replacement for the earlier Kubernetes manual installation and upgrade procedures.

Installation Overview

To install, you will download the installer binary and a values.yaml file, provide a few basic parameters, and launch the Installer. In a normal installation, the rest is automatically configured and deployed.

You can perform a quick install if your environment has access to the internet, or a partial or full airgapped installation, as needed. Each is described below.

See Frequently Used Installer Configurations to:

Customize or override settings
Use hostPath for static storage of Sysdig components
Use Kubernetes node labels and taints to run only Sysdig pods on selected nodes in a cluster

Install vs Upgrade

With Sysdig Platform 3.5.0, the installer has been simplified from previous versions. Upgrade differs from Install in that you run an installer diff to discover the differences between the old and new versions and then installer deploy for the new version.

If you are installing the Sysdig Platform for the first time, ignore the For Upgrade Only step in the process.

If you are upgrading, check the Upgrade notes before you begin.

Prerequisites

The installer must be run from a machine with kubectl/oc configured with access to the target cluster where the Sysdig platform will be installed. Note that this cluster may be different than where the Sysdig agent will be deployed.

Requirements for Installation Machine with Internet Access

Network access to Kubernetes cluster
Network access to quay.io
A domain name you are in control of.

Additional Requirements for Airgapped Environments

Edited values.yaml with airgap registry details updated
Network and authenticated access to the private registry

Access Requirements

Sysdig license key (Monitor and/or Secure)
Quay pull secret

Storage Requirements

You may use dynamic or static storage on a variety of platforms to store the Sysdig platform components (stateful sets). Different configuration parameters and values are used during the install, depending on which scenario you have.

Use Case 1: Default, undefined (AWS/GKE)

If you will use dynamic storage on AWS or GKE and haven’t configured any storage class there yet, then the Quick Install streamlines the process for you.

storageclassProvisioner: Enter aws or gke. The installer will create the appropriate storage class and then use it for all the Sysdig platform stateful sets.
storageclassName: Leave empty.

Use Case 2: Dynamic, predefined

It is also possible that you are using dynamic storage but have already created storage classes there. This dynamic storage could be AWS, GKE, or any other functioning dynamic storage you use. In this case, you would enter:

storageclassProvisioner: Leave empty; anything put here would be ignored.
storageclassName: Provide the name of the pre-configured storage class you want to use. The installer will use this storage class for all the Sysdig platform stateful sets.

Use Case 3: Static Storage

In cases where dynamic storage is not available, you can use static storage for the Sysdig stateful sets. In this case, you would use:

storageclassProvisioner: Enter hostpath, then define the nodes for the four main Sysdig components: ElasticSearch, Cassandra, MySQL, and Postgres.storageclassProvisioner
See Frequently Used Installer Configurations for details.

Quickstart Install

This install assumes the Kubernetes cluster has network access to pull images from quay.io.

Have your Sysdig Technical Account Manager download the installer binary that matches your OS from the the sysdigcloud-kubernetes releases page.
For Upgrades Only: Copy the current version of values.yaml to your working directory.]
```
./installer-image import -n sysdig --certs-directory certs -o values.yaml
```
If you will be editing for an OpenShift installation and want to review a sample, see openshift-with-hostpath values.yaml. .
Edit the following values:
- size: Specifies the size of the cluster. Size defines CPU, Memory, Disk, and Replicas. Valid options are: small, medium and large
- quaypullsecret: quay.io provided with your Sysdig purchase confirmation mail
- storageClassProvisioner: Review Storage Requirements, above.
  If you have the default use case, enter aws or gke in the storageClassProvisioner field. Otherwise, refer to Use Case 2 or 3.
- sysdig.license: Sysdig license key provided with your Sysdig purchase confirmation mail
- sysdig.dnsname: The domain name the Sysdig APIs will be served on. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- sysdig.collector.dnsName: (OpenShift installs only) Domain name the Sysdig collector will be served on. When not configured it defaults to whatever is configured for sysdig.dnsName. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- deployment: (OpenShift installs only) Add deployment: openshift to the root of the values.yaml file.
- sysdig.ingressNetworking: The networking construct used to expose the Sysdig API and collector.Options are:
  - hostnetwork: sets the hostnetworking in the ingress daemonset and opens host ports for api and collector. This does not create a Kubernetes service.
  - loadbalancer: creates a service of type loadbalancer and expects that your Kubernetes cluster can provision a load balancer with your cloud provider.
  - nodeport: creates a service of type nodeport.The node ports can be customized with:
    sysdig.ingressNetworkingInsecureApiNodePort
    sysdig.ingressNetworkingApiNodePort
    sysdig.ingressNetworkingCollectorNodePort
    When not configured, sysdig.ingressNetworking defaults to hostnetwork.
  If doing an airgapped install , you would also edit the following values:
- airgapped_registry_name: The URL of the airgapped (internal) docker registry. This URL is used for installations where the Kubernetes cluster can not pull images directly from Quay
- airgapped_repository_prefix: This defines custom repository prefix for airgapped_registry. Tags and pushes images as airgapped_registry_name/airgapped_repository_prefix/image_name:tag
- airgapped_registry_password: The password for the configured airgapped_registry_username. Ignore this parameter if the registry does not require authentication.
- airgapped_registry_username: The username for the configured airgapped_registry_name. Ignore this parameter if the registry does not require authentication.
[For Upgrades Only:]
[Generate and review the diff of changes the installer is about to introduce:
```
./installer diff
```
This will generate the differences between the installed environment and the upgrade version. The changes will be displayed in your terminal.
If you want to override a change, based on your environment’s custom settings, then contact Sysdig Support for assistance.]
Run the installer:
```
./installer deploy
```
See Output (below) to finish.

Save the values.yaml file in a secure location; it will be used for future upgrades.

There will also be a generated directory containing various Kubernetes configuration yaml files that were applied by the Installer against your cluster. It is not necessary to keep the generated directory, as the Installer can regenerate it consistently with the same values.yaml file.

Airgapped Installation Options

The installer can be used in airgapped environments, either with a multi-homed installation machine that has internet access, or in an environment with no internet access.

Airgapped with Multi-Homed Installation Machine

This assumes a private docker registry is used and the installation machine has network access to pull from quay.io and push images to the private registry.

The Prerequisites and workflow are the same as in the Quickstart Install (above) with the following exceptions:

In step 2, add the airgap registry information
After step 3, make the installer push Sysdig images to the airgapped registry by running:
```
./installer airgap
```
That will pull all the images into the images_archive directory as tar files and push them to the airgapped registry.
If you are upgrading, run the diff as directed in Step 4.
Run the installer:
```
./installer deploy
```

Full Airgap Install

This assumes a private docker registry is used and the installation machine does not have network access to pull from quay.io, but can push images to the private registry.

In this situation, a machine with network access (called the “jump machine”) will pull an image containing a self-extracting tarball which can be copied to the installation machine.

Access Requirements

Sysdig license key (Monitor and/or Secure)
Quay pull secret
Anchore license file (if Sysdig Secure is licensed)

Requirements for jump machine

Network access to quay.io
Docker
jq

Requirements for installation machine

Network access to Kubernetes cluster
Docker
Network and authenticated access to the private registry
Edited values.yaml with airgap registry details updated
Host Disk Space Requirements:/tmp > 4 GB; directory from which the installer is run >8GB; and /var/lib/docker > 4GB.
NOTE: The environment variable TMPDIR can be used to override the /tmp directory.

Docker Log In to quay.io

Retrieve Quay username and password from Quay pull secret. For example:

AUTH=$(echo <REPLACE_WITH_quaypullsecret> | base64 --decode | jq -r '.auths."quay.io".auth'| base64 --decode)
QUAY_USERNAME=${AUTH%:*}
QUAY_PASSWORD=${AUTH#*:}

Log in to quay.ioUse the username and password retrieved above.
```
docker login -u "$QUAY_USERNAME" -p "$QUAY_PASSWORD" quay.io
```

Workflow

On the Jump Machine

Follow the Docker Log In to quay.io steps, above.
Pull the image containing the self-extracting tar:
```
docker pull quay.io/sysdig/installer:-uber
```

Extract the tarball:

docker create --name uber_image quay.io/sysdig/installer:-uber
docker cp uber_image:/sysdig_installer.tar.gz .
docker rm uber_image

Copy the tarball to the installation machine.

On the Installation Machine:

Copy the current version values.yaml to your working directory.

wget https://raw.githubusercontent.com/draios/sysdigcloud-kubernetes/installer/installer/values.yaml

Edit the following values:
- size: Specifies the size of the cluster. Size defines CPU, Memory, Disk, and Replicas. Valid options are: small, medium and large
- quaypullsecret: quay.io provided with your Sysdig purchase confirmation mail
- storageClassProvisioner: Review Storage Requirements, above.
  If you have the default use case, enter aws or gke in the storageClassProvisioner field. Otherwise, refer to Use Case 2 or 3.
- sysdig.license: Sysdig license key provided with your Sysdig purchase confirmation mail
- sysdig.dnsname: The domain name the Sysdig APIs will be served on. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- sysdig.collector.dnsName: (OpenShift installs only) Domain name the Sysdig collector will be served on. When not configured it defaults to whatever is configured for sysdig.dnsName. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- deployment: (OpenShift installs only) Add deployment: openshift to the root of the values.yaml file.
- sysdig.ingressNetworking: The networking construct used to expose the Sysdig API and collector.Options are:
  - hostnetwork: sets the hostnetworking in the ingress daemonset and opens host ports for api and collector. This does not create a Kubernetes service.
  - loadbalancer: creates a service of type loadbalancer and expects that your Kubernetes cluster can provision a load balancer with your cloud provider.
  - nodeport: creates a service of type nodeport.The node ports can be customized with:
    sysdig.ingressNetworkingInsecureApiNodePort
    sysdig.ingressNetworkingApiNodePort
    sysdig.ingressNetworkingCollectorNodePort
- airgapped_registry_name: The URL of the airgapped (internal) docker registry. This URL is used for installations where the Kubernetes cluster can not pull images directly from Quay
- airgapped_repository_prefix: This defines custom repository prefix for airgapped_registry. Tags and pushes images as airgapped_registry_name/airgapped_repository_prefix/image_name:tag
- airgapped_registry_password: The password for the configured airgapped_registry_username. Ignore this parameter if the registry does not require authentication.
- airgapped_registry_username: The username for the configured airgapped_registry_name. Ignore this parameter if the registry does not require authentication.
Copy the tarball file to the directory where you have your values.yaml file.
Run:
```
installer airgap --tar-file sysdig_installer.tar.gz
```
NOTE: This step will extract the images into the images_archive directory relative to where the installer was run and push the images to the airgapped_registry.
[For Upgrades Only:]
[Generate and review the diff of changes the installer is about to introduce:
```
./installer diff
```
This will generate the differences between the installed environment and the upgrade version. The changes will be displayed in your terminal.
If you want to override a change, based on your environment’s custom settings, then contact Sysdig Support for assistance.]
Run the installer:
```
./installer deploy
```
See Output (below) to finish.

Save the values.yaml file in a secure location; it will be used for future upgrades.

Updating Vulnerability Feed in Airgapped Environments

NOTE: Sysdig Secure users who install in an airgapped environment do not have internet access to the continuous checks of vulnerability databases that are used in image scanning. (See also: How Sysdig Image Scanning Works.)

As of installer version 3.2.0-9, airgapped environments can also receive periodic vulnerability database updates.

When you install with the “airgapped_” parameters enabled (see Full Airgap Install instructions), the installer will automatically push the latest vulnerability database to your environment. Follow the steps below to reinstall/refresh the vuln db, or use the script and chron job to schedule automated updates (daily, weekly, etc.).

To automatically update the vulnerability database, you can:

Download the image file quay.io/sysdig/vuln-feed-database-ubi:latest from the Sysdig registry to the jump box server and save it locally.
Move the file from the jump box server to the airgapped environment (if needed)
Load the image file and push it to the airgapped image registry.
Restart the pod sysdigcloud-feeds-db
Restart the pod feeds-api

The following script (feeds_database_update.sh) performs the five steps:

#!/bin/bash
QUAY_USERNAME="<change_me>"
QUAY_PASSWORD="<change_me>"

# Download image
docker login quay.io/sysdig -u ${QUAY_USERNAME} -p ${QUAY_PASSWORD}
docker image pull quay.io/sysdig/vuln-feed-database-ubi:latest
# Save image
docker image save quay.io/sysdig/vuln-feed-database-ubi:latest -o vuln-feed-database-ubi.tar
# Optionally move image
mv vuln-feed-database-ubi.tar /var/shared-folder
# Load image remotely
ssh -t user@airgapped-host "docker image load -i /var/shared-folder/vuln-feed-database-ubi.tar"
# Push image remotely
ssh -t user@airgapped-host "docker tag vuln-feed-database-ubi:latest airgapped-registry/vuln-feed-database-ubi:latest"
ssh -t user@airgapped-host "docker image push airgapped-registry/vuln-feed-database-ubi:latest"
# Restart database pod
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-db --replicas=0"
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-db --replicas=1"
# Restart feeds-api pod
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-api --replicas=0"
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-api --replicas=1"

Schedule a chron job to run the script on a chosen schedule (e.g. every day):

0 8 * * * feeds-database-update.sh >/dev/null 2>&1

Output

A successful installation should display output in the terminal such as:

All Pods Ready.....Continuing
Congratulations, your Sysdig installation was successful!
You can now login to the UI at "https://awesome-domain.com:443" with:

username: "configured-username@awesome-domain.com"
password: "awesome-password"

There will also be a generated directory containing various Kubernetes configuration yaml files which were applied by installer against your cluster. It is not necessary to keep the generated directory, as the installer can regenerate consistently with the same values.yaml file.

Additional Installer Resources

To see all the configuration parameters available, as well as their definitions, values, and examples, see configuration_parameters.md on GitHub.
For advanced options, including static storage and patching, see Frequently Used Installer Configurations.
Example values.yaml files:
- single-node values.yaml
- openshift-with-hostpath values.yaml

2.5.2.1.1 -

Frequently Used Installer Configurations

SMTP Configs for Email Notifications

The available fields for SMTP configuration are documented in the configuration_parameters.md. Each includes SMTP in its name. For example:

sysdig:
  ...
  smtpServer: smtp.sendgrid.net
  smtpServerPort: 587
  #User,Password can be empty if the server does not require authentication
  smtpUser: apikey
  smtpPassword: XY.abcdefghijk...
  smtpProtocolTLS: true
  smtpProtocolSSL: false
  #Optional Email Header
  smtpFromAddress: sysdig@mycompany.com

To configure email settings to be used for a notification channel, copy the parameters and appropriate values into your values.yaml.

Configure AWS Credentials Using the Installer

The available fields for AWS credentials are documented in the configuration_parameters.md. They are:

sysdig:
  accessKey: my_awesome_aws_access_key
  secretKey: my_super_secret_secret_key

Use hostPath for Static Storage of Sysdig Components

The Installer assumes the usage of a dynamic storage provider (AWS or GKE). In case these are not used in your environment, add the entries below to thevalues.yamlto configure static storage.

Based on the size entered in the values.yaml file (small/medium/large), the Installer assumes a minimum number of replicas and nodes to be provided. You will enter the names of the nodes on which you will run the Cassandra, ElasticSearch, mySQL and Postgres components of Sysdig in the values.yaml, as in the parameters and example below.

Parameters

storageClassProvisioner:``hostPath.
sysdig.cassandra.hostPathNodes: The number of nodes configured here needs to be at minimum 1 when configured size is small, 3 when configured size is medium and 6 when configured size is large.
elasticsearch.hostPathNodes: The number of nodes configured here needs to be at minimum 1 when configured size is small, 3 when configured size is medium and 6 when configured size is large.
sysdig.mysql.hostPathNodes: When sysdig.mysqlHA is configured to true, this must be at least 3 nodes. When sysdig.mysqlHA is not configured, it should be at least 1 node.

sysdig.postgresql.hostPathNodes: This can be ignored if Sysdig Secure is not licensed or used in this environment. If Secure is used, then the parameter should be set to 1, regardless of the size setting

Example

storageClassProvisioner: hostPath
elasticsearch:
  hostPathNodes:
    - my-cool-host1.com
    - my-cool-host2.com
    - my-cool-host3.com
    - my-cool-host4.com
    - my-cool-host5.com
    - my-cool-host6.com
sysdig:
  cassandra:
    hostPathNodes:
      - my-cool-host1.com
      - my-cool-host2.com
      - my-cool-host3.com
      - my-cool-host4.com
      - my-cool-host5.com
      - my-cool-host6.com
  mysql:
    hostPathNodes:
      - my-cool-host1.com
  postgresql:
    hostPathNodes:
      - my-cool-host1.com

Run Only Sysdig Pods on a Node Using Taints and Tolerations

If you have a large shared Kubernetes cluster and want to dedicate a few nodes for just the Sysdig backend component installation, you can use the Kubernetes concept of taints and tolerations.

The basic process is:

Assign labels and taints to the relevant nodes.
Review the sample node-labels-and-taints values.yaml in the Sysdig github repo.
Copy that section to your own values.yaml file and edit with labels and taints you assigned.

Example from the sample file:

# To make the ‘tolerations’ code sample below functional, assign nodes the taint
# dedicated=sysdig:NoSchedule. E.g:
# kubectl taint my-awesome-node01 dedicated=sysdig:NoSchedule
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: sysdig
      effect: "NoSchedule"
# To make the Label code sample below functional, assign nodes the label
# role=sysdig.
# e.g: kubectl label nodes my-awesome-node01 role=sysdig
  nodeaffinityLabel:
    key: role
    value: sysdig

Patching

Patching can be used to customize or “tweak” the default behavior of the Installer to accommodate the unique requirements of a specific environment. Use patching to modify the parameters that are not exposed by thevalues.yaml. Refer to the configuration_parameters.md for more detail about various parameters.

The most common use case for patching is during upgrades. When generating the differences between an existing installation and the upgrade, you may see previously customized configurations that the upgrade would overwrite, but that you want to preserve.

Patching Process

If you have run generate diff and found a configuration that you need to tweak (e.g. the installer will delete something you want to keep, or you need to add something that isn’t there), then follow these general steps:

Create an overlays directory in the same location as the values.yaml.
This directory, and the PATCH.yaml you create for it, must be kept. The installer will use it during future upgrades of Sysdig.
Create a .yaml file to be used for patching. You can name it whatever you want; we will call it PATCH.yaml for this example.
Patch files must include, at a minimum:
- apiVersion
- kind
- metadata.name
of the object to be patched.
Then you add the specific configuration required for your needs. See one example below.
You will need this patch definition for every Kubernetes object you want to patch.
Run generate diff again and check that the outcome will be what you want.
When satisfied, complete the update by changing the scripts value to deploy and running the installer (see Installer Upgrade (2.5.0+).

If you want to add another patch, you can either add a separate .yaml file or a new YAML document separated by ---.

The recommended practice is to use a single patch per Kubernetes object.

Example

Presume you have the following generated configuration:

apiVersion: v1
kind: Service
metadata:
  annotations: {}
  labels:
    app: sysdigcloud
    role: api
  name: sysdigcloud-api
  namespace: sysdigcloud
spec:
  clusterIP: None
  ports:
  - name: api
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: sysdigcloud
    role: api
  sessionAffinity: None
  type: ClusterIP

To Add to the Generated Configuration

Suppose you want to add an extra label my-awesome-label: my-awesome-value to the Service object. Then in the PATCH.yaml, you would put the following:

apiVersion: v1
kind: Service
metadata:
  name: sysdigcloud-api
  labels:
    my-awesome-label: my-awesome-value

Run the installer again, and the configuration would be as follows:

apiVersion: v1
kind: Service
metadata:
  annotations: {}
  labels:
    app: sysdigcloud
    role: api
    my-awesome-label: my-awesome-value
  name: sysdigcloud-api
  namespace: sysdigcloud
spec:
  clusterIP: None
  ports:
  - name: api
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: sysdigcloud
    role: api
  sessionAffinity: None
  type: ClusterIP

To Remove from the Generated Configuration

Supposed you wanted to remove all the labels. Then in the PATCH.yaml, you would put the following:

apiVersion: v1
kind: Service
metadata:
  name: sysdigcloud-api
  labels:

Run the installer again, and the configuration would be as follows:

apiVersion: v1
kind: Service
metadata:
  annotations: {}
  name: sysdigcloud-api
  namespace: sysdigcloud
spec:
  clusterIP: None
  ports:
  - name: api
    port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: sysdigcloud
    role: api
  sessionAffinity: None
  type: ClusterIP

2.5.2.2 -

Manual Install 3.0.0+ (Kubernetes)

For customers, the instructions in this section are for review purposes only.

When installing the Sysdig platform with Kubernetes as the orchestrator, you install each backend component with separate kubectl commands.

Installation with the Installer tool is recommended from version 2.5.0 onwards.

To perform a manual install on OpenShift, see Manual Install (OpenShift).

The manual install on Kubernetes 1.9+ is described below.

Prerequisites

Access to a running Kubernetes cluster 1.9+
(Note: if your environment is installed elsewhere, such as your own data center, contact Sysdig Professional Services to customize the installation instructions appropriately.)
Two items from your Sysdig purchase-confirmation email:
- Your Sysdig license key
- Your Sysdig quay.io pull secret
kubectl installed on your machine and communicating with the Kubernetes cluster
(Note that your kubectl and Kubernetes versions should match to avoid errors.)
An External Load Balancer (required for production – see below)
If installing in a cloud-provider environment (such as AWS, GCloud, or Azure), you will deploy an HAProxy load balancer and point a DNS record to that load balancer.
If installing in your own data center, then you will need two DNS records, one for the collector and one for the UI.
A DNS server and control over a DNS name that you can point to Sysdig

Consider Elasticsearch Default Privileges

By default, the Elasticsearch container will be installed in privileged (root-access) mode. This mode is only needed so the container can reconfigure the hosts' Linux file descriptors if necessary. See Elasticsearch’s description here.

If you prefer not to allow Elasticsearch to run with root access to the host, you will need to:

Set your own file descriptors on all Linux hosts in the Kubernetes cluster.
If one host were to go down, Kubernetes could choose a different node for Elasticsearch, so each Linux host must have the file descriptors set.
Set privileged:false in the elasticsearch-statefulset.yaml file.
See the step under Coonfigure Backend Components, below, for details.

Configure Storage Class

If you are using EKS or GKE, default storage classes are provided; check for them (step 1).

In other environments, you may need to create a storage class (step 2).

Finally, enter the storageClassName in the appropriate .yaml files (step 3).

Verify whether a storage class has been created, by running the command:
```
kubectl get storageclass
```

If no storage class has been defined, create a manifest for one, and then deploy it.

For example, a manifest could be named sysdigcloud-storageclass.yaml and contain the following contents (for a storage class using GP2 volumes in AWS):

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp2
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
  labels:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: EnsureExists
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2

Now run the command:

kubectl apply -f sysdigcloud-storageclass.yaml

Download the Source Files to a New Namespace

Sysdig provides the necessary scripts, images, and .yaml files in a GitHub repository. The first step is to clone those files and check out the latest version. (These examples use 1234.)

Find the current release tag from https://github.com/draios/sysdigcloud-kubernetes/releases/latest.

Run the command:

git clone https://github.com/draios/sysdigcloud-kubernetes.git
cd sysdigcloud-kubernetes
git checkout tags/<1234>

Create a namespace called sysdigcloud:
```
kubectl create namespace sysdigcloud
```

Add External Load Balancer

Create a TCP load balancer (i.e., AWS NLB) that forwards ports 80, 443, 6443 to the Kubernetes worker nodes, with a healthcheck to /healthz on port 10253.

This can be done in three ways:

Use an existing external load balancer. Sysdig relies heavily on DNS; you need a DNS record pointing to the load balancer.
Create a load balancer in your cloud provider. (For example in AWS, see https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-network-load-balancer.html.) You need a DNS record that points to the load balancer. This is the fully qualified domain name required later in the config.yaml, api-ingress.yaml and/or api-ingress-with-secure.yaml.
Create a yaml with the following content and apply it to the sysdigcloud namespace. This automatically creates a load balancer in the cloud provider environment, with an external DNS name.
This is the fully qualified domain name required later in the config.yaml, api-ingress.yaml and/or api-ingress-with-secure.yaml.
```
---
apiVersion: v1
kind: Service
metadata:
  name: haproxy-ingress-lb-service
spec:
  type: LoadBalancer
  ports:
  - name: http
    port: 80
    targetPort: 80
  - name: https
    port: 443
    targetPort: 443
  - name: https2
    port: 6443
    targetPort: 6443
  selector:
    run: haproxy-ingress
```

Apply the changes to the sysdigcloud namespace.

kubectl -n sysdigcloud apply -f <yourlbfile.yamlservice.yaml>

To get the DNS name, run the command:

$ kubectl get svc -o wide -n sysdigcloud

The output shows the External-IP (DNS name):

NAME                         TYPE           CLUSTER-IP       EXTERNAL-IP                           PORT(S)                                        AGE       SELECTOR
haproxy-ingress-lb-service   LoadBalancer   100.66.118.183  sample123.us-east-1.elb.amazonaws.com  80:31688/TCP,443:32324/TCP,6443:30668/TCP      1d        run=haproxy-ingress

DNS Entry (For Test Environments without a Load Balancer)

Not for production environments.

Create a DNS entry for your Sysdig install using the fully qualified domain name that contains all the external IPs as A records. This will use DNS round-robin to load balance your clients to the Kubernetes cluster.

Prepare the Environment

The install images, scripts, and other files are located in a GitHub repository:https://github.com/draios/sysdigcloud-kubernetes

Step 1 Configure Backend Components

The ConfigMap (config.yaml) is populated with information about usernames, passwords, SSL certs, and various application-specific settings.

The steps below give the minimum edits that should be performed in a test environment.

It is necessary to review and customize the entries in config.yaml before launching in a production environment.

See To Make Configuration Changes for the kubectl format to use for post-install edits, such as adding third-party authenticators like LDAP.

If you are not installing Sysdig Secure, set the following attributes to false in the config.yaml:

nats.enabled: “false”
nats.forward.enabled: "false"

Add your license key:
In config.yaml, enter the key that was emailed to you in the following parameter:
```
# Required: Sysdig Cloud license
  sysdigcloud.license: "
```

Change the super admin name and password, which are the super admin credentials for the entire system. See here for details.

Find the settings in config.yaml here:

  sysdigcloud.default.user: test@sysdig.com
  # Required: Sysdig Cloud super admin user password
  # NOTE: Change upon first login
  sysdigcloud.default.user.password: test

Change the mysql.password from change_me to desired credentials.

mysql.password: change_me
  # Required: Cassandra endpoint DNS/IP. If Cassandra is deployed as a
    Kubernetes service, this will be the service name.
  # If using an external database, put the proper address (the address of a
    single node will be sufficient)

**Edit the collector endpoint and api-url:**Change the defaults (sysdigcloud-collector and sysdigcloud-api:443) to point to the DNS name you have established for Sysdig.
Note: The collector port should remain 6443.
```
collector.endpoint: <DNS_NAME>
collector.port: "6443"
api.url: https://<DNS_NAME>:443
```

Recommended: edit the file to set the JVM options for Cassandra, Elasticsearch, and API, worker, and collector as well.

(To use the AWS implicit key, edit the JVM options as described in AWS: Integrate AWS Account and CloudWatch Metrics (Optional).)

For installations over 100 agents, it is recommended to allocate 8 GB per JVM.

  cassandra.jvm.options: "-Xms8G -Xmx8G"
  elasticsearch.jvm.options: "-Xms8G -Xmx8G"
  sysdigcloud.jvm.api.options: "-Xms8G -Xmx8G"
  sysdigcloud.jvm.worker.options: "-Xms8G -Xmx8G"
  sysdigcloud.jvm.collector.options: "-Xms8G -Xmx8G"

Note: If you do not wish to use SSL between the agent and the collector, use the following settings instead:

cassandra.jvm.options: "-Xms8G -Xmx8G"
elasticsearch.jvm.options: "-Xms8G -Xmx8G"
sysdigcloud.jvm.api.options: "-Xms8G -Xmx8G -Ddraios.agents.installParams.sslEnabled=false"
sysdigcloud.jvm.worker.options: "-Xms8G -Xmx8G -Ddraios.agents.installParams.sslEnabled=false"
sysdigcloud.jvm.collector.options: "-Xms8G -Xmx8G -Ddraios.agents.installParams.sslEnabled=false"

Optional: Change ElasticSearch container setting to non-privileged.
See Consider Elasticsearch Default Privileges, above.
To change the default setting, edit the file elasticsearch-statefulset.yaml and set privileged: false.
```
containers:
        - name: elasticsearch
          image: quay.io/sysdig/elasticsearch:5.6.16.15
          securityContext:
            privileged: false
```

Deploy the configuration map and secrets for all services by running the commands:

For Sysdig Monitor:

kubectl -n sysdigcloud apply -f sysdigcloud/config.yaml

To add Sysdig Secure:

kubectl -n sysdigcloud apply -f sysdigcloud/scanning-secrets.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/anchore-secrets.yaml

Apply the secret for the policy advisor:

kubectl -n sysdigcloud apply -f sysdigcloud/policy-advisor-secret.yaml

Configure DNS name in api-ingress.yaml (or api-ingress-with-secure.yaml if using Secure). (Files located in sysdigcloud/)
Edit: host: <EXTERNAL-DNS-NAME> to suit your DNS name
Define namespace in ingress-clusterrolebinding.yaml. (File located in sysdigcloud/ingress_controller/) Edit namespace: sysdigcloud

Step 2 Add Storage Class to Manifests

Using either the existing storage class name from step 1, or the storage class name defined in the previous step, edit the storageClassName in the following .yaml files:

For Monitor:

datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml
datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml
datastores/as_kubernetes_pods/manifests/mysql/mysql-deployment.yaml

With Secure:

datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

Step 3 (Secure-Only): Edit mysql-deployment yaml

If using Sysdig Secure:

Edit the MySQL deployment to uncomment the MYSQL_EXTRADB_* environment variables.

This forces MySQL to create the necessary scanning database on startup.

File location: datastores/as_kubernetes_pods/manifests/mysql/mysql-deployment.yaml

 - name: MYSQL_EXTRADB_SCANNING_DBNAME
              valueFrom:
                configMapKeyRef:
                  name: sysdigcloud-config
                  key: scanning.mysql.dbname
            - name: MYSQL_EXTRADB_SCANNING_USER
              valueFrom:
                configMapKeyRef:
                  name: sysdigcloud-config
                  key: scanning.mysql.user
            - name: MYSQL_EXTRADB_SCANNING_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: sysdigcloud-scanning
                  key: scanning.mysql.password

The scanning service will not start unless MySQL creates the scanning database.

Step 4 Deploy Your Quay Pull Secret

A specific Quay pull secret is sent via email with your license key.

Edit the file sysdigcloud/pull-secret.yaml and change the place holder <PULL_SECRET> with the provided pull secret.

Deploy the pull secret object:

kubectl -n sysdigcloud apply -f sysdigcloud/pull-secret.yaml

Step 5 Set Up SSL Connectivity to the Backend

SSL-secured communication is used between user browsers and the Sysdig API server(s), and between the Sysdig agent and the collectors.

To set this up, you must:

Use existing standard certs for API and collector, or
Create self-signed certificates and keys for API and collector

To Disable SSL between Agent and Collector

To disable SSL between agents and collectors, set JVM options when configuring backend components.

To Create Self-Signed Certs

Run these commands (edit to add your API_DNS_NAME and COLLECTOR_DNS_NAME):

openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj "/C=US/ST=CA/L=SanFrancisco/O=ICT/CN=<API_DNS_NAME>" -keyout server.key -out server.crt
openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj "/C=US/ST=CA/L=SanFrancisco/O=ICT/CN=<COLLECTOR_DNS_NAME>" -keyout collector.key -out collector.crt

To Create Kubernetes Secrets

This uses two different certificates, one for the API/UI, and one for the collector.

kubectl -n sysdigcloud create secret tls sysdigcloud-ssl-secret --cert=server.crt --key=server.key
kubectl -n sysdigcloud create secret tls sysdigcloud-ssl-secret-collector --cert=collector.crt --key=collector.key

Step 6 (Optional) Use CA Certs for External SSL Connection

The Sysdig platform may sometimes open connections over SSL to certain external services, including:

LDAP over SSL
SAML over SSL
OpenID Connect over SSL
HTTPS Proxies

If the signing authorities for the certificates presented by these services are not well-known to the Sysdig Platform (e.g., if you maintain your own Certificate Authority), they are not trusted by default.

To allow the Sysdig platform to trust these certificates, use the command below to upload one or more PEM-format CA certificates. You must ensure you’ve uploaded all certificates in the CA approval chain to the root CA.

kubectl -n sysdigcloud create secret generic sysdigcloud-java-certs --from-file=certs1.crt --from-file=certs2.crt

Install Components

Install Datastores and Backend Components

For Sysdig Monitor:

Create the datastore statefulsets for Elasticsearch and Cassandra. Elasticsearch and Cassandra are automatically set up with --replica=3 generating full clusters.

kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-service.yaml
kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml
kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-service.yaml
kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml

Wait for those processes to be running, then create the database and caching systems: MySQL, and Redis.

kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/mysql/mysql-deployment.yaml
kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/redis/redis-deployment.yaml

To add Sysdig Secure: Create the PostgreSQL database:

kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-service.yaml
kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

Wait until datastore pods are in ready state:
Run the command:
```
kubectl -n sysdigcloud get pods
```
Then look in the READY column to ensure all pods are ready. For example, displaying a 1/1 means 1 of 1 pods is ready

Apply the NATS service and deployment to deliver events to Sysdig backend components:

kubectl -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/nats-streaming/nats-streaming-deployment.yaml
kubectl -n sysdigcloud apply -f  datastores/as_kubernetes_pods/manifests/nats-streaming/nats-streaming-service.yaml

Apply the API deployment. Pause until all containers in the API pod are running, then apply the collector and worker deployments.

kubectl -n sysdigcloud apply -f sysdigcloud/api-deployment.yaml

kubectl -n sysdigcloud apply -f sysdigcloud/collector-deployment.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/worker-deployment.yaml

Create the service for the API and collector:

kubectl -n sysdigcloud apply -f sysdigcloud/api-headless-service.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/collector-headless-service.yaml

Sysdig Secure only Create anchore-engine deployments and service (used in scanning):

kubectl -n sysdigcloud apply -f sysdigcloud/anchore-service.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/anchore-core-config.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/anchore-core-deployment.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/anchore-worker-config.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/anchore-worker-deployment.yaml

Wait 60 seconds to ensure the Anchore components are up and running. Then deploy custom Sysdig Secure scanning components:

kubectl -n sysdigcloud apply -f sysdigcloud/scanning-service.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/scanning-api-deployment.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/scanning-alertmgr-service.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/scanning-alertmgr-deployment.yaml

Sysdig Secure only Create services, deployments, and a janitor job for the activity audit and policy advisor features:

kubectl -n sysdigcloud apply -f sysdigcloud/policy-advisor-service.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/activity-audit-api-service.yaml

kubectl -n sysdigcloud apply -f sysdigcloud/activity-audit-api-deployment.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/policy-advisor-deployment.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/activity-audit-worker-deployment.yaml

kubectl -n sysdigcloud apply -f sysdigcloud/activity-audit-janitor-cronjob.yaml

Connecting to the Cluster

Add Cluster-Admin to User (GKE/GCloud Only)

kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user $(gcloud config get-value account)

Add Ingress Controller

For Sysdig Monitor:

To permit incoming connections to the Sysdig API and collector, deploy the following ingress yamls.

kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-clusterrole.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-clusterrolebinding.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-role.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-rolebinding.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-serviceaccount.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/default-backend-service.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/default-backend-deployment.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-configmap.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-tcp-services-configmap.yaml
kubectl -n sysdigcloud apply -f sysdigcloud/ingress_controller/ingress-daemonset.yaml

If NOT using Sysdig Secure, then apply the following ingress.yaml:

kubectl -n sysdigcloud apply -f sysdigcloud/api-ingress.yaml

For Sysdig Secure:

If you ARE using Secure, replace the api-ingress.yaml with the following line:

kubectl -n sysdigcloud apply -f sysdigcloud/api-ingress-with-secure.yaml

Install Complete

When the terminal messages indicate that installation was successfully completed:

Point your browser to https://API_DNS_NAME.
You will be prompted to log in with the Admin credentials you set in Step 2 Configure Backend Components.
Log in as Super Admin.
The Welcome Wizard is launched and prompts you to install your first Sysdig agent.

Install the agent(s).

The Welcome Wizard should be populated with install parameters from your environment (access key, collector name, and collector port). For example:

{docker run -d --name sysdig-agent --restart always --privileged --net host --pid host -e ACCESS_KEY=xxxxxxxxxx -e COLLECTOR=abc.us-west.elb.amazonaws.com -e COLLECTOR_PORT=6443 -e CHECK_CERTIFICATE=false -e TAGS=example_tag:example_value -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro --shm-size=350m sysdig/agent

To Make Configuration Changes

Replace kubectl with oc for OpenShift.

Update the Config Map

There are two ways to change the original installation parameters in the config map: edit or overwrite.

To edit the config map, run the following command:
```
kubectl edit configmap/sysdigcloud-config --namespace sysdigcloud
```
A text editor is presented with the config map to be edited. Enter parameters as needed, then save and quit.
Then restart the config map (below).
To overwrite the config map that is edited on the client-side, (e.g. to keep it synced in a git repository), use the following command:
```
kubectl replace -f sysdigcloud/config.yaml --namespace sysdigcloud
```
Then restart the config map (below).

Restart Configmap

After updating the configmap, the Sysdig components must be restarted for the changed parameters to take effect. This can be done by forcing a rolling update of the deployments.

A possible way to do so is to change something innocuous, which forces a rolling update. E.g.:

kubectl -n sysdigcloud patch deployment [deploymnet] -p \
 "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"date\":\"$(date +'%s')\"}}}}}"

Replace kubectl with oc for OpenShift.

2.5.2.3 -

Installer (Kubernetes | OpenShift) 2.5.0-3.2.2

For Sysdig installations on Kubernetes or OpenShift, version 2.5.0 and above.

The Sysdig Installer tool is a Docker image containing a collection of scripts that help automate the on-premises deployment of the Sysdig platform (Sysdig Monitor and/or Sysdig Secure), for environments using Kubernetes or OpenShift. Use the Installer to install or upgrade your Sysdig platform. It is recommended as a replacement for the earlier Kubernetes manual installation and upgrade procedures.

Installation Overview

To install, you will log in to quay.io, download a values.yaml file, provide a few basic parameters in it, and launch the Installer. In a normal installation, the rest is automatically configured and deployed.

You can perform a quick install if your environment has access to the internet, or a partial or full airgapped installation, as needed. Each is described below.

See Frequently Used Installer Configurations to:

Customize or override settings
Use hostPath for static storage of Sysdig components
Use Kubernetes node labels and taints to run only Sysdig pods on selected nodes in a cluster

Prerequisites

Requirements for Installation Machine with Internet Access

Network access to Kubernetes cluster
Docker
Bash
jq
Network access to quay.io (See Docker Login to quay.io, below.)
A domain name you are in control of.

Additional Requirements for Airgapped Environments

Edited values.yaml with airgap registry details updated
Network and authenticated access to the private registry

Access Requirements

Sysdig license key (Monitor and/or Secure)
Quay pull secret

Storage Requirements

Use Case 1: Default, undefined (AWS/GKE)

If you will use dynamic storage on AWS or GKE and haven’t configured any storage class there yet, then the Quick Install streamlines the process for you.

storageclassProvisioner: Enter aws or gke. The installer will create the appropriate storage class and then use it for all the Sysdig platform stateful sets.
storageclassName: Leave empty.

Use Case 2: Dynamic, predefined

storageclassProvisioner: Leave empty; anything put here would be ignored.
storageclassName: Provide the name of the pre-configured storage class you want to use. The installer will use this storage class for all the Sysdig platform stateful sets.

Use Case 3: Static Storage

In cases where dynamic storage is not available, you can use static storage for the Sysdig stateful sets. In this case, you would use:

storageclassProvisioner: Enter hostpath, then define the nodes for the four main Sysdig components: ElasticSearch, Cassandra, MySQL, and Postgres.storageclassProvisioner
See Frequently Used Installer Configurations for details.

Retrieve the Quay username and password from Quay pull secret.

For example

AUTH=$(echo <REPLACE_WITH_quaypullsecret> | base64 --decode | jq -r '.auths."quay.io".auth'| base64 --decode)
QUAY_USERNAME=${AUTH%:*}
QUAY_PASSWORD=${AUTH#*:}

**Log in to quay.io.**Use the username and password retrieved above.
```
docker login -u "$QUAY_USERNAME" -p "$QUAY_PASSWORD" quay.io
```

Quickstart Install

This install assumes the Kubernetes cluster has network access to pull images from quay.io.

Copy the current version values.yaml to your working directory.
```
wget https://raw.githubusercontent.com/draios/sysdigcloud-kubernetes/installer/installer/values.yaml
```
If you will be editing for an OpenShift installation and want to review a sample, see openshift-with-hostpath values.yaml.
Edit the following values:
- size: Specifies the size of the cluster. Size defines CPU, Memory, Disk, and Replicas. Valid options are: small, medium and large
- quaypullsecret: quay.io provided with your Sysdig purchase confirmation mail
- storageClassProvisioner: Review Storage Requirements, above.
  If you have the default use case, enter aws or gke in the storageClassProvisioner field. Otherwise, refer to Use Case 2 or 3.
- sysdig.license: Sysdig license key provided with your Sysdig purchase confirmation mail
- sysdig.dnsname: The domain name the Sysdig APIs will be served on. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- sysdig.collector.dnsName: (OpenShift installs only) Domain name the Sysdig collector will be served on. When not configured it defaults to whatever is configured for sysdig.dnsName. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- deployment: (OpenShift installs only) Add deployment: openshift to the root of the values.yaml file.
- sysdig.ingressNetworking: The networking construct used to expose the Sysdig API and collector.Options are:
  - hostnetwork: sets the hostnetworking in the ingress daemonset and opens host ports for api and collector. This does not create a Kubernetes service.
  - loadbalancer: creates a service of type loadbalancer and expects that your Kubernetes cluster can provision a load balancer with your cloud provider.
  - nodeport: creates a service of type nodeport.The node ports can be customized with:
    sysdig.ingressNetworkingInsecureApiNodePort
    sysdig.ingressNetworkingApiNodePort
    sysdig.ingressNetworkingCollectorNodePort
    When not configured, sysdig.ingressNetworking defaults to hostnetwork.
  If doing an airgapped install , you would also edit the following values:
- airgapped_registry_name: The URL of the airgapped (internal) docker registry. This URL is used for installations where the Kubernetes cluster can not pull images directly from Quay
- airgapped_repository_prefix: This defines custom repository prefix for airgapped_registry. Tags and pushes images as airgapped_registry_name/airgapped_repository_prefix/image_name:tag
- airgapped_registry_password: The password for the configured airgapped_registry_username. Ignore this parameter if the registry does not require authentication.
- airgapped_registry_username: The username for the configured airgapped_registry_name. Ignore this parameter if the registry does not require authentication.

Run the installer. (This step differs in Airgapped Installation, below.)

docker run \
  -e HOST_USER=$(id -u) \
  -e KUBECONFIG=/.kube/config \
  -v ~/.kube:/.kube:Z \
  -v $(pwd):/manifests:Z \
  quay.io/sysdig/installer:

See Output (below) to finish.

Save the values.yaml file in a secure location; it will be used for future upgrades.

Airgapped Installation Options

The installer can be used to install in airgapped environments, either with a multi-homed installation machine that has internet access, or in an environment with no internet access.

Updating Vulnerability Feed in Airgapped Environments

As of installer version 3.2.0-9, airgapped environments can also receive periodic vulnerability database updates.

To automatically update the vulnerability database, you can:

Download the image file quay.io/sysdig/vuln-feed-database-ubi:latest from the Sysdig registry to the jump box server and save it locally.
Move the file from the jump box server to the airgapped environment (if needed)
Load the image file and push it to the airgapped image registry.
Restart the pod sysdigcloud-feeds-db
Restart the pod feeds-api

The following script (feeds_database_update.sh) performs the five steps:

#!/bin/bash
QUAY_USERNAME="<change_me>"
QUAY_PASSWORD="<change_me>"

# Download image
docker login quay.io/sysdig -u ${QUAY_USERNAME} -p ${QUAY_PASSWORD}
docker image pull quay.io/sysdig/vuln-feed-database-ubi:latest
# Save image
docker image save quay.io/sysdig/vuln-feed-database-ubi:latest -o vuln-feed-database-ubi.tar
# Optionally move image
mv vuln-feed-database-ubi.tar /var/shared-folder
# Load image remotely
ssh -t user@airgapped-host "docker image load -i /var/shared-folder/vuln-feed-database-ubi.tar"
# Push image remotely
ssh -t user@airgapped-host "docker tag vuln-feed-database-ubi:latest airgapped-registry/vuln-feed-database-ubi:latest"
ssh -t user@airgapped-host "docker image push airgapped-registry/vuln-feed-database-ubi:latest"
# Restart database pod
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-db --replicas=0"
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-db --replicas=1"
# Restart feeds-api pod
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-api --replicas=0"
ssh -t user@airgapped-host "kubectl -n sysdigcloud scale deploy sysdigcloud-feeds-api --replicas=1"

Schedule a chron job to run the script on a chosen schedule (e.g. every day):

0 8 * * * feeds-database-update.sh >/dev/null 2>&1

Airgapped with Multi-Homed Installation Machine

This assumes a private docker registry is used and the installation machine has network access to pull from quay.io and push images to the private registry.

The Prerequisites and workflow are the same as in the Quickstart Install (above) with the following exceptions:

In step 2, add the airgap registry information

In step 3, run the installer as follows:

docker run \
  -e HOST_USER=$(id -u) \
  -e KUBECONFIG=/.kube/config \
  -e IMAGE_EXTRACT_PUSH=true \
  -v ~/.kube:/.kube:Z \
  -v $(pwd):/manifests:Z \
  -v /var/run/docker.sock:/var/run/docker.sock:Z \
  -v ~/.docker:/root/docker:Z \
  quay.io/sysdig/installer:

Full Airgap Install

This assumes a private docker registry is used and the installation machine does not have network access to pull from quay.io, but can push images to the private registry.

In this situation, a machine with network access (called the “jump machine”) will pull an image containing a self-extracting tarball which can be copied to the installation machine.

Requirements for jump machine

Network access to quay.io
Docker
jq

Requirements for installation machine

Network access to Kubernetes cluster
Docker
Bash
tar
Network and authenticated access to the private registry
Edited values.yaml with airgap registry details updated
Host Disk Space Requirements:/tmp > 4 GB; directory from which the installer is run >8GB; and /var/lib/docker > 4GB.
NOTE: The environment variable TMPDIR can be used to override the /tmp directory.

Workflow

On the Jump Machine

Follow the Docker Log In to quay.io steps.
Pull the image containing the self-extracting tar:
```
docker pull quay.io/sysdig/installer:-uber
```

Extract the tarball:

docker create --name uber_image quay.io/sysdig/installer:-uber
docker cp uber_image:/sysdig_installer.tar.gz .
docker rm uber_image

Copy the tarball to the installation machine.

On the Installation Machine:

Copy the current version values.yaml to your working directory.

wget https://raw.githubusercontent.com/draios/sysdigcloud-kubernetes/installer/installer/values.yaml

Edit the following values:
- size: Specifies the size of the cluster. Size defines CPU, Memory, Disk, and Replicas. Valid options are: small, medium and large
- quaypullsecret: quay.io provided with your Sysdig purchase confirmation mail
- storageClassProvisioner: Review Storage Requirements, above.
  If you have the default use case, enter aws or gke in the storageClassProvisioner field. Otherwise, refer to Use Case 2 or 3.
- sysdig.license: Sysdig license key provided with your Sysdig purchase confirmation mail
- sysdig.dnsname: The domain name the Sysdig APIs will be served on. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- sysdig.collector.dnsName: (OpenShift installs only) Domain name the Sysdig collector will be served on. When not configured it defaults to whatever is configured for sysdig.dnsName. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- deployment: (OpenShift installs only) Add deployment: openshift to the root of the values.yaml file.
- sysdig.ingressNetworking: The networking construct used to expose the Sysdig API and collector.Options are:
  - hostnetwork: sets the hostnetworking in the ingress daemonset and opens host ports for api and collector. This does not create a Kubernetes service.
  - loadbalancer: creates a service of type loadbalancer and expects that your Kubernetes cluster can provision a load balancer with your cloud provider.
  - nodeport: creates a service of type nodeport.The node ports can be customized with:
    sysdig.ingressNetworkingInsecureApiNodePort
    sysdig.ingressNetworkingApiNodePort
    sysdig.ingressNetworkingCollectorNodePort
- airgapped_registry_name: The URL of the airgapped (internal) docker registry. This URL is used for installations where the Kubernetes cluster can not pull images directly from Quay
- airgapped_repository_prefix: This defines custom repository prefix for airgapped_registry. Tags and pushes images as airgapped_registry_name/airgapped_repository_prefix/image_name:tag
- airgapped_registry_password: The password for the configured airgapped_registry_username. Ignore this parameter if the registry does not require authentication.
- airgapped_registry_username: The username for the configured airgapped_registry_name. Ignore this parameter if the registry does not require authentication.
Copy the tarball file to the directory where you have your values.yaml file.
Run the tar file:
bash sysdig_installer.tar.gz
NOTE: The above step extracts images, runs the installer, and pushes images to the remote repository in a single step. The extract/push images can be redundant for successive installer runs. Setting IMAGE_EXTRACT_PUSH=false runs only the installer:
IMAGE_EXTRACT_PUSH=false bash sysdig_installer.tar.gz
See Output (below) to finish.

Save the values.yaml file in a secure location; it will be used for future upgrades.

Output

A successful installation should display output in the terminal such as:

All Pods Ready.....Continuing
Congratulations, your Sysdig installation was successful!
You can now login to the UI at "https://awesome-domain.com:443" with:

username: "configured-username@awesome-domain.com"
password: "awesome-password"

Additional Installer Resources

To see all the configuration parameters available, as well as their definitions, values, and examples, see configuration_parameters.md on GitHub.
For advanced options, including static storage and patching, see Frequently Used Installer Configurations.
Example values.yaml files:
- single-node values.yaml
- openshift-with-hostpath values.yaml

2.5.2.4 -

Manual Install (OpenShift)

For customers, the instructions in this section are for review purposes only.

As of Sysdig Platform v 2.5.0, a semi-automated install option is available and is preferred.

This section describes how to install the backend components of the Sysdig platform using an existing OpenShift cluster. It applies to backend versions 1929 and higher.

Introduction

The Sysdig platform includes both Sysdig Monitor and Sysdig Secure, which are licensed separately. All installations include Sysdig Monitor, while some of the Secure components are installed and configured as additional steps within the overall installation process. When installing the Sysdig platform on OpenShift manually, you will install each backend component with separate oc commands.

Prerequisites

Overview

Access to a running OpenShift 3.11+ instance
Two items from your Sysdig purchase-confirmation email:
- Your Sysdig license key
- Your Sysdig quay.io pull secret
octools installed on your machine and communicating with the OpenShift cluster. (Note that your oc and OpenShift versions should match to avoid errors.)

DNS Preparation

If you want more information on OpenShift’s DNS requirements; see the OpenShift documentation.

Option 1: DNS without Wildcard
You need to request two different DNS records from your DNS team: one for the Sysdig API/UI and another for the Sysdig collector. These records should point to your infrastructure nodes and are the two routes that will be exposed, i.e., sysdig.api.example.com and sysdig.collector.example.com.
Option 2: DNS with Wildcard
With wildcard DNS, you do not have to make an official request from the DNS team. Your implementation team can pick any two DNS names to use for the API/UI and Collector. These will be exposed to the infrastructure nodes once the configuration is completed. (i.e. sysdig.api.example.com and sysdig.collector.example.com.)

SSL Certificate Preparation

Step 5: Set Up SSL Connectivity to the Backend discusses how to implement SSL; decide ahead of time whether you will use SSL with wildcard or without.

SSL with Wildcard
With wildcard SSL, you use the same certificate for both the API and the collector.
SSL without Wildcard
You need two SSL certs, one for each DNS record.

Consider Elasticsearch Default Privileges

If you prefer not to allow Elasticsearch to run with root access to the host, you will need to:

Set your own file descriptors on all Linux hosts in the Kubernetes cluster.
If one host were to go down, Kubernetes could choose a different node for Elasticsearch, so each Linux host must have the file descriptors set.
Set privileged:false in the elasticsearch-statefulset.yaml file.
See the step under Coonfigure Backend Components, below, for details.

Prepare the Environment

Step 1 Download and Unpack the Latest Release

Download the latest release from https://github.com/draios/sysdigcloud-kubernetes/releases/latest
Unpack the .tar ball.
The source link has the format: https://github.com/draios/sysdigcloud-kubernetes/archive/<v1234>.tar.gz. To unpack it, run the following commands (replacing version number as appropriate):
```
wget https://github.com/draios/sysdigcloud-kubernetes/archive/<v1234>.tar.gz
tar zxf <1234>.tar.gz
cd sysdigcloud-kubernetes-<1234>
```
Create a new project called sysdigcloud and copy the cloned folders into it:
```
oc new-project sysdigcloud
```

Apply the correct security contexts to the namespace. (This allows you to run privileged containers in the sysdigcloud namespace)

oc adm policy add-scc-to-user anyuid -n sysdigcloud -z default
oc adm policy add-scc-to-user privileged -n sysdigcloud -z default

Step 2: Configure Backend Components

The ConfigMap (config.yaml) is populated with information about usernames, passwords, SSL certs, and various application-specific settings.

The steps below give the minimum edits that should be performed in a test environment.

It is necessary to review and customize the entries in config.yaml before launching in a production environment.

See Making Configuration Changes, below, for the oc format to use for post-install edits, such as adding 3rd-party authenticators such as LDAP.

Add your license key:
In config.yaml, enter the key that was emailed to you in the following parameter:
```
# Required: Sysdig Cloud license
  sysdigcloud.license: ""
```

Change the super admin name and password, which are the super admin credentials for the entire system. See here for details.

Find the settings in config.yaml here:

 sysdigcloud.default.user: test@sysdig.com
  # Required: Sysdig Cloud super admin user password
  # NOTE: Change upon first login
  sysdigcloud.default.user.password: test

**Edit the collector endpoint and API URL:**Change the placeholder to point to the DNS names you have established for Sysdig.
Remember that you must have defined one name for the collector and another for the API URL.
Note: Change the collector port to 443.
```
collector.endpoint: <COLLECTOR_DNS_NAME>
collector.port: "443"
api.url: https://<API_DNS_NAME>:443
```

Recommended: edit the file to set the JVM options for Cassandra, Elasticsearch, and API, worker, and collector as well.

(To use the AWS implicit key, edit the JVM options as described in AWS: Integrate AWS Account and CloudWatch Metrics (Optional).)

For installations over 100 agents, it is recommended to allocate 8 GB of heap per JVM.

  cassandra.jvm.options: "-Xms8G -Xmx8G"
  elasticsearch.jvm.options: "-Xms8G -Xmx8G"
  sysdigcloud.jvm.api.options: "-Xms4G -Xmx8G"
  sysdigcloud.jvm.worker.options: "-Xms4G -Xmx8G"
  sysdigcloud.jvm.collector.options: "-Xms4G -Xmx8G"

Note: If you do not wish to use SSL between the agent and the collector, use the following settings instead:

cassandra.jvm.options: "-Xms8G -Xmx8G"
elasticsearch.jvm.options: "-Xms8G -Xmx8G"
sysdigcloud.jvm.api.options: "-Xms8G -Xmx8G -Ddraios.agents.installParams.sslEnabled=false"
sysdigcloud.jvm.worker.options: "-Xms8G -Xmx8G -Ddraios.agents.installParams.sslEnabled=false"
sysdigcloud.jvm.collector.options: "-Xms8G -Xmx8G -Ddraios.agents.installParams.sslEnabled=false"

Optional: Change ElasticSearch container setting to non-privileged.
See Consider Elasticsearch Default Privileges, above.
To change the default setting, edit the file elasticsearch-statefulset.yaml and set privileged: false.
```
containers:
        - name: elasticsearch
          image: quay.io/sysdig/elasticsearch:5.6.16.15
          securityContext:
            privileged: false
```
Deploy the configuration maps and secrets for all services by running the commands:
For Sysdig Monitor:
```
oc -n sysdigcloud apply -f sysdigcloud/config.yaml
```

**(Sysdig Secure only) Edit and apply secrets for Anchore and the scanning component:**Edit theyaml files:

scanning-secrets.yaml

stringData:
  scanning.mysql.password: change_me

anchore-secrets yaml

stringData:
  anchore.admin.password: change_me
  anchore.db.password: change_me

policy-advisor-secret.yaml

stringData:
  padvisor.mysql.password: change_me

Then apply the files:

oc -n sysdigcloud apply -f sysdigcloud/scanning-secrets.yaml
oc -n sysdigcloud apply -f sysdigcloud/anchore-secrets.yaml
oc -n sysdigcloud apply -f sysdigcloud/policy-advisor-secret.yaml

Edit the API DNS name in either api-ingress.yaml or api-ingress-with-secure.yaml (if using Secure).

The files are located in sysdigcloud/

 spec:
   rules:
     - host: <API_DNS_NAME>
...

 tls:
     - hosts:
         - <API_DNS_NAME>
       secretName: sysdigcloud-ssl-secret

Edit the collector DNS name in the file openshift-collector-router.yaml. Use the collector DNS name you created in the Prerequisites.
The file is located in sysdigcloud/openshift/.
```
spec:
  host: <COLLECTOR_DNS_NAME>
```

Step 3 (Secure-Only): Edit mysql-deployment.yaml

If using Sysdig Secure :

Edit the MySQL deployment to uncomment the MYSQL_EXTRADB_* environment variables. This forces MySQL to create the necessary scanning database on startup.

File location: datastores/as_kubernetes_pods/manifests/mysql/mysql-deployment.yaml

 - name: MYSQL_EXTRADB_SCANNING_DBNAME
              valueFrom:
                configMapKeyRef:
                  name: sysdigcloud-config
                  key: scanning.mysql.dbname
            - name: MYSQL_EXTRADB_SCANNING_USER
              valueFrom:
                configMapKeyRef:
                  name: sysdigcloud-config
                  key: scanning.mysql.user
            - name: MYSQL_EXTRADB_SCANNING_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: sysdigcloud-scanning
                  key: scanning.mysql.password

The scanning service will not start unless MySQL creates the scanning database.

Step 4: Deploy Your Quay Pull Secret

A specific Quay pull secret is sent via email with your license key.

Edit the file sysdigcloud/pull-secret.yaml and change the place holder <PULL_SECRET> with the provided pull secret.

vi sysdigcloud/pull-secret.yaml

    ---
    apiVersion: v1
    kind: Secret
    metadata:
    name: sysdigcloud-pull-secret
    data:
    .dockerconfigjson: <PULL_SECRET>
    type: kubernetes.io/dockerconfigjson

Deploy the pull secret object:

oc -n sysdigcloud apply -f sysdigcloud/pull-secret.yaml

Step 5: Set Up SSL Connectivity to the Backend

SSL-secured communication is used between user browsers and the Sysdig API server(s), and between the Sysdig agent and the collectors.

To set this up, you must:

Use an existing wildcard SSL certificate and key, or
Use existing standard certs for API and collector, or
Create self-signed certificates and keys for API and collector

If you are not using wildcard SSL, you have to use two separate certificates, one for API URL and one for the collector.

To disable SSL between agent and collector:
To disable SSL between agent and collectors, you set a JVM option when configuring backend components (below).

To create self-signed certs:

Run these commands (edit to add your API_DNS_NAME and COLLECTOR_DNS_NAME):

openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj "/C=US/ST=CA/L=SanFrancisco/O=ICT/CN=<API_DNS_NAME>" -keyout server.key -out server.crt
openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj "/C=US/ST=CA/L=SanFrancisco/O=ICT/CN=<COLLECTOR_DNS_NAME>" -keyout collector.key -out collector.crt

To use an existing wildcard cert:
Obtain the respective server.crt and server.key files.

To Create Kubernetes Secrets for the Certs

With Wildcard

Uses the same certificate for both the API/UI and the collector.

Run these commands:

oc -n sysdigcloud create secret tls sysdigcloud-ssl-secret --cert=server.crt --key=server.key
oc -n sysdigcloud create secret tls sysdigcloud-ssl-secret-collector --cert=server.crt --key=server.key

Without Wildcard

Uses two different certificates, one for the API/UI, and one for the collector.

Run these commands:

oc -n sysdigcloud create secret tls sysdigcloud-ssl-secret --cert=server.crt --key=server.key
oc -n sysdigcloud create secret tls sysdigcloud-ssl-secret-collector --cert=collector.crt --key=collector.key

Step 6: (Optional) Use CA Certs for External SSL Connections

The Sysdig platform may sometimes open connections over SSL to certain external services, including:

LDAP over SSL
SAML over SSL
OpenID Connect over SSL
HTTPS Proxies

oc -n sysdigcloud create secret generic sysdigcloud-java-certs --from-file=certs1.crt --from-file=certs2.crt

Install Components (OpenShift)

Edit storageClassName Parameters

You need a storage class; step 2 shows how to create one if needed.

Enter the storageClassName in the appropriate .yaml files (see step 3).

Verify whether a storage class has been created, by running the command:
```
oc get storageclass
```

If no storage class has been defined, create a manifest for one, and then deploy it.

For example, a manifest could be named sysdigcloud-storageclass.yaml and contain the following contents (for a storage class using GP2 volumes in AWS):

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp2
  labels:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: EnsureExists
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2

Now run the command:

oc apply -f sysdigcloud-storageclass.yaml

Using either the existing storage class name from step 1, or the storage class name defined in step 2, edit the storageClassName in the following .yaml files:

For Monitor:

datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml
datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml
datastores/as_kubernetes_pods/manifests/mysql/mysql-deployment.yaml

With Secure:

datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

In each file, the code snippet looks the same:

volumeClaimTemplates:
 - metadata:
     name: data
   spec:
     accessModes: ["ReadWriteOnce"]
     resources:
       requests:
         storage: 50Gi
     storageClassName: <STORAGECLASS_NAME>

Install Datastores and Backend Components

For Sysdig Monitor

Create the datastore statefulsets for Elasticsearch and Cassandra. Elasticsearch and Cassandra are automatically set up with --replica=3 generating full clusters.

oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-service.yaml
oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml
oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-service.yaml
oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml

Wait for those processes to be running, then create the MySQL and Redis databases:

oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/mysql/mysql-deployment.yaml
oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/redis/redis-deployment.yaml

To add Sysdig Secure: Create the PostgreSQL database:

oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-service.yaml
oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

Wait until datastore pods are in ready state, then deploy the backend deployment sets (worker, collector, and API).
Run the command:
```
kubectl -n sysdigcloud get pods
```
Then look in the READY column to ensure all pods are ready. For example, displaying a 1/1 means 1 of 1 pods is ready.

Apply the NATS service and deployment to deliver events to Sysdig backend components:

oc -n sysdigcloud apply -f datastores/as_kubernetes_pods/manifests/nats-streaming/nats-streaming-deployment.yaml
oc -n sysdigcloud apply -f  datastores/as_kubernetes_pods/manifests/nats-streaming/nats-streaming-service.yaml

Then deploy the backend deployment sets (worker, collector, and API). Pause for 60 seconds after creating the API deployment.

oc -n sysdigcloud apply -f sysdigcloud/api-deployment.yaml

oc -n sysdigcloud apply -f sysdigcloud/openshift/openshift-collector-deployment.yaml
oc -n sysdigcloud apply -f sysdigcloud/worker-deployment.yaml

Create the service for the API and collector:

oc -n sysdigcloud apply -f sysdigcloud/api-headless-service.yaml
oc -n sysdigcloud apply -f sysdigcloud/openshift/openshift-collector-service.yaml

For Sysdig Secure Wait for the API, worker, and collector to come up before proceeding.

Then create anchore-engine deployments and service (used in scanning):

oc -n sysdigcloud apply -f sysdigcloud/anchore-service.yaml
oc -n sysdigcloud apply -f sysdigcloud/anchore-core-config.yaml
oc -n sysdigcloud apply -f sysdigcloud/anchore-core-deployment.yaml
oc -n sysdigcloud apply -f sysdigcloud/anchore-worker-config.yaml
oc -n sysdigcloud apply -f sysdigcloud/anchore-worker-deployment.yaml

Wait 60 seconds to ensure the core-deployment is in Running status, then deploy the rest of the Secure-related yamls:

oc -n sysdigcloud apply -f sysdigcloud/scanning-service.yaml
oc -n sysdigcloud apply -f sysdigcloud/scanning-api-deployment.yaml
oc -n sysdigcloud apply -f sysdigcloud/scanning-alertmgr-service.yaml
oc -n sysdigcloud apply -f sysdigcloud/scanning-alertmgr-deployment.yaml

Sysdig Secure only Create services, deployments, and a janitor job for the activity audit and policy advisor features:

oc -n sysdigcloud apply -f sysdigcloud/policy-advisor-service.yaml
oc -n sysdigcloud apply -f sysdigcloud/activity-audit-api-service.yaml

oc -n sysdigcloud apply -f sysdigcloud/activity-audit-api-deployment.yaml
oc -n sysdigcloud apply -f sysdigcloud/policy-advisor-deployment.yaml
oc -n sysdigcloud apply -f sysdigcloud/activity-audit-worker-deployment.yaml

oc -n sysdigcloud apply -f sysdigcloud/activity-audit-janitor-cronjob.yaml

Configure Access for Connectivity to the Cluster

Apply the appropriate ingress yaml. (The API_DNS name was entered in step 7, in Step 2: Configure Backend Components This configures the route to the Sysdig UI.

For Sysdig Monitor

oc -n sysdigcloud apply -f sysdigcloud/api-ingress.yaml

With Sysdig Secure:

oc -n sysdigcloud apply -f sysdigcloud/api-ingress-with-secure.yaml

Configure connectivity to the collector for the agent:

oc -n sysdigcloud apply -f sysdigcloud/openshift/openshift-collector-router.yaml

To Make Configuration Changes

Replace kubectl with oc for OpenShift.

Update the Config Map

There are two ways to change the original installation parameters in the config map: edit or overwrite.

To edit the config map, run the following command:
```
kubectl edit configmap/sysdigcloud-config --namespace sysdigcloud
```
A text editor is presented with the config map to be edited. Enter parameters as needed, then save and quit.
Then restart the config map (below).
To overwrite the config map that is edited on the client-side, (e.g. to keep it synced in a git repository), use the following command:
```
kubectl replace -f sysdigcloud/config.yaml --namespace sysdigcloud
```
Then restart the config map (below).

Restart Configmap

After updating the configmap, the Sysdig components must be restarted for the changed parameters to take effect. This can be done by forcing a rolling update of the deployments.

A possible way to do so is to change something innocuous, which forces a rolling update. E.g.:

kubectl -n sysdigcloud patch deployment [deploymnet] -p \
 "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"date\":\"$(date +'%s')\"}}}}}"

Replace kubectl with oc for OpenShift.

2.5.2.5 -

Install with Replicated

Sysdig will deprecate support for Replicated installs in the coming months. If you are a new customer considering installing with Replicated, please contact Sysdig support.

Understand the Choice Points

When planning an on-premises installation, the following choice points must be decided upon.

Infrastructure Managers: To install Sysdig on-premises, administrators choose one of two infrastructure managers:
- Kubernetes (see Installer (Kubernetes | OpenShift), or
- Replicated: an easy-to-use orchestrator that includes a GUI management tool.
  This guide describes how to install the Replicated client and use it to install and manage the Sysdig platform.
Single-Host or Multi-Host Install: For test or proof-of-concept installations, a single-host install will include all components; for production, a distributed environment is needed.
Airgapped or non-airgapped environment:
If your environment is accessible to the Internet during the install process, then the installation options include both script-based or GUI-based.
In airgapped environments (no Internet access), you must download components into your airgapped repository, and can only use the GUI-based installation.
See Airgapped Installation.
Where to put the Replicated Management Console: When installing on-premises using Replicated as the orchestrator, the following Replicated components will be installed on your system:
- Replicated UI (on a host you designate to host the Replicated Management Console)
- Replicated retraced containers that handle logging (on the Management Console host only)
- Replicated operator component (will go on all hosts)

In a multi-host installation, one server will be the Replicated Management Console host. The system load for these components is minor.

No matter which installation options you choose, you will use the Replicated GUI post-installation to:

Start/stop Sysdig components
Upgrade Sysdig backend components
Collect and view logs (support bundles).

Understand the Installation Process

Review and complete the Pre-Install requirements.
If installing on multiple nodes, decide which node will host the Replicated Management Console.
If using an airgapped environment, set up for an Airgapped Installation.
Install the Replicated Clienton a host.
Log In to the Replicated Management Console and set the Replicated Management Console Password.
Configure Sysdig Admin Password and Basic Settings.
Configure Sysdig Application Advanced Settings (if necessary).
Complete Distributed Install Steps (if necessary).
Restart the host(s).

2.5.2.5.1 -

Airgapped Installation

Sysdig will deprecate support for Replicated installs in the coming months. If you are a new customer considering installing with Replicated, please contact Sysdig support.

To install the Sysdig platform on-premises, in an environment that has no inbound or outbound paths available to internet traffic, you must use the Replicated GUI-based installation option. No script-based option is currently available.

Perform the following steps to download the required Sysdig installation files, the Replicated components, and the Sysdig license file, and save them to a repository on your airgapped server. Then perform the setup steps in the Replicated Management Console, as described below.

Prerequisites

A server instance with Docker version 1.7.1 or later installed is required prior to installation.

The Replicated .airgap installation script does not install docker-engine. Sysdig recommends using the latest version of Docker available for the server operating system.

For more information on installing Docker in an airgapped environment, refer to the Installing Docker in an Airgapped Environment documentation.

Instructions

Download Components to a Repository

Download the latest Sysdig installation files using the links provided by the Sysdig Sales Engineer:
- The Sysdig platform application .airgap package
- The Sysdig application license file (.rli)
- (Optional) The Sysdig Agent Docker image
Download the latest Replicated installation file from:
https://s3.amazonaws.com/replicated-airgap-work/replicated.tar.gz
Copy all downloaded files to a designated location on your airgapped server. For example:
/var/tmp/sysdig
(Note this path to be used when you complete the Install Components (Replicated).)
Open a command shell on the airgapped server and extract the replicated.tar.gz file:
```
sudo tar xzvf replicated.tar.gz
```

Install and Set Up Replicated Management Infrastructure

Run the following command to install the Replicated infrastructure manager:
sudo cat ./install.sh | sudo bash -s airgap
In a browser, navigate to the Replicated Management Console: https://server_address:8800 **(**Replace server_address with the server name/IP address.)
Accept the default self-signed certificate, or provide a custom one, and click Continue.
On the next screen, once the “preflight” checks have been resolved, select the Airgapped option, and click Continue.
Upload the .rli license file.
Provide a path to the Sysdig application .airgap file.
Should you need to upgrade an airgapped license at a future time, see Upgrade an On-Premises License. For general license information, see Subscription.

Complete the Installation Steps

Continue with “Setting the Replication Management Password” and the rest of the installation steps in Install Components (Replicated).

2.5.2.5.2 -

Install Components (Replicated)

Sysdig will deprecate support for Replicated installs in the coming months. If you are a new customer considering installing with Replicated, please contact Sysdig support.

You can use the Replicated UI to install the Sysdig platform on either a single host or on multiple hosts. If multi-host, decide which machine will also run the Replicated Admin Console and begin there.

If your environment is “airgapped” (no access to inbound or outbound internet traffic), there are some setup steps you must perform before doing the GUI-based Replicated installation.

See Airgapped Installation for details.

Install the Replicated Client

Log in to the chosen machine with a shell and run a command to install the Replicated components. You can also install Docker if it’s not already on the environment.

Log into the designated server instance with SSH.

Run the following commands:

a. To install the Replicated Infrastructure and Docker:

sudo curl -sSL https://install.sysdigcloud.com/docker | sudo bash

b. If Docker is already installed on the server instance, add-s --no-dockerto the command:

sudo curl -sSL https://install.sysdigcloud.com/docker | sudo bash -s -- no-docker

c. If installing the Replicated Infrastructure behind a proxy, modify the installation command as shown below:

sudo curl -sSL -x http://<proxy>:<port> -o /tmp/sdc-onpremises-installer.sh https://install.sysdigcloud.com/docker && bash /tmp/sdc-onpremises-installer.sh http-proxy=http://<proxy>:<port>

Define Basic Settings & License Info

Log In to Replicated Admin Console/ “admin console” and Set SSL Certificate

As prompted, open the Replicated Client at https://<yourserver>:8800.
Supply the DNS hostname for the Replicated Admin Console.
Accept the self-signed certificate, or upload a custom SSL certificate and private key.
Note: If a self-signed certificate is uploaded, it must include the end user, all intermediate, and the root certificates, as the certificate will be used by the Sysdig platform, as well as for the Replicated Admin Console.
To later replace a self-sign cert with a custom cert, see Replace a Self-Signed Cert with Custom Cert.
Click the Choose License button, and upload the Sysdig license file supplied from Sysdig Sales.
Choose Online installation option if prompted.

Set the Replicated Admin Console Password

Once the Sysdig license validation is complete, secure the Replicated Admin Console using a local password, LDAP user account, or anonymous access (insecure).

Sysdig recommends securing the console with either a local password or LDAP user account.

Click Continue.

Configure Sysdig Super Admin Password and Basic Settings

After clicking Continue, the Settings page is displayed. Here you enter the configuration information that will be used by Replicated to orchestrate the Sysdig installation.

|| ||

Define Advanced Settings

These settings are typically defined with consultation from a Sysdig Sales Engineer.

Any JVM options to be passed to the application, such as memory constraint settings for the Java Virtual Machine components, proxy settings, etc.

At a minimum, it is recommended to define the memory constraints, in the format:

-Xms###g Xmx###g.

Note that if multiple components are on a single machine, adjust the percentages as needed so JVMs all fit in a node.

Cassandra JVM options: recommended allocating 50% of the host’s memory to this JVM
(in a multi-node environment)
Elasticsearch JVM options: recommended allocating 50% of the host’s memory to this JVM
(in a multi-node environment)
Sysdig Cloud application JVM options: recommended to allocate up to 80% of the host’s memory to this JVM.
This is also used to set proxy settings; see HTTP/HTTPS and Proxy Support.
It is also used to set an implicit key in AWS; see AWS: Integrate AWS Account and CloudWatch Metrics (Optional).
NOTE: If you do not want to use SSL between the agent and the collectors, you append the following settings to the Sysdig Cloud application JVM options entry:
```
-Ddraios.agents.installParams.sslEnabled=false
```
For example:
```
-Xms8G Xmx8G -Ddraios.agents.installParams.sslEnabled=false
```

Ports and Security

Sysdig UI port: default 80. Port used for the Sysdig Monitor/ Sysdig Secure GUI.
Sysdig UI secure port: default 433. SSL port used for Sysdig Monitor/ Sysdig Secure GUI.
Force HTTPS: This turns off the unsecured port (80) access.
Forward Sysdig application logs to stdout: switches logging from the application log files to Linux standard output (stdout).
Sysdig collector port: default 6443. Port used for agent metrics collection. See also Agent Installation.
In earlier versions, the Sysdig Agent connected to port 6666. This behavior has been deprecated, as the Sysdig agent now connects to port 6443.
Sysdig secure collector port: default 6443. Port used for agent metrics collection. See also Agent Installation.
Exposed port for HTTP traffic inbound to Sysdig Platform backend container: 27878 – do not change without the recommendation of Sysdig Support.
Exposed port for Collector traffic inbound: 27877 – do not change without the recommendation of Sysdig Support.

Database Entries

Store Sysdig Captures in Cassandra (recommended): Default checked. Used for Sysdig trace file storage when capture function is used. If you do not store files in the Cassandra DB, you can alternately configure an AWS S3 bucket storage location.
See also: Storage: Configure AWS Capture File Storage (Optional) and Captures.
Sysdig data directory: default /opt. Where Cassandra, MySQL, and Elasticsearch databases will be created on a host.
Cassandra CQL native client’s port: The default port is 9042. Change the default port if you are running your own Cassandra cluster with non-standard ports.
Cassandra replication factor: The value should be either 1 or 3, never 2.
Sysdig MySQL user: default admin, recommend changing
Sysdig MySQL password: Enter a unique password and store securely.
This password is needed for future updates and will not be visible in the Replicated Admin Console. Retain this password for future use.
Sysdig MySQL max connections: The default is 1024.
Cassandra CQL native client’s port: The default is 9042.
External MySQL service: The secure end of your MySQL service. This is external to the Sysdig platform.
External Cassandra service: The secure end of your Cassandra service. This is external to the Sysdig platform.
External Redis service: The secure end of your Redis Service. This is external to the Sysdig platform.
Sysdig Redis password: The password associated with the Redis account.
External Elasticsearch service URL: An external service URL with user name and password embedded
OAuth allowed domains: List of secure Elasticsearch domains.
Google OAuth client ID: Used when integrating Google-based user login.
See Google OAuth (On-Prem)
Google OAuth client secret: Used when integrating Google-based user login. See Google OAuth (On-Prem)
SSO CA certificate: CA certificate for single sign-on.
Datastore Authorization and SSL: See Authenticating Backend Components on Cassandra and Authenticating Backend Components on Elasticsearch.

When fields are complete, click Save.

After Saving, click Start Now to apply settings to the environment immediately. Click **Cancel**to apply settings at a later time.

Authenticating Backend Components on Cassandra

As of version 2.4.1, authenticating Sysdig backend components on Sysdig’s Cassandra nodes or for your own Cassandra nodes is supported. In order to authenticate the backend components to Cassandra, enable the option, specify credentials of the identity you want to establish with Cassandra, and enable secure communication. This is the additional layer of defense against unauthorized access to the datastore.

Enable Cassandra authentication: Select this option if you want to authenticate Sysdig backend components to use Cassandra datastore. The option by default is disabled.
Cassandra password for authentication: The password associated with the username. If running Sysdig’s Cassandra database, create a password here. If you are using your own Cassandra database, enter the appropriate user password for Sysdig access.
Enable Cassandra TLS: (Mandatory) Establish TLS communication between the Sysdig backend components and the Cassandra node. The option by default is unchecked.
Cassandra username for authentication: The username of the identity that you want to establish with Cassandra. If running Sysdig’s Cassandra database, create a user here. If you are using your own Cassandra database, enter the appropriate user account for Sysdig access.

Authenticating Backend Components on Elasticsearch

As of version 2.4.1, authenticating Sysdig backend components on both Sysdig’s Elasticsearch cluster or for your own Elasticsearch cluster is supported. In order to authenticate the backend components to Elasticsearch datastore, configure TLS-based authentication. You generate certificates and keys for Elasticsearch server, client, and admin user, and specify them along with Elasticsearch user credentials while setting up Sysdig platform. This is the additional layer of security to safeguard the datastore.

Before you configure Elasticsearch authentication, ensure that you set up Sysdig Agent for data collection and TLS generate certificates.

Generate TLS Certificates

Log into Quay:

Locate your Quay pull_secret. Contact Support if you are unable to locate it.

Get your credentials by running:

# Note: For MacOS users, change "base64 -d" to "base64 -D"echo <quay_pull_secret> | base64 -d | awk NR==4 | cut -d'"' -f4 | xargs | base64 -d

The Output should look as follows:

sysdig+<your_username>:<your_password>

Log into Quay by running the following:

docker login quay.io -u sysdig+<your_username> -p <your_password>

Run the following docker command to generate the root/admin certificates for Elasticsearch to a directory within the current working directory:
```
docker run -d -v "$(pwd)"/generated_elasticsearch_certs:/tools/out quay.io/sysdig/elasticsearch:1.0.1-es-certs
```
The following files are generated in the generated_elasticsearch_certs directory. Retain the certificates and key files to upload as part of the TLS configuration as described in Configure TLS Authentication.
- Elasticsearch root CA
  - root-ca.pem
  - root-ca.key
- Elasticsearch Admin (Kirk)
  - kirk.pem
  - kirk.key
- EElasticsearch Client (Spock)
  - spoke.pem
  - spoke.key

Configure TLS Authentication

Sysdig Replicated install supports Search Guard to establish secure authentication with Elasticsearch datastore. You set up two users in order to access Elasticsearch datastore on behalf of the Sysdig backend components: Admin user and read-only user.

Amin user: The admin user will have the read and write permissions on Elasticsearch clusters and indices. Sysdig backend components use this identity to write data to Elasticsearch clusters. This is the same as the Search Guard admin user.

Read-only user: As the name implies, the read-only user will only have the read permission on Elasticsearch indices. Sysdig Agent uses this identity to read data from Elasticsearch datastore. This is the same as the Search Guard sg_readonly user that is created as part of the installation.

Enable Elasticsearch Authentication and TLS: Select this option to enable authentication and secure communication between Sysdig backend components and the Elasticsearch datastore. To gain access to Elasticsearch datastore, you must prove your identity, by using credentials and certificates. The Elastic Stack authenticates users by identifying the users behind the requests that hit the datastore and verifying that they are who they claim to be.
Elasticsearch admin username: The admin user is created by default. You can edit the user name if desired. The default user is admin.
Elasticsearch admin password: The password associated with the Elasticsearch admin user.
Elasticsearch read-only username: Specify the username for the read-only access to the Elasticsearch indices. If running your own secure Elasticsearch cluster, enter the username for the read-only Search Guard user.
Elasticsearch read-only password: The password associated with Elasticsearch read-only username.

When fields are complete, click Save.

After saving, click Restart Now to apply settings to the environment immediately.

Click Cancel to apply settings at a later time.

Configure Sysdig Agent

If you are monitoring Elasticsearch with sysdig-agent, ensure the sysdig-agent configuration file, dragent.yaml, has the following Elasticsearch configuration in the data.dragent.yaml.app_checks section below:

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: https://<DNS_or_ip_address_to_elasticsearch>:9200
      username: <your_read_only_username>
      password: <your_read_only_password>
      ssl_verify: false

Example for Docker Environment

Follow these steps if you are running the Agent in a Docker container:

READONLY_USERNAME=<your_readonly_username>
READONLY_PASSWORD=<your_readonly_username_password>
ELASTICSEARCH_PORT=9200
URL_TO_SECURE_ELASTICSEARCH=https://<your_url_to_secure_elasticsearch>
ADDITIONAL_CONF="$(echo "app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: $ELASTICSEARCH_PORT
      comm: java
    conf:
      url: $URL_TO_SECURE_ELASTICSEARCH:$ELASTICSEARCH_PORT
      username: $READONLY_USERNAME
      password: $READONLY_PASSWORD
      ssl_verify: false
" | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/\\n/g')"

Remove the existing Agent container:
Make sure that you remove the existing Agent container instead of just stopping it. By default, the Agent container is named sysdig-agent. If you stop the Agent container and attempt to create a new one, you will get a name-conflict error:
docker: Error response from daemon: Conflict. The container name “/sysdig-agent” is already in use by container <ontainer-id>. You have to remove (or rename) that container to be able to reuse that name.

Run the Agent container with the new additional config. For example:

docker run \
    --name sysdig-agent \
    --restart always \
    --privileged \
    --net host \
    --pid host \
    -e ACCESS_KEY=1234-your-key-here-1234 \
    -e COLLECTOR=collector_ip \
    -e COLLECTOR_PORT=6443 \
    -e SECURE=true \
    -e TAGS=dept:sales,local:NYC \
    -e ADDITIONAL_CONF="$ADDITIONAL_CONF" \
    -v /var/run/docker.sock:/host/var/run/docker.sock \
    -v /dev:/host/dev \
    -v /proc:/host/proc:ro \
    -v /boot:/host/boot:ro \
    -v /lib/modules:/host/lib/modules:ro \
    -v /usr:/host/usr:ro \
    sysdig/agent

You may encounter an error in the sysdig-agent logs stating that an unverified HTTPS request has been made. You can safely ignore the error for now.

Example for Non-Containerized Environment

Do the following if you are running the Agent directly on the machine (non-containerized environment):

Add the app_check configuration to your /opt/draios/etc/dragent.yaml configuration:

app_checks:
  - name: elasticsearch
    check_module: elastic
    pattern:
      port: 9200
      comm: java
    conf:
      url: https://<DNS_or_ip_address_to_elasticsearch>:9200
      username: <your_read_only_username>
      password: <your_read_only_password>
      ssl_verify: false

Restart the agent:
```
service dragent restart
```

Single-Host Installation Wrap-Up

After completing the Settings and restarting, no further installation steps are required for a single-host install.

The dashboard will remain in Starting mode for approximately 4-5 minutes, depending on the internet connection bandwidth, while Sysdig application software is downloaded and installed. Once the installation is complete, the dashboard will move to Started mode.

Click the Open link to navigate to the Sysdig Monitor login panel.
Input the Super Admin user login credentials defined in the basic settings, above.

Next Steps

To start, stop, and update the application, or to retrieve support information, use the Replicated Admin Console: https://<yourserver>:8800.
To login as a user and see metrics for hosts with the Sysdig Agent installed, use the Sysdig Monitor Web Interface: https://<yourserver>:80
- If you have not yet done so, install Sysdig Agents to monitor your environment. See Agent Installation for details.

Multi-Host Installation Wrap-Up

After configuring the settings and clicking Start Now, an error will indicate the need to assign and install the remaining components. You will need to define the hosts/nodes to be used and will assign the Sysdig components to be installed on them. The steps below describe the actions on one host; they must be repeated on all applicable hosts and all the Sysdig components must be assigned.

Choose the Cluster tab in the Replicated Admin Console.
From here, you can tag components to be run on the local host, and/or add new nodes.
To add and configure new nodes:
Click Add Node.
The Add Node worksheet is displayed. Here you enter the IP address and then tag the Sysdig component(s) to be installed on that node.
Replicated will compile either an installation script or a Docker run command out of your entries, which you will copy and use on the given node.

On the Add Node worksheet page, do the following:

Choose Installation script or Docker run command option.

Enter private and/or public IP address, depending on the type of access you want to permit.

Select the Sysdig components to be installed by checking the appropriate “Tags” buttons.

Descriptions in the table below:

Name	Tag	Role Description
api	api	Application Programming Interface server
cassandradb	cassandra	Cassandra database server
elasticsearch	elasticsearch	Elasticsearch server for events storage/search
collector	collector	Agent metrics collector
lb_collector	lb_collector	Load balancer for collector service; handles connections from the agents
lb_api	lb_api	Load balancer for API service; handles user connection requests to the Sysdig application. Use the address for this node as the DNS entry for the cluster.
mysql, redis	mysql & redis	MySQL & Redis databases
worker	worker	Metrics history processor
emailrenderer	emailrenderer	Email renderer
nginxfrontend	nginxfrontend	Frontend static server

When setting up a DNS entry for the cluster, use the address for the ‘lb_api' node.

At the bottom of the page, a curl script or Docker run command is compiled for you.

Copy the command and issue it on the targeted host.

Repeat this procedure on all desired hosts.
Restart the Sysdig application from the Replicated console.
The dashboard will be in “Starting” mode for several minutes while software is downloaded and installed onto each server component (depending on your internet connection bandwidth).
You should see green check marks for each host next to the Provisioned and Connected columns, as the software is installed and the node connects successfully to the Replicated Admin server.
Once the installation is fully completed, the infrastructure admin dashboard will be in “Started” mode and will also show the “Open” link that will bring you to Sysdig Monitor web interface login screen.
At the login screen, use the credentials configured earlier (Default User) to log in and start using the Sysdig application on-premises solution.
To start, stop, and update the application or retrieve support information use the Replicated Admin dashboard: https://server_address:8800
To log in as a user and see metrics about hosts where Sysdig agents are installed, use the Sysdig Monitor UI: https://server_address:80

2.5.2.5.3 -

Post-Install Configuration

Sysdig will deprecate support for Replicated installs in the coming months. If you are a new customer considering installing with Replicated, please contact Sysdig support.

These configurations are optional.

Replace a Self-Signed Cert with Custom Cert

This process differs depending on how you installed the Sysdig Platform.

For Kubernetes Installer Installs

If you installed the Sysdig Platform on Kubernetes or OpenShift using the Installer, the Installer automatically generates a self-signed cert on the fly. To use a different certificate you would:

Add your cert and key to the /certs directory ex: (server.crt, server.key)

Update values.yaml:

sysdig:
  certificate:
    crt: certs/server.crt
    key: certs/server.key

Rerun the Installer.

The configuration_parameter.md Readme gives full details on sysdig.certificate.crt and sysdig.certificate.key.

For Kubernetes Manual Installs

If you installed the Sysdig Platform manually on Kubernetes or OpenShift, the steps for managing the certs are described in Step 5 of the installation procedures:

For Replicated Installs

If you installed the Sysdig Platform using Replicated and you accepted the self-signed certificate for SSL/TLS communication when installing the Sysdig components (see Define Basic Settings & License Info ), you can exchange for a custom certificate as follows:

Log in to the Replicated Management Console and select the Gear icon > Console Settings.
Click Upload certificate and it will automatically replace the original self-signed certificate.

Optional: Custom Self-Signed Certificat

Sysdig Monitor/Cloud/etc uses a self-signed SSL/TLS security certificate, unless a custom certificate is provided.

The example command below creates a custom, unsigned certificate called MyCert.pem; the certificate has a private key called MyCert.key, and is valid for five years:

sudo openssl req -new -x509 -sha256 -days 1825 -nodes -out ./MyCert.pem -keyout ./MyCert.key

2.5.2.6 -

Troubleshooting On-Premises Installation

Collect Troubleshooting Data

When experiencing issues, you can collect troubleshooting data that can help the support team. The data can be collected by hand, or Sysdig provides a very simple get_support_bundle.sh script that takes as an argument the namespace where Sysdig is deployed and will generate a tarball containing some information (mostly log files). The script is located in the GitHub repository: https://github.com/draios/sysdigcloud-kubernetes. (See also Get Help | Using Sysdig Support (On-Prem).)

$ ./scripts/get_support_bundle.sh sysdigcloud
Getting support logs for sysdigcloud-api-1477528018-4od59
Getting support logs for sysdigcloud-api-1477528018-ach89
Getting support logs for sysdigcloud-cassandra-2987866586-fgcm8
Getting support logs for sysdigcloud-collector-2526360198-e58uy
Getting support logs for sysdigcloud-collector-2526360198-v1egg
Getting support logs for sysdigcloud-mysql-2388886613-a8a12
Getting support logs for sysdigcloud-redis-1701952711-ezg8q
Getting support logs for sysdigcloud-worker-1086626503-4cio9
Getting support logs for sysdigcloud-worker-1086626503-sdtrc
Support bundle generated: 1473897425_sysdig_cloud_support_bundle.tgz

Docker Connectivity Issues (IPv4/IPv6)

Some issues with IPv4 and IPv6 interconnectivity between on-premises containers and the outside world have been detected.

IP packet forwarding is governed by the ip_forward system parameter. Packets can only pass between containers if this parameter is 1. Usually, you will simply leave the Docker server at its default setting --ip-forward=trueand Docker will go set ip_forward to 1 for you when the server starts up. If you set --ip-forward=false and your system’s kernel has it enabled, the --ip-forward=false option has no effect.

To check the setting on your kernel use:

sysctl net.ipv4.conf.all.forwarding

To turn it on use:

sysctl net.ipv4.conf.all.forwarding=1

Please see this article from docker for more details on Docker Connectivity.

Proxy/Firewall Issues

Prior to installing ensure your proxy settings are valid for the session. You can use curl, lynx, or wget to test internet connectivity:

export http_proxy="http://user:password@proxy_server:port"
export https_proxy="https://user:password@proxy_server:port"
echo $http_proxy

You can then attempt a curl or docker hub call to ensure outside connectivity.

Firewall

Prior to installation, you may want to disable local firewall (iptables) to rule out local connectivity issues.

However here are some details around Sysdig connectivity and backend connectivity requirements.

Sysdig Connectivity:

6443 Agent communication

443 Sysdig Monitor UI access

8800 Management console access

Here are specifics around what is used for connectivity for the Sysdig backend for on-premises solution:

https://www.replicated.com/docs/kb/supporting-your-customers/firewalls/

File Write Permissions Issues (SELINUX or APP ARMOR)

During the install, you may see errors writing to volumes such as (/var or /opt) from either the onprem install scripts or Docker. You should disable SELINUX (CENTOS/RHEL) or Apparmor (UBUNTU/DEBIAN) during the course of the install so the valid directories can be created. This can be accomplished by:

Centos (SELINUX)

From the command line, edit the /etc/sysconfig/selinux file. This file is a symlink to /etc/selinux/config. The configuration file is self-explanatory. Changing the value of SELINUX or *SELINUXTYPE*changes the state of SELinux and the name of the policy to be used the next time the system boots.

[root@host2a ~]# cat /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - SELinux is fully disabled.
SELINUX=permissive
# SELINUXTYPE= type of policy in use. Possible values are:
#       targeted - Only targeted network daemons are protected.
#       strict - Full SELinux protection.
SELINUXTYPE=targeted

# SETLOCALDEFS= Check local definition changes
SETLOCALDEFS=0

See SELinux Modes for more information.

UBUNTU/Debian (AppArmor)

AppArmor can be disabled, and the kernel module unloaded by entering the following:

sudo systemctl stop apparmor.service
sudo update-rc.d -f apparmor remove

To re-enable AppArmor enter:

sudo systemctl start apparmor.service
sudo update-rc.d apparmor defaults

Advanced Troubleshooting - Firewall, IPtables, IP forwarding

In the preflight check step with Replicated, if you come across the error:

getsockopt: no route to host

Please do the following:

For CentOS 7/RedHat:

service firewalld stop
systemctl disable firewalld
sysctl -w net.ipv4.ip_forward=1
iptables -F
setenforce 0
service docker restart

For Ubuntu:

sysctl -w net.ipv4.ip_forward=1
systemctl stop apparmor.service
update-rc.d -f apparmor remove
ufw disable
iptables -F
service docker restart

2.5.3 -

On-Premises Upgrades

This section describes how to upgrade an on-premise installation, depending on whether it was installed using a Kubernetes or a Replicated orchestrator.

As needed, version-specific upgrade or migration instructions will be added to this section.

Oversight Services Now Offered for All Installs and Upgrades

Assess your environment to ensure it is configured correctly
Review your infrastructure to validate the appropriate storage capacities are available
Review and provide recommendations for backing up your Sysdig data
Work with you to ensure our teams are ready to assist you during the install and upgrade process
Provide the software for the install
Be available during the process to ensure a successful deployment

You can always review the process in the documentation on GitHub (v. 3.6.0+) or the standard docs site (for older versions).

If you are a new customer looking to explore Sysdig, please head over here to sign up for a trial on our SaaS Platform. Alternatively, you can contact us here.

Explore the Upgrade Topics

This section provides roadmaps for upgrading Sysdig Platform components. Review the topics appropriate to your environment.

Topics	Description
Supported Upgrade Paths	Understand the upgrade and migration paths for on-prem installations.
Installer Upgrade (3.5.0-3.5.1)	Upgrading Sysdig Platform to v 3.5.0 from v. 3.0, 3.2.x using the installer tool. There is no manual install option as of version 3.5.0.
Installer Upgrade (2.5.0+)	Upgrading Sysdig Platform v2.5.0 - 3.2.2 by using the Installer tool. As of version 2.5.0, the Sysdig platform on Kubernetes or OpenShift should be upgraded using the Installer tool.
Manual Upgrade (3.0.0+)	Upgrading Sysdig Platform v3.0.0 and above manually on Kubernetes.
Upgrading v2.4.1- v2.5.0 on Kubernetes	Upgrading the Sysdig Platform versions between 2.4.1and 2.5.0 manually on Kubernetes.
Upgrading v2.3.0 on Kubernetes	Upgrading Sysdig Platform v2.3.0 manually on Kubernetes.
Upgrading v2435 on Kubernetes	Upgrading Sysdig Platform v2435 manually on Kubernetes.
Basic Upgrade on Replicated	Upgrading the mandatory components of the Sysdig Platform on a Replicated environment.

Supported Migration Paths

In general, Sysdig tests and supports direct upgrade from five releases back to the current version. Release-specific requirements are listed in the table below.

For Kubernetes Environments

Install Version	Incl. Hotfixes	Supported Upgrade From	Notes	Baseline Documentation
3.6.0 (by request)		3.2.2, 3.5.1	Platform changes: new inline scanner version, interactive session expiration. Sysdig Secure modules added/changed, including Compliance, Event Forwarding, Capture improvements, Image Scan results. Sysdig Monitor improvements in UI.	GitHub Readme
3.5.1 (by request)		3.0, 3.2.x, (3.5.0 if it was installed)	New/changed modules in both Sysdig Secure and Sysdig Monitor, including: New Getting Started and Overview, new Dashboards, overhauled Secure Events Feed, new navigation bar icons and layout, changed Image scanning navigation and usage, new Secure vulnerability feed and benchmark test,	Installer Upgrade (3.5.0-3.5.1) with oversite assistance from Sysdig Support
3.2.2		2.5.0, 3.2.0	Hot fix	Installer Upgrade (2.5.0+) Installer Upgrade (2.5.0+)
3.2.0	3.2.1, 3.2.2	2.5.0, 3.0.0	In Sysdig Secure: Data retention limits for scan results, vulnerabiity comparison in scan results, redesigned Captures page, RBAC capability, activity audit improvement. In Sysdig Monitor and Secure: S3-compatible storage for Capture files.	Installer Upgrade (2.5.0+)
3.0.0		2.4.1, 2.5.0	Beta Activity Audit feature, Beta Policy Advisor feature.	Installer Upgrade (2.5.0+)
2.5.0		2.3.0, 2.4.1	New Installer tool for upgrading; new documentation site; inline scanning for Secure, other enhancements	Installer Upgrade (2.5.0+)
2.4.1		2.3.0	Report service added; upgrade to Anchore license required	Upgrade v 2.4.1
2.3.0		1929, 2435	Ability to secure Elasticsearch and the Cassandra DB with password authentication or SSL/TLS protection.	Manual Upgrade (2.3.0)
2435	(2304) (2266) (2172)	1929, 1765	Architecture changes to Dashboards & API pods.	Manual Upgrade (v2435) Note that this replaces 2172, 2266, and 2304.
1929				legacy
1765			Migration Tool (MySQL Connector) Architecture & Port 443 change	legacy
1630	((1586)			legacy
1511	(1472) (1402)			legacy
1245				legacy
1149			Migration Tool (Unified Events)	legacy
1091				legacy

For Replicated Environments

Most Replicated environments can be upgraded directly to the current version. See also: Basic Upgrade (Replicated).

It is recommended to follow upgrade best practices:

Keep upgrades current.
Test upgrades in a non-mission-critical or staging environment before rolling into production.

2.5.3.1 -

Installer Upgrade (3.5.0-3.5.1)

As of version 3.5.0/3.5.1, Sysdig has changed its upgrade procedure.

For customers, the instructions in this section are for review purposes only.

Overview

With version 3.5.0, both installing and upgrading with the installer has been simplified from previous versions. Upgrade differs from Install in that you run an installer diff to discover the differences between the old and new versions and then installer deploy for the new version.

Some guidance from Sysdig Support may be warranted in highly customized installations.

Upgrade Steps

Upgrading is performed just like a fresh install, with the addition of the generate diff step. Refer to the appropriate workflow, depending on your environment:

Postgres Version Update v10.x to 12.x

Version 3.5.0 upgrade includes an automatic Postgres version upgrade. Depending on the size of your database, that can take some time.

The data migration takes approximately 1 min per 1 GiB of data. The speed of data migration ultimately depends on the underlying storage media.

To complete the Postgres upgrade, you must also ensure there is sufficient disk space in the volume when using a local-disk storage provisioner in Kubernetes. For example, if your current Postgres size is 100 GiB, ensure there is at least another 100 GiB space free in the volume. This is required temporarily for copying the data during the upgrade.

3.5.0 - 3.5.1 Elasticsearch Upgrade

Upgrading from version 3.5.0 to 3.5.1 also upgrades Elasticsearch. Due to the Elasticsearch update strategy of ondelete, the pods will only be upgraded when they are restarted:

image: quay.io/sysdig/elasticsearch:6.8.3.7
image: quay.io/sysdig/elasticsearch:6.8.3.9

2.5.3.2 -

Installer Upgrade (2.5.0+)

Overview

For customers, the instructions in this section are for review purposes only.

The Installer tool can be used to upgrade a Sysdig implementation. Just as in an installation, you must meet the prerequisites, download the values.yaml, edit the values as indicated, and run the installer. The main difference is that you run it twice: once to discover the differences between the old and new versions and the second time to deploy the new version.

As this is a new feature, some guidance from Sysdig Professional Services may be warranted in highly customized installations.

Supported Installer Versions

On-Prem Version	Installer Version
`3.0.0`	`3.0.0-7`
`3.2.0`	`3.2.0-9`
`3.2.2`	`3.2.2-1`

Upgrade Steps

To upgrade:

Download the latest installer binary that matches your OS from the sysdigcloud-kubernetes releases page.

Copy the current version of values.yaml to your working directory.

wget https://raw.githubusercontent.com/draios/sysdigcloud-kubernetes/installer/installer/values.yaml

Edit the following values:
- scripts: set to generate diff.
  This setting will generate the differences between the installed environment and the upgrade version. The changes will be displayed in your terminal.
- size: Specifies the size of the cluster. Size defines CPU, Memory, Disk, and Replicas. Valid options are: small, medium and large
- quaypullsecret: quay.io provided with your Sysdig purchase confirmation mail
- storageClassProvisioner: The name of the storage class provisioner to use when creating the configured storageClassName parameter. When installing, if you use AWS or GKE as your storage provisioner for Kubernetes, enter aws or gke in the storageClassProvisioner field. If you do not use one of those two dynamic storage provisioners, enter: hostPath and then refer to the Advanced examples for how to configure static storage provisioning using this option.
- sysdig.license: Sysdig license key provided with your Sysdig purchase confirmation mail
- sysdig.anchoreLicensePath: The path relative to the values.yaml where the Anchore enterprise license yaml is located. (For Sysdig Secure users only.)
- sysdig.dnsname: The domain name the Sysdig APIs will be served on. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- sysdig.collector.dnsName: (OpenShift installs only) Domain name the Sysdig collector will be served on. When not configured it defaults to whatever is configured for sysdig.dnsName. Note that the master node may not be used as the DNS name when using hostNetwork mode.
- deployment: (OpenShift installs only) Add deployment: openshift to the root of the values.yaml file.
- sysdig.ingressNetworking: The networking construct used to expose the Sysdig API and collector.Options are:
  - hostnetwork: sets the hostnetworking in the ingress daemonset and opens host ports for api and collector. This does not create a Kubernetes service.
  - loadbalancer: creates a service of type loadbalancer and expects that your Kubernetes cluster can provision a load balancer with your cloud provider.
  - nodeport: creates a service of type nodeport.The node ports can be customized with:
    sysdig.ingressNetworkingInsecureApiNodePort
    sysdig.ingressNetworkingApiNodePort
    sysdig.ingressNetworkingCollectorNodePort
  If doing an airgapped install , you would also edit the following values:
- airgapped_registry_name: The URL of the airgapped (internal) docker registry. This URL is used for installations where the Kubernetes cluster can not pull images directly from Quay
- airgapped_registry_password: The password for the configured airgapped_registry_username. Ignore this parameter if the registry does not require authentication.
- airgapped_registry_username: The username for the configured airgapped_registry_name. Ignore this parameter if the registry does not require authentication.

Run the installer.

For environments with access to the internet:

docker run -e HOST_USER=$(id -u) -e KUBECONFIG=/.kube/config
-v ~/.kube:/.kube:Z -v $(pwd):/manifests:Z
quay.io/sysdig/installer:<version>

For other supported installer versions, see Supported Installer Versions.

For partial-airgap (installation machine has access to the internet):

docker run -e HOST_USER=$(id -u) -e KUBECONFIG=/.kube/config
  -v ~/.kube:/.kube:Z
  -v $(pwd):/manifests:Z
  -v /var/run/docker.sock:/var/run/docker.sock:Z
  -v ~/.docker:/root/docker:Z
  quay.io/sysdig/installer:<version>

For other supported installer versions, see Supported Installer Versions.

For full airgapped environment:

Run steps 1-4 in the Full Airgap Install, then run:

bash sysdig_installer.tar.gz

If you are fine with the differences displayed, then set scripts to deploy and rerun the installer as in Step 3.
If you want to override a change, based on your environment’s custom settings, then contact Sysdig Support for assistance.
The datastores Cassandra and ElasticSearch have an onDelete update strategy and must be manually restarted to complete the upgrade.

2.5.3.3 -

Manual Upgrade (3.0.0+)

As of August 2020, Sysdig has changed its upgrade procedure.

For customers, the instructions in this section are for review purposes only.

Sysdig platform on-premise releases are listed here. Each release has a version number and specific release notes.

This release has the following significant changes:

Added NATS service to deliver events to the Sysdig backend
Added services for the beta Policy Advisor, which permits a user to auto-generate Pod Security Policies and perform dry tests or “simulations” of them before committing them to an environment.
Added services for activity audit, which allows users to view different data sources in-depth for monitoring, troubleshooting, diagnostics, or to meet regulatory controls
Some Anchore reporting components are not needed anymore and have been removed.

Download the New Version

Download the new version from Sysdig’s GitHub and unzip it.

wget https://github.com/draios/sysdigcloud-kubernetes/archive/<version_number>.tar.gz &&  tar xvf <version_number>.tar.gz

Edit New Files to Match Your Customized Files

It is important to use the latest YAML files for a successful upgrade.

Edit the following files within the sysdigcloud directory to match any customizations you may have made in your existing production system.

Please do not edit the image: property.

Sysdig Component Files

Ensure that any passwords or user names are transferred from your existing config.yaml to the new one. Suggested areas to review are listed below.

config.yaml:

The following variables are always customized in Sysdig installations:

api.url
collector.endpoint
sysdigcloud.license
mysql.password

Modifying following variables is optional but commonly done:

cassandra.jvm.options
elasticsearch.jvm.options
sysdigcloud.jvm.api.options
sysdigcloud.jvm.collector.options
sysdigcloud.jvm.worker.options

Check deployment YAML files for CPU/memory settings.
Update the spec.replicas definition in the following files:
- sysdigcloud/api-deployment.yaml
- sysdigcloud/collector-deployment.yaml
- sysdigcloud/worker-deployment.yaml
If running Sysdig Secure:
- sysdigcloud/anchore-core-config.yaml
- sysdigcloud/anchore-worker-config.yaml
- sysdigcloud/anchore-core-deployment.yaml
- sysdigcloud/anchore-worker-deployment.yaml
- sysdigcloud/scanning-api-deployment.yaml
- sysdigcloud/scanning-alertmgr-deployment.yaml

Postgres File (Sysdig Secure Only)

postgres-statefulset.yaml : Edit the storage class name in this file.
The file is located in datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulsets.yaml
Storage class name appears as spec.volumeClaimTemplates[].spec.storageClassName

Elasticsearch and Cassandra Files

elasticsearch-statefulset.yaml: For example, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:

spec.replicas and spec.template.spec.containers[elasticsearch].env[ELASTICSEARCH_GOSSIP_NODES_NUM].value
spec.template.spec.containers[].resources
spec.volumeClaimTemplates[].spec.resources.requests.storage
spec.volumeClaimTemplates[].spec.storageClassName

cassandra-statefulset.yaml: As with Elasticsearch, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:
```
spec.replicas
spec.template.spec.containers[].resources
spec.volumeClaimTemplates[].spec.resources.requests.storage
spec.volumeClaimTemplates[].spec.storageClassName
```

Apply the Files

The --force flag deletes the object and re-creates it whereas the --replace flag automatically creates an object if it doesn’t exist.

For the upgrade, assume NAMESPACE=sysdigcloud.

Install the NATS Components

In version 3.0, a NATS datastore was introduced for handling events inside the Sysdig platform:

kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/nats-streaming/nats-streaming-deployment.yaml
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/nats-streaming/nats-streaming-service.yaml

Upgrade Sysdig Monitor

Run the kubectl commands to apply the relevant files to your cluster.

kubectl -n $NAMESPACE apply -f sysdigcloud/config.yaml

kubectl -n $NAMESPACE replace --force -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml
kubectl -n $NAMESPACE replace --force -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml

Pause to allow Elasticsearch and Cassandra to come up. then continue:

kubectl -n $NAMESPACE apply -f sysdigcloud/api-deployment.yaml

Pause to allow api to come up, then continue:

kubectl -n $NAMESPACE apply -f sysdigcloud/collector-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/worker-deployment.yaml

Upgrade Sysdig Secure

Run the kubectl commands to apply the relevant files to your cluster.

kubectl -n $NAMESPACE replace --force -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-deployment.yaml

kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-alertmgr-deployment.yaml

Create secrets for the new policy advisor and activity audit components by deploying the policy-advisor-secret.yaml.

kubectl -n $NAMESPACE apply -f sysdigcloud/policy-advisor-secret.yaml

Deploy the components:

kubectl -n $NAMESPACE apply -f sysdigcloud/policy-advisor-service.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/activity-audit-api-service.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/activity-audit-api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/policy-advisor-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/activity-audit-worker-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/activity-audit-janitor-cronjob.yaml

You can delete the Anchore reporting components to free up system resources:

kubectl -n $NAMESPACE delete -f sysdigcloud/anchore-enterprise-license.yaml
kubectl -n $NAMESPACE delete -f sysdigcloud/anchore-reports-config.yaml
kubectl -n $NAMESPACE delete -f sysdigcloud/anchore-reports-deployment.yaml
kubectl -n $NAMESPACE delete -f sysdigcloud/anchore-reports-service.yaml

2.5.3.4 -

Manual Upgrade (2.4.1- 2.5.0)

As of August 2020, Sysdig has changed its upgrade procedure.

For customers, the instructions in this section are for review purposes only.

Sysdig platform on-premise releases are listed here. Each release has a version number and specific release notes.

This release has the following significant component change:

The Report service is now available for Sysdig Secure. Installing it requires first applying an Anchore license and then applying the appropriate report yamls, as listed below.

Download the New Version

Download the new version from Sysdig’s GitHub and unzip it.

Note that as of this release, versioning standards have changed from a single build number (e.g. v1929) to semantic versioning (e.g. 2.3.0)

wget https://github.com/draios/sysdigcloud-kubernetes/archive/<version_number>.tar.gz &&  tar xvf <version_number>.tar.gz

Edit New Files to Match Your Customized Files

It is important to use the latest YAML files for a successful upgrade.

Edit the following files within the sysdigcloud directory to match any customizations you may have made in your existing production system.

Sysdig Cloud Files

Customization involves copying the existing settings from your environment and modifying the files listed in this section.

Update the following files with customizations from your existing environment:

sysdigcloud/config.yaml: Pull configurations from your sysdigcloud-config configmap to the downloaded sysdigcloud/config.yaml.
The following variables are mandatory for Sysdig installations:
```
api.url
collector.endpoint
sysdigcloud.license
```
The following variables are optional but commonly modified for Sysdig installations:
```
cassandra.jvm.options
elasticsearch.jvm.options
sysdigcloud.jvm.api.options
sysdigcloud.jvm.collector.options
sysdigcloud.jvm.worker.options
```
If you have modified the previous config.yaml, copy the modified options such as the external endpoints. You must also check deployment YAML files for CPU/memory settings.
Copy configurations from your existing deployment and update the spec.replicas definition in the following files:
- sysdigcloud/api-deployment.yaml
- sysdigcloud/collector-deployment.yaml
- sysdigcloud/worker-deployment.yaml

If running Sysdig Secure:
Please do not edit the image: property.
- sysdigcloud/anchore-core-config.yaml
- sysdigcloud/anchore-worker-config.yaml
- sysdigcloud/anchore-core-deployment.yaml
- sysdigcloud/anchore-worker-deployment.yaml
- sysdigcloud/scanning-api-deployment.yaml
- sysdigcloud/scanning-alertmgr-deployment.yaml

Postgres File (if running Sysdig Secure)

Update the following file with customizations from your existing environment:

Please do not edit the image: property.

Modify the storage class name, spec.volumeClaimTemplates[].spec.storageClassName in the datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml file.

Elasticsearch and Cassandra Files

In version 2.3.0, Elasticsearch and Cassandra yaml configurations have been updated. Update the new files with customizations from your existing environment.

Please do not edit the image: property.

elasticsearch-statefulset.yaml - For example, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:

spec.replicas and spec.template.spec.containers[elasticsearch].env[ELASTICSEARCH_GOSSIP_NODES_NUM].value
spec.template.spec.containers[].resources
spec.volumeClaimTemplates[].spec.resources.requests.storage
spec.volumeClaimTemplates[].spec.storageClassName

cassandra-statefulset.yaml - As with Elasticsearch, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:
```
spec.replicas
spec.template.spec.containers[].resources
spec.volumeClaimTemplates[].spec.resources.requests.storage
spec.volumeClaimTemplates[].spec.storageClassName
```

Apply the Files

Run the kubectl commands to apply the relevant files to your cluster.

Upgrade for Sysdig Monitor

The --force flag deletes the object and re-creates it whereas the --replace flag automatically creates an object if it doesn’t exist.

NAMESPACE=sysdigcloud
kubectl -n $NAMESPACE apply -f sysdigcloud/config.yaml
kubectl -n $NAMESPACE replace --force -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml
kubectl -n $NAMESPACE replace --force -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/collector-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/worker-deployment.yaml

Upgrade for Sysdig Secure

For versions 2.4.1 and higher: To use the Reports functionality in Sysdig Secure, it is necessary to enter a license key in the anchore-license.yaml. If you are upgrading or installing and do not have an anchore license please contact support. This license is used for additional 3rd party vulnerability feed entitlements.

Edit the license YAML file: sysdigcloud/anchore-enterprise-license.yaml. Replace <LICENSE> with the key received from Sysdig.

---
apiVersion: v1
kind: Secret
metadata:
  name: anchore-enterprise-license
data:
  # <LICENSE> is derived from `cat anchore-license.yaml | base64`
  anchore-license.yaml: <LICENSE>
type: Opaque

Run the command:

kubectl -n sysdigcloud apply -f sysdigcloud/anchore-enterprise-license.yaml

Apply the Files

Run the following commands, preserving the order:

kubectl -n $NAMESPACE replace --force -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-alertmgr-service.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-alertmgr-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-enterprise-license.yaml #version 2.4.1 or higher
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-reports-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-reports-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-reports-service.yaml

2.5.3.5 -

Manual Upgrade (2.3.0)

As of August 2020, Sysdig has changed its upgrade procedure.

For customers, the instructions in this section are for review purposes only.

Sysdig platform on-premise releases are listed here. Each release has a version number and specific Release Notes.

This release has the following significant component changes:

Includes the option of securing Elasticsearch and/or Cassandra with username/password authentication and TLS-encrypted data in transit. This prevents both unauthorized access to the clusters and network eavesdropping. The upgrade instructions below incorporate this new capability when using the Sysdig-provided Cassandra and Elasticsearch components.
If you are running your own Cassandra or Elasticsearch clusters, you can skip the section Elasticsearch and Cassandra Files.
Updates of the Postgres database and Anchore engine (if you are running Sysdig Secure).
The following parameter has been renamed in config.yaml: elasticsearch.url to elasticsearch.hostname
The value of elasticsearch.hostname does not include the protocol (e.g.http://); just use the hostname itself. For example, if you had elasticsearch.url: ``http://sysdigcloud-elasticsearch, it would now be elasticsearch.hostname: sysdigcloud-elasticsearch.

Download the New Version

Download the new version from Sysdig’s GitHub and unzip it.

Note that as of this release, versioning standards have changed from a single build number (e.g. v1929) to semantic versioning (e.g. 2.3.0)

wget https://github.com/draios/sysdigcloud-kubernetes/archive/<version_number>.tar.gz &&  tar xvf <version_number>.tar.gz

Edit New Files to Match Your Customized Files

It is important to use the latest YAML files for a successful upgrade.

Edit the following files within the sysdigcloud directory to match any customizations you may have made in your existing production system.

Sysdig Cloud Files

Customization involves copying the existing settings from your environment and modifying the files listed in this section.

Update the following files with customizations from your existing environment:

sysdigcloud/config.yaml: Pull configurations from your sysdigcloud-config configmap to the downloaded sysdigcloud/config.yaml.
The following variables are mandatory for Sysdig installations:
```
api.url
collector.endpoint
sysdigcloud.license
```
The following variables are optional but commonly modified for Sysdig installations:
```
cassandra.jvm.options
elasticsearch.jvm.options
sysdigcloud.jvm.api.options
sysdigcloud.jvm.collector.options
sysdigcloud.jvm.worker.options
```
If you have modified the previous config.yaml, copy the modified options such as the external endpoints. You must also check deployment YAML files for CPU/memory settings.
Copy configurations from your existing deployment and update the spec.replicas definition in the following files:
- sysdigcloud/api-deployment.yaml
- sysdigcloud/collector-deployment.yaml
- sysdigcloud/worker-deployment.yaml

If running Sysdig Secure:
Please do not edit the image: property.
- sysdigcloud/anchore-core-config.yaml
- sysdigcloud/anchore-worker-config.yaml
- sysdigcloud/anchore-core-deployment.yaml
- sysdigcloud/anchore-worker-deployment.yaml
- sysdigcloud/scanning-api-deployment.yaml
- sysdigcloud/scanning-alertmgr-deployment.yaml

Postgres File (if running Sysdig Secure)

Update the following file with customizations from your existing environment:

Please do not edit the image: property.

Modify the storage class name, spec.volumeClaimTemplates[].spec.storageClassName in the datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml file.

Elasticsearch and Cassandra Files

In version 2.3.0, Elasticsearch and Cassandra yaml configurations have been updated. Update the new files with customizations from your existing environment.

Please do not edit the image: property.

elasticsearch-statefulset.yaml - For example, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:

spec.replicas and spec.template.spec.containers[elasticsearch].env[ELASTICSEARCH_GOSSIP_NODES_NUM].value
spec.template.spec.containers[].resources
spec.volumeClaimTemplates[].spec.resources.requests.storage
spec.volumeClaimTemplates[].spec.storageClassName

cassandra-statefulset.yaml - As with Elasticsearch, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:
```
spec.replicas
spec.template.spec.containers[].resources
spec.volumeClaimTemplates[].spec.resources.requests.storage
spec.volumeClaimTemplates[].spec.storageClassName
```

Apply the Files

Run the kubectl commands to apply the relevant files to your cluster.

Note: if you run into an error replacing the statefulsets, you may need to delete the existing one before applying the new configuration. See the Statefulset Deletion and Creation section below.

Upgrade for Sysdig Monitor

NAMESPACE=sysdigcloud
kubectl -n $NAMESPACE apply -f sysdigcloud/config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/collector-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/worker-deployment.yaml
kubectl -n $NAMESPACE replace -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml
kubectl -n $NAMESPACE replace -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml

Upgrade for Sysdig Secure

kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-alertmgr-service.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-alertmgr-deployment.yaml
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

Statefulset Deletion and Creation

If you are unable to update the existing statefulsets with the commands above, you may need to delete them before applying the new configuration.

# Elasticsearch
kubectl -n $NAMESPACE delete statefulset sysdigcloud-elasticsearch
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml

# Cassandra
kubectl -n $NAMESPACE delete statefulset sysdigcloud-cassandra
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml

# Postgres (if running Sysdig Secure)
kubectl -n $NAMESPACE delete statefulset sysdigcloud-postgresql
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

Replace Existing Statefulset Pods

The replace command above only replaces the Kubernetes configuration, but not the running pods themselves. For the changes to take effect, perform the following steps:

For Elasticsearch, run:

kubectl -n $NAMESPACE delete pod -l role=elasticsearch
kubectl -n $NAMESPACE delete pod -l role=cassandra
# If running Sysdig Secure
kubectl -n $NAMESPACE delete pod -l role=postgresql

Check that all the new pods come up Ready by running the commands below separately:

kubectl -n $NAMESPACE get pod -l role=elasticsearch
kubectl -n $NAMESPACE get pod -l role=cassandra
# If running Sysdig Secure
kubectl -n $NAMESPACE get pod -l role=postgresql

This may take a few minutes.

2.5.3.6 -

Manual Upgrade (v2435)

As of August 2020, Sysdig has changed its upgrade procedure.

For customers, the instructions in this section are for review purposes only.

Sysdig platform on-premise releases are listed here. Each release has a version number and specific Release Notes.

Sysdig On-Premise version 2435 replaces v2304, v2266, and v2172. Versions 2304 and 2266 are hotfix releases. Version 2172 is a major release.

Sysdig On-Premise version 2435 includes the following changes:

Dashboards upgraded from v1 to v2: This update happens automatically.
However, if you have saved v1 dashboards and need to reapply them, follow these instructions: Migrate Saved Dashboards from V1 to V2.
Architecture Change in the ContainersIn previous releases, there was a single backend container which ran several processes.
As of version 2435, the processes have been divided into unique containers, following container best practices.
As a result, it is necessary to apply the entire configuration, not simply change the image version. Follow the instructions below.

If you have licensed and will run only Sysdig Monitor, then you upgrade fewer components than if you also use Sysdig Secure, as described below.

Download the New Version

Use get to download the new version from Sysdig’s GitHub and unzip it.

For example:

wget https://github.com/draios/sysdigcloud-kubernetes/archive/<version_number>.tar.gz &&  tar xvf <version_number>.tar.gz

Edit New Files to Match Your Customized Files

Edit the following files within the sysdigcloud directory to match any customizations you may have made in your existing production system.

config.yaml

Edit the Sysdig user name, default user, API URL, Sysdig license, collector endpoint, from your config.yaml to the new config.yaml .

sysdigcloud.default.user: test@sysdig.com
collector.endpoint: onprem.sysdigcloud.com
collector.port: "6443"
api.url: https://onprem.sysdigcloud.com:443

deployment YAML files

Edit the CPU limits and replicas in the deployment YAML files: api-deployment.yaml, collector-deployment.yaml, worker-deployment.yaml

Note that the values in the sample below are examples only; edit them to match the requirements of your deployment.

spec:
 replicas: 1

....

resources:
  limits:
    cpu: "4"
    memory: 4Gi
  requests:
    cpu: "1"
    memory: 1G

Apply the Files

Run the kubctl commands to apply the relevant files to the environment.

This upgrade updates dashboards from v1 to v2. The process requires 20-30 minutes on large systems, and the environment remains live throughout the rolling upgrade.

DO NOT create or delete dashboards during the upgrade.

After upgrading, if you have saved v1 dashboards previously and need to upload them to the v2 environment, see Migrate Saved Dashboards from V1 to V2.

Upgrade for Sysdig Monitor Only

kubectl -n sysdigcloud apply -f config.yaml
kubectl -n sysdigcloud apply -f api-deployment.yaml
kubectl -n sysdigcloud apply -f collector-deployment.yaml
kubectl -n sysdigcloud apply -f worker-deployment.yaml

Upgrade for Sysdig Monitor + Sysdig Secure

kubectl -n sysdigcloud apply -f config.yaml
kubectl -n sysdigcloud apply -f api-deployment.yaml
kubectl -n sysdigcloud apply -f collector-deployment.yaml
kubectl -n sysdigcloud apply -f worker-deployment.yaml
kubectl -n sysdigcloud apply -f scanning-api-deployment.yaml
kubectl -n sysdigcloud apply -f scanning-alertmgr-deployment.yaml

2.5.3.7 -

Basic Upgrade (Replicated)

Support for new Replicated installations will be deprecated in the coming months. Feel free to contact Sysdig Support with questions.

Non-Airgapped Installation

Upgrading is very simple when your environment has access to the Internet during the installation process (non-airgapped).

It is highly recommended to follow upgrade best practices:

Keep upgrades current
Upgrade progressively without skipping versions, and
Test upgrades in a non-mission-critical or staging environment before rolling in to production.

Review the On-Premises Upgrades for supported upgrade paths.

Upgrade Replicated Components

Check Current Version

The Replicated infrastructure installs its own container based agents that deploy and manage the various Sysdig back-end components. To confirm the currently running version of the Replicated agent, perform replicated --version at the command line on each host. [Reference Replicated.com]

Upgrade Replicated Client

Log in to the Replicated Management Console and stop the Sysdig application (Sysdig Monitor and Sysdig Secure, if applicable) before upgrading the Replicated client.
Run the following command on the management host to upgrade the replicated infrastructure:
```
sudo curl -sSL https://get.replicated.com/docker | sudo bash
```

Run the following command on the remaining nodes in the cluster:

sudo curl -sSL https://get.replicated.com/operator | sudo bash

Upgrade Sysdig Application

Installation Sequence:

Pre-Version 860: Install upgrades sequentially, one version at a time.
Version 860 - 1091: You can directly install the version you want.*
Version 1091 - Sept 2018 release: All users must upgrade from 1091 - Sept 2018 and run the Unified Events migration tool.
Version 2266: See Note, below.

*Sequential installs (even when not strictly required) ensure consistent database migrations and allow for easier troubleshooting, should problems occur. Sysdig recommends staying fairly up-to-date on the release cycle to avoid “stacking up” upgrades.

Log in to the Replicated Management Console > Dashboards.
Click View Update.
The release history is listed, and “New” for any new releases.
Click Install for the desired release.

Upgrades to version 2304

After upgrading to version 2304, you must add a node for emailrenderer and nginxfrontend to the replicated cluster, and then run a command on the node.

Private and Public IP Addresses: Provide the IPs where the containers will run.
Select emailrenderer and nginxfrontend.
Run the curl command as noted in the image above, including the optional parameters as needed for your environment.

Airgapped Installation

Upgrade Replicated Components

Check Current Version

Upgrade Replicated Client

Download the latest Replicated agent installation package:

curl https://s3.amazonaws.com/replicated-airgap-work/replicated.tar.gz > replicated.tar.gz

In a command shell, extract the Replicated installer:
```
sudo tar xzvf replicated.tar.gz
```
Run the ‘install.sh’ script on the management host:
```
sudo cat ./install.sh | sudo bash -s airgap
```
Run the ‘operator_install.sh’ script on all remaining nodes:
```
sudo cat ./operator_install.sh | sudo bash -s airgap
```

Upgrade Sysdig Application

Download the new Sysdig application .airgap installer, using the link and password supplied for the initial installation.
Copy the .airgap file to the update directory on the management host.
To check or configure the update path, log in to the Replicated Management Console and click Console Settings > Airgapped Settings section under the gear icon.
In the Replicated Management Console, select the Dashboard tab and click Check Now.
Click Install for the desired version.

2.5.4 -

Find the Super Admin Credentials and API Token

Sysdig on-premises installations contain several configuration options only available to the initial admin user, or “super” admin user. This section outlines the steps for locating the super admin user (if you do not know who it is) and using the super user login token.

The Sysdig Monitor web interface does not currently provide a way to identify the super user. If you are trying to use the API to make a configuration change and it fails due to insufficient privileges, you can use the API to locate the super user.

Find Super Admin Credentials

Two approaches:

1. Access the API endpoint to list users directly via curl and parse the JSON output to locate the user with “ROLE_ADMIN” listed in the “roles” section.

# curl -k \
    -H 'Authorization: Bearer xxxxxxxx-yyyy-zzzz-aaaa-bbbbbbbbbbbb' \
    https://<your-sysdig-monitor-hostname>/api/users \
    | python -m json.tool

Output:

{
    "users": [
        {
        ....
            "roles": [
                "ROLE_ADMIN",
                "ROLE_CUSTOMER",
                "ROLE_USER"
             ],
             "username": "your-super-admin@example.com"
        },
        ...
    ]
}

2. Use this example Python script that leverages the Sysdig Monitor API.

export SDC_SSL_VERIFY="false"
export SDC_URL="https://<your-sysdig-monitor-hostname>"
# python list_admins.py xxxxxxxx-yyyy-zzzz-aaaa-bbbbbbbbbbbb

Output:

Admin users
-----------
your-super-admin@example.com
regular-admin@example.com

Super Admins
------------
your-super-admin@example.com

Find Sysdig API Token

As with any user, you can then obtain the API token by logging in as the “super” admin to the Sysdig UI.

When using the Sysdig API with custom scripts or applications, an API security token (specific to each team) must be supplied.

Log in to Sysdig Monitor or Sysdig Secure and select Settings.
Select User Profile.The Sysdig Monitor or Sysdig Secure API token is displayed (depending on which interface and team you logged in to).
You can Copy the token for use, or click the Reset Token button to generate a new one.
When reset, the previous token issued will immediately become invalid and you will need to make appropriate changes to your programs or scripts.

2.5.5 -

Configure Interactive Session Expiration

(For On-Premises installations): When you want inactive sessions to deactivate after a time-out period, you need to set four interlinked configuration parameters with the installer. Two of these parameters handle the session expiration in the backend and two of them control the frontend tracker that handles the session expiration when the user’s browser is idle. To achieve session expiration for a specific period of time (for example, 30 minutes), these parameters should be aligned to the same value.

The parameters, with sample settings, are:

sysdig
 inactivitySettings:
    trackerEnabled: true
    trackerTimeout: 1800
  api:
    jvmOptions: -Ddraios.security.rememberMe.tokenValiditySeconds=1800 -Ddraios.security.session.timeoutMinutes=30

Parameter	Description	Values
`sysdig.inactivitySettings.trackerEnabled`	Must be set to enable frontend activity tracker in general, boolean	`false` by default
`sysdig.inactivitySettings.trackerTimeout`	Timeout in seconds before the inactive interactive session expires, valid only if `sysdig.inactivitySettings.trackerEnabled` is set to `true`	1800 seconds by default
`draios.security.rememberMe.tokenValiditySeconds`	Must match the `trackerTimeout` value	1800 if `trackerTimeout` default is used
`draios.security.session.timeoutMinutes`	Convert `validitySeconds` to minutes	30 if `trackerTimeout` default is used

The jvmOptions parameters handle the backend session expiration, while the sysdig.inactivitySettings.trackerEnabled and

sysdig.inactivitySettings.trackerTimeout handle the frontend activity tracker.

2.5.6 -

Upgrade an On-Premises License

On-premises environments may require a license upgrade to renew, extend an expiration date, enable new features, add a service (Sysdig Secure), or change the number of licensed agents.

For Kubernetes On-Prem Installations

As described in the Kubernetes installation instructions, the license file is simply entered as one of many configuration user settings in the ConfigMap (config.yaml) (manual install) or values.yaml (installer-based).

The relevant parameter is:

 # Required: Sysdig Cloud license
  sysdigcloud.license: ""

To apply the new license, update the yaml file with the new license and then restart all Sysdig API, Worker, and Collector pods.

For Replicated On-Prem Installations

Upgrade an Airgapped License

If you are running an airgapped environment, you must download the license file to a local directory, then follow the steps below.

Log in to the Replicated Management Console and choose Console Settings from the gear icon drop-down menu.
Review the current airgapped settings and note the pathname to the license file.
If you followed the Airgapped Installation instructions, your current .rli license file will be in the/var/tmp/sysdigdirectory as shown below.
(The name of your .rli file will vary.)
From the Linux shell, cd to the directory shown as the Update Path and replace your prior RLI license file with the new one, saving the prior one to a backup filename.
NOTE: the new license file may have a different name than the prior one, and may have a non-RLI extension if it was sent to you as an email attachment (to avoid being removed by firewalls).
The steps below are an example of renaming the license as necessary for the environment shown above (your filenames will vary).
```
/var/tmp/sysdig# mv my-replicated-license.rli prior-replicated-license-1.rli
/var/tmp/sysdig# mv newer-license.allow my-replicated-license.rli
```
Continue in the Replicated Management Console to sync the license and restart the Sysdig application, as described in the non-airgapped instructions below.

Upgrade a Non-Airgapped License

If your environment can access the Internet, upgrading the license is a simple sync and restart.

Log in to the Replicated Management Console.
Select View License from the Gear drop-down menu.
Click Sync License.
For license terms to take effect, restart the Sysdig application from the Replicated Management Console.
Navigate to theDashboard on the pull-down menu and click theStop Nowbutton, followed by theStart Nowbutton.

2.5.7 -

Authentication and Authorization (On-Prem Options)

This section is for the Sysdig On-Premises software platform only. If you are using SaaS (cloud-based) Sysdig applications, see Authentication and Authorization (SaaS) instead.

Sysdig Monitor and Sysdig Secure are designed to work with several user authentication/authorization methods:

Type	Enabled by Default	Integration Steps Required
User email/password	Yes	No
Google OAuth	No	Yes
SAML	No	Yes
OpenID Connect	No	Yes
LDAP	No	Yes

A sample user’s view:

In the SaaS environment, Googlelogin can be enabled with a simple drop-down selection; the integration has already been performed.

To enable a third-party authentication method for both Sysdig Monitor and Sysdig Secure, you must configure the SSO settings separately for each.

Workflow

With the new Authorization UI, the basic process of enabling a Single Sign-On (SSO) option is:

Determine which SSO option (GoogleOAuth, SAML, OpenID, LDAP) your enterprise uses, and which IdP service (Okta, OneLogin, etc.) is used if any.
Configure any associated IdP settings on the IdP side.
Enter the required connection settings for the chosen SSO on the appropriate Authentication tab in Sysdig Settings.
You can also configure the settings using a script, if preferred.
Select the SSO option from the Enabled Single Sign-On drop-down and click Save Authentication.
If enabling for both Sysdig Monitor and Sysdig Secure, perform the necessary steps on the second application.

View of the Authentication page for Google OAuth in the on-prem environment.

2.5.7.1 -

Google OAuth (On-Prem)

These instructions are specific to On-Premises Deployments of the Sysdig platform. If you are using the cloud-based (SaaS) Sysdig platform, refer to Google OAuth (SaaS) instead.

Google supports OAuth 2.0, which allows users to log in to third-party applications such as Sysdig using Google credentials. By default, the created user will not have Admin rights within the Sysdig application, though these rights can be subsequently assigned.

Prerequisites

The Sysdig platform on-premises installation must have a DNS name associated with it. Google does not support applications that do not have an associated DNS name.

DNS Name

Replicated

For Replicated-based installations, enter your Hostname in the Settings tab.

Kubernetes

For Kubernetes-based installations, ensure the api.url ConfigMap element contains your hostname (older installations), or use the sysdig.dnsname (newer Installer-based).

For the examples that follow, DNS_NAME refers to this hostname you configured in your platform settings.

In Google Console: Obtain OAuth Client Credentials

Log in to the Google API Console.
Create your project.
Select Credentials from the left-hand navigation, and choose the OAuth consent screen from the navigation bar.
When prompted, select Internal or External User Type and click Create.
Choosing Internal will limit the users to those with accounts belonging to the same domain as the email used to create the project, e.g. mycompany.com. Note that if some of your users have a different domain, e.g. mycompany.uk, you will want to choose the External user type.
On the subsequent Oauth Consent screen, enter the required Email address and Product name, as well as other additional optional information, then click Save.
From the Credentials tab, click the Create Credentials drop-down and select OAuth client ID.
When prompted for Application type, select Web application, then enter the following parameters:
- Name: Use a meaningful name, such as “Sysdig”.
- Authorized Javascript Origins: Enter https://DNS_NAME:API_PORT
- Authorized Redirect URLs: Enter one or more of the following values:
  If configuring Sysdig Monitor, enter: https://DNS_NAME:API_PORT/api/oauth/google/auth
  If configuring Sysdig Secure, enter: https://DNS_NAME:API_PORT/api/oauth/google/secureAuth
Click Create.
A success message with client ID and client secret will be displayed. Copy these to a safe place, as you will need them in the next step.

Configure Settings in Sysdig Platform

There are three options for configuring OAuth settings on the Sysdig side: a UI page, scripts, or entries in your Replicated or Kubernetes orchestrator.

Option 1 UI-Based: Configure Google OAuth in Settings

To enable baseline Google Oauth functionality:

Enter Google OAuth Basic Settings

Log in to Sysdig Monitor or Sysdig Secure as “super” Admin and select Settings.
Select Authentication.
Select the Google OAuth tab.
Enter the relevant parameters and click Save.
Application ID: the Client ID you were sent.
Application Secret: the Client Secret you were sent
URL Redirect:
If configuring Sysdig Monitor, enter: https://DNS_NAME:API_PORT/api/oauth/google/auth
If configuring Sysdig Secure, enter: https://DNS_NAME:API_PORT/api/oauth/google/secureAuth Allowed Domains: Comma-separated list of domains permitted to log in. For example, mycompany.com, myxompanyalias.com.

Select Google OAuth for SSO

Select Google Oauth from the Enabled Single Sign-On dropdown
Click Save Authentication.
Repeat for Sysdig Monitor or Sysdig Secure, if you want to enable on both applications.

Option 2 Script-Based: Configure OAuth Using Scripts

The configuration of the Google OAuth feature can be viewed, updated, and deleted by the “super” Admin. A google_oauth_config.sh helper script is available in the SSO folder at sysdig-cloud-scripts repository to assist in completing this configuration. Invoking the script with no options will display help text.

# ./google_oauth_config.sh -h
Usage: ./google_oauth_config.sh [OPTIONS]

Affect Google Oauth login settings for your Sysdig software platform installation

To use the helper script, modify env.sh to set the required values for API_TOKEN of the “super” Admin user and the URL for accessing the Sysdig platform API (which will be the same URL that your users access for the Sysdig Monitor application).

Depending if the API_TOKEN has been obtained from the Sysdig Monitor or Sysdig Secure application UI, the settings will be applied to the consequent product.

Initially no Google Oauth settings are set. A initial run of the script would confirm that:

# ./google_oauth_config.sh
No google-oauth settings are set
Run for further info: ./google_oauth_config.sh -h

Add the -s option to set the Google Oauth configuration for a particular Sysdig application. When setting the config, you’ll use additional options to provide the config details you saved in the earlier Google Oauth step.

Config Detail	Option
Client ID	`-i`
Client Secret	`-e`
Allowed Domains	`-a`
Redirect URL	`-r`

If the configuration is successfully posted to the Sysdig platform, the new configuration will be echoed back.

Depending if the API_TOKEN has been obtained from the Sysdig Monitor or Sysdig Secure application UI, the settings will be applied to the relevant product.

# ./google_oauth_config.sh -s -i "t2em0alq7l13n1hevua48ehieenkb06q.apps.googleusercontent.com" -e "ucP_WY908-k" -r "https://sysdigtest.com:443/api/oauth/google/auth" -a "[\"sysdig.com\"]"
{
  "authenticationSettings": {
    "id": 1,
    "version": 1,
    "createdOn": 1547709552000,
    "type": "google-oauth",
    "scope": "SYSTEM",
    "settings": {
      "clientId": "t2em0alq7l13n1hevua48ehieenkb06q.apps.googleusercontent.com",
      "clientSecret": "ucP_WY908-k",
      "redirectUrl": "https://sysdigtest.com:443/api/oauth/google/auth",
      "allowedDomains": [
        "sysdig.com"
      ]
    }
  }
}

Once you’ve completed this configuration, clicking the Google Login button at the login screen of the appropriate Sysdig application(s) should redirect to Google Oauth login page.

If you wish to delete your Google Oauth configuration, invoke the -d option. If successful, the disabled configuration will be printed.

# ./google_oauth_config.sh -d
{
  "authenticationSettings": {
    "id": 1,
    "version": 1,
    "createdOn": 1547709552000,
    "type": "google-oauth",
    "scope": "SYSTEM",
    "settings": {
      "clientId": "t2em0alq7l13n1hevua48ehieenkb06q.apps.googleusercontent.com",
      "clientSecret": "ucP_WY908-k",
      "redirectUrl": "https://sysdigtest.com:443/api/oauth/google/auth",
      "allowedDomains": [
        "sysdig.com"
      ]
    }
  }
}

Option 3 Orchestrator-Based: Enter Settings Using Orchestrator

Replicated

If you used the Replicated infrastructure manager to install the Sysdig platform:

Log in to the Replicated Management Console, click to the Settings tab, then check the box to expand theAdvanced Settings.
Enter the Google OAuth client ID and Google OAuth client Secret in the appropriate fields.
(Optional) In a comma-separated list, enter the OAuth-allowed email domains that should be permitted to authenticate. If set, only Google users whose email addresses are in these domains will be able to login to your Sysdig installation. If this setting is left blank, any user that successfully authenticates via Google will be permitted to login to your Sysdig installation.
Click Save.The Sysdig platform will then restart to enable the settings.

Kubernetes

Enter the OAuth allowed domains, Client ID, and Client Secret into the appropriate elements of the Kubernetes ConfigMap. Use appropriate Kubernetes methods to push the updated settings and restart the backend containers to make the changes take effect.


`# Optional: OAuth allowed domains (comma separated list of domains)` `sysdigcloud.oauth.allowed.domains.list:` `""` `# Optional: Sysdig Cloud Google OAuth Client ID` `sysdigcloud.google.oauth.client.id:` `""` `# Optional: Sysdig Cloud Google OAuth Client Secret` `sysdigcloud.google.oauth.client.secret:` `""`

User Experience

Note the following requirements for successful Google OAuth login:

The user must have already logged in successfully at least once to your environment (such as via email-based Invitation and having set an initial password)
The user’s login username in the Sysdig platform must precisely match the user’s Google email address (that is, it cannot be a shortened/altered Google email alias)

For such a user to log in via Google OAuth, click the Log in with Google button.

See also User and Team Administration for information on creating users.

2.5.7.2 -

SAML (On-Prem)

These instructions are specific to On-Premises Deployments of the Sysdig platform. If you are using the cloud-based (SaaS) Sysdig platform, refer to SAML (SaaS) instead.

SAML support in the Sysdig platform allows authentication via your choice of Identity Provider (IdP).

This section describes how to integrate and enable SAML with both Sysdig Monitor and Sysdig Secure.

For specific IdP integration information, refer to:

Basic Enablement Workflow

Step	Options	Notes
1. Know which IdP your company uses and will be configuring.	Okta (SAML On-Prem) OneLogin (SAML On-Prem) Azure Active Directory (SAML On-Prem) ADFS (SAML On-Prem)	These are the IdPs for which Sysdig has performed detailed interoperability testing and confirmed how to integrate using their standard docs. If your IDP is not listed, it may still work with the Sysdig platform. Contact Sysdig Support for help.
2. Decide the login flow you want users to experience (choose from three options):	Click SAML button	From `https://HOSTNAME/` or `https://HOSTNAME/secure/`
	Type/bookmark a URL in browser	Monitor:`https://HOSTNAME/api/saml` Secure: `https://HOSTNAME/api/saml?product=SDS`
	Log in from an IdP interface	The individual IdP integration pages describe how to add Sysdig to the IdP interface. You will need your Sysdig customer number on hand. Normally 1 for on-premises.
3. Perform the configuration steps in your IdP interface and collect the resulting config attributes.	Okta (SAML On-Prem) OneLogin (SAML On-Prem) Azure Active Directory (SAML On-Prem) ADFS (SAML On-Prem)	Collect metadata URL (or XML) and test it. If you intend to configure IDP-initiated login flow, have your Sysdig customer number on hand. It will be referenced in later configuration steps as `CUSTOMER_ID_NUMBER`. Normally 1.
4 a. Log in to Sysdig Monitor (as "super" admin) and enter the necessary configuration information in the UI. Enable SAML as your SSO. 4b. Log in to Sysdig Secure (as "super" admin) and repeat the above.

Administrator Steps

Configure IdP

Select the appropriate IdP from the list below, and follow the instructions:

UI-Based: Configure SAML in Settings

At this time, the Authorization UI is available only for Sysdig Monitor.

To enable baseline SAML functionality:

Enter SAML Connection Settings

Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.
Select Authentication.
Select the SAML tab.
Enter the relevant parameters (see table below) and click Save.
It is strongly recommended that “Signed Assertion” and “Validate Signature” are enabled to ensure that the SAML SSO process is as secure as possible.

Connection Setting	Options	Description	Sample Entry
Metadata	URL	The URL provided at the end of the IdP configuration steps.
	XML	An option that can be used for an IdP that doesn’t support extracting metadata XML via URL.
Signed Assertion	off/on	Should Sysdig check for assertions signed in responses (to assist in validating correct IdP).	ON
Email Parameter	email	Name of parameter in the SAML response for user email ID. Sysdig uses this to extract the user’s email from the response.	email
Validate Signature	off/on	Sysdig backend should verify that the response is signed.	ON
Verify Destination	off/on	Flag to control whether Sysdig should check the “destination” field in the SAMLResponse. Recommend ON, as a security measure. May be OFF in special cases, such as a proxy in front of the Sysdig back end.	ON

Select SAML for SSO

Select SAML from the Enabled Single Sign-On dropdown
Click Save Authentication.
Repeat entire enablement process for Sysdig Monitor or Sysdig Secure, if you want to enable on both applications.

Script-Based: Configure SAML Using Scripts

The configuration of the SAML feature can be viewed, updated, and deleted by the “super” Admin. A saml_config.sh helper script is available in the SSO folder at sysdig-cloud-scripts repository to assist in completing this configuration. Invoking the script with no options will display help text.

# ./saml_config.sh
Must specify the Sysdig App whose SAML configuration will be viewed/set

Usage: ./saml_config.sh [OPTIONS]

Affect SAML login settings for your Sysdig software platform installation

If no OPTIONS are specified, only the help output is displayed.

Depending if the API_TOKEN has been obtained from the Sysdig Monitor or Sysdig Secure application UI, the settings will be applied to the relevant product.

Initially no SAML settings are set. A initial run of the script would confirm that:

# ./saml_config.sh
No saml settings are set
Run for further info: ./saml_config.sh -h

Add the -s option to set the SAML configuration for a particular Sysdig application. When setting the config, you’ll also include the metadata URL you saved in the earlier IDP configuration step (-m option) and specify the name of a supported IDP configuration (-i option), which will tailor other details of your SAML configuration to the specifics of that IDP. If the configuration is successfully posted to the Sysdig platform, the new configuration will be echoed back.

An example of creating the two separate SAML configurations for both Monitor and Secure, each using Okta IDP settings:

# ./saml_config.sh -s -m 'https://dev-824158.oktapreview.com/app/exkfpgqiskSoGZrjE0h7/sso/saml/metadata' -i okta
{
  "authenticationSettings": {
    "id": 1,
    "version": 1,
    "createdOn": 1547539750000,
    "type": "saml",
    "scope": "SYSTEM",
    "settings": {
      "metadataUrl": "https://dev-824158.oktapreview.com/app/exkfpgqiskSoGZrjE0h7/sso/saml/metadata",
      "metadata": null,
      "validateSignature": true,
      "emailParameter": "email",
      "signedAssertion": true,
      "verifyDestination": true,
      "createUserOnLogin": true
    }
  }
}

If you are using an IDP other than those available with the -i option, contact Sysdig Support for assistance with determining the correct settings.

Once you’ve completed this configuration, clicking the SAML button at the login screen of the appropriate Sysdig application(s) should redirect to your IDP for authentication.

If you wish to delete your SAML configuration, invoke the -d option. If successful, the disabled configuration will be printed.

# ./saml_config.sh -a monitor -d
{
  "authenticationSettings": {
    "id": 1,
    "version": 1,
    "createdOn": 1547539750000,
    "type": "saml",
    "scope": "SYSTEM",
    "settings": {
      "metadataUrl": "https://dev-824158.oktapreview.com/app/exkfpgqiskSoGZrjE0h7/sso/saml/metadata",
      "metadata": null,
      "validateSignature": true,
      "emailParameter": "email",
      "signedAssertion": true,
      "verifyDestination": true,
      "createUserOnLogin": true
    }
  }
}

Configure SAML Single Logout

Sysdig supports SAML Single Logout (SLO).

SLO Process

Caveats

SLO is currently supported only in US-West and EU-Central regions.
Sysdig does not support HTTP Post binding for single logout, and therefore, SLO with Okta is not functional at this point.

Configure IdP

Configure logout URLs:
- Monitor: <base_URL>/api/saml/slo/logout
- Secure: <base_URL>/api/saml/slo/secureLogout
Choose HTTP Redirect as the binding method.
This option is an alternative to the HTTP POST method, which Sysdig does not support currently.
If your IdP mandates, upload the public key for Sysdig.
Contact Sysdig Support to retrieve the public key associated with your deployment.
Certain IDPs, such as Azure, don’t require uploading the public key.

Configure Sysdig

Log in to Sysdig Monitor or Sysdig Secure as an administrator and select Settings.
For on-prem deployments, log in as the super admin.
Navigate to Settings > Authentication, and select SAML under Connection Settings.
Enter the SAML configuration.
Ensure that Enable SAML single logout is toggled on.
Click Save.
Ensure that you select SAML from the Enable Single Sign On drop-down.

Optional: Auto-creation of user records

When a user successfully authenticates via SAML, if a user record does not yet exist in the Sysdig platform database for their email address, one will be created at that time (default behavior). Some environments may not like this approach and may instead only want to permit logins for users whose records already exist (such as may have been already created via email invite or creation via the API).

To disable the auto-creation of user records after SAML authentication, add the -n option to your command line when applying your settings. This will set createUserOnLogin to false .

# ./saml_config.sh -s -n -m 'https://dev-824158.oktapreview.com/app/exkfpgqiskSoGZrjE0h7/sso/saml/metadata' -i okta
{
  "authenticationSettings": {
    "id": 2,
    "version": 1,
    "createdOn": 1547539856000,
    "type": "saml",
    "scope": "SYSTEM",
    "settings": {
      "metadataUrl": "https://dev-824158.oktapreview.com/app/exkfpgqiskSoGZrjE0h7/sso/saml/metadata",
      "metadata": null,
      "validateSignature": true,
      "emailParameter": "email",
      "signedAssertion": true,
      "verifyDestination": true,
      "createUserOnLogin": false
    }
  }
}

User Experience

As noted in the Basic Workflow, above, you can offer users three ways to log in with a SAML configuration:

They can begin at the Sysdig SaaS URL and click the SAML button.
In the US East, Monitor: https://HOSTNAME/ or Secure: https://HOSTNAME/secure/.
See SaaS Regions and IP Ranges for URLs for other regions.
They will be prompted to enter a Company Name, so the Sysdig platform can redirect the browser to your IdP for authentication.
You can provide an alternative URL to avoid the user having to enter a company name, in the format:
Sysdig Monitor: https://<HOSTNAME>/api/saml
Sysdig Secure: https://<HOSTNAME>/api/saml/secureAuth
This format is for the US East region. See SaaS Regions and IP Ranges for URLs for other regions.
You can configure an IdP-initiated login flow when configuring your IdP. The users then select the Sysdig application from your IDP’s app directory and do not browse directly to a Sysdig application URL at all.

Environments that wish to have all users access Secure by default could use this example script to frequently “sync” the team memberships.

See also User and Team Administration for information on creating users.

Caveats

SAML Assertion Encryption/Decryption is not currently supported.
SAML Single Logout is not supported. Therefore, users should take care to log out directly from Sysdig application(s).

2.5.7.2.1 -

Okta (SAML On-Prem)

Review SAML (On-Prem) before you begin.

Sysdig-Specific Steps for Okta Configuration

IDP-Initiated Login Flow

If you don’t intend to configure IDP-initiated login flow, check the boxes for “Do not display application icon to users” and “Do not display application icon in the Okta Mobile app”.

SSO, URI, and RelayState Values

Enter the values shown in the table below, replacing HOSTNAME with the hostname through which your users access the Sysdig application(s) and PORT with the TCP port # (typically 443).

To configure IDP-initiated login flow, replace CUSTOMER-ID-NUMBER with the number retrieved as described in Find Your Customer Number. (Normally the Customer ID will be 1 in on-prem installations.)

Setting

Value for Sysdig Monitor

Value for Sysdig Secure

Single sign on URL

https://HOSTNAME:PORT/api/saml/auth

https://HOSTNAME:PORT/api/saml/secureAuth

Audience URI (SP Entity ID)

https://HOSTNAME:PORT/api/saml/metadata

Default RelayState

(optional - only configure if you intend to use IDP-initiated login flow)

#/&customer=CUSTOMER-ID-NUMBER

You must include the port number in the IDP-side configuration, even though port 443 is the typical default for https:// URLs.

Email and Name Values

Instead of those shown in the Okta example, add these values:

Name	Value
`email`	`user.email`
`first name`	`user.firstName`
`last name`	`user.lastName`

Note that the attributes are case sensitive, so use caution when entering them.

URL Metadata Value

Copy the URL and paste in the Metadata entry on the SAML Configuration page in the SAML connection settings.

Test Metadata (Optional)

To ensure the metadata URL you copy at the end of the IDP configuration procedure is correct, you can test it by directly accessing it via your browser.

<?xml version= "1.0" ?> <EntityDescriptor xmlns= "urn:oasis:names:tc:SAML:2.0:metadata" entityID= "https://app.onelogin.com/saml/metadata/680358" >

`<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">`

...

2.5.7.2.2 -

OneLogin (SAML On-Prem)

Review SAML (On-Prem) before you begin.

Sysdig-Specific Steps for OneLogin Configuration

Adding the SAML Test Connector

Test Connector Configuration Page Settings

Field	Value for Sysdig Monitor	Value for Sysdig Secure
RelayState (optional - only configure if you intend to use IDP-initiated login flow)	`#/&customer=CUSTOMER-ID-NUMBER`	`#/&customer=CUSTOMER-ID-NUMBER`
Recipient	`https://HOSTNAME:PORT/api/saml/auth`	`https://HOSTNAME:PORT/api/saml/secureAuth`
ACS (Consumer) URL Validator	`https://HOSTNAME:PORT`	`https://HOSTNAME:PORT`
ACS (Consumer) URL	`https://HOSTNAME:PORT/api/saml/auth`	`https://HOSTNAME:PORT/api/saml/secureAuth`

You must include the port number in the IDP-side configuration, even though port 443 is the typical default for https:// URLs.

Field Name	Value
`first name`	`First Name`
`last name`	`Last Name`

Note that the Field Names are case sensitive , so be careful to enter them as all lowercase.

The following shows an example of a correctly-configured field for First Name:

Issuer URL

Click to the SSO tab, copy the Issuer URL, and paste in the Metadata entry on the SAML Configuration page in the SAML connection settings.

Test Metadata (Optional)

To ensure the metadata URL you copy at the end of the IDP configuration procedure is correct, you can test it by directly accessing it via your browser.

<?xml version= "1.0" ?> <EntityDescriptor xmlns= "urn:oasis:names:tc:SAML:2.0:metadata" entityID= "https://app.onelogin.com/saml/metadata/680358" >

`<IDPSSODescriptor xmlns:ds=` `"http://www.w3.org/2000/09/xmldsig#" ` `protocolSupportEnumeration=` `"urn:oasis:names:tc:SAML:2.0:protocol"` `>names:tc:SAML:` `2.0` `:metadata` `" entityID="` ` https://app.onelogin.com/saml/metadata/ ` `680358` `">`

...

2.5.7.2.3 -

Azure Active Directory (SAML On-Prem)

This topic explains how to configure SAML Single Sign On (SSO) with Azure Active Directory (AD) and helps you configure Sysdig to allow users to access Sysdig application by using SSO.

Prerequisites

Administrator privileges on Sysdig and Azure.

Configure the Sysdig Application in Azure AD

Log in to the Azure AD portal.
Select Azure Active Directory, then click Enterprise Applications.
The Enterprise applications - All application screen is displayed.
Click New Application.
On the Add an Application screen, select Non-gallery application.
Give your application a name, and click Add at the bottom of the page.
On the menu, select Single sign-on.
Choose SAML as the sign-on method.
Edit the Basic SAML Configuration as follows:
1. In the configuration page, click the edit icon.
2. Specify the following:
  - Identifier (Entity ID): Uniquely identifies the Sysdig application. Azure AD sends the identifier to the Sysdig application as the audience parameter of the SAML token. Sysdig validates this as part of the SSO process.
    For example, the identifier for Sysdig Monitor for the EU region is https://eu1.app.sysdig.com.
    See SaaS Regions and IP Ranges for the complete list of entity IDs for different regions.
  - Reply URL: Specifies where Sysdig expects to receive the SAML token.
    For example, the identifier for Sysdig Monitor for the EU region is https://eu1.app.sysdig.com/api/saml/auth.
  - See SaaS Regions and IP Ranges for the complete list of reply URLs for different regions.
  - Relay State: Specifies to the application where to redirect the user after authentication is completed. Typically the value is a valid URL for Sysdig. If you are configuring SSO for SaaS, change the relay state to reflect the correct customer number associated with your Sysdig application. For on-prem installations, the customer number is always 1.
    The format is:
```
#/&customer=1234
```
  - Sign on URL: It is the sign-in page for the Sysdig application that will perform the service provider-initiated SSO. Leave it blank if you want to perform identity-provider-initiated SSO.
  For more information on configuration parameters, see Configure SAML-based single sign-on to non-gallery applications.

Sysdig-Specific Steps for Active Directory Configuration

Under SAML Signing Certificate, copy the App Federation Metadata URL.
Log in to your Sysdig instance as an admin.
For on-prem deployments, log in as the super admin.
Navigate to Settings > Authentication, and select SAML under Connection Settings.
Enter the following:
- Metadata: Enter the App Federation Metadata URL you copied.
- Email Parameter: Set the value to emailaddress.
  Azure AD claims are:
```
saml = AD
givenname = user.givenname
surname = user.surname
emailaddress = user.mail
name = user.userprincipalname
Unique User Identifier = user.userprincipalname
```
  In the Sysdig application, you need to set the email to emailaddress which is what Azure AD sends to Sysdig in the SAML assertion. Alternatively, Azure AD can be modified to send another attribute.
Click Save.
Select SAML from the Enable Single Sign On drop-down.

Create a User in Azure Active Directory Domain

Log in to the Azure AD portal.
Click Azure Active Directory, and note down the domain name.
Select Azure Active Directory, then Users.
The Users - All Users screen is displayed.
Select New Users .
You can either create a new user or invite an existing AD.
Enter name, username, and other details, then click Create.
In the Profile page, add the Email and Alternate Email parameters. The values can match

Assign the User to the Sysdig Application

Navigate to the Sysdig application.
Click Users and Group, then click the Add user button.
Select the Users and Groups checkbox, then choose the newly created user to add to the application.
Click Select, then Assign at the bottom of the screen.

Enable Authentication Settings in the Sysdig Instance

Ensure that Flag to enable/disable create user on login is enabled. Typically this setting is enabled by default.

if you are on Sysdig Platform versions 2.4.1 or prior, contact Sysdig Support to help with user creation.