This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Release Notes

Sysdig Monitor & Sysdig Secure Release Notes

Sysdig components are released on varying schedules. The links below point to the ever-updated changes.

It is recommended to follow upgrade best practices:

  • Keep upgrades current

  • Upgrade progressively without skipping versions, and

  • Test upgrades in a non-mission-critical or staging environment before rolling into production.

Installed Components Changelog

Sysdig Agent Release Notes

Serverless Agent Release Notes

Sysdig SaaS Changelog

SaaS: Sysdig Monitor Release Notes

SaaS: Sysdig Secure Release Notes

Sysdig On-Prem Changelog

Sysdig On-Premises Release Notes

Sysdig On-Premises Release Support

Falco Rules Changelog

Falco Rules Changelog

RSS Feed for Sysdig Documentation

Subscribe to the RSS feed for Sysdig Release Notes to stay updated with the latest product releases. Use your favorite news aggregation apps, such as RSS Feed Reader, to get notified immediately when we post new content.

Add the desired RSS feed URL to your reader and it should notify you whenever a new release note is available.

1 - Sysdig Agent Release Notes

12.8.1 August 29, 2022

Defect Fixes

Fix Vulnerabilities in Promscrape V1

Upgraded the Prometheus version and resolved vulnerabilities in promscrape v1.

Read information on users and groups from /host/etc/passwd and /host/etc/group when agent is running as a container.

Show Falco Events as Expected

Fixed a problem when the Falco output string for a rule is cut on the first ascent or empty field.

12.8.0 August 02, 2022

Feature Enhancements

Add a New Metric to Indicate Retrieving Kubernetes State

Added an internal metric, statsd_dragent_subproc_cointerface_ready to indicate when the agent has pulled Kubernetes state from the API server.

Read Certificate Chain

Previously, the agent would only accept the first certificate in a cert chain and would attempt to verify all other certificates from the configured certificate store. This behavior is compliant with the TLS specification, but idiomatic usage in the wild requires the agent to accept intermediate certificates provided in the handshake as well. The agent will now accept these certificates if provided.

Support for dup() Syscalls

The support for the dup() family of syscalls has been enhanced and additionally, support for dup2 and dup3 is now available.

Falco Rules Optimizer

Falco Rules Optimizer can now be optionally enabled. This feature increases the speed of syscalls evaluation against Falco rules by introducing indexing on the rules conditions and by caching partial rule condition evaluations. This feature is only available in Sysdig agent, but not in open-source Falco. The feature is enabled by setting falco_optimizer.enabled to true (default value is false).

New Falco Rules Parser

Starting from version 12.8.0, Sysdig agent uses a new Falco rules parser from OSS Falco. The new OSS Falco parser performs stricter grammar parsing and would fail on the following cases:

  • when \n is used instead of , in a list
  • when "[" is present in a rule definition
  • when \034 surrounded by " is present in a rule definition
  • when or operation between lists is used instead of, with in operator. For example: condition: open_write and fd.filename is (list1 or list2)

If any of the above cases are present in custom rules file, the agent fails to parse the respective rule and outputs the following error:

Error, security_mgr:791: Could not load policies_v2 message:.

In this case, the custom rules have to be edited to correct or remove unparsable rules

Defect Fixes

Process Kubernetes Audit Events as Expected

Agent no longer throw errors while processing Kubernetes audit events when Kubernetes audit rules contain the endswith condition.

Upgrade Go Language Packages

Go language packages have been upgraded to fix vulnerabilities

Fix Vulnerabilities

Fixed the following vulnerabilities with Promscrape V2:

  • CVE-2015-3627
  • CVE-2021-3121
  • CVE-2020-14040
  • CVE-2014-6407
  • CVE-2014-9356
  • CVE-2014-9357
  • CVE-2022-23648
  • CVE-2022-27191
  • CVE-2021-41103
  • CVE-2020-15257
  • CVE-2014-9358
  • CVE-2021-21334
  • CVE-2020-13401
  • CVE-2014-5277
  • CVE-2020-13401
  • CVE-2020-8565
  • CVE-2021-32760
  • CVE-2021-20329
  • CVE-2019-11254
  • CVE-2021-4189
  • CVE-2020-8565
  • CVE-2021-4189
  • CVE-2021-3737
  • CVE-2021-3634
  • CVE-2021-3634
  • CVE-2021-3737
  • CVE-2022-1996

Detect Prometheus Targets Correctly

Fixed a problem that was causing new prometheus targets to not be detected until an agent restart.

Intermittent Scraping Failure No Longer Causes Missing Metrics

Fixed an issue with missing metrics when there are intermittent metrics scraping failures.

Show Falco Events as Expected

Sysdig agent now throttles redundant secure events for compliance policies reducing the event noise.

Show Username Correctly in Policy Events

Fixed an agent build issue that caused password and group functions unvailable and therefore, linked the password and group from `/host/etc’ inside the agent container to have the username correctly shown in policy events.

Fix a Logging Issue in Promscrape V2

Fixed a logging issue with Promscrape v2. Log levels are taking effect as expected when passed in with --log.level.

Agents No Longer Incorrectly Behave as Delegated

Fixed an issue that might cause all the agents to behave as delegated.

12.7.1 July 06, 2022

Defect Fixes

Fixed memdump.size Issue

Fixed the memdump.size configuration, which was not being honored.

Fixed Promscrape Crash Issue

Fixed a crash issue in Promscrape v2 when a node has a large number of pods and multiple containers per pod.

Fixed Issue Affecting Two Agent Modes

Fixed a problem that can cause agent subprocesses to be killed in nodriver mode. This affects the custom-metrics-only and monitor_light modes. For more information, see Configure Agent Modes.

12.7.0 June 28, 2022

Feature Enhancements

New Helm Chart

Sysdig released a unified helm chart, sysdig-deploy with the following benefits:

  • Easier to deploy multiple components with one chart, rather than using multiple separate charts
  • Fewer errors by way of using common configuration for components
  • Auto-detection of certain configurations, including eBPF for GKE COS and endpoint region.

We will maintain the old version of helm chart, sysdig chart for a period of six months. In this period, the sysdig chart will be updated with new component versions and and defect fixes.

Live Logs

Sysdig Monitor displays Live Logs in Advisor to allow for troubleshooting Kubernetes, which is the equivalent of running kubectl logs. Live logs are displayed on-demand and not stored by Sysdig.

Support Prometheus v2.32

Updated Prometheus scraper to version 2.32.

Metrics Collected in Custom Metrics Only Mode

When custom-metrics-only mode is used, no process metrics are collected. Additionally, only the metrics related to resources (CPU, memory) are collected for containers and host.

Known Issues

While the agent is running, you might encounter an error similar to the following:

Error, security_rule:610: Could not parse rule xx from rules json array.

The rule number in the error message might change depending on how many rules are defined.

This is a known issue related to failing to parse an experimental rule. The parser will skip this rule and will log the error message as above. The agent performance and policy evaluation will not be affected.

Defect Fixes

Remove Ceph App Checks

Fixed a problem where errors for obsoleted app checks would be shown when Ceph was running on the host.

Disable Timeseries Caching

Removed a configuration option which caused Prometheus jobs to not report timeseries if the scrape failed temporarily.

Builds eBPF Probes in Bottlerocket

Fixed an issue that prevented ebpf probes from having built by the agent in Bottlerocket Environments.

Reports Infrastructure State Correctly

Fixed an issue where Sysdig agent would opens a stream to Cointerface even when it is disabled. This resolves the issue of infrastructure state having reset costantly.

Sends Only Supported Metrics in Nodriver Mode

Fixed an issue where unused container and process metrics were sent while in nodriver mode.

Change Log Level to DEBUG When Excessive Log Level Occurs

The excessive logging level occurs under specific conditions, for example, a pod whose used memory results in zero. This case seems to be normal for small pods using very little memory. Fix has been provided so that, when these conditions are detected, the log level for the message that is polluting the logs is brought from INFO to DEBUG.

Reports Container Resource Limits and Requests Correctly

Fixed an issue where container resource limits and requests would appear as zero when no limit or request was configured.

12.6.0 May 16, 2022

Defect Fixes

Reloading Promscrape V2 No Longer Causes Dropping Scrape Targets

Reloading promscrape v2 no longer causes dropping some scrape targets from sending metrics.

Losing Node No Longer Generates Duplicate Node Events

Resolved an issue that caused generating duplicate events when a Kubernetes node is lost.

Agents Connect to SaaS Backend Through HTTP Proxy on Older Hosts

Fixed an issue related to SSL certificate verification when connecting through an HTTP proxy on older host OS, such as CentOS 7.

Agent Refreshes Service Account Token as Expected

Connection with the Kubernetes API Server works as expected. The Kubernetes client is configured to refresh the bearer token.

12.5.0 May 02, 2022

Feature Enhancements

Default Availability of Slim Agent

The agent installation is defaulted to the slim agent. Slim agent reduces the surface area for potential vulnerabilities as compared to the full agent, which implies increased security for your monitoring environment. For more information, see Agent Installation.

To continue using the regular agent, set slim.enabled to false in your helm chart.

Monitoring Kubernetes Resources

Sysdig agent v12.5.0 and above no longer collect the HPA kube state metrics by default. To enable the agent to collect HPA kube state metrics, you must edit the agent configuration file, dragent.yaml, and include it along with the other resources you would like to collect. For more information, see Enable Kube State Metrics.

Container DriftControl: Detect and Prevent Drift in Container Runtime

Sysdig agent can now detect when a new executable was added to a container after a container has started up. The agent collects when a file was downloaded and made executable. When using prevention mode, the agent can also deny the process from ever running. A policy can also be used to define binaries that should be denied/excluded from being denied if they have been added after the container has started.

See also: Drift Policy

Disable Syscalls for Secure Modes

Switch syscall events are disabled for secure and secure light modes.

Known issues

  • An error message is displayed when the agent detects ceph and attempts to run an obsoleted app check.
  • The Sysdig agent for ARM can restart when multiple containers are started in rapid succession on the host.

Defect Fixes

Agent on zLinux No Longer Restarts Due to Incorrect Detection of tid Colliions

The agent on s390x architecture (zLinux) has been fixed so the agent does not restart needlessly due to incorrect detection of too many tid collisions.

Reports Correct CronJob Version When Adding CronJob Parents

Fixed an issue causing CronJobs to be reported not as the parents of Job objects.

Agent No Longer Crashes During Abnormal Termination

Fixed an issue causing the agent to crash with a stack backtrace during certain abnormal termination situations.

Slow-Starting JVMs Are Terminated Correctly

An incorrect detection of too many tid collisions on s390x architecture (zLinux) will no Longer cause the agent to restart periodically.

Kubernetes Events Are Collected as Expected

Fixed an issue that could prevent Kubernetes events from being correctly fetched.

Disable Watching HorizontalPodAutoscaler

Watching Horizontal Pod Autoscalers has been disabled by default to decrease load on Kubernetes API server. For more information, see Enable Kube State Metrics.

False Positive CVEs for Go Packages No Longer Reported

The Go compiler version has been upgraded to prevent getting flagged with (false-positive) CVEs associated with older Go versions.

Secure Events Reports Correct Cluster Information

The Secure events no longer report Kubernetes cluster name default when no cluster exists in the environment.

12.4.0 April 04, 2022

Feature Enhancements

Support for New Architectures

Installing agent on the following architecture are supported:

  • ARM (aarch64)

    aarch64 environments support AWS Graviton

  • s390x (zLinux)

For more information, see Host Requirements for Agent Installation.

ARM support includes AWS EC2 Graviton platform

Custom-Metrics-Only Mode

A new agent mode, custom-metrics-only, has been introduced. It enables all custom metrics and Kubernetes state metrics but disables all the driver-based metrics.

Prevent Processing Policy Updates

Prevent processing policy update messages to reduce CPU usage when no changes are required in the agent.

Known Issues

Increased Resource Consumption due to Misconfiguration of Node Lease

Incorrect configuration of Kubernetes lease can result in elevated memory usage in the Sysdig agent pods as well as increased load on the Kubernetes API server due to multiple agents querying for more information simultaneously. This also results in a significant amount of additional and unnecessary load on the Sysdig backend. To resolve this issue,

  • Upgrade to Sysdig agent 12.5.0 which adapts to the non-optimal Kubernetes configuration.
  • Configure the Kubernetes lease functionality. If you are using Helm, the latest versions of the Sysdig Agent Helm chart defaults to configuring the lease functionality automatically. If you do not use Helm, the DaemonSet and ClusterRole YAML files are available in our gitbub repository. For further assistance, contact Sysdig Support.

Agent Restarts Periodically on zLinux

An incorrect detection of too many tid collisions on s390x architecture (zLinux) can cause the agent to restart periodically. To workaround this issue, set the following configuration option:

watchdog:
  analyzer_tid_collision_check_interval_s: 86400

This configuration change reduces the number of restarts to once a day instead of every 10 minutes, which is the default value for the above configuration option.

This issue has been fixed in Sysdig agent v12.5.0.

Defect Fixes

Validate Promscrape Scrape Jobs

Validate scrape jobs associated with promscrape integration before scraping the endpoints to avoid unnecessary errors with irrelevant scrape jobs.

Remove App Check Warning Messages When App Checks Are Disabled

Remove unnecessary warning messages about app checks limits when app checks are disabled.

Slow-Starting JVMs Are No Longer Terminated

Slow starting JVMs can be terminated by sdjagent. For example, -XX:+AlwaysPreTouch with large heaps. This fix introduces additional configuration options to tune the delay between sdjagent detecting a started JVM process and an attempt to connect.

jmx:
  monitor_connect_timeout_ms: 5000
  management_agent_connect_delay_ms: 0

EVE Connector Works as Expected in Kubernetes

Fixed metadata incompatibility in profiling with Kubernetes versions above 1.20.

Name Change to Configuration Parameter

The falcobasline.max_drops_buffer_rate_percentage parameter has been corrected to falcobaseline.max_drops_buffer_rate_percentage. Notice the missing e in falcobasline in falcobasline.max_drops_buffer_rate_percentage. However, the backward compatibility is ensured, and therefore, falcobasline.max_drops_buffer_rate_percentage can still be used.

12.3.1 March 03, 2022

Defect Fixes

Noisy Messages Silenced

Removed a kernel message from the driver that could generate spam when the syscall event buffer is full.

12.3.0 February 17, 2022

Feature Enhancements

Binaries Category for Falco Baseline

A new category, binaries is added to the Falco baselines feature.

Support for Workload Information in Falco Baseline

Add workload information to Kubernetes context for Falco baselines.

Default Monitoring of Kubernetes Resources

The following kubernetes resources are monitored by default:

- persistentvolumeclaims
- persistentvolumes
- storageclasses
- horizontalpodautoscalers

Known Issues

IPv6 Addresses Are Saved Incorrectly When Adding Rules

Adding a new rule causes problem saving IPv6 address for both fd.net and fd.ip.

Defect Fixes

Fix Truncated Capture Files

Fixed a problem which caused the agent to generate truncated capture files.

Container Action Pause Work on Kops/GKE Clusters

Fixed the logic that determines the cgroup path for a container in containerd and made the freezer subsystem available to the agent in order to be able to pause/unpause it.

Agent Profiling Works as Expected

High CPU load no longer prevents generating CPU and memory profiles in the agent.

Agents Are Not Reset with Signal 11

Large and negative file descriptors are handled correctly so agents are no longer reset with signal 11.

12.2.1 February 07, 2022

Feature Enhancements

Manage Collecting Metadata from Individual Container Engines

Access to individual container engines from within the agent for fetching metadata can now be disabled via agent configuration. For example, to disable docker, use the following configuration:

container_engines:
  docker: false

Known Issues

The Pause policy action is not working as expected in GKE, EKS, and Openshift4 environments.

Defect Fixes

Policy Action “Kill” Is Correctly Triggered in GKE Environments

Policy action on GKE with containerd works as expected:

  • The container is stopped if HTTP proxy is enabled.
  • The status of the container is checked upon stop requests. If the status is not CONTAINER_EXITING, termination of the container is attempted with exponential backoff.

Agents Assign Username Correctly for Container Events

Fixed an issue that prevented the proc.name field from extracting the right user from the container started events. This issue was found in agent versions 12.2.0 and above.

12.2.0 January 25, 2022

Feature Enhancements

Improve Install Script to Support eBPF

A new option, bpf or -b is added to the native install script of Sysdig agent to support eBPF.

Enable 10s Flush by Default

By default, the agent collects metrics at 1-second granularity, then aggregates and sends them to the backend in 10-seconds intervals. If you want to use agent versions 12.2.0 or above with the on-prem Sysdig Platform versions below 3.5.0, set the 10s_flush_enable configuration to false to prevent compatibility issues.

The backend in our SaaS deployments continues to enable 10-second flush automatically for all agent versions 10.0.0 or above.

Improved Log Messages

Improved the log messages to report the errors encountered while configuring subprocess_resource_limits.

Known Issues

Processing Secure policy updates in the agent can take longer than it did in the previous releases, and in some rare scenarios, it causes agent restarts.

Defect Fixes

Fix CVE-2020-29652 in Cointerface

Updated crypto go module to fix CVE-2020-29652.

Promscrape V2 No Longer Crashes on Pods with Multiple Containers

Prevent promscrape_v2 from crashing when a pod has multiple containers.

skip_events_by_type Works as Expected

Fixed an issue in the kernel probe, which prevented the skip_events_by_type feature from correctly filtering events by system call type.

Kubernetes State Is Transmitted as Expected

Fixed an issue where Kubernetes information and metrics would not be sent from the agent. This scenario arose when the agent was deployed in a namespace other than sysdig-agent, and the agent daemonset did not include the podinfo volume.

Agent Successfully Connects to JMX

Fixed an issue where agent wouldn’t connect to JMX on some applications/JVMs. This issue was originally observed on the WebSphere application and Liberty JVM.

Agent Updates Container Status as Expected

Fixed an issue where the agent would not update the container status it first received from the API server. The agent now updates the container statuses as it receives them from the API server.

Check for Invalid Log Level in sdjagent

Fixed an issue where using a log level of none caused sdjagent to crash.

App Checks Run as Expected on Non-Containerized Agent Installations

Fixed an issue preventing app checks to run on non-containerized agent installation.

Native Install Doesn’t Support eBPF

Prevents insertion of Sysdig probe kernel module when the agent is installed with eBPF by using rpm or deb package.

Prevents Connection Attempts When Agent Encounters Errors

Connection attempts are prevented when the agent encounters errors while handling handshake messages.

12.1.1 November 22, 2021

Defect Fixes

Falco Action Works as Expected

The kill container Falco action works as expected for containerd in Azure.

12.1.0 November 08, 2021

Feature Enhancements

Ability to Build eBPF Probes for Debian 11 Kernels

The agent container has been enhanced to build probes for Debian 11 kernels.

Prebuilt Probes for Debian 11 Kernels

Prebuilt probes are added for Debian 11 kernels.

Prebuilt Probes for Fedora Kernels

Prebuilt probes are added for latest Fedora kernels.

Ability to Build eBPF Probes for Linux Kernel v5.10

The agent container can now build eBPF probes for Linux kernel version 5.10.

Enhanced Agent Containers for Probes on New Kernels with glibc v2.33

The agent container has been enhanced to build probes for new kernel versions that use glibc v2.33.

File Metrics in Audit Tap

Metrics related to file are included in audit tap.

Promscrape Memory Usage Limit

You can now limit the promscrape memory usage. The default is set to 640 MB. For more information, see Sysdig Agent.

Remove Self-Signed Certificate for Agent to Collector Connection

Self-signed certificate support has been removed for agent connection to the collector. See End of Support.

Defect Fixes

Image Profile Shows Results Correctly

The imageid is reported correctly when using a CRI engine.

Duplicate Environment Variable Hashes No Longer Appear in Audit Tap

The discrepancy between reported environment variables and hash in audit tap has been fixed.

Kubernetes Daemonset and Replicaset Association Works as Expected

Fixed an issue that could invalidate the association between Kubernetes Daemonset and Replicaset.

Agent Updates Prometheus Configurations Correctly

Fixed a problem that was causing Prometheus configurations to be merged incorrectly when certain integrations were updated from the backend.

12.0.4 October 29, 2021

Defect Fixes

Secure Policies Load as Expected

Fixed an issue present in 12.0.3 where Secure policies might not be loaded correctly by the agent.

12.0.3 October 22, 2021

Defect Fixes

Leases Fallback Works as Expected on OpenShift v3

Fixed an issue where Kubernetes clusters that don’t support leases failed to report Kubernetes data due to not falling back to the previous behavior.

Update the Cluster Install Scripts for Leases on OpenShift

Modified the OpenShift agent installer to add the sysdig-agent cluster role and to assign it to the sysdig-agent service account. The new cluster role allows the agent to utilize the coldstart leases.

12.0.2 September 30, 2021

Defect Fixes

Network Security Communication Works As Expected

In some environments Sysdig agents could not send any Network Security (Kubernetes Network Policies) communications upon not completing CIDR auto-discovery. This issue has been fixed.

Agent No Longer Crashes in Orchestrated Environments

Fixed a problem related to a race condition in orchestrated environments, such as OpenShift v3, due to which the agent might crash repeatedly at the agent start.

12.0.1 September 27, 2021

Defect Fixes

OpenShift 4 Clusters Able To Retrieve Metadata Without Leases

Fixed an issue where OpenShift clusters would fail to report Kubernetes data when the agent service-account did not have the permission to create leases. With this fix, the Sysdig agent falls back to the previous behavior to retrieve the metadata.

12.0.0 September 15, 2021

Feature Enhancements

Allow Sysdig Backend to Manage Prometheus Configuration

Allow Sysdig backend to manage Prometheus configuration. For more information, see the following:

Agent Console Supports Troubleshooting Prometheus Configuration

The Agent Console now supports troubleshooting Prometheus configuration.

To support this feature, Agent Console is enabled by default. This helps both users and Sysdig support to troubleshoot Sysdig agent issues. Sensitive user configuration is obfuscated and not viewable.

For more information, see Using the Agent Console.

Support for Node Leases

Sysdig agent supports using Kubernetes Lease to control how and when connections are made to the Kubernetes API Server.

For more information, see the following:

Support for Podman Environments

Sysdig agent is supported in Podman environments. For more information, see Prerequisites for Podman Environments.

Add Startup Delay to Agent to Kubernetes API Server Connection

Added a delay prior to the agent connecting to the Kubernetes API server. The delay time is set based on the number of nodes in the cluster to prevent overloading the API server. This is to support environments where node leases cannot be used.

Known Issues

None

Defect Fixes

Stale Capture Files No Longer Exhaust Local File System

Prevent incomplete and stale capture files from being left behind and thereby avoiding storage consumption for such files.

Honor CPU Quotas

Moved the main dragent process to the default cgroup so that CPU quotas can cover all the agent processes.

Containers Are Detected as Expected

Fixed issue where containers are not detected if SystemdCgroup = true is not enabled in the containerd configuration.

Report Correct Container Metadata

Fixed a problem that caused some container metadata such as the image repository and image tag to be reported incorrectly.

Upgrading from 10.8.0 to 11.3.0 No Longer Fails

Provide a http_proxy configuration option to address connection problems post-OpenSSL upgrade from v11.0 to v11.1.

11.4.1 August 03, 2021

This is a hotfix release.

Defect Fixes

Fixed a problem that broke app checks in agent-slim by adding the missing dependencies.

11.4.0 July 28, 2021

Feature Enhancements

Probe Builder

The probe builder can now be used to build kernel modules for the Sysdig agent. It can run on any host with Docker installed, including (with some preparation) air-gapped hosts.

Probe Builder is now enabled and available at https://github.com/draios/probe-builder. See the Readme for more information.

Promscrape v2

Promscrape v2 (used when prom_service_discovery is enabled for Prometheus) has been changed to discover only Kubernetes pods running on the same node as the agent. This should help reduce the load on the Kubernetes API servers in large clusters.

Added Missing Fields for Unified Workload Metrics

Added Kubernetes metric fields indicating the availability of daemon sets (status.numberAvailable, status.numberUnavailable, and status.updatedNumberScheduled) and replica sets (status.availableReplicas) to support workload-level metrics (SaaS only).

Known Issues

App checks in agent-slim don’t work due to missing dependencies. This problem will be addressed in an upcoming hotfix release.

Defect Fixes

Multiple Hosts No Longer Report the Same Pod

Fixed an issue causing multiple hosts to report the same pod if its UUID is the same on both hosts.

Duplicate StasD Metrics Are Reported Correctly

Fixed an issue related to handling duplicate StatsD metrics corresponding to a container that is reported by a host.

Stale Markers Are Sent properly for Dropped Targets

Properly generate stale markers for Prometheus metrics when a scrape target is no longer available and when using promscrape.v1.

Report a Positive Time Delta Value

Fixed a defect that could result in an invalid file.time.in, file.time.out, file.time.other, and file.time.total values.

Agent No Longer Crashes When App Check or Prometheus Is Enabled

Fixed a defect that could cause crashing the agent when app checks or Prometheus is enabled.

Secure Captures No Longer Causes Host Shutdown

Prevent agent restarts caused by apparent stalls encountered in the sample handler thread.

11.3.0 June 10, 2021

Feature Enhancements

Console Logging

Introduced per-component-level console logging feature. See Manage Console Logging for Agent Components.

Slim Agent for eBPF Probes

agent-kmodule and agent-kmodule-thin can now be used to build eBPF probes.

Replication Controller Fields

Added missing replication controller fields to the aggregator Actions.

Non-Delegated Agents Retrieve Less Data From the API Server

Use Kubernetes leases to better control the load on the Kubernetes API Server. This is disabled by default.

Defect Fixes

Agent No Longer Generates Core Dumps on Java

Prevents java process core dumps caused by the Sysdig agent while trying to access /tmp directory.

Support Container Action on Containerd

Container actions are now properly supported on containerd (CRI-O and other CRI engines that already had support). Actions for unsupported container engines are now properly reported to the Sysdig backend and a warning message is logged in the agent logs.

Recovery During Agent Shutdown

Introduced a detection and recovery mechanism for hangs during agent shutdown.

Promscrape V2 Termination No Longer Causes Agent Crash

Fixed a problem causing the agent to crash after promscrape_v2 is terminated.

Agent No Longer Restarts in Kubernetes Environment

The agent tries to fetch the metadata of the AWS instance in which it is running in order to tag metrics generated with the information unique to the AWS instance. If the metadata structure is not as expected, the agent continuously restarts due to an error in fetching such metadata. This issue has been fixed.

Profiling Works as Expected

Fixed an issue that disabled support for performance profiles in the agent.

11.2.1 May 06, 2021

This is a hotfix release.

Defect Fixes

Report Container User Information

Start tracking container user information and make that information accessible in container events. These events denote having a container started. This feature works for Docker as well as CRI-O container engines.

Reporting container user information does not work in OpenShift 4.x because it does not provide necessary CRI-O information.

11.2.0 April 26, 2021

Feature Enhancements

Agent CLI

Sysdig supports Agent CLI, a command-line interactive tool, to troubleshoot agents. This tool helps Sysdig support to solve user issues quickly and efficiently. It is currently disabled by default and requires the customer to turn it on.

For more information, see Using the Agent Console

Scraping Prometheus Metrics

Scraping Prometheus metrics is supported in the following cases:

  • Advertised ports on container IP addresses

  • Advertised ports on host IP addresses

  • Advertised ports on pod IP addresses

Slim Agent for IKS

Use the following:

Reduce Load on Kubernetes API Server

Terminated pods are no longer collected in order to reduce the load on the Kubernetes API server.

Audit Server Listens on All Interfaces

The audit server now by default listens on all the interfaces for Kubernetes audit events. This makes integration with Kubernetes audit events in the agent easier without the need for configuration changes.

Improved Noise-Reduction Filter for Activity Audits

The noise-reduction filter for Activity Audit has been improved. All the filtered data is duplicated.

Defect Fixes

CRI-O Versions Report Correct Image ID

The new CRI-O versions (1.19+, possibly 1.18) now properly report container.image.id.

Log Level Changes for Duplicate Host Container Groups

Demoted logs about duplicate host container_groups from warning to debug level

Fix CVE-2021-28831

Fix CVE-2021-28831 in the Slim Agent container.

11.1.3 April 13, 2021

This is a hotfix release.

Defect Fixes

Prevent Agent CrashLoopBackoff Error Caused by Smaller initialDelaySeconds Values

The readiness probe improvement in version 11.1.2 delayed the transition of the agent pod to a ready state until communication with the Kubernetes API server was established. But this delay could cause a CrashLoopBackoff due to liveness or readiness probes configured with an initialDelaySeconds set to less than 90.

In Agent version 11.1.3 the transition to the ready state does not wait for communication with the Kubernetes API server to be established unless the behavior is enabled via a new configuration option: k8s_wait_before_ready.

11.1.2 March 30, 2021

Known Issues

Prevent Agent CrashLoopBackoff Error Caused by Smaller initialDelaySeconds Values

The readiness probe improvement in version 11.1.2 delayed the transition of the agent pod to a ready state until communication with the Kubernetes API server was established. But this delay could cause a CrashLoopBackoff due to liveness or readiness probes configured with an initialDelaySeconds set to less than 90.

Workaround

If you are using agent version 11.1.2, set initialDelaySeconds for both liveness and readiness probes to a value that is greater than or equal to 90.

Feature Enhancements

Enhanced Connection with Kubernetes API Server

Kubernetes reconnect logic has been improved to automatically backoff (1 min, 2 min, 4 min… 1hr) if the connection is continuously dropped when using Thin Cointerface. This reduces the load that the agent imposes on the Kubernetes API Server in clusters with heavily burdened API servers.

Reduced Load on Kubernetes API Server

The agent’s readiness probe has been improved to not report ready until after the agent connects to the Kubernetes API server. This reduces the load that the agent imposes on the Kubernetes API server when starting up during RollingUpdate.

11.1.1 March 26, 2021

Defect Fixes

Agent Reports Memory Usage Accurately for Containers

Fixed an issue where the agent would incorrectly report memory.bytes.used for containers that use more than 4GB.

Runtime Policies Work as Expected

The runtime policies that have a policy type and capture action are handled as expected.

11.1.0 March 23, 2021

Defect Fixes

Agent Tags in Policy Scopes

Agent tags are supported in runtime policy scopes.

Metric Limits Are Updated As Expected

Fixed a problem where metric limits were not updated from the defaults. This is unlikely to happen if agents are connected to the SaaS backend.

Configured Tags in Prometheus Scraper

Fixed a problem in the old Prometheus scraper (used when promscrape is disabled) to ensure that configured tags are properly added to the metrics.

JMX Metrics for Short-Lived Java Processes

Fixed an issue where short-lived Java processes could cause the Sysdig Agent to stop collecting JMX metrics.

Misconfiguration No Longer Leads to Agent Constantly Querying Kubernetes API Server

Fixed a problem where the agent would continuously send requests to the Kubernetes API server to query the endpoints API. This occurs when the agent’s clusterrole is incorrectly configured. With this fix, the agent will no longer repeat the attempt if it is unable to connect to the Kubernetes API during boot.

Scope Runtime Policies

The runtime policies are now correctly scoped by kubernetes.cluster.name. The fix in 10.6.0 was incomplete.

Agent Correctly Reports Replicasets

Fixed an issue where the agent could lose track of a replicaset and report incomplete metadata.

Agent Issues Over HTTP Proxy

  • Fixed an agent connection issue over plaintext HTTP proxy with encryption.

  • Fixed an agent connection issue via HTTP proxy connections over SSL.

11.0.0 February 18, 2021

Feature Enhancements

Thin Cointerface to Reduce Memory Usage

Thin cointerface reduces the memory required to handle the Kubernetes metadata on both the agent and the Kubernetes API Server. The reduction in memory usage is significant for Kubernetes clusters with a large number of pods (in the range of 10,000 or more) or clusters that heavily use Replication Controllers.

Using this feature returns the same data to the Sysdig backend and does not affect any Sysdig features. The thin cointerface feature is disabled by default.

To enable:

  1. Add the following in either the sysdig-agent’s configmap or via the dragent.yaml file:

    thin_cointerface_enabled: true
    
  2. Restart the agent.

See also: Reduce Memory Consumption in Agent.

Reduce the Volume of Agent Log Messages

Some high-frequency information level log messages are converted to debug level to reduce the volume of messages generated at the default information level.

File Logging Capability

Per-component file logging capability for an additional set of agent components has been enabled.

For more information, see Manage File Logging for Agent Components.

Reduce Agent Memory Consumed by Prometheus

The number of Prometheus time series ingested has been limited to reduce agent memory consumption. This limit is applied after Prometheus relabeling rules are applied but before the agent’s metric filter and metric limit.

Defect Fixes

Missing Metrics Due to Aggregation in Agent Fixed

Fixed an issue where processes with certain names were improperly aggregated, which in turn caused missing metrics in certain situations.

Cointerface Fix

Fixed an issue that caused the agent’s cointerface process to restart continuously while processing kubernetes label selectors.

10.9.1 January 21, 2021

Defect Fixes

Thin Cointerface Works as Expected

Fixed a defect in the Thin Cointerface feature which could cause Kubernetes metadata to stop updating. Because Thin Cointerface is turned off by default, the change affects only a small number of users who have this feature turned on.

10.9.0 January 13, 2021

Feature Improvements

Support for Kubernetes Cronjobs

Kubernetes cronJobs are supported when reporting network communications.

Defect Fixes

Runtime Policies and Rules Are Loaded with No Errors

Fixed a race condition that could prevent runtime policies and rules from being loaded properly if multiple messages from the Sysdig backend are received consecutively.

Cluster Overview Displays Compliance Score

Fixed an issue where Statsd metrics related to compliance would have no associated Kubernetes metadata and were not visible on Cluster Overview.

10.8.0 December 18, 2020

Defect Fixes

Filtering Long Container Labels

Filtering long container labels works as expected with no parsing failures or undesirable agent restarts.

Correct kubernetes.pod.restart.rate Metric

Fixed an issue that could cause kubernetes.pod.restart.rate metric to be incorrect.

Prometheus Metrics With Multiple Process Listening Concurrently

Fixed a problem that caused scraping Prometheus metrics to fail when another process was listening to the TCP port 9090 on a host interface.

StatsD Metrics Reports Correct Value

Fixed a problem that caused Statsd metrics to report incorrect values.

Correct Environment Variable Hash in Audit Tap

Fixed an issue that could cause the environment variable hash associated with the exported processes in audit tap to have an incorrect value.

Improve JMX Availability Check

The sdjagent process in the agent no longer consumes excessive CPU resources.

10.7.0 November 20, 2020

Feature Improvements

Policies and Baselines V1 Messages Are Deprecated

Sysdig agent no longer supports the old backend message types that were originally deprecated in on-prem release 2.4.0 (August 2019).

Load Falco Rules on a Separate Thread

Partially load Falco rules in the background to avoid interrupting event processing.

Workflow for Unacknowledged Metrics

The agent is restarted if a metrics acknowledgment hasn’t been received from the Sysdig backend components in 8 minutes. This can happen if networking issues cause the agent to believe it has an active connection when the backend has closed the connection.

Run Single Agent RPM Per Host

Prevents multiple agent services from being launched on the same RHEL-based hosts.

Known Issues

The host.container.start.count metric acts as a counter metric and its value increases monotonically.

Defect Fixes

OpenShift Hardening Guide Correctly Detects Master and Worker Nodes

Running the OpenShift Hardening Guide functionality of the Kubernetes Benchmark will now correctly detect master vs worker nodes, and run the appropriate Benchmark tests.

Agent No Longer Terminates Non-Agent Processes

In some rare situations when process creation in the Agent’s JMX module failed due to issues caused by resource limits, it could inadvertently stop unrelated processes running on the host. This problem has been fixed.

10.6.0 October 30, 2020

Feature Improvements

Python 2.7 Is No Longer Supported in Agent Containers

Python 2.7 has been removed from the agent and agent-slim containers.

This is a breaking change for users who are using an agent container and have set the python_binary configuration to /usr/bin/python2.7.

To prevent breaking the setup, do one of the following:

  • Remove the python_binary configuration option.

  • Set python_binary to /usr/bin/python3.

Sysdig agent continues to support python 2.7 if installed as a service and the host has python 2.7.

Kubernetes Benchmarks

Updated kube-bench to support Kubernetes benchmarks and targets. For a complete list of benchmarks, see Benchmarks (Legacy) .

  • Kubernetes benchmark 1.6

    • Master

    • Control plane

    • Node

    • etcd

    • Policies

  • Google Kubernetes Engine (GKE) Benchmark 1.0

    • Master

    • Control plane

    • Node

    • etcd

    • Policies

    • Managed services

  • Amazon Elastic Kubernetes Service (EKS) Benchmark 1.0

    • Control plane

    • Node

    • Policies

    • Managed services

Configuring Prometheus Metric Expiration Time

Configuring metrics expiration time is supported by promscrape.v2 for Prometheus metrics gathered by using Prometheus service discovery.

Support for Scoping Policies by Kubernetes Cluster Name

Add support for scoping policies by kubernetes.cluster.name. The cluster name must still be manually configured by using the configuration option, k8s_cluster_name: <CLUSTER NAME>.

Improved Prometheus Service Discovery

Made kubernetes node matching more reliable for Prometheus Service Discovery by comparing IP addresses as opposed to node names in the default configuration.

Defect Fixes

CVE Fixes

Addressed a known vulnerability in the jackson-databind package version 2.9.10.6 by upgrading to version 2.11.3 in agent containers.

Reduce Severity of NoClassDefFoundError Log from Error to Info

Changed the java NoClassDefFoundError class from Error to Info to reduce spamming the logs at the Error level. This happens commonly when the agent attempts to read metrics from a java v11 application which was not started with the com.sun.management.jmxremote option.

StatsD Metrics No Longer Show Larger Than Expected Values

Fixed a problem that caused StatsD metrics to be double the expected value.

Remove Warning Logs

Removed warning logs about ambiguous source labels when using the Prometheus service discovery with multi-container pods.

10.5.2 October 21, 2020

Defect Fixes

Memory Leak No Longer Occurs in the Agent

Fixed an issue that could potentially cause a slow increase in the agent’s memory usage over time when the thin_cointerface_enabled configuration option is enabled.

10.5.1 October 08, 2020

Feature Improvements

Added New Rules to the Prometheus Configuration to Honor Pod Annotations

Improved the default Prometheus configuration for promscrape.v2 to honor pod annotations.

Known Issues

Logs warning messages in the agent log file when promscrape.v2 is enabled.

Defect Fixes

Pods Are No Longer Associated with Incorrect Deployments

Fixed a problem that could cause a pod to be associated with incorrect deployments.

10.5.0 September 24, 2020

New Features

Enable Communication Between Agent and Collector Through a Proxy Server

Sysdig agent to the collector communication can be established via an HTTP or an HTTPS Proxy server.

For more information, see Enable HTTP Proxy for Agents.

Default Prometheus Configuration File

A new version of promscrape, promscrape.v2 , has been introduced to offer native Prometheus service discovery capabilities. To support this, a default prometheus.yaml file has been added with Kubernetes pod discovery rules to use when native Prometheus service discovery is enabled. See Enable Prometheus Native Service Discovery for more information.

Secure Mode

Sysdig agent now supports secure mode that offers Secure only features. See Secure Mode for more information.

Known Issues

None.

Defect Fixes

CVE Fixes

Addressed vulnerabilities reported in the agent and agent-slim containers, including the one for CVE-2017-18640 in a dependency library related to image scanning.

Agent No Longer Hangs While Handling Connection Errors

Fixed an issue that caused the agent to hang while handling some types of connection errors. When this issue is encountered, restarting the agent will allow it to reconnect.

Upgrading to Sysdig agent v10.5.0 or higher is strongly recommended to avoid this problem.

Scraping Prometheus Endpoints in Docker Containers

Prometheus metrics can now be scraped from endpoints in Docker containers with remapped port numbers.

Prevent Agent Crashes in Large Systems

The agent now starts faster on systems with thousands of processes and hundreds of containers.

Warning for Prometheus Metric Limit

The agent logs a warning once in a minute when the Prometheus metric limit is reached.

Transmitting Prometheus Metrics Works As Expected When Service Discovery Is Enabled

Fixed a problem that could randomly result in Prometheus metrics not being sent when Prometheus service discovery is enabled.

Appcheck Metrics No Longer Go Missing

Fixed a problem that would cause certain app check metrics to be missing when 10-second aggregation in the agent is enabled.

Agent Now Times Out If Connection Attempt to Collector Does Not Work

Added a timeout to the handshake protocol between agent and collector.

Agent Now Collects JMX Metrics from New Process Following a Java Service Restart

Fixed a problem that randomly caused JMX metrics to be not collected due to transient errors encountered during the startup of new Java processes.

Pod to Service Connection

Fixed a problem that caused the UI to show a pod under an incorrect service if other services exist in different namespaces with the same selectors. This happened when the thin_cointerface_enabled property was set to true.

Syscall Fast Rule Triggers as Expected

Fixed the evaluation of secure fast engine syscall rules when the If Not Matching rule is selected.

10.4.1 August 26, 2020

Defect Fixes

Kubernetes Pods No Longer Lose Association with Resources

Fixed a problem that could cause Kubernetes pods to lose association with their deployment or other related resources.

10.4.0 August 19, 2020

New Features

Ability to Scrape Prometheus Metrics from Container IP Addresses

The agent can now scrape Prometheus metrics from the docker containers that expose ports only on specific IP addresses besides the localhost.

Use Forwarder Is Enabled by Default

The use_forwarder option is now enabled by default. See Collect StatsD Metrics Under Load.

Set JMX Limits

The default value (300) of per-process JMX bean limits can now be changed as follows:

jmx:
  max_per_process_beans: 500

Known Issues

Handling Benchmark Task When StatsD Metrics Collection Is Disabled

When Statsd is disabled, do not attempt to send metrics related to benchmarks tasks. This also means that benchmarks dashboards will not have data when Statsd is disabled.

Kubernetes Pods Can Lose Association with Resources

A problem that could cause Kubernetes pods to lose association with their deployment or other related resources has been identified in Agent version 10.4.0. A new version, 10.4.1, that will address this problem is currently in development.

Defect Fixes

Kubernetes Audit Server and Agent Process Restart Congruently

Embedded web server for Kubernetes audit events restarts as expected when the agent process is restarted.

Updated the version of the jackson-databind package to fix vulnerabilities discovered in the slim agent v10.3.0

10.3.1 August 06, 2020

Defect Fixes

Kubernetes Benchmark Tasks No Longer Fail

The kube-bench binary that was identified as broken due to the change in the output format has been fixed.

kube-bench that performs the Kubernetes Benchmarks tasks has changed the output format, causing the existing Benchmark tasks to fail in v10.3.0. With this fix, the agent will no longer throw errors related to this issue and the new Kubernetes Benchmark results will appear in the UI as expected.

Probes Works As Expected for v5.8 Kernels

Fixed an issue with building probes for Linux v5.8.0 kernel.

10.3.0 July 28, 2020

New Features and Enhancements

Changes to the Monitor Mode

URL segmentation for metrics has been moved from the default monitor mode to the troubleshooting mode. Due to this change, dashboard panels with per URL metric will show no data. See Additional Metrics Values Available in Troubleshooting.

Sysdig Probe Location Changes

The Sysdig probe URL is changed to download.sysdig.com.

If the Sysdig probe URL is included in the allow list for outbound firewall access, you must change the endpoints to reflect the new probe location.

Agent Connects to Promscrape through UNIX Socket By Default

The agent now connects to promscrape through a UNIX socket by default as opposed to the TCP port 9876.

New Configuration File Paths for Kube Proxy

The version of kube-bench has been upgraded to 0.2.4. The changes include an additional configuration file path for Hyperkube kube-proxy to support OpenShift.

Known Issues

Kubernetes Benchmark Tasks Fail

The kube-bench binary is broken due to the change in the output format and the issue will be fixed in an upcoming release.

kube-bench that performed the Kubernetes Benchmarks tasks changed the output format, causing the existing Benchmark tasks to fail. The new Kubernetes benchmark results will not appear in the UI, and the agent will report errors related to Kubernetes benchmark tasks.

Defect Fixes

EndPoints-Independent Metrics Limits for Prometheus

Prometheus metric limits have been modified to ensure that endpoints with fewer timeseries are not affected when another endpoint hits the limit. Reporting of Prometheus timeseries statistics has also been updated.

Prometheus Count Metrics for Summary and Histogram

The calculated Prometheus _count metrics are reported for summaries and histograms even when the _sum values are missing. This feature is not applicable to raw metrics.

A .count metric (which is the rate of change of _count values) and a .avg (which is the average of new samples when _count increases) are calculated for summaries and histograms. Earlier, those .count and .avg metrics are reported only if the raw Prometheus metrics include both _sum and _count values. In this release, changes have been made such that _sum values are no longer required to calculate Prometheus _count metrics for summaries and histograms.

Reporting Running Pod Counts

Fixed an issue pertaining to the reporting of running pod counts for replication controllers, deployments, and ReplicaSets.

Segmenting Kubernetes Jobs Metrics By Namespace

Fixed an issue that prevented having Kubernetes jobs segmented by namespace.

Agent No Longer Stalls Under High Load

Fixed an issue that caused the agent to stall under high load.

Restarting Agent No Longer Causes Exception

Fixed an issue that caused an exception at agent restart while collecting CPU metrics.

10.2.0 June 25, 2020

New Features and Enhancements

Prometheus Scraping

Periodic logging of statistics for Prometheus timeseries has been added. When a metric limit is hit, all the timeseries metrics associated with the endpoint are dropped.

App Checks and Prometheus Metrics

Processes with app checks or Prometheus metrics are now included by default in the top processes to be sent to the Sysdig collector.

Performance Improvement

A variety of performance improvements have been rolled out to accelerate the evaluation of Falco rules and fast engine rules for the common case of events not matching any rules/policies.

Detect JSVC Processes as Java Programs

The agent has been enhanced to detect JSVC processes as java programs to enable the collection of JMX metrics.

Troubleshooting Metrics Removed from Default Mode

The net.mongodb.* and net.sql.* metrics have been moved from the default monitor mode to the troubleshooting mode. For more information, see Additional Metrics Values Available in Troubleshooting.

Deprecated Metrics

The following deprecated App Checks have been removed and will no longer be supported.

  • Network

  • RiakCS

  • TokuMX

  • Ceph

  • Gearmand

  • Gunicorn

  • Kyoto Tycoon

  • Teamcity

  • Riak

  • Solr

  • OpenStack

Defect Fixes

Fixed a Race Condition

Fixed a potential race condition that could occur when receiving multiple policies and related messages from the Sysdig collector at nearly the same time.

Benchmark Task Configuration

The agent no longer runs a built-in set of benchmark tasks. The agent will only run benchmark tasks when configured to do so by a Sysdig Secure backend.

Prometheus Metrics From Idle Processes Are No Longer Dropped

Prometheus metrics from idle processes are no longer dropped even if the target processes are not active enough to be in the top processes. Additionally, the app_checks_always_send parameter, which can force report the idle processes with metrics, now works as expected for metrics gathered by promscrape.

Removed Authentication Credentials

Removed sensitive authentication credentials related to app checks from debug log messages.

Kubernetes Events Are No Longer Dropped

Kubernetes events are no longer dropped under some high load conditions.

Memcached App Checks Collects Slabs and Items Stats

Fixed a problem that prevented the collection of slab and item stats in the Memcache app checks in certain Python environments.

Metrics No Longer Report Incorrect Zero Values

The following metrics now no longer return incorrect zero values:

  • kubernetes.resourcequota.cpu.requests.hard

  • kubernetes.resourcequota.cpu.requests.used

  • kubernetes.resourcequota.memory.requests.hard

  • kubernetes.resourcequota.memory.requests.used

Agent Automatically Restarts Upon Protocol Mismatch Errors

The agent used to require manual intervention to recover from protocol mismatch errors received from the Sysdig Backend. This error can occur when the agent and Sysdig Backend are not in sync. The agent has been enhanced to automatically restart when this error is encountered, so manual intervention is no longer required.

10.1.1 June 02, 2020

Defect Fixes

Enable Network Topology

Network stats metrics that were moved to the troubleshooting mode in Agent v10.1.0 have been re-enabled by default. The metrics will now be available in the monitor mode, which in turn will enable the network topology by default.

For information on agent modes, see Configure Agent Modes.

10.1.0 June 01, 2020

New Features

Support for Linux v5.6 Kernels

Added support for Linux 5.6 kernels.

JMX Support for Java v11, 12, 13 and 14 JRE

Added JMX support for Java 11, 12, 13, and 14 JRE. For containerized Java apps with JRE, run the app with the -Dcom.sun.management.jmxremote option.

Added Rate Limiting Configurations

Added rate limiting configurations to the agent to avoid connection timeouts for metrics and secure messages.

Added New Metrics

Added a new metric to display the kernel version of the host where the agent is running.

  • host.uname

    This metric can be segmented by host.uname.kernel.name, host.uname.kernel.release , and host.uname.kernel.version. For more information, see host.uname.

Added Container Name to the Containerd Event Description

Added container name to the containerd events description. In some rare cases, the container name associated with a containerd event might be unavailable due to metadata lookup delay.

Removed Authentication Credentials

Removed sensitive authentication credentials related to app checks from debug log messages.

Removal of Deprecated App Checks

The following deprecated app checks will be removed in an upcoming release:

  • Network

  • RiakCS

  • TokuMX

  • Ceph

  • Gearmand

  • Gunicorn

  • Kyoto Tycoon

  • Teamcity

  • Riak

  • Solr

  • OpenStack

Enable Removed Metrics

Some metrics related to network and file will not be available by default. You can enable them by editing the dragent.yaml file.

Edit the Configuration File
  1. Open the dragent.yaml file.

  2. Add the following configuration parameter:

    feature:
      mode: troubleshooting
    
  3. Restart the agent.

Removed Metrics in Agent v10.1.0

The following metrics will not be reported by default in agent v10.1. When segmented by a particular label, these metrics will not have some values. The table summarizes the metrics and missing values when they are segmented by a particular label.

MetricsUnreported Metrics Values When Segmented by
file.error.total.countfile.name and file.mount labels
file.bytes.total
file.bytes.in
file.bytes.out
file.open.count
file.time.total
host.count
host.error.count
proc.count
proc.start.count
net.bytes.innet.connection.server, net.connection.direction, net.connection.l4proto and net.connection.clientlabels
net.bytes.out
net.connection.count.total
net.connection.count.in
net.connection.count.out
net.request.count
net.request.count.in
net.request.count.out
net.request.time
net.request.time.in
net.request.time.out
net.bytes.total

Defect Fixes

Promscrape No Longer Breaks Metrics Collection Over HTTPS

Fixed promscrape to honor the ssl_verify configuration option.

Slim Agent Container No Longer Prevents Certain App Checks From Emitting Metrics

Fixed an issue with the agent-slim container that prevented postgres and pgbouncer app checks from emitting metrics.

Reduced the Frequency of Log Messages

Reduced the frequency of a log message to reduce spam and enhanced a statsd related log message to provide more information about incorrectly formatted strings.

Use Exact Rule Names When Adding Rules to Runtime Policies

Consider only exact matches when linking secure runtime policies to Falco rules to fix this issue.

Corrected Calculation of net.bytes.* Metrics

Fixed calculation of net.bytes.* metrics at the host level when using calico interfaces or VPN tunnels.

10.0.0 May 01, 2020

New Features

Kubernetes Benchmark Master Programs

Added the ability to run Kubernetes Benchmark Master Programs on additional Kubernetes distributions.

New Scraping Mechanism for Prometheus

A new process, called promscrape, has been introduced to scrape Prometheus metrics by default. The mechanism, based on the open-source Prometheus, improves compatibility and performance. It also allows per-endpoint metric filtering and relabeling through metric_relabel_configs.

For more information, see Working with Prometheus Metrics.

Non-Root Access to Log Files

Added the ability to make draios.log files readable by users other than root. This can be enabled with the following configuration in dragent.yaml.

log:
  globally_readable: true

New Runtime Policy Action

Added the ability to kill containers as a runtime policy action. See Manage Policies for details.

Defect Fixes

Fixed the Path Parameter Issue in Prometheus Configuration

Fixed the use of the path parameter in Prometheus configuration when using promscrape. With this fix, the configured path is passed to promscrape by the agent when it is set up for a target rule in dragent.yaml.

Service Annotation Based Prometheus Scraping

Prometheus scraping can now be triggered based on service annotations by default.

Added a Missing Module to the agent-slim Container

Added the missing posix-ipc module to the slim agent. This fixed an issue that prevented App Checks from running in the agent-slim container on v9.9.0.

No Metric Limit on Scraped Prometheus Metadata

Prometheus scraping metadata is no longer counted toward, or limited by, metric limits when using promscrape.

Fix for Percentile Metrics

Fixed a defect that caused percentile metrics to not work properly.

9.9.1 April 16, 2020

Defect Fixes

Added the Missing Module to the Slim Agent

Added the missing Posix module to the slim agent. This fixed an issue that prevented App Checks from running in the agent-slim container on v9.9.0.

9.9.0 April 13, 2020

Core Features and Fixes

Python 3 Set as Default and Some App Checks Deprecated

Python 3 is the new default Python version for app checks, instead of Python 2. Python 2 can still be used by setting the following option in your dragent.yaml:

python_binary: <path to python 2.7 binary>

For containerized agents, this path will be: /usr/bin/python2.7

The following app checks are deprecated as of 9.9.0:

  • Network

  • RiakCS

  • TokuMX

  • Ceph

  • Gearmand

  • Gunicorn

  • Kyoto Tycoon

  • Teamcity

  • Riak

  • Solr

  • Openstack

See Integrate Applications (Default App Checks).

Fixed Kernel Issue when Deploying Agent on GKE

Fixed a potential CPU stall on kernels with versions greater >= 4.19 using eBPF probe.

Fixed Flooded Agent Logs

Fixed an issue that caused excessive logging in the agent log file.

9.8.0 March 31, 2020

All the public-facing URLs that were pointing to https://s3.amazonaws.com/download.draios.com/ have been updated to point to https://download.sysdig.com/.

Change the URL in the whitelisting firewall/proxy setting to reflect https://download.sysdig.com/. Otherwise, the agent install on Linux will fail.

Fixes

Metrics Reporting

Fixed an issue in the agent wherein the kubernetes.namespace.pod.desired.count and kubernetes.namespace.pod.available.count metrics were not reporting any values.

HDFS App Check Deprecated

The HDFS (Hadoop Distributed File System) App Check had been deprecated and removed. Users of the HDFS App Check can switch to hdfs_namenode and hdfs_datanode App Checks.

Metric Calculation

Fixed an issue related to calculating the kubernetes.pod.restart.rate metric.

Network Congestion

Isolated the Kubernetes Audit HTTP server from the Audit Event processing path to reduce the chances of slowing down the connections from the Kubernetes API server. This should reduce the likelihood of multiple outstanding connections from the Kubernetes API server.

Certifi Python Module

Added a missing certifi Python module to the agent container.

9.7.0 March 09, 2020

New Features

Support for Openshift Hardening Guide

Added Openshift Hardening Guide as a benchmark program. It is available as an option for CIS Kubernetes Benchmark.

Support for Linux Benchmark

Added Linux benchmarking as an available benchmark program.

New Metrics for Redis and MongoDB App Checks

The following metrics are introduced:

  • RedisDB

    • redis.mem.startup

    • redis.mem.overhead

  • MongoDB

    • mongodb.tcmalloc.generic.current_allocated_bytes

    • mongodb.tcmalloc.generic.heap_size

    • mongodb.tcmalloc.tcmalloc.aggressive_memory_decommit

    • mongodb.tcmalloc.tcmalloc.central_cache_free_bytes

    • mongodb.tcmalloc.tcmalloc.current_total_thread_cache_bytes

    • mongodb.tcmalloc.tcmalloc.max_total_thread_cache_bytes

    • mongodb.tcmalloc.tcmalloc.pageheap_free_bytes

    • mongodb.tcmalloc.tcmalloc.pageheap_unmapped_bytes

    • mongodb.tcmalloc.tcmalloc.spinlock_total_delay_ns

    • mongodb.tcmalloc.tcmalloc.thread_cache_free_bytes

    • mongodb.tcmalloc.tcmalloc.transfer_cache_free_bytes

For more information, see Metrics Introduced with Agent v9.7.0 and RedisDB Metrics

Fixes

Slim Agent Vulnerabilities

Fixed the vulnerabilities detected in the agent-slim v9.6.1 image. These issues are related to the python2 and jackson-databind packages. These packages were upgraded to the versions with fixes.

Run App Checks on Hosts with Python 2.6

Fixed a defect that prevented app checks from running on hosts that install Python 2.6.

9.6.1 February 28, 2020

Fixes

Metrics calculation

Fixed an issue that caused an error in the calculation of some metrics such as net.* in agent version 9.6.0.

Red Hat-based host issue

Fixed an issue that caused the kernel module build associated with agent version 9.6.0 to fail on Red Hat-based hosts.

9.6.0 February 26, 2020

Upgrades

Integrations improved

Added new metrics and configuration options for HAProxy and Consul app check integrations. See HAProxy, and Consul for details.

Fixed a problem Go app check which caused it to fail with an exception error.

Metrics added

Added Kubernetes metric kubernetes.namespace.pod.running.count to track the number of pods in running state. See Kubernetes Dashboards.

Reduced load on the Kubernetes API server

The version of client-go was updated and now defaults to encoded protobuf messaging instead of JSON to improve performance.

Configuration option new_k8s now enabled by default.

Default collector port changed

The default port for the collector was changed from 6666 to 6443.

This could affect your firewall port settings; you may want to review them before upgrading the agent.

Fix for the dynamic back-end configuration of Kubernetes Audit Logging caused some agent deploys to fail

The agent is enhanced to listen on /k8s-audit for Kubernetes audit events and the path can be configured via the config option security:{k8s_audit_server_path_uris: [path1, path2]}.

Fixes

Prometheus metrics fix

Fixed a problem that inhibited the agent from scraping multiple ports on a single process for Prometheus metrics.

Inaccurate cpu.used reporting fixed

Fixed a problem that caused the agent to erroneously report very high CPU usage in some environments.

9.5.0 January 28, 2020

Note that the versioning scheme for agent releases has been updated with this release. Previous versions used the format 0.<version number><hotfix>, such as 0.94.0.

Sysdig is aligning version numbers to the rest of the product. The new version number reflects the maturity of the Agent software over the last several years. Going forward, all Agent versions will be numbered as Major.minor.hotfix

We encourage users to be on the latest version of the Agent. Starting with the next release of the Agent, we will support n-3 versions back based on the minor number. For example, if the next release is v 9.6.0, we will support n-3 versions back, e.g to 9.3.0 (old version scheme = 0.93.0).

Fixes and Upgrades

Added new configuration option and metrics for Elasticsearch integrations

In the Elasticsearch app check, the parameter index_stats can be used to collect metrics from individual indices. See Example 4 in Elasticsearch and Elasticsearch Metrics for details.

Added new metrics for NGINX Plus integrations

More than 60 new metrics have been added to the NGINX app check. See NGINX Plus Metrics for details.

Made Go-based event handling the default

See Process Kubernetes Events. As of agent 9.5.0, the default setting for go_k8s_user_events is true and there is no need explicitly to enable it. To switch back to the older events monitoring (C++ based), set the value to false in your agent config (dragent.yaml).

Enhanced log tracing for include/exclude processes filter.

No user action is required; see Include/Exclude Processes to use the filter.

Fixed agent termination issue

Fixed a problem that was causing an internal process within the agent to repeatedly restart.

Improved memory buffer handling

The agent will now auto-disable memdump functionality when the memory buffer is too small.

Agent start/stop improvements on CRI-O and Openshift 4.x

The agent can now correctly perform the pause and stop container actions on clusters running OpenShift 4.x and CRI-O.

0.94.0 December 20, 2019

Fixes and Upgrades

Fixed issue in the agent install scripts

The agent install scripts have been updated to mount /etc/modprobe.d from the host into the agent container. This prevents a problem where the agent loaded drivers that were excluded from the host.

Added user events for additional resource types

Added events monitoring for statefulsets, services, and horizontal pod auto-scalers (HPAs) when the Golang-based events monitoring feature is enabled. To enable, see Process Kubernetes Events.

Added regex support for Kafka integrations

Added regex capability for consumer groups and topics in Apache Kafka configurations. See Example 6 in Apache Kafka.

Increased the Prometheus max_tags default value

The Prometheus max_tags configuration has been increased from 20 to 40.

Made change to guarantee support for older cpuset configurations.

Changed CRIO cpuset calculations to use the configured cpuset.cpus value instead of cpuset.effective_cpus. This guarantees support on older cpuset configurations.

Corrected an issue that resulted in the suffix “_total” to be stripped to Prometheus counter metric names.

0.93.1 November 25, 2019

Fixes and Updates

Fixed installation issue on native RHEL 7.x installs

The agent installer script has been updated to refer to an updated epel repository.

Improved JMX metrics reporting

Fixed an issue when retrieving JMX metrics which could result in missing samples.

(Sysdig Secure): Improvement in Kubernetes Audit events

Fixed runtime policy scopes for Kubernetes audit events.

(Sysdig Secure) Fixed audit event exception

The system now catches JSON object-type exceptions when parsing Kubernetes audit events.

Improved error message

Improved the error message reported when the Sysdig agent cannot find a pre-installed kernel header or cannot download a sysdigcloud-probe.

Performance improvement in dragent logging

0.93.0.1 November 15, 2019

Fixes

Fixed issue with Prometheus metrics names

Corrected a problem that resulted in the suffix _total to be removed from Prometheus counter metric names.

0.93.0 November 6, 2019

New Features

Mask the customer ID in log files

The Customer ID is no longer output in the agent log, to avoid inadvertent exposure when sharing of log files.

Kubernetes role node label included by default

The kubernetes.node.label.kubernetes.io/role label is available by default

Update Kubernetes API used, in order to expand support of Kubernetes v1.16

Replaced usage of the extensions/v1beta1 Kubernetes API with apps/v1 in the agent. This is required for supporting Kubernetes v1.16 using the agent’s legacy Kubernetes integration (when new_k8s is not enabled).

Introduced a new config option in ElasticSearch app check

Introduced a new config option to generate cluster-wide primary shard metrics from a master node: pshard_stats_master_node_only. See Elasticsearch (Example 3).

Enhanced Postgresql app check

The Postgres app check has been enhanced to provide new metrics and examples. See PostgreSQL.

Agent preparation for upcoming Policy Advisor feature in Sysdig Secure

The agent will support new Rules generated by Sysdig’s Kubernetes Policy Advisor. This agent is the minimum version required to use the upcoming feature.

Updates and Fixes

Improved system events handling for Ubuntu 19.10

On kernels 5.1 and newer, some syscall events were incorrectly dropped. This has been fixed.

Stopped Kubernetes pause containers (pods) from being reported

Fixed an issue where Kubernetes pause containers were also showing up in Kubentes events. This fix filters them out from the events being reported.

Fixed rare issue on OpenShift

Fixed an issue where, in a rare case, a dropped event could cause a kernel deadlock and crash the node.

Fixed issue preventing kernel module creation for Debian Buster

This change adds support for building the Sysdig Monitor agent kernel module for Debian Buster.

Improved event timestamp in Kubernetes

This fix ensures that user events get the correct timestamp with Kubernetes v1.16 when thego_k8s_user_events option is set to true.

Updated Kubernetes API used, in order to expand support of Kubernetes v1.16

In dragent.yaml, the Kubernetes API extensions/v1beta1 is updated to apps/v1. This enables agent support for Kubernetes v1.16 even when the new_k8s option is set to false.

Fixed a Kubernetes event reporting issue

Fixed an issue with Kubernetes Events where the host MAC scope was not populated correctly, resulting in not showing up on the dashboard.

Improved Kubernetes events handling from delegated agents

When using go_k8s_user_events, kubernetes events from non-delegated agents are no longer sent.

Eliminated legacy “BASELINES” message

Stopped processing legacy BASELINES messages from the backend collector.

Performance improvement at startup

The agent now defers initialization of Secure-related components slightly to reduce excess resource usage at startup.

0.92.3 October 7, 2019

Updates and Fixed Issues

Included Example of a Prometheus Matching Rule Using HTTPS

The Sysdig agent will use HTTPS for scraping when target’s annotation has “kuberentes.pod.annotation.prometheus.io/scheme: https”.

Kubernetes versions older than 1.9 no longer supported.

The Sysdig agent has replaced the use of the extensions/v1beta1 Kubernetes API with apps/v1.

Included Example of a Prometheus Matching Rule Using HTTPS

The Sysdig agent will use HTTPS for scraping when target’s annotation has “kuberentes.pod.annotation.prometheus.io/scheme: https”.

The RabbitMQ app check has a new config option: filter_by_node

Without this option, each node reports cluster-wide information (as presented by rabbitmq itself). This option makes it easier to view the metrics in the UI by removing redundant information reported by individual nodes. See RabbitMQ for details.

0.92.2 September 26, 2019

New Features

Asynchronous metadata collection for CRI-O and containerd

The collection of container metadata from CRI-based runtimes was previously synchronous with other agent tasks.

**Prioritize and filter how process metrics are reported in Sysdig Monitor. **

In addition to filtering data by container, it is also possible to filter independently by process. Broadly speaking, this refinement helps ensure that relevant data is reported while noise is reduced. See Include/Exclude Processes for details.

As of this version, App Checks on hosts with Python 2.6 will no longer be supported.

Fixed Issues

  • **Fix for Agent termination during resource discovery from the Kubernetes API Server **

    Fixed an issue where the Agent stopped and shut down if there an error occurred during resource discovery from the Kubernetes API Server. This fix simply reports the error and continues with the discovered resources.

  • Fix for Kubernetes delegation error

    Fixed an issue that caused Kubernetes delegation to not work after the cointerface process restarts following a crash.

  • Fix for accounting Network errors

    Network-related errors are now correctly accounted for instead of being treated as file-open errors.

  • New Prometheus Client Version

    Updated prometheus_client to version 0.7.1. This should result in improved performance while ingesting Prometheus metrics.

  • Fix for dropping StatsD Metrics

    A defect in earlier versions of Sysdig Monitor with the statsd.use_forwarderoption could drop some StatsD metrics from containers. This change resolves that problem; the agent will begin fetching metrics from containers 10 seconds after first identifying that the container exists. The 10 second delay allows containers to start StatsD servers within their network namespaces if they choose.

    The timeout can be overridden using the statsd.container_server_creation_delay_s option, which specifies the delay in seconds.

  • Fixed resource metrics for CRI-O containers

    The following metrics reporting correctly in the Monitor UI: memory.limit.bytes, memory.limit.used.percent, and cpu.quota.used.percent. The CRI extra_queries option now enabled by default. See Runtime Support: CRI-O and Containerd for details.

Sysdig Secure

  • **Fix for enlarging Sysdig Capture **

    Fixed an issue where a Sysdig capture would grow endlessly if a security policy was set to Capture 0 seconds after an event.

  • Fix for processing system events

    Fixed problem where gettimeofday syscall was called in compliance code while processing system events. This could potentially cause performance problems in Linux distros that called down to the kernel for gettimeofday responses, such as some versions of Amazon Linux.

Sysdig Platform

  • New RPM dependency

    Changed RPM dependency to Python 2 to support installation on RHEL 8.

0.92.1 August 16, 2019

Fixed Issues

Sysdig Monitor

  • Fixed issue with cluster name in Monitor UI

    Cluster name was being populated incorrectly for Kubernetes event scopes.

  • Fixed Kubernetes events issue

    Fixed Kubernetes event collection issue that occurred when using the go_k8s_user_events option. This option was introduced in agent version 0.91.

Sysdig Platform

  • RHEL 7.7 and 8.0+ support The kernel module now builds for RHEL 7.7 and 8.0+

  • Fixed issue with StatsD metrics collection limits Some versions of the Sysdig agent allowed fewer than the configured number of StatsD metrics because Sysdig Secure-related StatsD metrics were counted towards the configured limit.

    This change corrects that behavior so that the configured limit applies only to StatsD metrics that do not originate from Sysdig components.

Sysdig Secure

  • Fixed a profiling-related issue that impacts Sysdig Secure 2.4

    Sysdig Secure 2.4 will include a new Profiling feature, and 0.92.1 fixes a bug where profiling could remain disabled after periods of high load. In order to use Profiling, it is required to upgrade to agent 0.92.1 or higher.

0.92 August 7, 2019

New Features

Preparatory enhancements for upcoming Sysdig Secure Policy Editor Although the feature UI will not be released until version 2.4.0, Sysdig encourages all users of Sysdig Secure to upgrade to agent 0.92 in preparation for the new Policy Editor feature. Agent 0.92 will accept policies messages from both the current backend as well as a backend that supports the new policy editor.

Ability to compress metrics data for internal transfer

With app checks integrations, when the volume of metrics data collected was too large to send over the agent’s internal queue, app checks could fail. This problem is solved by introducing an option to compress app checks metrics data, which reduces the internal load. See Compress Metrics Data for details on how to enable this option.

Fixed Issues

Sysdig Monitor

Fix for occasionally dropped metrics In earlier releases of Sysdig Monitor, the agent sometimes failed to parse metrics containing negative values for some fields.

This change updates the behavior to drop fields that have unsupported negative values, and to generate a log message when such fields are encountered.

Sysdig Platform

  • Fix for MySQL versions 8.0.14+

    Fixed a bug that caused the MySQL app check to fail with an error.

  • Fixed agent crash issue exposed by recent Linux kernels

    Affected kernels include the 5.2.x line, 5.1.8+, and 4.19.49+.

  • Fixed a bug in HTTP parserIn the (uncommon) situation where absoluteURI is used in the Request-URI, fixed a bug that was causing a faulty URL.

0.91 July 17, 2019

New Features

Improved securityRemoved obsolete and vulnerable Python 2.6-compatible libraries from Docker images.

More efficient Kubernetes event handling.

The agent has added functionality to allow more efficient processing of Kubernetes user events.

See Process Kubernetes Events to enable.

Reduced CPU usage on Kubernetes clusters Extended performance optimizations for processing Kubernetes Services, which will reduce agent CPU usage in large clusters.

Container filtering enhanced. Smart filters and aggregated filtering options are now available. See Prioritize/Include/Exclude Designated Containers.

Fixed Issues

Monitor

  • Fixed issue with Prometheus metrics gathering intervals

    The agent will now respect the configured interval for scraping Prometheus metrics from remote endpoints, as opposed to doing it every second.

  • Fixed limit/requests calculations for init containers

    Fixed memory calculations for Kubernetes init container limits and requests

  • Improved Healthcheck monitoringAgent has improved ability to detect commands identified as a part of Kubernetes Liveness/Readiness Probes, in addition to Docker Health Checks.

  • Improved error messaging

    Warning messages for container group inconsistencies were demoted to debug level, as they are harmless and do not need to clutter the error reporting stream.

  • Fixed issue with container “incomplete” reporting status

    Starting with version 0.90.0, the agent would report containers for which it had not yet fetched metadata as “incomplete.” This would then propagate to the Monitor UI. This restores the behavior where the agent leaves the unknown fields unset.

  • Resolved REST server issue

    Fixed problem where an enabled port would respond to HTTP requests when not desired.

  • Fixed issue with StatsD metrics collection

    Previous versions of the Sysdig agent, when configured to use the StatsD fowarder ({{statsd.use_forwarder: true}}) truncated messages that it received from containers to 2048 bytes, resulting in the potential for dropped and corrupted metrics. This change resolves that problem. See details under StatsD Integration.

It is recommended to follow upgrade best practices:

  • Keep upgrades current

  • Test upgrades in a non-mission-critical or staging environment before rolling into production.

2 - Serverless Agent Release Notes

For Installation and Upgrade steps, see AWS Fargate Serverless Agents.

3.0.3 September 19, 2022

New feature

Added task label to the metric serverlessdragent.workload_agent.count to enable grouping multiple containers in a single task.

3.0.2 September 02, 2022

Defect Fixes

Fixed preventing workload starting if no policies in place

To avoid workload starvation the instrumentation can now start the workload if security policies are not in place.

The workload starting policy can be easily configured, see Configure workload starting policy.

Fixed workload starvation-detection watchdog

Instrumentation watchdog no longer needs to be configured anymore via the watchdog.sinsp_worker_timeout_s parameter.

Fixed /proc scan failure

Instrumentation /proc scan no longer fails when the SSM Agent runs as root and the instrumented task runs as non-root user.

New instrumentation logging level parameter

The instrumentation logging level can now be easily configured via a new parameter exposed in the Instrumentation stack.

3.0.1 June 30, 2022

Defect Fixes

Log Levels Updated

The instrumentation logger for the Fargate Serverless Agent can now be configured to the following log levels:

  • silent
  • error
  • warning
  • info
  • debug
  • trace

See also: Manage Serverless Agent Logs

3.0.0 June 17, 2022

Defect Fixes

Fixed DEBUG Logging Error

The instrumented task should no longer be blocked from starting when using DEBUG logging with log-forwarding enabled, and better error messages have been added for failures when log forwarding.

Fixed Termination Error

Instrumentation tasks now terminate correctly on fatal errors and trigger the ECS restart policy.

Cleaned Up Serverless Agent Metadata

Redundancies in the serverless agent metadata (labels and tags) were corrected:

  • AWS-related metadata are grouped below aws.* tags
  • Container-related metadata are grouped below container.* tags
  • Custom tags are grouped below agent.* tags

New Features

New Container-Based Installer

The Serverless Agent 3.0.0 provides a new container-based installer to simplify the deployment of the instrumentation & orchestration stacks. (Serverless Agent 3.0.0 supports the existing command-line-based installer as well.) See also: AWS Fargate Serverless Agents.

Instrumentation Logs Format

The Serverless Agent 3.0.0 supports both the json and text format for the forwarded instrumentation logs. See also Manage Serverless Agent Logs.

2.3.0 March 15, 2022

Defect Fixes

Container Metadata Now Automatically Provided to Avoid Errors

The following metadata values are now automatically passed by serverless agents:

- container.image.repo*
- container.image.tag**
- container.image.digest**
- container.image.id*

*value is always provided in same way **value depends on how the image is referred to when deploying the instrumented container, i.e. repo:tag vs repo@digest.

Example:

:latest When specifying an image such as falcosecurity/event-generator:latest the metadata configuration =:

- container.image.repo = falcosecurity/event-generator
- container.image.tag = latest
- container.image.digest = null
- container.image.id = sha256:aaabbbcccddd

:named image When specifying an image such asfalcosecurity/event-generator@sha256:aaabbbcccddd the metadata configuration =:

- container.image.repo = falcosecurity/event-generator
- container.image.tag = null
- container.image.digest = sha256:aaabbbcccddd
- container.image.id = sha256:aaabbbcccddd

Fixed Display Problem in Insights Composite View for Fargate Events

Secure events from the Fargate serverless agent are now correctly labeled with Account ID and Region, allowiing them be grouped correctly in the Insights Composite view.

Fixed Occasional Problem with Starting Instrumented Tasks

Added retry and fallback logic to avoid restarts when a log-forwarding endpoint isn’t present.

Manual Instrumentation of Workload Agents

Improved documentation for manual instrumentation of workload agents, including handling logs.

2.2.0 December 2, 2021

Defect Fixes

Fixed Workload Agent Start Issue

The system no longer allows the workload agent to connect to the orchestrator agent if policies have not been loaded. This prevents the workload from starting without policies in place in the event of network disruption.

New Features

Easier Setup of Alternative Port for Orchestrator

Because the 6667 port is hardcoded in multiple places in the orchestrator CTF, users who needed to assign a different port to the orchestrator agent faced a cumbersome process. The orchestrator port can now be configured via either SYSDIG_ORCHESTRATOR_PORT (default) or the SysdigOrchestratorAgentPort (new) parameter in the CloudFormation template.

Instrumentation Logs Collected Separately from Workload Logs

Fargate instrumentation logs are by default collected in a separate log group, which is created when installing the CFN instrumentation macro.

2.1.0 September 27, 2021

Defect Fixes

Fixed Task Stall Issue

Fixed a memory leak in the Serverless Agent instrumentation that could cause the instrumented task to stall. The problem is more likely to be encountered when a large number of captures are generated in quick succession.

Resolved an Agent Error when Reading File Descriptors

Reduced the log level of a benign warning message to debug.

2.0.0 July 7, 2021

New Features

Captures Available

Announcing the availability of the Captures feature in Fargate.

Defect Fixes

Fixed/Enabled Policy Scoping on Instrumented Fargate Tasks

At this time, only container-related scope labels such as container.id or container.name are supported.

Delay Event Source Startup by Default

The system now waits for policies to be available before launching the instrumented task, to fully secure workloads

Fixed Exit Codes for Faulty Workloads

The exit codes of the instrumented tasks are now faithfully propagated.

Better Handling of cmd and entrypoint Errors

Log more informative errors when cmd and/or entrypoint are not available for serverless agent instrumentation.

Fixed S3 Bucket Error

Fixed an issue in the serverless agent installer that caused a failure while attempting to create an S3 bucket in us-east-1 region.

1.0.1 April 15, 2021

Segmentation Fault Error Fixed

Fixed a problem that caused a segmentation fault error inside a Fargate task due to Sysdig instrumentation.

Container Definition Fields Now Support Complex Values

Added support for complex values inside Name and Image fields of the container definition. See also the ECS Task Definition docs from Amazon.

March 15, 2021: Serverless Agents Introduced

Sysdig Serverless Agent 1.0.0 for Fargate ECS

The “container-as-a-service” serverless environment calls for new agent models, and Sysdig provides them. Whereas in ECS, users still manage the underlying instances, with AWS Fargate the host is never visible and users simply run their workloads. And while this model is convenient, it can introduce risk as many people leave the containers unattended, without monitoring security events within that can exfiltrate secrets, compromise business data, impact performance, and increase their AWS costs. In addition, it is not possible to install a standard agent in an environment where you do not have access to a host.

For these reasons, Sysdig has introduced a new “serverless agent” model that can be deployed in these container-based cloud environments. The first implementation is for Fargate (ECS).

Sysdig will be rolling out security features on the serverless agent over time. In v1.0.0, users will see:

  • Runtime Policies and Rules

  • Secure Events

To obtain secure event information and the associated Falco policies and rules in the Sysdig Secure UI from a Fargate environment, users install the serverless agent using a CloudFormation Template. Then log in to Sysdig Secure and review the events in the UI.

See also: AWS Fargate Serverless Agents and Serverless Agent Release Notes (for future updates).

3 - SaaS: Sysdig Monitor Release Notes

Monthly updated list of newly released features.

The dates shown are for the initial release of a feature.  The feature may not be rolled out to all regions concurrently. The availability of a feature in a particular region will depend on scheduling.

Supported Web Browsers

Sysdig supports, tests, and verifies the latest versions of Chrome and Firefox.

Other browsers may also work, but are not tested in the same way.

September 25, 2022

SAML Single Logout

SAML single logout, the facility to terminate multiple Sysdig user sessions simultaneously, is now available on all the regions. Furthermore, Sysdig now supports Okta for SAML single logout in federated authentication environments.

Case Sensitive Labels in PromQL Queries

To comply with PromQL specification for filtration expressions, label names in PromQL filtering expressions in Sysdig Monitor will be case sensitive. If the casing of a filtering label is incorrect, the query will return an empty response.

As majority of PromQL queries were crafted using auto-complete for existing label names, changes to the label casing will not have a major impact. However, there could be rare cases where auto-complete is ignored or PromQL queries are crafted via API, which should be reviewed to make sure casing is correct.

For example:

If the given label name is ‘host_hostname’, and you want to match the time series of the ‘sysdig_host_cpu_used_percent’ metric to the host ‘foo’, the correct query would be:

sysdig_host_cpu_used_percent{host_hostname='foo'}

Previously, both the following queries returned results.

sysdig_host_cpu_used_percent{HOST_hostname='foo'}
sysdig_host_cpu_used_percent{HoSt_HoStNamE=‘foo'}

The following are unimpacted by this change:

  • The alert and dashboard queries created by using the Form UI

  • Label values

    Label values are already case sensitive in Sysdig Monitor.

Google Chat Integration

You can now use Google Chat as a notification channel in Sysdig Monitor. See Configure a Google Chat Channel for more details.

Stacked Bars

Timechart panels support creating statcked bar charts. For more information, see Timechart.

Monitoring Integrations

Rename Dashboard Templates to Dashboard Library

In order to align with the rest of Monitor, Dashboard Templates has been renamed to Dashboard Library.

Integrations

  • Added the following integrations:

    • OpenShift API Server
    • Openshift 4 CoreDNS
    • Openshift 4 etcd
    • Calico
    • Cassandra
  • Split the k8s-control-plane integration to different integrations per application

  • Improved the Troubleshooting guide by removing scope from the promQL queries.

Dashboards and Alerts

  • Added OpenShift v4 API Server dashboard including the openshift-api scopes
  • Made Etcd and CoreDNS dashboards compatible with Kubernetes and Openshift v4 (both OKD and ROKS)
  • Changed the AWS Metrics ECS MetricStream template to include ECS in the name

Promcat.io

  • Updated Cassandra integration details with JMX exporter

August 17, 2022

New Permission for Changing Team Roles

Team management has been improved with the addition of the new permission, Team Membership Roles. This new permission will allow you to change the roles of team members separately while adding users to the teams.

For more information, see:

August 08, 2022

Advisor

Accelerate Troubleshooting by Up to 10x with Advisories

Advisories evaluate the thousands of data points being collected by the Sysdig agent, and displays a prioritized view of key problems in your infrastructure that affect the health and availability of your clusters and the workloads running on them.

See Sysdig Advisor: Making Kubernetes troubleshooting effortless on the Sysdig blog.

Entire Infrastructure Overview

Entire Infrastructure shows an aggregated view of all Advisories, active alerts, events, and a quick snapshot of the state of your Kubernetes infrastructure. This is shown before selecting a cluster or workload, and is the new default landing page of the Monitor product.

Display Improvements

Display and representation of data has been improved, including the use of new panel types. Information such as workload availability or resource limits are now displayed as a table instead of a chart.

Dashboard Manager

Sysdig introduces Dashboard Manager to organizes all the dashboards associated with your account. The page acts as the repository for all the dashboards that you have created, that your teams have shared with you, and that you have marked as favorite, as well as the dashboard templates available to you.

For more information, see Dashboard Manager.

Prometheus Alertmanager Notifications

You can now integrate Prometheus Alertmanager as a notification channel in Sysdig Monitor. See Prometheus Alertmanager Notifications for more details.

Contextual Tooltip

The Contextual Tooltip has been enhanced to display all segments. To enable this feature, toggle the Contextual Tooltip in Dashboards in the Settings > User Profile screen. The option is found under the Beta Features section.

Enhanced Label Selector

The label selector in Dashboards and Metrics Explorer has been enriched with the following sought after features:

  • Label documentation
  • Preview of label values
  • Suggested labels

New PromQL Variables

The following PromQL variables have been added:

  • $__interval_sec
  • $__range_sec

They are used for translating the rate time aggregation in a Form query into a PromQL query. For example:

avg(sum_over_time(sysdig_container_cpu_used_percent{$__scope}[$__interval])) / $__interval_sec

For more information, see Using PromQL.

Events Feed Enhancements

The Events module has been refreshed to show metrics and labels in Prometheus notation.

Monitoring Integrations

Integrations

  • Added the following integrations:

    • HAProxy Openshift integration
    • Istio integration
  • Removed metrics filtering in envoy job in Istio agent configuration. This will allow for collecting other custom metrics merged into the Envoy sidecar.

  • Enhanced the OpenShift HAProxy configuration to use ClusterRole

  • Added the following to Promcat.io:

    • HAProxy Openshift 4.7
    • Istio 1.14

Dashboards and Alerts

  • Enhanced RDS description for PostgreSQL

  • Enhanced the calculation of used vs request/limits in Kubernetes Capacity Planning Dashboard

  • Enhanced promQL in Kubernetes Dashboards to avoid operations occuring in ephemeral containers

  • Added updated Time Series Usage Dashboard Template to the repository

  • Removed the deprecated ‘OutOfDisk’ condition on Node Status and Performance Dashboard

  • Updated Kubelet metrics for Kubernetes v1.19 and above in Dashboard Templates

    • kubelet_running_container_count to kubelet_running_containers
    • kubelet_running_pod_count to kubelet_running_pods
  • Removed duplicated Dashboard Templates

Exporter

  • Upgraded exporters Jenkinsfile for scratch and ubi images
  • Fixed the error in JMX exporter image.
  • Fixed port information in Memcached exporter scratch image.
  • Added the following Security updates in UBI images of all the exporters:
    • Apache

      quay.io/sysdig/apache-exporter:v0.11.1-ubi

      quay.io/sysdig/apache-exporter:v0.11.1

    • Elasticsearch

      quay.io/sysdig/elasticsearch-exporter:v1.3.4-ubi

      quay.io/sysdig/elasticsearch-exporter:v1.3.4

    • Grok

      quay.io/sysdig/sysdig/grok-exporter:v1.0.4-ubi

      quay.io/sysdig/sysdig/grok-exporter:v1.0.4

    • JMX

      quay.io/sysdig/promcat-jmx-exporter:v0.17.3-ubi

      quay.io/sysdig/promcat-jmx-exporter:v0.17.3

    • Memcached

      quay.io/sysdig/memcached-exporter:v0.10.2-ubi

      quay.io/sysdig/memcached-exporter:v0.10.2

    • MongoDB

      quay.io/sysdig/mongodb-exporter:v0.11.9-ubi

      quay.io/sysdig/mongodb-exporter:v0.11.9

    • MySQL

      quay.io/sysdig/mysql-exporter:v0.14.1-ubi

      quay.io/sysdig/mysql-exporter:v0.14.1

    • NGINX

      quay.io/sysdig/nginx-exporter:v0.10.1-ubi

      quay.io/sysdig/nginx-exporter:v0.10.1

    • Node exporter

      quay.io/sysdig/node-exporter:v1.2.4-ubi

      quay.io/sysdig/node-exporter:v1.2.4

    • NTP

      quay.io/sysdig/ntp-exporter:v2.0.4-ubi

      quay.io/sysdig/ntp-exporter:v2.0.4

    • PHP-FPM

      quay.io/sysdig/php-fpm-exporter:v2.3.2-ubi

      quay.io/sysdig/php-fpm-exporter:v2.3.2

    • PostgreSQL

      quay.io/sysdig/postgresql-exporter:v0.10.8-ubi

      quay.io/sysdig/postgresql-exporter:v0.10.8

    • Redis

      quay.io/sysdig/redis-exporter:v1.43.1-ubi

      quay.io/sysdig/redis-exporter:v1.43.1

July 13, 2022

Integrate AWS CloudWatch Metric Streams

Sysdig has rolled out support for AWS CloudWatch Metric Streams. Based on Kinesis Firehose, AWS CloudWatch Metric Streams is a real-time metrics aggregation and delivery tool for AWS cloud services. Sysdig Monitor now can ingest metrics directly from Kinesis Firehose, allowing you to fully integrate all your existing AWS service metrics into Sysdig Monitor. Configuring AWS CloudWatch Metric Streams to send metrics to Sysdig can either be done by using the AWS CloudFormation template available directly on the Monitor UI, by manually deploying the CloudFormation template, or by manually selecting Sysdig as an HTTP receiver through the AWS Kinesis Fire configuration.


In addition, we have also released 9 out-of-the-box dashboards and alerts for the following AWS CloudWatch Metric Streams services:
  • AWS ALB
  • AWS EBS
  • AWS ELB
  • AWS Fargate
  • AWS Lamda
  • AWS RDS
  • AWS S3
  • AWS SQS

For other services, custom dashboards and alerts can be configured for all the service metrics coming in from AWS CloudWatch Metric Streams.

For more information, see Cloud Integrations.

July 06, 2022

Live Logs

Sysdig introduces Live logs support for Kubernetes in Advisor to help you debug infrastructure problems. Advisor displays live logs for a container, which is the equivalent of running kubectl logs. This strengthens Sysdig Monitor capabilities for troubleshooting, allowing you to debug problems, such as pods in a CrashLoopBackOff state and consolidates tooling, and reducing the need to switch to other tools for troubleshooting and root cause analysis.

Live logs requires Sysdig agent v12.7.0 or above. For more information, see Live Logs.

Enhanced Alerts Editor

Sysdig introduces a new Alert Editor with an improved user experience thanks to a redesigned look and feel. We’ve also added the ability to link a dashboard and a runbook to the alert definition to expedite troubleshooting.


We are deprecating the existing Anomaly Detection and Group Outlier alert types. Previously created alerts of this type can still be viewed and edited. We will be bringing new alert types in the future.

The new Alerts Editor will be available only in environments where the new metric store is enabled. For more information, see Alerts.

PromQL Panel Enhancements

The Compare To function is now supported in Timechart and Number PromQL panels.

Monitoring Integrations

Integrations

  • Added the following integrations:
    • HaProxy
    • PHP-fpm
  • Split Kubelet PVC-and-Storage integration into two different integrations, PVC and Storage.
  • Enabled Kubelet-PVC metrics by default.
  • Updated agent jobs for kube-controller-manager and kube-scheduler to support HTTPS and authentication.
  • Added Helm chart for ElasticSearch exporter with CA certificates option.

Dashboards and Alerts

  • Added dashboard and alert templates for HAProxy
  • Changed the rules to toggle showing Kubernetes dashboards to prevent hiding when encountering unstable metrics or disconnected agents
  • Fixed waiting time in Portworx alert templates with predict linear functions
  • Fixed used request in the Cluster Capacity Planning dashboard

Exporter

  • New exporter image for PHP-FPM:
    • quay.io/sysdig/php-fpm-exporter:v2.3.0
    • quay.io/sysdig/php-fpm-exporter:v2.3.0-ubi
  • Updated the JMX exporter image
    • quay.io/sysdig/promcat-jmx-exporter:v0.17.0
    • quay.io/sysdig/promcat-jmx-exporter:v0.17.0-ubi

June 7, 2022

Enhanced Metric and Label Selection

The metric and label selectors in Dashboards and Metrics Explorer have been improved to provide easier search and find what you are looking for.

Improvements include:

  • Suggested labels now show only relevant labels for a selected metric.
  • Displays 500 labels by default for a selected metric. Previously it was 50.
  • Supports inline editing of metric and label names.
  • Provides improved search relevancy.

Dashboard enhancements

Translate Form-Query to PromQL

You no longer require advanced Prometheus knowledge to build complex PromQL queries in Sysdig Monitor. With single click, you can translate form query to PromQL, and build PromQL-based dashboards in no time. For more information, see Build PromQL Panels from Form Query.

PromQL Support for Toplist

Toplist panels support running PromQL queries.

Multi-Query Support for Stacked Area Charts

Timechart now supports visualizing multiple queries as stacked areas in the same y-axis.

With this feature, it’s easier to visualize and compare sparse metrics.

Lazy Loading of Dashboard Panels

Dashboards now supports lazy loading panels. Lazy loading greatly reduces the initial page loading time by only loading panels once they become visible on screen.

Monitoring Integrations

Integrations

  • Added the following integrations:

    • Fluentd
    • NTP
  • Improved CoreDNS Prometheus job to be detected in IKS clusters

  • Changed troubleshooting metrics in some integrations for metrics inside the filter of the Prometheus job

Dashboards and Alerts

  • Added the following templates for dashboard and alert:

    • Fluentd
    • NTP
  • Changed OOTB K8s dashboards to use “is” vs “in” scoping to improve performance.

  • Changed the following dashboards:

    • Cluster/Namespace Available Resources
    • Cluster Capacity Planning
    • Pod Rightsizing & Workload Capacity Optimization
    • Pod Scheduling Troubleshooting
    • Kubernetes HPA
  • Added the containers with limits/requests only in certain panels in the Cluster Capacity Planing dashboard

  • Limited the use of the label job to some panels in the Kubernetes CoreDNS dashboard

Exporters

  • Added support for CA files in ElasticSearch exporter Helm chart
  • Removed duplicated securityContext in ElasticSearch exporter Helm chart
  • Changed the ElasticSearch wizard and Helm chart to use secrets for URL of the ElasticSearch server
  • Bumped Helm chart repository version to include NTP exporter and fixes in Elasticsearch
  • The following Exporter images for NTP exporter have been added:
    • quay.io/repository/sysdig/ntp-exporter:v2.0.3
    • quay.io/repository/sysdig/ntp-exporter:v2.0.3-ubi
  • New version of grok exporter with security updates:
    • quay.io/sysdig/grok-exporter:v1.0.2
    • quay.io/sysdig/grok-exporter:v1.0.2-ubi

May 23, 2022

Custom Roles

A custom role is an admin-defined role that allows Sysdig administrators to bundle a set of permissions and assign those permissions to individual users or teams. Custom roles allow for finer-grained definition beyond the standard out-of-the-box Sysdig Roles. Once defined, a custom role can be assigned to any user inside a particular team, and also be configured as the default role for new users in that team. For more information, see Custom Roles.

The addition of custom roles into the platform is transparent, meaning that standard roles and assignments that already exist will not experience any changes.

May 4, 2022

Sysdig Platform Audit

We are glad to announce that Sysdig Platform now supports the capability of tracking, logging, and reporting on all changes in the system.

  • Track all activities on the API level
  • Retention period: 90 days
  • Simple API for retrieving audit information (no UI)
  • Events Forwarding support to be included in the near future (to be announced)
  • Enabled by default for all SaaS users

See Sysdig Platform Audit for more information.

Sysdig Platform Login Banner

We would like to announce that Sysdig Monitor and Secure now allow you to define a Login Message that will be presented to all users. Added to boost Sysdig compliance/enterprise readiness, requested originally by the IRS.

  • Users are not allowed to access the system until they acknowledge the message
  • One login banner per account
  • Only Admin users can enable/update the message
  • Single banner for both Monitor and Secure (for Platform customers)
  • Available on SaaS for all users

See Configure Login Message for more information.

April 13, 2022

Advisor

Advisor brings your metrics, alerts, and events into a focused and curated view to help you operate and troubleshoot Kubernetes infrastructure. To help you solve problems faster, over time, Advisor will surface your infrastructure issues that you should pay attention to. For more information, see Advisor.

Metrics Explorer

Metrics Explorer has been rebuilt from the ground up to focus on advanced metric exploration and querying.


Improvements to Metrics Explorer include:
  • Simple querying that builds PromQL queries under the hood. Metrics Explorer is the easiest way to build PromQL queries.
  • Graph multiple metrics at once for correlation. For example, CPU usage vs Kubernetes limits.
  • Queries are ungrouped by default, showing the individual time series for a metric. This allows you to spot any problems faster. For example, 1 of 50 Cassandra nodes with high pending compactions. Instead of segmenting, you now group by one or more labels, for example, workload, pod, and container.
  • When selecting a scope in the tree, only those metrics that are applicable to that entity are displayed.
  • Metrics are now more logically categorized by metric namespace (prefix).
  • Resolution has been improved. For example a 1-hour view now shows 10-seconds data. Additionally, the concept of time re-alignment has been removed.

For more information, see Explorer.

February 10, 2022

Improved Usability with New Navigation

The Sysdig Monitor UI has been enhanced to provide you with a smoother and smarter left-hand navigation experience.

Check out a video walk-through of the new feature!

  • Collapsible main menu: Allows you to toggle the visibility of menu options. The collapsible left-hand navigation prevents long lists from displaying by default and gives you a clear structure that is easy to scan and locate.

  • Hoverable sub-menu: With each module that has additional menu options, hover over the respective module to quickly navigate.

New Menu Option for Integrations

A dedicated Integrations menu option provides an easy way to access both inbound and outbound integrations with Sysdig.

  • Inbound: Access Monitoring Integrations quickly and understand which applications and services are running. You can also manage your AWS Account and review the Sysdig agent installation.
  • Outbound: Manage the Notification Channels and S3 Capture Storage.

Revamped User Menu

The User menu provides the following:

  • Option to efficiently switch between Sysdig Teams.
  • Access Management to the Administrator.
  • Sysdig API Tokens to the authenticated user.
  • Documentation and What’s new links

The Settings sub-menu link is provided to review all the available options for the current user.

January 26, 2022

Support for PVC Metrics

Contact your Sysdig representative or Sysdig Support to enable PVC metrics in your environment.

With Sysdig agent v12.2.0 or above installed in your monitoring environment, Sysdig Monitor can help you surveil your Kubernetes PV/PVCs objects. Use the PVC dashboard and alert templates to get an insight into your PV usage, such as disk usage, inodes, storage latency, errors, and so on.

For more information, see Configure PVC Metrics.

New KSM Troubleshooting Metrics

Sysdig provides the following new troubleshooting metrics:

  • kube_workload_pod_status_phase
  • kube_workload_pod_status_reason
  • kube_pod_status_unschedulable
  • kube_pod_container_status_waiting
  • kube_pod_container_status_waiting_reason
  • kube_pod_container_status_terminated
  • kube_pod_container_status_terminated_reason

These metrics give insights into why pods are stuck or crashing (CrashLoopBackOff, OOMKilled, DeadlineExceeded etc.). To support this:

  • The Kubernetes Alerts Library has been updated to provide additional alerts for errors such as CrashLoopBackOff.

  • New panels has been added to the Kubernetes Workload Status & Performance dashboard.

In environments running older versions of Sysdig agent, the Kubernetes Dashboards will display a banner prompting you to upgrade to agent v12.2.0 or above for these metrics to be automatically collected.

origin/main

December 17, 2021

Update on Log4j Vulnerability (CVE-2021-44228)

Sysdig confirms that all services that compose Sysdig’s Cloud Platform running Apache’s vulnerable Log4j library have been patched to 2.16. We have not detected any successful attempts at exploitation of this attack vector during that time window.

December 15, 2021

Update on Log4j Vulnerability (CVE-2021-44228)

The sysdig agent does not include the Log4j library

Sysdig is using an alternative framework for logging, called Logback. The logback framework isn’t vulnerable to this issue.

Sysdig components include a log4j library in our standard distribution that was vulnerable. This library is included for compatibility reasons only, is not used for primary logging, and our security team has determined we are not vulnerable based on our application architecture and existing mitigating controls.

Sysdig can confirm that all services that compose Sysdig’s Cloud Platform running Apache’s vulnerable Log4j library have been patched to the latest version or adds additional mitigating controls suggested by vendors. We have not detected any successful attempts at exploitation of this attack vector during that time window.

Details regarding upgrades

We:

  • explicitly set commonsLog4jVersion = 2.15.0
  • update all of log4j-to-slf4j, log4j-api, and log4j-core to version 2.15.0

December 12, 2021

A Statement on Log4j Vulnerability (CVE-2021-44228)

Security researchers recently disclosed the vulnerability CVE-2021-44228 in Apache’s log4j, which is a common Java-based library used for logging purposes

Sysdig is using an alternative framework for logging called Logback. The logback framework isn’t vulnerable to this issue.

Sysdig components include a log4j library in our standard distribution that appears to be vulnerable. It has been confirmed that this library is included for compatibility reasons only and is not used for primary logging. As a result this should not pose any risks.

Patches will be provided to upgrade the log4j libraries that are included for compatibility reasons.

If you have any questions or concerns, please reach out to your Sysdig contact.

October 12, 2021

Expose Custom Data on Webhook Notification

Sysdig gives you the ability to specify custom data and attach it to the alert notification. For more information, see Configure a Webhook Channel.

Prometheus Recording Rules

Sysdig now supports Prometheus recording rules for metric aggregation and querying. To enable this feature in your environment, contact Sysdig Support.

For more information, see Recording Rules.

Team Scope for Prometheus Remote Write

Sysdig gives you the ability to determine the granularity of data collected by Prometheus Remote Write to which team members will have the visibility. You can specify what data team members can see by specifying tag/value expressions for the metrics. The drop-down defaults to “is”, but can be changed to “is not”, “in”, “contains”, and so on. Complex policies can be created by clicking drop-down to create AND chains of several expressions.

You can view the saved team scope by hovering on the corresponding team from the User menu.

Enhanced User Experience for Monitor Integrations

The UI for Monitoring Integrations has been enhanced to include guided configuration for exporters.

  • For each integration, you can use the wizard to specify the required information and you will be provided with a single-line command to install the exporter in your cluster. You no longer have to see the documentation or the given exporter source code to guess the name of the variable to configure the credentials of your database or the SSL certificate in the connection string.

  • If you want to deploy it through your CI/CD pipeline and you cannot run commands directly in production, you also have the option to generate the manifests to upload to your repository. If you prefer a package management approach, you also have the option to use Helm charts for the Monitoring Integrations.

  • After deploying an exporter, you can see whether it is working on the wizard. Sysdig Monitor automatically detects the metrics arriving in your account and associates them to the workload. This way, it is easy to visually detect the applications that are correctly reporting metrics and those that need some attention.

For more information, see Configure Monitoring Integrations.

Dashboard Enhancements

  • Ability to edit dashboard and panel name inline in the Panel Editor.

  • Ability to add dashboard template to favorites.

  • Moved the legacy dashboard templates to Deprecated section
  • Supports RabbitMQ Integration. Configure it using the Monitoring Integrations.
  • Added new dashboard templates for the following:
    • Fargate Usage
    • Go applications
    • Sysdig Admission Controller
    • RabbitMQ Integrations
    • Kubernetes Controller Manager
    • Kubernetes Scheduler
    • CoreDNS

Alert Enhancements

  • Added new alerts for RabbitMQ and CoreDNS integrations and for Go applications.

August 10, 2021

Monitoring Integrations

Sysdig discovers the services running in your environment and gives you visibility into deeper application performance and health telemetry by configuring a managed Monitoring Integration through PromCat. You can easily view which services you can configure an integration for, check the status of existing integrations, and leverage curated content in the alerts library and out-of-the-box dashboards.

See (Limited Availability) Configure Monitoring Integrations for more details.

Alerts Library

The Alerts Library in Sysdig Monitor gives you a recommended list of alerts to configure based on the services running in your infrastructure. The curated content from the Sysdig removes the need for guessing which alerts to configure and takes you from zero to full monitoring coverage faster.

For more information, see Alerts Library.

Alert Enhancements

The usability of the Alert page has been enhanced to include:

  • Ability to create and edit alert groups based on the service that they are representing. The alerts created from alert templates will have groups automatically assigned to them.

  • Efficient visual cues to see alert activities and identity the alerts that are not resolved. A bell icon next to an alert indicates that it has not been resolved. Alerts that are active over the past two weeks will have an event chart under the Activities Over Last Two Weeks column and an event feed on the alert details slider.

For more information, see Manage Alerts.

Enhanced Kubernetes Dashboards

We have introduced several improvements to the out-of-the-box Kubernetes dashboards:

  • Workload dashboards are refreshed with relevant status and golden signals.

  • Improved UX with panel location and color code.

  • Some workflows are simplified to make it easier for beginners in Kubernetes

  • Improved capacity planning capabilities.

  • Text boxes are easier to read and locate near the relevant panels.

July 19, 2021

Customized Session Expiration

Session expiration is the amount of time a user can remain idle before the session is automatically ended or expired. After the session expires, the user must log in to the Sysdig application again.

Sysdig now gives you the ability to make a shorter or longer idle session expiration for Sysdig applications. When a user browser is idle for a certain period of time, they will get automatically logged out. For more information, see Configure Customized Session Expiration.

Enhanced Session Logout

To offer superior user security, the logout procedure has been enhanced. When the users log out of a Sysdig application, they will be automatically be logged out of both Monitor and Secure applications.

June 01, 2021

PromQL Library

We have compiled a list of PromQL queries to give you one-click insights into the health and performance of your infrastructure. The library also includes a PromQL 101 category to give you hands-on exposure to PromQL. For more information, see PromQL Library.

Prometheus Remote Write

Sysdig supports ingesting metrics from Prometheus servers by using remote_write capabilities. In Sysdig terminology, the remote endpoints that can read Prometheus metrics are known as Prometheus Remote Write. Prometheus Remote Write does not require the Sysdig agent to be installed in the Prometheus environment. This facility expands Sysdig monitoring capability beyond Kubernetes and regular Linux kernels to environments where the Sysdig agent cannot be installed.

For more information, see Prometheus Remote Write.

Dark Mode

The dark appearance, known as Dark Mode, is available in Sysdig applications.

Sysdig can now automatically match your OS preferences. Available in Sysdig platform on-premises, or in SaaS in the US East and rolling out globally. For more information, see Configure Theme Preference.

Improved Dashboard Templates

The following Dashboard templates have been enhanced to display the data better, return improved results, and add golden signals.

  • Kubernetes

  • Application

    • Ngnix

    • Ceph

    • Ngnix Ingress

    • ElasticSearch

    • Redis

May 10, 2021

Silencing Alert Notifications

Sysdig Monitor allows you to silence alert notifications for a given scope for a predefined amount of time, and schedule silence in advance. When silenced, the alert will still be triggered and posted on the Events feed and in the graph overlays but will indicate it has been silenced. The types of notification channels you can use are Email, Slack, and Amazon SNS.

You will be notified 30 minutes before the start time and 30 minutes before the end time of a silence window. You will also be able to easily extend or end an active silence. To access the feature, navigate to Alerts > Silence on the Monitor UI.

For more information, see Silence Alert Notifications.

Workload Label

Sysdig Monitor now supports two new labels, kubernetes.workload.name and kubernetes.workload.type which can be used for scoping Dashboards and configuring Gropings.

Earlier, each type of object (deployment, replicaset, statefulset, etc.) was unique, and in turn, you needed to use different types of Kubernetes Dashboards and a different Grouping resulting in n/a , where distinct types of Kubernetes objects are listed.

For more information, see Unified Workload Labels.

New Kubernetes Dashboards

Available Resources Calculator

Ensure there is sufficient capacity in a cluster to deploy a new application.

Application Status&Overview

Understand the status of applications (workloads) running in a cluster by monitoring performance, pod health, and resource usage

Cluster Capacity Planning

Monitor the capacity of Kubernetes clusters ensuring they’re correctly sized to support new applications when they’re deployed.

Container Resource Usage&Troubleshooting

Understand the performance of the different containers running in pods across your infrastructure and identify any that are behaving anomalously.

Node Status&Overview

Monitor the health, resource usage, and network statistics for nodes running in clusters

Pod Rightsizing&Capacity Optimization

Optimize your infrastructure and better control cluster spend by ensuring pods are sized correctly. Understand if you can free up resources by reducing memory and/or CPU requests.

Pod Scheduling Troubleshooting

If a pod cannot be scheduled due to insufficient resources, use this dashboard to identify where the resource bottleneck is.

Pod Status&Overview

Monitor the health, resource usage, and network statistics for pods running as part of workloads.

April 26, 2021

Extended Label Set

Running PromQL queries is now smoother and faster with the extended label set. The extended label set is created by augmenting the incoming data with the rich metadata associated with your infrastructure and making it available in PromQL. You now no longer have to write complex queries in order to troubleshooting infrastructure issues or building dashboards and alerts. For more information, see Run PromQL Queries Faster with Extended Label Set.

Microsoft Team Channel

You can now use Microsoft Team s as a notification channel in Sysdig Monitor. See Configure a Microsoft Teams Channel for more details.

S3-Compatible Storage for Capture Files

Configuring S3-compatible storage, such as Minio or IBM Cloud Object Storage, for your Sysdig captures is now supported on Sysdig Monitor. The capability can be turned on by configuring the system appropriately, as given in (SaaS) Configure Custom S3 Storage Endpoint.

Webhook Channel Enhancements

Sysdig supports the following on a Webhook channel integration:

  • Insecure connections: You now have the ability to skip the TLS verification.

  • Custom headers: If your Webhook integrations require additional headers or data you can append to the alert format by using a custom header on the UI. This option is in addition to the existing API facility to add custom headers programmatically.

View LogDNA Alerts as Sysdig Events

If your environment has both LogDNA and Sysdig, you can view relevant LogDNA Alerts as Events in Sysdig. These Sysdig Events behave like any other type of Events in Sysdig They will be overlaid on Sysdig graphs, listed in the Event Feed, and can be used to create an Alert in the Sysdig Platform. The link provided in the Event Details redirects you to the LogDNA Platform, in case further investigation is needed. For more information, see LogDNA Events.

March 03, 2021

PromQL Query Explorer

PromQL Explorer helps you understand metrics and their labels and values, and create queries faster before using them in Dashboards and Alerts.

PromQL can be used not only with metrics collected from Prometheus endpoints but also with Sysdig native metrics collected by the agent. For more information, see PromQL Query Explorer.

IBM Cloud Functions

You can now use IBM Cloud Functions as a notification channel in Sysdig Monitor. See Configure IBM Cloud Functions Channel for more details.

SAML Single Logout

Sysdig supports SAML Single Logout. This feature enables you to configure automatic logout from the Identity Provider when users log out of Sysdig. This feature is currently available for SaaS regions US-West and EU-Central. For more information, see Configure SAML Single Logout.

Enhanced Dashboard Scope Session

When returning to a previously visited Dashboard the UI retains your last used scope.

February 05, 2021

Import Prometheus Alert Rules

You have now the ability to import Prometheus alert rules into Sysdig Monitor. The ease of YAML import makes it significantly convenient to tap into Prometheus ecosystem resources, such as promcat.io.

For more information, see Import Prometheus Alert Rules.

UX Improvements

Sysdig Monitor interface has been enhanced to provide the following capabilities:

  • Edit dashboard scopes in a panel editor.

  • Set a dashboard template as the team entry point.

January 05, 2021

Improved Alerts

The Alert interface has been improved to allow faster browsing and easier management. For more information, see Alerts.

December 16, 2020

Statement RE: Solarwinds and Sysdig’s Security

We have seen requests for statements regarding tooling in the wake of the Solarwinds and related compromises. Sysdig does not use these tools internally. To maintain a secure SDLC process for own product we use Sysdig Secure as well as source code analysis tools. We also maintain our own branch of key OSS components to ensure software is fully vetted before it’s delivered to customers.

November 19, 2020

Explore Workflow Enhancements

The Explore interface has been improved to allow faster troubleshooting.

You are now launched directly into the drill-down view when you navigate to Explore. You will still be able to group and navigate your infrastructure by using the hierarchical scope tree.

The new Grouping editor helps you create and manage your infrastructure groupings.

For more information, see Explore.

Transfer Dashboard Ownership

Administrators have now the ability to transfer dashboard ownership to another user. For more information, see Transfer Dashboard Ownership.

Enhancements for Navigating Dashboards

You can now pin the dashboard menu to the sidebar in the Sysdig Monitor UI. Pinning makes it easier to navigate and browse different Dashboards. In addition, the Dashboard interface has been enhanced to retain your preference for open or closed categories to help you locate the desired items faster.

October 22, 2020

Visualizing Missing Data on Dashboards

Dashboards now show null or missing data values as gaps instead of zero. Optionally, missing data can be displayed as a dotted or solid line in both Form-based and PromQL panels. StatsD metrics will continue to show null values as zero unless overridden by the settings. For more information, see Display Missing Data.

Time Navigation in Events Feed

You can now browse and find historic events easily by using time navigation.

Zooming Out Dashboards

You now have the ability to zoom out Dashboards. This feature doubles the selected timeframe by 2x for a better context surrounding a problem when troubleshooting an incident.

July 27, 2020

Sysdig Essentials

We have introduced a new product tier, Sysdig Essentials. This tier includes everything required to achieve the five essential requirements for practicing Secure DevOps:

  • Image Scanning

  • Runtime Security

  • Compliance

  • Kubernetes and container monitoring

  • Application and cloud service monitoring

To learn more about Essentials, register for our webinar, Deploy Faster by Automating Container Security, Monitoring and Compliance.

With the introduction of Essentials, It’s also easier to get started with a trial program and manage your Sysdig subscription.

Learn the difference between Essentials and Enterprise, including pricing and features, at Pricing.

Rebranded Login Page

The login page has been updated with the Sysdig Kraken and the new logo.

Sysdig Monitor Enhancements

Hosts Overview

To complement Sysdig Kubernetes Overviews, Hosts Overview has been released. Host Overview provides a unified view of the performance and health of physical hosts in your infrastructure.

New and Improved Empty States

A number of different splash screens have been introduced to guide you through getting up and running with features across the application.

Sysdig Platform Enhancements

SAML Single Sign-On

The initial email to the following types of users will take them directly to the Single-Sign-On URL, and not the registration page.

  • SAML SSO Users

  • The users that are invited to the platform (as opposed to having them automatically created via Sysdig on-demand provisioning for SSO)

Earlier, landing on the registration page was confusing to users because they had to set up their initial password.

June 17, 2020

This 3.2.6 release focuses on the general availability of New Dashboard with a rich set of features and enhancements. Learn more about the release from the blog post, New and improved dashboards .

New Dashboards is GA

Sysdig Monitor offers a new version of dashboards. Its improved editing experience provides you with more flexibility and the new set of functionalities offers additional ways to visualize and consume your Sysdig data.

Features and Enhancements

Improved User Experience

The New Dashboard offers a more fluid, natural dashboard building experience. The UI has been redesigned to introduce two types of panels—form-based and PromQL-based— to make visualizing your metrics effortless. Use a PromQL-based panel to build dashboards for Prometheus raw metrics and custom metrics. The form-based panel for legacy queries. For more information, see About the Dashboard UI.

Dashboard Sharing

You can now share your dashboard with members within your Sysdig team or share it across teams with fine-grained access controls. Define who should be able to see the dashboards and what level of access they should be granted: view only or collaborator with edit privileges. For more information, see Sharing New Dashboards

Time Series Name Templating

Customize the time series names on the legend on the panel editor by using the labels associated with Prometheus metrics and segments to gain context faster. For more information, see Create a New Panel.

Multi-Metric, Multi-Segmentation Options

Configure multiple queries within a single panel, and configure each query with multiple segmentation and scoping options. Individual queries can be customized to render as a line or stacked area. For more information, see Using PromQL.

Event Overlay

Contextualize metrics and understand the “why” faster with a unified view of both metrics and events. Configure event overlay to display events from Kubernetes environments as well as alert events, and any other events ingested using Sysdig’s open REST API. For more information, see Display Dashboard Specific Events.

Dashboard Templates

You can quickly view your infrastructure through the lens of one of Sysdig’s curated dashboards, or use it as a base to start building your own. You can find dashboard templates for managing Kubernetes capacity and health, hosts and server performance, applications and services telemetry, and the security posture of your infrastructure with data fed from Sysdig Secure. See Dashboard Templates to learn more.

Mapping Values to Text

Instantly understand what’s going on by mapping number panel values to text. If you have a metric that returns 1 for up, and 0 for down, map those values to “UP” and “DOWN” respectively. By defining thresholds and mapping to text, you don’t need to be concerned about the values. This is critically valuable when dashboards are shared between team members. For more information, see Text.

Granular Axes and Legend Controls

You have more flexibility when customizing the axes, as well as better support for time series with long names. You can now configure the legend by toggling its visibility and moving it to the bottom of the panel. See About the Dashboard UI.

Major Changes

Significant changes have been introduced to enhance the usability of the existing functionalities. Review the changes before you explore the functionalities.

Topology Maps

Topology maps are no longer available in Dashboard. Access Topology maps through Explore, as you explore your microservices and Kubernetes applications.

Dashboard Wizard

My Dashboards are no longer accessible in Explore. Additionally, Dashboard Wizard has been removed. Instead, the concept of Templates has been introduced in Dashboards to help you get started with a library of templates addressing key use cases.

Histogram and Summary Metric Type

Histogram and summary metrics are no longer supported in the Histogram panel type. You can continue to use them within Explore. If you have enabled PromQL, we encourage you to use Prometheus functions for visualizing histograms.

Use the new Prometheus histograms with the histogram_quantile metrics on a time-series graph.

APIs and Integrations

API endpoints for the legacy dashboards (v2) will soon be deprecated. If you are directly integrating into the API, please contact Sysdig for guidance. Additionally, our Python SDK and CLI have been updated to support the new dashboards APIs.

PromQL Support

PromQL support for querying Prometheus metrics has been rolled out to a subset of Sysdig Monitor users. See Using PromQL.

Intelligent $__interval

Use $__interval within a PromQL query and Sysdig will intelligently use the most appropriate sampling depending on the time range you have selected. This configuration ensures that we balance providing access to the most granular data available while downsampling when you select a long time range to panels load as fast as possible.

Scope variables

Configure scope variables at the dashboard level to quickly filter metrics based on cluster, namespace, workload, and more. When using PromQL queries, the scope can be injected by using dynamic variables. This configuration is significant when troubleshooting as it allows you to switch context quickly without reconfiguring queries.

Smart Autocompletion and Syntax Highlighting

Autocomplete suggests metrics, operators, and functions, while syntax highlighting helps keep you on the right path and helps highlight problems within a PromQL query. This is invaluable in dynamic environments and allows you to craft the right queries faster.

Configurable Default Team Role

You can now define the default user role to apply when a new member is added to the team. The Admin can change this default on a per-team basis. See also: Create a Team.

RBAC and Team Assignment for Notification Channels

Previously, notification channels in Sysdig Secure and Monitor were treated as global entities, visible and editable for most users of the platform regardless of team configurations.

We are enhancing the management and RBAC controls in the following ways:

  • Notification channels can now be “global” or limited to a particular team

  • Global channels can be managed by admins and can be viewed/used by other roles, while team-limited channels are available only to team members

  • Team Manager , Advanced User, and Service Manager (Secure) roles can create/update/delete team-scoped notification channels, they can also read and use the global ones

  • Standard and View Only roles can read team-limited and global notification channels

  • Admins will be able to create global notification channels and migrate channels from “global” to “team-limited”, and also from one team to another.

See also: Set Up Notification Channels and the Share With field in each individual channel setting page.

May 15, 2020

The New Get Started Page

The Get Started page provides the key steps to ensure that you are getting the most value out of Sysdig Monitor. We’ll update this page with new steps as we add new features to Sysdig Monitor.

The Get Started page also serves as a linking page for:

  • Documentation

  • Release Notes

  • The Sysdig Blog

  • Self-Paced Training

  • Support

You can access the page at any time by clicking the rocketship icon in the left navigation bar. See Getting Started with Sysdig Monitor.

AWS Role Delegation

Sysdig Monitor can now utilize the Amazon Web Service (AWS) AssumeRole functionality and discover cloud assets, grab CloudWatch metrics from your AWS account, and use custom S3 bucket for storing captures. Upon integrating with an AWS role, you can delegate access to AWS resources that are not associated with your Sysdig AWS account.

Role delegation is an alternative to the existing integration method using the access keys. This method is considered secure as sharing developer access keys with third-parties is not recommended by Amazon.

For more information, see Integrate with AWS Role Delegation.

April 16, 2020

Default Dashboards for Istio 1.5

Default dashboards (Overview and Services dashboards) are now available for Istio v1.5 in addition to the existing ones for Istio v1.0.

November 21, 2019

Overview Is GA

Overview is now generally available. Overview leverages Sysdig’s unified Kubernetes data platform to monitor, secure, and troubleshoot your Kubernetes clusters and workloads.

Cluster Overview

Major highlights of Overview GA include but are not limited to:

  • Multi-cloud view of the health, risk, and capacity of your Kubernetes infrastructure— a single pane of glass for Kubernetes Clusters, Nodes, Namespaces, and Workloads across a multi- and hybrid-cloud environment. You can easily filter by any of these entities and view associated events and health data. View the infrastructure organized by Clusters, Nodes, Workloads

  • Shows metrics prioritized by event count and severity, allowing you to get to the root cause of the problem faster.

  • Drill down to Dashboards for instant insights.

To learn about the capabilities of the Overview feature, see Overview.

Beta Features: Prometheus and New Dashboards

Introducing Prometheus and New Dashboards available in Beta. Contact sales@sysdig.com to join the Beta Program.

[BETA] Prometheus Capabilities

Sysdig now supports native Prometheus time series ingestion. Run Prometheus queries inside Sysdig Monitor and create visualization by using the new Beta Dashboards that support it. This enables you to use Sysdig Monitor as a standard Prometheus data source for other visualization tools, such as Grafana. For more information, see Using PromQL.

PromQL Dashboard

With this support, Prometheus and Sysdig metrics can now be supported in regular Prometheus expressions.

[BETA] New Dashboards

Sysdig Monitor provides an enhanced New Dashboard to use with Prometheus. For more information, see Dashboards.

New Dashboards

The New Dashboards offer:

  • Flexibility to position the Legend.

  • Ability to run multiple queries.

  • Inherit the Dashboard scope to individual panels.

  • Multi-select items in the Legend to narrow down the lines you want to focus on. Use command-click on Mac and Control-click on non-Mac machines.

  • Features new query types: Form-based and PromQL expressions with the easy toggling facility.

  • Enhanced auto-layout with the ability to re-position panels.

To access the New Dashboards:

  1. Click the Dashboards tab on the left navigation panel.

  2. Click Add Dashboard (+)

  3. Click Beta Dashboards.

Enhanced Out-of-the-box Dashboards

In an attempt to improve the Dashboards experience, the following changes have been introduced:

The following Dashboards are added:

  • Kubernetes Cluster Overview: Provides nodes and workloads availability and highlights the high-level health of your Clusters. It also summarizes resources consumption (CPU, memory) across Nodes and Namespaces to pinpoint possible anomalies and node disk utilization

  • Kubernetes Node Overview: Provides availability of the Nodes, indicating potential issues reported by Kubernetes; a summary of resource (CPU and Memory) allocation and utilization, as well as Network and Disk utilization.

  • Kubernetes Namespace Overview: Provides a high-level summary of availability, and resource allocation and utilization across all the Workloads in the selected Namespace.

  • Kubernetes Deployment Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods for each Workload.

  • Kubernetes StatefulSet Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods for each StatefulSet.

  • Kubernetes DaemonSet Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods.

  • Kubernetes Job Overview: Provides a detailed summary of job status, completion trend, pod restarts, as well as resource allocation and utilization across pods.

  • Kubernetes ReplicaSet Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods for each ReplicaSet.

  • Kubernetes Pod Overview: Provides a detailed summary of pod status, pod restarts, and resource allocation and utilization in a selected pod.

  • Kubernetes Workloads CPU Usage and Allocation: Helps you verify that CPU requests are properly configured and actual utilization is expected.

  • Kubernetes Workloads Memory Usage and Allocation: Helps you verify that memory requests are properly configured and actual utilization is expected.

  • Kubernetes CPU Allocation Optimization: Helps you verify that infrastructure resources are available for future needs and are not wasted.

  • Kubernetes Memory Allocation Optimization: Helps you verify that infrastructure resources are available for future needs and are not wasted.

The following Dashboards are retained:

  • Health Overview (applicable to all the objects in the environment)

  • Horizontal Pod Autoscaler (the default Dashboard when selecting an HPA)

  • Resource Quota

  • Service Health (the default dashboard when selecting a service)

  • Cluster and Node Capacity

The following Dashboards are removed:

  • State Overview

  • Daemonset State

  • Namespace State

  • Stateful State

  • Nodes State

  • Deployment State

  • Deployment Health

  • Nodes Health

  • Namespace Health

  • Pod State

  • Pod Health

  • Replica Set Health

For more information, see Dashboard Templates.

What’s n/a?

The Sysdig Monitor UI displays n/a in several scenarios associated with labeling. The Explore UI has now been enhanced to add a tooltip for n/a to help you understand the scenario. See The Meaning of n/a for more information.

Filtering Events by Scope

Events are now filtered by Scope to show the most relevant Events in Explore and Dashboards. This is an extension of the existing Event Scope functionality. You can toggle between showing Event feed from the entire infrastructure and only from the particular scope you are interested in within the infrastructure. Event scoping for Dashboards and Explore is enabled by default.

By default, Events are filtered to show only the relevant ones. However, you can turn the filtering off and see Events from the complete scope. To do so:

  1. Click the Dashboard Settings (three dots) icon and select Events.

  2. Use the toggle button to turn off Filter events by dashboard Scope.

  3. Click Save.

Similarly, you can filter Events by Scope in Explore.

Known Issues

  • Time Chart may encounter some response time delays

  • Not all the functionality from the existing dashboards will be available in the new dashboards. The following functionalities are not yet fully functional or not yet available:

    • Gauge chart

    • Text Panel

    • Top Chart

    • Table

October 11, 2019

Ability to “Favorite” a Dashboard

Users can click the star icon to mark a “Favorite” dashboard, which will then be listed under “My Favorites” in the Dashboard view.

Enhancement: Additional Metrics Segmentation

This change enables Sysdig Monitor to segment metrics file.bytes.in and file.bytes.out by file.mount and file.name.

Enhancement: New Documentation Site at docs.sysdig.com

Sysdig’s documentation platform has been upgraded and moved to docs.sysdig.com.

Improvements include:

  • Look and feel: Updated to match the rest of the Sysdig branding

  • Search: Enhanced search speed, accuracy, and ease

  • Structure and content: Enhancements to content have been added and are being continuously updated

  • Feedback: Buttons on each page enable users to communicate directly with the documentation team.

August 14, 2019

New Default Kubernetes Grouping

Groupings for Kubernetes have been modified. This updated Grouping is available to new teams. Default groupings are immutable–-they cannot be modified or deleted other than by copying. Modifying a copy is allowed.

New Groupings:

  • Clusters and Nodes (cluster.name > node.name > pod.name > container.name)

  • Deployments (cluster.name > namespace.name > deployment.name > pod.name > container.name)

  • Services ( cluster.name > namespace.name > service.name > pod.name > container.name)

  • Statefulsets (cluster.name > namespace.name > statefulset.name > pod.name > container.name)

  • Daemonsets (cluster.name > namespace.name > daemonset.name > pod.name > container.name)

  • ReplicaSets (cluster.name > namespace.name > deployment.name > replicaset.name > pod.name)

  • HPAs (cluster.name > namespace.name > hpa.name > pod.name > container.name)

For more information, see Grouping, Scoping, and Segmenting Metrics.

Enhanced Event Notification

The ability to customize the subject and body of alert notifications with variables has been extended to Event notifications. Event titles and notification messages are in sync in the following cases:

  • Event feed on the Events page

  • Event overlay on Dashboards page

For more information, see Events.

Units for Metrics

The format of metric units are the same for the following:

  • The CPU and Memory metrics for Host and Container.

  • Kube-state CPU and Memory metrics.

Introducing the same format now makes the comparison of those metrics easier on a chart.

Container Segmentation

Sysdig now supports segmenting all net.* metrics at container or pod level by low level net.* dimensions, such as net.http.url or net.http.status.code. Container-based teams now display segmentations for net.http.* metrics as expected. The net.http.url and net.http.status.codes are displayed if you select a container-based team as it does for a host-based team for the same cluster.

Display Instance Name

Instance name in the Sysdig Monitor UI is now visible during creating and editing it. Instance names are displayed right below the username in the user dialog for switching teams.

Default Dashboard for Cluster and Node Capacity

Kubernetes Cluster and Node Capacity Dashboard has been refreshed to add actual usage of CPU and Memory compared to Requests, Limits and Allocatable capacity.

Aggregation for Kubernetes Nodes Health

Aggregation method has been refreshed for Kubernetes Node metrics. The Kubernetes Node Health dashboard has been updated with metric aggregations that are ‘summed’ across all containers running on the node to reflect accurate node level data.

July 11, 2019

Enhanced Dashboard Menu

The Dashboard menu features a drawer-style popover that displays a list of Dashboards you own and those shared by your team. With the popover menu, you can add new Dashboards and search for existing ones. Click a Dashboard name to access the relevant Dashboard page where you can continue with the regular Dashboard settings.

Customize Alert Notification Template

Sysdig Monitor alerts now provide an option to customize the messages that are sent with alert notifications in email and other channels, such as Pagerduty and Webhook.

Use the Alert Editor to input dynamic variables, such as hostname, or a hyperlink, and to add custom messages in plain text to the notifications for intended recipients. You can modify both the subject and the body of the alert notification with a hyperlink or a variable. For example, you can add an agent id or a link to a Dashboard to the message. This can help provide context for troubleshooting the errors that triggered the alert.

For more information, see Customizing Alert Notification.

Prometheus Remote Scraping

Sysdig Monitor can now collect Prometheus metrics from remote endpoints with minimal configuration.

Remote endpoints (remote hosts) refer to hosts where the Sysdig agent cannot be deployed, e.g., a Kubernetes master node on managed Kubernetes services such as GKE and EKS, where user workload cannot be deployed. To enable remote scraping on such hosts, simply identify an agent to perform the scraping and declare the endpoint configurations in the agent configuration file.

The collected Prometheus metrics are reported under and associated with the agent that performed the scraping, rather than with a process.

Enhancements to Kafka AppCheck

Kafka integrations can now support authentication and SSL/TLS. If authentication or SSL/TLS are enabled in Kafka, see Apache Kafka Example 5 for how to enable configuration details on the Sysdig side.

Two New Metrics for Accurate Pod Counts

Two new Kubernetes metrics, kubernetes.namespace.pod.desired.count and kubernetes.namespace.pod.available.count, have been added at the Namespace level to track desired and available pod counts.

For earlier release notes, please see Sysdig Monitor SaaS Release Notes, here.

4 - SaaS: Sysdig Secure Release Notes

You may also want to review the update log for Falco rules used in the Policy Editor: Falco Rules Changelog.

Dates shown are for the initial release of a feature. The feature may not be be rolled out to all regions concurrently and availability of a feature in a particular region will depend on scheduling.

Supported Web Browsers

Sysdig supports, tests, and verifies the latest versions of Chrome and Firefox.

Other browsers may also work, but are not tested in the same way.

September 20, 2022

Disable a Rule within a Policy

Starting today, customers can disable (and re-enable) individual rules within threat detection policies. This allows:

  • Using a subset of rules within a managed policy or managed ruleset without giving up the ability to receive new rule updates.
  • Temporarily disabling a noisy rule until the cause is investigated or an appropriate exception is put in place.

September 19, 2022

Actionable Compliance - Control Library Preview Released

Sysdig is pleased to announce the Preview release of CSPM Control Library in Actionable Compliance.

This is a technical preview release and the feature is open for all customers.This feature includes:

  • Visibility of all available controls
  • Filter for specific controls by control attributes

Read more about the feature here.

August 29, 2022

Actionable Compliance - Custom Policies Preview Released

Sysdig is pleased to announce the Preview release of CSPM Custom Policies in Actionable Compliance.

This is a technical preview release and the feature is open for all customers.

This feature includes:

  • Clone an existing policy and edit its metadata
  • Create, Edit & Delete a custom policy
  • Create, Edit & Delete requirements in a custom policy
  • Link & Unlink available controls to policy requirements

You can read more about the feature in Sysdig’s documentation.

Coming soon in Actionable Compliance:

  • Control Library
  • Creating your own custom control in a custom policy

August 17, 2022

New Permission for Changing Team Roles

Team management has been improved with the addition of the new permission, Team Membership Roles. This new permission will allow you to change the roles of team members separately while adding users to the teams.

For more information, see:

August 10, 2022

Machine Learning Policies

A new machine-learning-based detection capability is available in Sysdig Secure.

While we strongly believe in our Falco-based rule approach, and do not consider machine learning to be the best way to detect every threat, we understand that specific use cases such as Cryptominer detections require a different approach. This is the first detection capability available in our Machine Learning policies. Read more about how to configure them and how they work here.

Read more in our dedicated press release.

August 4, 2022

Agent Overview Page Released in Data Sources (Preview)

An Agents overview page in the Data Sources |Integrations interface has been made available in Technology Preview for all customers. This new page shows all of the Sysdig Agents that have reported into the Sysdig backend, and enables the user to quickly determine:

  • Which agents are up-to-date, out of date, or approaching being out of date
  • Which managed clusters have been detected in your cloud environment, but have not yet been instrumented with the Sysdig agent

The feature will remain in Technology Preview, as we add additional functionality and refine the workflows within the page.

See also: Data Sources | Sysdig Agents

Actionable Compliance - CSPM Policies Preview Released

Sysdig is pleased to announce the Preview release of CSPM Policies in Actionable Compliance. This is a technical preview release, and the feature is open for all customers.

This feature includes:

  • See what is being evaluated by the Actionable Compliance feature in the context of compliance standards (CIS, NIST, etc.)
  • Review the policy structure and the controls connected to it
  • Enable/disable controls
  • Filter controls by enablement status, violation severity, name, and control type

The features are under development and will soon include the ability to create custom CSPM policies as well.

Read more in CSMP Policies (Preview).

July 28, 2022

Managed Threat Detection Policies

Starting today all existing customers will see all existing policies labeld as Custom Policies and with a list of disabled Managed Policies. The existing custom policies work exactly as they have always worked, and do not require any action from the user to make changes. However to get the power of the Sysdig Threat Research team, we recommending moving over to the new managed policies. You can read more about the different types of managed policies here.

July 21, 2022

Actionable Compliance - Accept Risk Preview Released

Sysdig is pleased to announce the Preview release of Risk Acceptance in Actionable Compliance, This is a technical preview release, and the feature is open for all customers. This feature includes:

  • Improving compliance score by Accepting a risk on a failing resource in a control

  • Registering an Acceptance reason and expiration date

  • Editing and revoking acceptance

  • Compliance views - summary of accepted risks

  • Violation results mini-inventory - filtering by Accepted resources

You can read more about the feature in the documentation.

June 23, 2022

New Secure Event Forwarder Integrations: Elasticsearch & Microsoft Sentinel

Two new integrations have been released for Sysdig Secure’s Event Forwarder functionality:

June 2, 2022

Actionable Compliance Preview Released

Sysdig is pleased to announce the first Preview release of Actionable Compliance, the next phase of the Sysdig Secure compliance offering and the first capability to support KSPM, and in the future also CSPM. This is a technical preview release, and the feature is open for all customers.This feature includes:

  • Compliance views - a redesigned summary view for each built-in policy
  • Violation results - the first-ever mini-inventory to show violated resources with filtering capabilities
  • Actionable Remediation - automatically open a Pull Request to remediate a resource violation in its git stored source file (Infrastructure as Code)

Technical highlights:

  • Inventory based collection - a paradigm shift in how we collect CSPM data - bring it raw!

  • New agent collector - gathers all Kubernetes objects (workloads, subjects, roles, etc.) from the customer for Inventory future use

  • New node-analyzer container - collects the node’s Kubernetes, Linux & docker configurations

  • 8 new micro-services

  • OPA based policies - built-in policies (previously benchmarks) with OPA controls (previously rules) for Kubernetes, docker & Linux

You can read more about the feature in Sysdig’s documentation

May 23, 2022

Custom Roles

A custom role is an admin-defined role that allows Sysdig administrators to bundle a set of permissions and assign those permissions to individual users or teams. Custom roles allow for finer-grained definition beyond the standard out-of-the-box Sysdig Roles. Once defined, a custom role can be assigned to any user inside a particular team, and also be configured as the default role for new users in that team. For more information, see Custom Roles.

The addition of custom roles into the platform is transparent, meaning that standard roles and assignments that already exist will not experience any changes.

May 19, 2022

To facilitate a smooth transition from the Legacy Scanning Engine to the new Sysdig Secure Vulnerability Management, the Settings Menu now provides options for displaying the UI for the new, legacy, or both scanning engines.

Safe and transparent: This is a non-intrusive change; regardless of how you have the current New Vulnerabilities engine toggle set, the Sysdig Secure navigation menu will not be modified without explicit user intervention. And the toggles will only alter the user interface and not impact the function or running of the engine itself.

To enable/disable: See Which Scanning Engine to Use

If both are enabled: The two sets of features are clearly distinguished in the Navigation menu.

May 18, 2022

Policy Advisor Deprecation Notice

Sysdig Policy Advisor will be removed from all Sysdig accounts on June 17, 2022.

Policy Advisor was built during a time when PodSecurityPolicies (PSPs) were the only way to add Security Policies to a Kubernetes workload. PSPs have now been deprecated in Kubernetes 1.21, released more than a year ago.

May 17, 2022

Runtime Scanner 1.0.3 Released

  • Optimized requests performed on the Kubernetes API

See also: Vulnerabilities | Runtime

May 4, 2022

Sysdig Platform Audit

We are glad to announce that Sysdig Platform now supports the capability of tracking, logging and reporting on all changes in the system.

  • Track all activities on the API level
  • Retention period: 90 days
  • Simple API for retrieving audit information (no UI)
  • Events Forwarding support to be included in the near future (to be announced)
  • Enabled by default for all SaaS customers

See also: Sysdig Platform Audit

Sysdig Platform Login Banner

We would like to announce that Sysdig Monitor and Secure now allow you to define a Login Message that will be presented to all users. Added to boost Sysdig compliance/enterprise readiness, requested originally by the IRS.

  • Users are not allowed to access the system until they acknowledge the message
  • One login banner per customer
  • Only Admin users can enable/update the message
  • Single banner for both Monitor and Secure (for Platform customers)
  • Available on SaaS for all customers

See also: Configure Login Message

May 3, 2022

Insights Feature GA

This release marks the general availability (GA) of the Secure Insights feature. Some of the changes introduced include:

  • Better support for Azure events
  • AWS IAM permission integration
  • Bug fixes for policy tuner flow
  • Limit for displaying events in a time range removed

May 2, 2022

DriftControl Policies: Detect and Prevent Drift in Container Runtime

Sysdig agent can now detect when a new executable was added to a container after a container has started up. The agent collects when a file was downloaded and made executable. When using prevention mode, the agent can also deny the process from ever running. A policy can also be used to define binaries that should be denied/excluded from being denied if they have been added after the container has started.

See also: Drift Policy

April 28, 2022

Component Security Fixes

The following Sysdig Secure components were updated with the latest security patches (April 2022):

April 20, 2022

New Vulnerability Management Engine

Sysdig is pleased to announce the New Vulnerability Management engine, a major upgrade to the vulnerability and image scanning functionality for the Sysdig Secure product.

Major Highlights

  • Scanning times have been drastically reduced: 8x faster on average!

  • Additional data for vulnerabilities and remediation

    • CVSS scores and metrics: Network Attack Vector, Privileges required, etc.
    • Flagging of publicly available code Exploits
    • Suggested package fix version
  • Risk spotlight Focus on the vulnerabilities that Sysdig detects in active packages at runtime.

    • This is a new filter that only shows CVEs with active packages, to save time browsing infrastructure and to help focus on high-impact CVEs
  • New Vulnerability Reporting module

    • Up to 14 days retention of individual reports
    • “Generate now” immediate scheduling directly available from the UI
  • Flexible policies that can be attached to the different runtime and security contexts

How to Move to the New Scanning Engine

The new vulnerability management engine uses a different data storage, API, host components, and user interfaces than the legacy scanning.

  • Contact your Sysdig representative; she/he will guide you through the process of migrating your subscription and vulnerability management configuration to the new engine.
  • Full documentation available here.

March 8, 2022

Scanning Component Updates

The following components have been upgraded to the listed versions with bug fixes and security updates:

  • node-image-analyzer:0.1.16
  • secure-inline-scan:2.4.9
  • host-analyzer:0.1.6

The latest Helm chart includes these versions for Node Image Analyzer and Host Analyzer. Follow the usual process to upgrade the inline scanner.

March 3, 2022

New CIEM Features

User Risk Labels

Risk Labels are now surfaced to highlight insecure attributes for specific Users and Roles. They are listed within the Users & Roles page and within the User Details tab of a specific user.

Trend Charts in Overview

Time charts are now available within the Overview tab of Identity and Access. These help to visualize your Permission trends over time for Users, Policies, and Resources.

CSV Report Export

All of the pages within Identity and Access can now be exported as a CSV file. Select the Download CSV button found at the top right corner of all pages.

Effective Permission Calculation

AWS supports different types of policies to limit permissions on different scopes. Sysdig has added support for calculating effective permissions based on permission boundaries and organization level service control policy (SCP). This gives additional context when viewing permissions set on identities. For example, an identity that has been given administrator level identity policy will be limited in overall permissions if there is a permission boundary policy attached to it.

CIEM Data in Insights

Within the Cloud Activity and User Activity views in Insights, there is now an Identity and Access tab. This will help investigative flows to understand the context from an IAM perspective.

March 1, 2022

New: Data Sources Instrumentation

On the Data Sources > Managed Kubernetes page: For unconnected clusters, Sysdig has added quick instrumentation instructions using the known details about the cluster, such as the cloud account, region, and cluster name.

February 10, 2022

Improved Usability with New Navigation

Sysdig’s new navigation improves the usability of the left-hand navigation for faster and easier navigation of where you’re trying to go.

Check out a video walk-through of the new feature!

Improved Menu Handling

  • Hoverable Sub-Menu: With each module that has additional menu options, hover over the respective module to quickly navigate.

  • Collapsible Main Menu: Save space with the collapsible left-hand navigation.

New Menu Option: Integrations

A dedicated Integrations menu option provides an easy way to access both inbound and outbound integrations.

Inbound:
  • Access the Cloud Accounts page to quickly understand which applications and services are running, and where the Sysdig agent is installed.
  • Access Managed Kubernetes to get a catalog for all the managed Kubernetes clusters in your environment. The status shown is connected/unconnected based on whether the agent is installed or not.
  • 3rd Party: Manage your Git Integrations

Outbound: Manage your Event Forwarding, Notification Channels, and S3 Capture Storage

3rd Party: Manage your Git Integrations

Revamped User Menu

Now all the settings options are collected and exposed in one mega menu. Find the right page before navigating away from where you are.

February 2, 2022

Enhanced Unified Filter for Event Feed

The Sysdig Secure Event Feed is getting a new unified filtering experience, available now for SaaS accounts.

Easily toggle from the original to the cleaner, simpler enhanced version, where you will find:

  • Unified scopes, free text and any other filterable/searchable attributes on a single lean bar
    • Autocomplete on keys and values
    • Autocomplete/suggest operands
    • One-click quick filtering directly from the list of displayed elements
  • Saved filters in various formats– no more retyping common filter expressions
    • Favorite filters, stored per user and feature
    • Default filters, per user and feature
    • Recent filters, per user and feature

See also: Secure Events

January 26, 2022

Unified Compliance Reporting

We are pleased to announce a rework of our Compliance and Benchmarking capabilities. This change brings a number of improvements:

  • Compliance and Benchmark tasks are now scheduled, managed, and generate reports in an updated and unified interface, including simpler pathways to remediation and easier-to-navigate reports.
  • The logic used to check individual controls now checks for events signalling control failures, as well as ensuring the correct Runtime rules are configured to detect these events. This leads to a more comprehensive audit that captures activity as well as configuration.
  • New compliance standards and platforms: added
    • For workload, AWS, GCP, and Azure:
      • NIST 800-82 Rev2
    • For workload and AWS:
      • Fedramp (workload and AWS only)
      • HITRUST CSF 9.4.2 (workload and AWS only)
    • For GCP and Azure
      • GDPR
      • HIPAA
      • ISO 27001:2003
      • NIST 800-53 Rev4
      • NIST 800-53 Rev5
      • NIST 800-171
      • NIST 800-190
      • PCI / DSS v3.2.1
      • SOC 2

Prerequisites

  • Agent version >= 12.0.4

    If necessary, install or upgrade your agent to the appropriate version.

  • Node analyzer installed

NOTE: If you are upgrading from an earlier version of Sysdig Secure, your existing compliance and benchmark records will be migrated to the new version and retained on the same schedule as before.

See also: Compliance

New Feature: Review Applied Kubernetes Network Policies

Sysdig Secure has added the ability to view the KNPs that have been applied directly from the Network Security Policy UI.

You can:

  • Review the relevant policies applied to the pod-to-pod communication for the current view

  • Click View Policy to see the raw yaml output of the network policy applied to that workload.

See also: Netsec Policy Generation

January 2, 2022

Welcome Infrastructure-as-Code!

Infrastructure-as-Code (IaC) is an important part of today’s cloud-native infrastructure. We at Sysdig know that the earlier you identify possible posture issues, the better off you are.

The new feature allows you to integrate Kubernetes IaC checks into your Git pipeline. With just a few clicks, the standard compliance checks will be integrated into the Pull Request (PR) flow and alert developers when they create violations of the policy before they merge.

Supportability & Requirements

The new capability will use either an application or a webhook in your respective git provider.

  • Github - Github Application
  • Gitlab - Webhook
  • Azure DevOps - Webhook
  • Bitbucket - Webhook

For each provider you can define the repos and folders to protect, as well as branches on which to perform the evaluation.

See also: Git IaC Scanning

December 17, 2021

Update on Log4j Vulnerability (CVE-2021-44228)

Sysdig confirms that all services that compose Sysdig’s Cloud Platform running Apache’s vulnerable Log4j library have been patched to 2.16. We have not detected any successful attempts at exploitation of this attack vector during that time window.

December 15, 2021

Update on Log4j Vulnerability (CVE-2021-44228)

The sysdig agent does not include the Log4j library

Sysdig is using an alternative framework for logging called Logback. The logback framework isn’t vulnerable to this issue.

Sysdig components include a log4j library in our standard distribution that was vulnerable. This library is included for compatibility reasons only, is not used for primary logging, and our security team has determined we are not vulnerable based on our application architecture and existing mitigating controls.

Sysdig can confirm that all services that compose Sysdig’s Cloud Platform running Apache’s vulnerable Log4j library have been patched to the latest version or adds additional mitigating controls suggested by vendors. We have not detected any successful attempts at exploitation of this attack vector during that time window.

Details regarding upgrades We:

  • explicitly set commonsLog4jVersion = 2.15.0
  • update all of log4j-to-slf4j, log4j-api, and log4j-core to version 2.15.0

December 15, 2021

Sysdig Secure for Azure

Sysdig is pleased to announce the general availability of the Sysdig Secure for cloud capabilities for Azure.

  • Cloud Security Posture Management (CSPM): Based on CIS benchmarks tailored for your assets
  • Cloud Threat Detection: Identify threats in your Azure environment using Falco rules for Azure
  • Event Hub: Fully managed, real-time data ingestion service that’s simple, trusted, and scalable
  • Image Vulnerability Scanning: Automatic vulnerability scanning of images pushed to Azure Container Registry and images executed on Azure Container Instance Group.

For details, see Deploy Sysdig Secure for cloud on Azure.

December 12, 2021

A Statement on Log4j vulnerability (CVE-2021-44228)

Security researchers recently disclosed the vulnerability CVE-2021-44228 in Apache’s log4j, which is a common Java-based library used for logging purposes

Sysdig is using an alternative framework for logging called Logback. The logback framework isn’t vulnerable to this issue.

Sysdig components include a log4j library in our standard distribution that appears to be vulnerable. It has been confirmed that this library is included for compatibility reasons only and is not used for primary logging. As a result this should not pose any risks.

Patches will be provided to upgrade the log4j libraries that are included for compatibility reasons.

If you have any questions or concerns, please reach out to your Sysdig contact.

December 1, 2021

Image Analyzer 0.1.15 Inline Scanner 2.4.8 Released

Release 0.1.15 of the Node Image Analyzer

Release 2.4.8 of the Sysdig Inline Scanner

  • Updated to the latest security fixes
  • Fixed support for COPY, USER, and other instructions when the image is built using buildkit

November 5, 2021

Improved Handling of Forwarded Benchmark Events

Forwarded benchmark events now include AWS tags as key-value pairs (rather than a flattened string), making them easier to consume.

November 2, 2021

Inline Scanner 2.4.7 Update

Requirements

  • libseccomp >= 2.3.3 (on the Host/JenkinsWorker - where the docker command is executed)
  • docker version > v18.05.0-ce

Fixes

Fixed support for COPY, USER, and other instructions when image is built.

October 27, 2021

Cloud Infrastructure Entitlements Management (CIEM) for AWS

Sysdig Secure has added Permissions and Entitlements Management functionality. You can find it under Posture menu tab.

By combining the CIEM capabilities announced today with Sysdig’s existing capabilities, Sysdig customers can proactively prevent cloud permissions risks, scan for vulnerabilities and misconfigurations, and detect and respond to attacks across container and cloud environments.

  • Gain visibility into all cloud identities and their privileges: get a comprehensive view into access permissions across all AWS users and services
  • Enforce least privilege: eliminate excessive permissions by applying least-privilege policies to users and services with automatically generated IAM policies. Sysdig proposes policies based on analyzing which entitlements are granted versus which are actually used.
  • Simplify audit of access controls to meet compliance requirements: use reports for regular access reviews to evaluate active and inactive user permissions and activity.

Additional Information:

October 26, 2021

New Secure Event Forwarder Integrations: Google Chronicle, Google Pub/Sub & Amazon SQS

An extended set of output data integrations has been added to Sysdig Secure’s Event Forwarder functionality, in particular:

  • Integration with Google Chronicle. NOTE: Only Runtime policy events are available as data to send at this moment.
  • Integration with Google Pub/Sub and Amazon SQS, which can be used as temporary storage that will adapt the EFO push behaviour into a data pull endpoint.

See also:

October 25, 2021

Sysdig Secure for GCP

Sysdig is pleased to announce the general availability of the Sysdig Secure for cloud capabilities for GCP.

  • Cloud Security Posture Management (CSPM): Based on CIS benchmarks tailored for your assets
  • Cloud Threat Detection: Identify threats in your GCP environment using Falco rules for GCP
  • Audit Logs: Google Security Command Center integration to forward threats identified by Falco rules.
  • Image Vulnerability Scanning: Automatic vulnerability scanning of images pushed to Google Container Registry, Google Artifact Registry and images executed on Google Cloud Run.
  • Chronicle Integration: Events forwarding to Google Chronicle.
  • Installation via GCP Marketplace: You can install Sysdig from the GCP marketplace and pay using the payment method and credit of your GCP account.

See full details: Sysdig Secure for cloud and Deploy Sysdig Secure for cloud on GCP.

October 13, 2021

New Scanning Engine (Technology Preview)

Sysdig Secure is developing a new scanning engine that will deliver major improvements, additional capabilities, and scanning-centric workflows.

The first iteration is available to test and provides:

  • Much faster scan times: 4x to 10x faster initial image analysis
  • Extended vulnerability data, including CVSS scores, vectors containing the full exploitability data, availability of an associated public exploit, etc.
  • Inline scanner available as a stand-alone binary, no longer requires spawning a container
  • Better remediation advice, including ‘Which packages are the worst offenders in my image? Considering all the possible fix versions, which one should I apply?’
  • Improved, more intuitive user experience, with faster response times

Important: The new engine is still on “Preview” phase.

This means:

  • Not suitable for production. There is no forward compatibility guarantee for the data or configuration (yet)
  • Testing the new scanning preview will NOT affect any existing scanning workflows that leverage the current scanning backend. It is safe to enable the preview on any account.
  • Additional fundamental components are still in development; they will be released in an upcoming version.

To test the new engine, simply enable the flag under Settings >User Profile>Sysdig Labs.

See New Scanning Engine to download the Inline Scanner binary and begin.

September 17, 2021

Date Columns Added for Scheduled Scanning Reports

In Sysdig Secure, the Scheduled Reports for Scanning now display additional vulnerability metadata for both runtime and registry reports.

Specifically:

  • Disclosure date: Time when the vulnerability information was registered in the feed
  • Solution date: Time when the fix version for this vulnerability (if any) was registered in the feed

To avoid breaking compatibility with existing reports and external instrumentation, these fields will only be available for newly created reports; existing Scheduled reports (even if they are modified and saved again) will not contain these columns.

September 8, 2021

New and Updated Compliance Standards

Sysdig Secure has added three new compliance standards and updated another. See also: Compliance

Updates to PCI DSS v3.2.1 Compliance for Workload

We have implemented some changes to the PCI DSS v3.2.1 for workload compliance checks. The control coverage for PCI is now: 1.1.2, 1.1.3, 1.1.5, 1.1.6.b, 2.2, 2.2.a, 2.2.1, 2.2.2, 2.4, 2.6, 4.1, 6.1, 6.2, 6.4.2, 6.5.1, 6.5.6, 6.5.8, 7.2.3, 10.1, 10.2, 10.2.1, 10.2.5, 10.2.7, 10.5.5, 11.5.1

Checks added:

  • Check for Network Security enabled added to controls 1.1.2, 1.1.3 and 1.1.5

  • Check for Kubernetes audit enabled added to controls 4.1, 6.4.2 and 6.5.8

Rules added:

  • Rule Outbound or Inbound Traffic not to Authorized Server Process and Port added to control 2.2.1

  • Rule Attach to cluster-admin Role added to controls 7.2.3 and 10.5.5

  • Rules EphemeralContainers Created and Terminal shell in container added to controls 10.1 and 10.2.1

  • Rules ClusterRole With Pod Exec Created , ClusterRole With Wildcard Created and ClusterRole With Write Privileges Created added to control 10.2

  • Rule Launch Privileged Container added to control 10.2.5

  • Rules Container Drift Detected (chmod) and Container Drift Detected (open+create) added to control 11.5.1

Rules removed:

  • Rule All K8s Audit Events rule removed from controls 10.1, 10.2, 10.2.1, 10.2.7

New PCI DSS v3.2.1 Compliance for AWS

The PCI Quick Reference describes the full range of controls required to pass a PCI 3.2 audit. In this release, Sysdig Secure will add the following controls.

For AWS protection: 2.2, 2.2.2, 10.1, 10.2.1, 10.2.2, 10.2.5, 10.2.6, 10.2.7, 10.5.5, 11.4

New AWS Well Architected Framework Compliance

The AWS Well Architected Framework whitepaper defines best practices to build secure, high-performing, resilient, and efficient infrastructure for applications and workloads.

For workload protection, Sysdig Secure will check the following sections: OPS 4, OPS 5, OPS 6, OPS 7, OPS 8, SEC 1, SEC 5, SEC 6, SEC 7, REL 2, REL 4, REL 5, REL 6, REL 10, PERF 5, PERF 6, PERF 7

For AWS protection, Sysdig Secure will check the following sectionsOPS 6, SEC 1, SEC 2, SEC 3, SEC 8, SEC 9, REL 2, REL 9, REL 10

New AWS Foundational Security Best Practices v1 (FSBP) Compliance

AWS Foundational Security Best Practices v1 (FSBP) describes the full range of controls to detect when your deployed accounts and resources deviate from security best practices.

For AWS protection, Sysdig Secure will check the following sections: AutoScaling.1, CloudTrail.1, Config.1, EC2.6, CloudTrail.2, DMS.1, EC2.1, EC2.2, EC2.3, ES.1, IAM.1, IAM.2, IAM.4, IAM.5, IAM.6, IAM.7, Lambda.2, GuardDuty.1

New NIST 800-171 rev2 Compliance

The National Institute of Standards and Technology (NIST) Special Publication 800-171 rev2  describes the full range of controls required to pass a NIST 800-171 audit. It provides agencies with recommended security requirements for protecting the confidentiality of Controlled Unclassified Information (CUI) when the information is resident in nonfederal systems and organizations.

For workload protection, Sysdig Secure will check the following sections:3.1.1, 3.1.2, 3.1.3, 3.1.5, 3.1.6, 3.1.7, 3.1.12, 3.1.13, 3.1.14, 3.1.15, 3.1.16, 3.1.17, 3.1.20, 3.3.1, 3.3.2, 3.3.5, 3.3.8, 3.3.9, 3.4.3, 3.4.5, 3.4.6, 3.4.7, 3.4.9, 3.5.1, 3.5.2, 3.11.2, 3.12.1, 3.13.1, 3.13.2, 3.13.3, 3.13.4, 3.13.5, 3.13.6, 3.13.8, 3.14.1, 3.14.2, 3.14.3, 3.14.4, 3.14.5, 3.14.6, 3.14.7

For AWS protection, Sysdig Secure will check the following sections:3.1.1, 3.1.2, 3.1.3, 3.3.1, 3.3.2, 3.3.7, 3.5.7, 3.5.8, 3.14.6, 3.14.7

September 2, 2021

New Terraform Onboarding Options for Secure for cloud

Users can now onboard Sysdig Secure for cloud with their AWS accounts (single or organizational) using Terraform. See the feature description and the deployment/onboarding instructions.

August 12, 2021

Inline Scanner 2.4.6 Released

Version 2.4.6 of the inline scanner container has been released. See also: Integrate with CI/CD Tools.

Feature:

  • Added support for images with the (deprecated) manifest schema V1

July 30, 2021

Inline Scanner 2.4.5 released

Version 2.4.5 of the inline scanner container has been released. See also: Integrate with CI/CD Tools.

Fix:

  • Fixed an edge case in which using the --verbose flag with --format json caused a corrupted JSON output

July 28, 2021

Inline Scanner v2.4.4 Released

Version 2.4.4 of the inline scanner container has been released. See also: Integrate with CI/CD Tools.

Fixes:

  • Bumped ClamAV version to latest (0.103.3).

  • Updated base image to get updated security fixes (July 2021)

  • Added retry mechanism when pulling images from registries

  • Added --write-json PATH option to permit storing json log to file

  • Fixed Malware scan fails when image has not read the permissions on files

  • Fixed failure in getting images for registries that do not support tag listing

July 27, 2021

Admission Controller with Kubernetes Audit (k8s_audit Falco rules)

Today we announce the general availability of the Kubernetes Audit functionality as part of the Sysdig Secure Admission Controller.

Background:

Kubernetes admission controllers provide operators the ability to validate and/or mutate incoming API requests. Admission controllers are a core functionality of Kubernetes, and many are enabled by default.

Sysdig Secure has long provided Kubernetes API security using k8s_audit Falco rules to create policies against Kubernetes audit logs. However, there have been some complications:

  • Diverse setup requirements:

    Many Kubernetes distros are opinionated in the way to collect and access logs, some using dynamic backends (deprecated in Kubernetes 1.19, but still available in OCP up to 4.3), while more vanilla approaches use webhooks, and cloud providers require a bridge to collect logs via their own logging streams.

  • Distros diverging from Falco:

    With OCP 4.4+, we had no clear way to collect and validate audit logs against our Falco rules.

The Solution?

Tap directly in the Kubernetes API request via Admission Controllers and use the existing k8s_audit rules our customers have relied on for so long. See the installation instructions.

July 2, 2021

Inline Scanner 2.4.3 Released

Change:

  • Updated base image to get updated security fixes (June 2021).

Fixed:

  • Fixed incorrect version detection for Apache Struts 2 packages which was leading to false positives.

July 1, 2021

Node Image Analyzer v0.1.13 Released

Version 0.1.13 of the Node Image Analyzer has been released.

This release comes with the following improvements:

Fixed:

  • Fixed a GKE- and ContainerID-specific bug where the node image analyzer couldn’t scan the image due to missing blobs

  • Implemented a few-second pause at startup to allow for Istio sidecars to complete the initialization before creating connections

New

  • We use the Universal Base Image (UBI) Sysdig-approved image as the base, in order to ensure the highest patch level approved by our security team.

June 23, 2021

Enhancements to Compliance Module

Terminology note: Compliance standards are scoped to different platforms depending on the specific security rules they include, Broadly, these are divided into:

  • Workload types: Including any Falco rules for kernel system calls, Falco rules for Kubernetes audit logs, host benchmarks, and security features that affect hosts, containers, and kubernetes clusters

  • AWS/cloud type: Falco rules for CloudTrail and Cloud Custodian rules on Amazon Web Services

See also: Compliance.

Extended Existing Compliance Standards to AWS

For the following existing compliance standards, we have added rules for AWS cloud provider:

  • NIST 800-53 rev4 for AWS

  • NIST 800-53 rev5 for AWS

  • ISO 27001:2013 for AWS

  • SOC2 for AWS

  • HIPAA for AWS

Added New Compliance Standards

We have also added the following new compliance standards to Sysdig Secure’s offerings:

  • GDPR for AWS 

  • GDPR for workload

  • NIST 800-190 for workload

Trimmed Excess Rules from Some Standards

Certain rules have been re-evaluated and were removed because they did not significantly contribute to the security posture:

  • Logged in without Using MFA (merged with Console Login Without MFA)

  • Interpreted procs outbound network activity

  • Launch Suspicious Network Tool in Container

  • All K8s Audit Events

June 14, 2021

CIS RedHat OpenShift Container Platform v4 Benchmark

Support for CIS RedHat OpenShift Container Platform v4 Benchmark has been added to Sysdig Secure.

As part of this release Sysdig is allowing you to scan and validate compliance with 112 controls included in the CIS Bencmark requirements.

See also: Benchmarks

June 9, 2021

Sysdig Secure UX Improvements: “Investigate” Navigation & Activity Audit

Sysdig navigation just got a facelift. To help our Sysdig Secure users navigate easily, we:

  • Added the new menu item Network (previously found under the Policies menu), and

  • Grouped Activity Audit + Captures into Investigation to better describe the use-case it helps users resolve.

Activity Audit

The Activity Audit module also got several interface and user experience improvements:

  • Runtime scope moved to the top to align with other Secure interfaces and allow more space for activity data

  • Activity types (network, file, kubectl, command) can now be filtered directly from the graph using the legend

  • Attributes of the displayed elements can be filtered directly from the list, without displaying the side detail panel

June 4, 2021

Kubernetes Network Security: New Configuration and Improved User Experience

Sysdig’s Kubernetes Network Policy tool has been updated to include additional fine-tuning configurations and an improved user experience.

Additional Configuration Panel

  • Workload Labels: Depending on your workload labelling policy, some labels may not be relevant for generating a KNP policy. Use the additional config to include/exclude a particular set of labels per cluster/namespace to declutter your UI and the resulting policy.

  • Unresolved IP Configuration: Now it is possible to label raw IPs that are not mapping to your Kubernetes/OpenShift entities, i.e. external cloud provider services, so these labels will be automatically applied to the topology and ingress / egress tables.

  • Cluster CIDR configuration: If the CIDR configuration is not automatically detected by the agent, you can now directly configure internal subnets per cluster using the Sysdig interface.

Improved UX

  • Topology map: Additional information pop-up when hovering over a network connection or a network node, such as server process, source, destination, and more.

  • Unresolved IP filtering: In the ingress and egress tables, by type or using free text search.

Additionally, Network is now presented as a top-level item in the Sysdig Secure navigation.

May 27, 2021

Falco Policy Tuner - Beta

Sysdig is now releasing a managed version of the standalone Falco Tuner.

Previously, you had to run the tuner in your local environment, print suggestions, and manually update a rule with those suggestions. The new feature runs in the background and automatically tunes noisy rules and false positives. To streamline the creation of these exceptions, we’ve created a new object within Falco called exceptions.

Note: To enable the tuner, Admin access rights to Sysdig Secure are required.

Feature Enhancement: Falco Exceptions

Previously, exceptions were created using and not conditions inside a Falco rule, e.g.

- rule: Write below binary dir
  ...
  condition: >
    bin_dir and evt.dir = < and open_write
    and not package_mgmt_procs
    and not exe_running_docker_save
    and not python_running_get_pip
    and not python_running_ms_oms
    and not user_known_write_below_binary_dir_activities    
  ....

However, this process can be unwieldy and can result in unintended behavior. The new format, using exceptions, looks like this:

- rule: Write below binary dir
  ...
  condition: bin_dir and evt.dir = < and open_write
  ....
  exceptions:
    - name: package_mgmt_procs
      fields: proc.name
      comps: in
      values: package_mgmt_binaries # list of known binaries
  ...

See the full documentation here.

May 19, 2021

Regulatory Compliance for ISO27001:2013 and HIPAA Now Available

Two new compliance standards have been added to Sysdig Secure’s compliance feature:

See also:Compliance for information about the specific controls Sysdig covers for each security standard.

Inline Scanner v2.4.1 Released

Version 2.4.1 of the inline scanner container has been released. See also: Integrate with CI/CD Tools.

Fixes:

  • Updated ClamAV version to 0.103.2 to avoid end-of-life problems present in the former version, such as failure in updating the antivirus database

Additional type Descriptor Forwarding Activity Audit through Event Forwarder

The JSON payload when sending Activity Audit elements through the Event Forwarder will now contain an additional field name: type. This describes the type of the entry, respectively: command, connection, fileaccess, or kubernetes.

See also: Event Forwarding.

May 18, 2021

New and Improved Host OS and Container Scanning Tools

We at Sysdig are working hard to improve your security posture and compliance experience. As part of this commitment we are implementing a new framework to generate host benchmark results, introducing host scanning, and making backend improvements to the image scanning mechanism.

Installation Steps

The new features require a new component to be installed called the Node Analyzer. We’ve provided an installation script to automate the installation or to upgrade an existing Node Image Analyzer daemonset, if applicable.

Once you’ve installed or updated the components, the UI will automatically show Host Scanning and new Benchmarks functionality (Legacy Benchmarks can still be accessed.)

Host Scanning: New

In addition to Sysdig Secure’s rich array of tools for scanning container images, you can now scan the hosts as well.

  • Scan hosts for vulnerabilities, and detailed Software Bill of Materials (SBoM)

  • Support for OS (e.g. rpm) and non-OS (e.g. Java, Ruby, Python) packages

  • Compare and diff scan results

Host Benchmarks: Updated

  • More checks

  • Better results

  • Clustered aggregations - understand the posture of your environments, not just a single entity

Image Scanning: Updated

  • Automatically scan images if they have not been scanned

April 29, 2021

New Scan Results Page Layout

We have reorganized the visual layout of the Scan Results summaries to clearly distinguish policy evaluation from vulnerability matching and to better summarize the information.

Improvements include:

  • Vulnerabilities and Policies are now two different sections in the UI

  • Vulnerability match update time is displayed to further distinguish from the Policy Evaluation time

  • Policy breakdown is collapsed by default to reduce cognitive load

  • Re-evaluate policies button is now located in the impacted section only, as opposed to whole page

  • Apart from the vulnerability update time, the data remains unchanged from previous versions

See also: Review Scan Results.

April 26, 2021

Inline Scanner v2.4.0 Released

Version 2.4.0 of the inline scanner container has been released. See also: Integrate with CI/CD Tools.

Changes:

  • Updated base image to get updated security fixes.

New

  • Added HTTP_PROXY and HTTPS_PROXY environment variables support for malware scanning mode. This is required if you want to retrieve the malware database inline behind a proxy.

  • Added support for.dockercfgrepository auth method, accessible via the--registry-auth-dockercfgCLI flag.

Fixes:

  • Now using HTTP1.1 by default to bypass a cURL bug.

  • Provided fix for an error when using the docker-daemon storage type with a docker UID different than 1000.

March 30, 2021

Sysdig Secure for cloud

Sysdig Secure for cloud is available with Cloud Risk Insights for AWS, Cloud Security Posture Management based on Cloud Custodian for AWS and multi-cloud threat detection for AWS using Falco.

What’s Included in this release:

  • Insights: a powerful new visualization tool for threat detection, investigation, and risk prioritization, to help identify compliance anomalies and ongoing threats to your environment. With Insights, all findings generated by Sysdig across both workload and cloud environments are aggregated into a visual platform that streamlines threat detection and forensic analysis.

  • Threat Detection based on AWS CloudTrail: To detect threats, anomalies and suspicious activities with the flexible Falco engine. See also: Sept 29, 2020.

  • Cloud Security Posture Management with AWS Benchmarks: The AWS CIS Benchmarks assessment evaluates your AWS services  against the benchmark requirements and  returns the results and remediation activities you need to fix misconfigurations in your cloud environment.

    We’ve included several UI improvements to provide additional details such as:  control descriptions, affected resources, failing assets, and guided remediation steps, both manual and CLI-based.

  • Image Scanning for ECR and Fargate: one-click deployment– see also ECR April 13, 2020 and Fargate Sept. 28, 2020.

Free-Forever Cloud Security Tier

Sysdig is launching a new Free-forever cloud security tier for one single account.

https://sysdig.com/company/start-free/

  • Easy onboarding in minutes

  • Manage cloud posture with a daily run of CIS Benchmarks

  • Detect threats with out-of-the-box CloudTrail detection rules based on Falco

  • Scan containers (ECR/Fargate scanning) automatically and within your cloud environment for upto 250 images a month

March 24, 2021

Image Scanning Reports v3 [BETA]

The Image Scanning Reports feature has been thoroughly updated and has moved from a synchronous model to an asynchronous mode, in which you schedule the reports you need and then receive them through your normal notification channels (email, Slack, webhook.). The new version also includes:

  • A preview function to check report structure in the UI

  • A more advanced query builder

  • Extended set of data columns (i.e. CVSS base score and vector) and extended set of available filters (i.e. package type)

Reporting v3 supports two different types or reports:

  • Vulnerability report: Containing vulnerability, package and image data

    I.e. Vulnerabilities in my runtime with Severity ≥ High, a Fix available and not included in a vuln exception list.

  • Policy report: Containing scanning policies and evaluated images data

    I.e. Images in my internal registry failing the “NIST” scanning policy.

You need to enable this feature from the Sysdig Labs setting on the User Profile page.

See Scheduled Reports for more detail.

March 22, 2021

Feature Enhancement: Falco Policy Types

Sysdig Secure has introduced Policy Types– a separation of policies into logical groups, based on the sources used in the policy engine. When creating a policy, you choose a type and then only the relevant scopes and container actions will be presented. 

We have also introduced a new policy type to support threat detection with AWS CloudTrail rules.

For full details, see Manage Policies.

March 17, 2021

Scan Results: UX Enhancements & Added Functionality

Scan Results List

Summarized views based on image count, image fail / pass distribution and image origin distribution.

  • Registry filter dropdown, multi-select

  • Visible image counters: Images shown in the page vs total number of images available after applying filters

  • Visual charts: Pass/fail and origin distribution (also respecting filters)

Vulnerability List:

New table design to offer additional visual feedback and reduce data redundancy, plus additional vulnerability data.

New functionality:

  • Individual vulnerabilities can now be clicked to display additional information in a side panel:

    • The vulnerability feed source that was used for the matching

    • A description of the vulnerability

March 15, 2021

Sysdig Serverless Agent 1.0.0 for Fargate ECS

The “container-as-a-service” serverless environment calls for new agent models, and Sysdig provides them. Whereas in ECS, users still manage the underlying instances, with AWS Fargate the host is never visible and users simply run their workloads. And while this model is convenient, it can introduce risk as many people leave the containers unattended, without monitoring security events within that can exfiltrate secrets, compromise business data, impact performance, and increase their AWS costs. In addition, it is not possible to install a standard agent in an environment where you do not have access to a host.

For these reasons, Sysdig has introduced a new “serverless agent” model that can be deployed in these container-based cloud environments. The first implementation is for Fargate (ECS).

Sysdig will be rolling out security features on the serverless agent over time. In v1.0.0, users will see:

  • Runtime Policies and Rules

  • Secure Events

To obtain secure event information and the associated Falco policies and rules in the Sysdig Secure UI from a Fargate environment, users install the serverless agent using a CloudFormation Template. Then log in to Sysdig Secure and review the events in the UI.

See also: AWS Fargate Serverless Agents and Serverless Agent Release Notes (for future updates).

March 12, 2021

Deprecation Notice: Legacy Commands Audit & Legacy Policy events

  • The “Commands Audit” feature was deprecated in favor of Activity Audit in November 2019. This feature will be completely removed from the SaaS product April 2021 (next month).

    Sysdig agent version 0.93+, released in November 2019, is required by the Activity Audit feature.

  • The “Policy Events” feature was deprecated in favor of the new Events feed in June 2020. This feature will be completely removed from the SaaS product April 2021 (next month).

    Sysdig agent version 10.3.0+ is recommended.

UI Improvement on Rules Library and Rule Details

Usability improvements that display the policies in which a rule is used, from both the Rules Library list and the Rule Detail view. See Manage Rules for details.

March 2, 2021

Regulatory Compliance for SOC 2, NIST 800-53 rev4 and rev5

Three new compliance standards have been added to Sysdig compliance feature: SOC 2, NIST 800-53 rev4 and NIST 800-53 rev5.

The compliance validator now also includes new checks for the following features: Admission Controller, Network Security Policies and Node Analyzer.

See the Compliance documentation for usage details and the controls implemented.

February 23, 2021

Windows Scanning Released

A beta version of the Windows Scanning Inspector has been released. This is a new feature from Sysdig for scanning Windows containers.

This is a standalone scanning engine. There is no centralized UI, management, or historical data. These features are planned for a future release.

See also: Windows Container Image Scanning [BETA].

Features

  • Identify Windows container image vulnerabilities from:

    • Windows OS CVEs
  • Windows or Linux hosts

  • Reports in JSON and PDF

  • Policy support

    • Severity

    • Fix available

    • Days since fixed

UI-Based Admission Controller Released

Kubernetes’ admission controllers help you define and customize which requests are allowed on your cluster. An admission controller intercepts and processes requests to the Kubernetes API prior to persistence of the object, but after the request is authenticated and authorized.

Sysdig’s Admission Controller (UI-based) builds upon Kubernetes and enhances the capacity of the image scanner to check images for Common Vulnerabilities and Exposures (CVEs), misconfigurations, outdated images, etc., elevating the scan policies from detection to actual prevention. Container images that do not fulfill the configured admission policies will be rejected from the cluster before being assigned to a node and allowed to run.

See also: Admission Controller.

Main Features

  • Granular admission policies: Defining a global policy per cluster, but also at the level of particular namespaces or image paths (i.e. registries) Registry and repository whitelist

  • Only allow images that pass the scanning evaluation criteria

  • Only allow images that have been evaluated recently

  • Only allow images that have been scanned before creation is requested to Kubernetes

  • Registry and repository whitelist

  • Scan unscanned requested images immediately (optional)

February 20, 2021

Network Micro-Segmentation: Support for CronJobs, Weave, & Cilium CNIs

The Sysdig Network Security Policy Tool has been upgraded to add support for CronJob pod Owners.

With the addition of CronJob support, communication is aggregated to the CronJob (scheduler) level, rather than the Job. Therefore, when administrators review the activity in the Network Security Policy menu, they will see the higher-level CronJobs listed, and not an excess number of individual Job entries.

This update also adds support for Weave and Cilium CNIs on top of Calico support.

Malware Detection during Inline Image Analysis

As part of the inline scanner version 2.3.1 release, malware scanning was added as a configurable detection that can be performed during inline analysis.

The default behavior if this feature is enabled and malware is found is to consider the scanning failed, report malware details, and abort analysis:

See Perform Inline Malware Scanning for recommended parameters and output options.

February 16, 2021

Registry Credentials: Support for Multiple Credentials

Sysdig Secure now supports assigning multiple credentials to the same registry depending on the relative internal registry path that is used to pull the image.

A wildcard can be added to the end of the path, indicating that any image located under the partial path inside the registry (/rg-2-1er in the example) will use the registry credentials configured here. This additional flexibility is useful, for example, for IBM registries which can have a different set of permissions depending on the namespace.

See also: Manage Registry Credentials.

February 10, 2021

Inline Scanner v2.3 Released

Version 2.3 of the Inline Scanner has been released.

Fixes:

  • Avoid prefixing the image names with localbuild when not strictly necessary

New:

  • Improved version detection for specific software packages: logback, SpringFramework and Tomcat Java

  • Allow setting of openssl security level via OPENSSL_SECLEVEL env var to support old certificates

  • More robust image ID identifier, avoiding unnecessary image re-scans along the container lifecycle

  • Added malware detection feature

February 4, 2021

Enhanced Activity Audit Filters

We have improved the noise-reduction filter for the Activity Audit feature in Sysdig Secure. The feed will now automatically filter out duplicate entries with a high number of occurrences. No information is lost, as the filtered noise is only duplications of entries in the feed.

A sudden reduction in the number of Activity Audit entries per time slot is expected as a result of this filter.

January 28, 2021

Node Image Analyzer v0.1.9 Released

Version 0.1.9 of the Node Image Analyzer has been released.

This release comes with the following improvements:

Fixes:

  • Fixed an issue that prevented some images from being processed on GKE clusters using Docker and Containerd

  • Fixed an issue that prevented some images that don’t have full tags from being processed on OpenShift

  • Improved version detection for Logback, SpringFramework and Tomcat Java packages

  • Fixed an issue that resulted in the image analyzer crashing without a proper error message when an incorrect Docker socket path was provided

New:

December 23, 2020

Sysdig Secure Jenkins Plugin v 2.1 Released

Version 2.1 of the Sysdig Secure Jenkins Plugin has been released!

New

  • Sysdig Jenkins Plugin 2.1 leverages the inline scanner v2 under the hood, which improves the scanning performance and execution times.

  • This version also introduces proxy support for both master and worker nodes.

See also: Integrate with CI/CD Tools and Integrate with Jenkins.

December 16, 2020

Node Image Analyzer v 0.1.7 Released

Version 0.1.7 of the Node Image Analyzer has been released.

This release fixes image analysis errors for OpenShift clusters configured in FIPS mode.

See also: Scan Running Images

Statement RE: Solarwinds and Sysdig’s Security

We have seen requests for statements regarding tooling in the wake of the Solarwinds and related compromises. Sysdig does not use these tools internally. To maintain a secure SDLC process for own product we use Sysdig Secure as well as source code analysis tools. We also maintain our own branch of key OSS components to ensure software is fully vetted before it’s delivered to customers.

December 14, 2020

Perform Image Scanning as a GitHub Action

A new version of the Sysdig Secure inline scanning action has been released. This Github action allows you to perform image analysis on locally built container images.

The action uses the new secure-inline-scan 2.x, which provides better performance and more input options. See Inline Scan 2.2 Release Note.

The action provides the following benefits:

  • Image evaluation results can be consumed using the Sysdig Secure UI or locally as check-run annotations.

  • Support for SARIF report output.

    This provides native integration with Github’s code scanning, for example: executing the codeql-action/upload-sarif action.

See also: Inline Scanning.

December 11, 2020

New Runtime Policy Events JSON Format

The JSON format for the runtime policy events has been upgraded to include full scope information, rule labels, and a single-line representation for the event field’s keys and values.

To preserve backwards compatibility with existing integrations, the former JSON format is still available (and used by default on migration).

From the Event Forwarder page, under “Data to Send,” the old JSON format is labeled “Policy Events (Legacy)” and the new one as “Runtime Policy Events.”

See also: Event Forwarding.

December 7, 2020

Node Image Analyzer v 0.1.6 Released

Version 0.1.6 of the Node Image Analyzer has been released.

New

  • Proxy configuration supportRunning Node Image Analyzer Behind a Proxy

  • Added support to scan images that lack a Repo tag, such as OpenShift 4.x distribution images.

Fixes

  • Updated dependencies and base images to keep up with latest fixes

December 2, 2020

Inline Scanner v2.2

Version 2.2 of the Inline Scanner has been released.

New:

  • The vulnerability report information has been added to the container output, together with the image details and policy evaluation

  • When scanning an image using the digest pullstring, it will be stored using the truncated digest as a tag. For example:

    postgres@sha256:839d6212e7aadb9612fd216374279b72f494c9c4ec517b8e98d768ac9dd74a15 will show up in the interface as postgres:839d6212e7aa
    

Fixed:

Fixed a permissions issue when running the container with a user other than root.

See also: Integrate with CI/CD Tools.

November 23, 2020

Inline Scanner v2.1

V2.1 of the inline scanner container has been released.

New: Ability to analyze scratch-based images

Fixes:

  • Fixed a bug retrieving the PDF output for previously scanned images

  • Addressed several vulnerabilities found in the inline scanner container

See also: Integrate with CI/CD Tools.

November 19, 2020

Kubernetes-Native Network Security with Sysdig Secure (Beta)

A new feature has been added to Sysdig Secure for authoring and refining Kubernetes network policies (KNPs) that:

  • Automatically extracts the connection information, by observing the cluster networks and microservices communications

  • Offers a visual flow to fine-tune the Kubernetes network policies, incorporating the user’s adjustments

  • Automatically generates the KNP YAML to be applied, without requiring previous Kubernetes policy knowledge from the user.

As soon as the feature is enabled, the Sysdig agent starts collecting and processing application communications, which are then enriched using Kubernetes metadata and presented in two different ways:

  • Topology maps: a visual representation of the network flow between the Kubernetes entities (Services, Deployments, StatefulSets, DaemonSets, Jobs)

  • Ingress / Egress tables: for additional detail on each inbound/outbound communication and policy tuning.

Once the user has finished editing the desired policy, Sysdig will automatically compute the associated KNP YAML:

  • Enforcement is delegated to the Kubernetes control plane, favoring policy-as-code and avoiding direct tampering with cluster communications

  • Allow-only approach ensures that any communication which is not explicitly allowed by the policy will be forbidden

Prerequisites

Sysdig agent version 10.7+

Supported Orchestrator Distributions and CNI Plugins:

  • Vanilla Kubernetes (kops, kube-admin) using Calico

  • OpenShift 4.x using OVS

  • Google GKE using Calico

  • Amazon EKS using Calico

  • Rancher Kubernetes using Calico

Please contact us to enable this feature for your Sysdig Secure accounts.

See also: Network Security Policy Tool .

November 13, 2020

Terraform Provider Update (v0.5.4)

Terraform v0.5.4 update is available in Sysdig Labs.

The following minor bug fixes are included:

  • sysdig_secure_policy resource can configure the response action Kill Container

  • Fixed severity field in sysdig_secure_policy resource, to accept all possible values

October 29,2020

Scan Results List Updated

The UI for the list of scanned images has been updated to include several functionality and design improvements:

  • Status column (Passed or Failed) is now filterable

  • Image Origin (Inline Scanner, Node image analyzer, etc.) is now visible, filterable, and has multi-select option

  • Image registry is now visible on the table

  • Ability to sort by date-added (default) or image name

  • Flexible free-text search: filter by registry/repo:tag, repo:tag, repo, etc.

See also: Review Scan Results.

October 26, 2020

Inline Scanner 2.0

A new version of the Sysdig inline scanner script has been released.

Major improvements:

  • The inline analysis container doesn’t need to spawn any additional containers

    • This removes the requirement for the Docker client, docker-in-docker, etc.

    • This enables usage in environments where docker-in-docker is not feasible or hard to instrument (e.g., Tekton).

  • Additional analysis workflows and formats:

    • Added support to analyze a docker archive

      • A .tar.gz file containing the image, i.e. the output from a “docker save”

      • Example execution

    • Added support to analyze OCI images (both and directory and archive)

    • Added support to retrieve an image from the container storage (CRI-O and others)

Additional improvements:

  • Faster image ingestion

  • More verbose logs available for troubleshooting and diagnosis

  • Machine-readable JSON output via --format JSON command

To upgrade an earlier Sysdig Inline Scanning version to 2.0, you need to take into account the new invocation parameters, which are not backward compatible.

Sysdig Inline scanner can be used stand-alone or as a step inside a CI/CD pipeline (Jenkins, Tekton, CircleCI, etc). In the upcoming weeks, we will update the different integrations to provide out-of-the-box support for the 2.0 version.

October 22, 2020

Forwarding the Activity Audit Information

The Sysdig Secure Event Forwarder has added support to forward Activity Audit data to external platforms.

Benchmarks support for Kubernetes Benchmark 1.6

  • Kubernetes Bench upgraded to version 1.6

  • Using the Kubernetes benchmark, we now provide customer-selected benchmark checks for GKE and EKS (rather than just the Kubernetes default).

October 9, 2020

Regulatory Compliance Control Validation & PCI Checks

A new feature has been added to Sysdig Secure for checking controls from various compliance standards. For the first release, we provide checks against specific controls in PCI 3.2. Future releases will include SOC2, NIST-800-53, and more. See also: Compliance in Sysdig documentation.

Compliance Validator and Reports

The validator checks many Sysdig Secure features, including: image scanning policies, Falco runtime policies and rules, scheduled benchmark testing, Admission Controller, Network Security Policies, Node Image Analyzer, and more. Over time we will add new compliance coverage.

Disclaimer: Sysdig cannot check all controls within a framework, such as those related to physical security.

This feature is a beta release. A Sysdig Secure admin must enable it from the Sysdig Labs interface under Settings.

PCI Control Details

The PCI Quick Reference describes the full range of controls required to pass a PCI 3.2 audit. In this release, Sysdig Secure will check the following subset:

Controls 1.1.2, 1.1.3, 1.1.6.b, 2.2, 2.2.1, 2.2.2, 2.2.a, 2.4, 2.6, 4.1, 6.1, 6.2, 6.4.2, 6.5.1, 6.5.6, 6.5.8, 7.1.2, 7.2.3, 10.1, 10.2, 10.2.1, 10.2.2, 10.2.3, 10.2.6, 10.2.7, 10.3, 10.5.5, 10.6.1, 11.4, 11.5.a, 11.5.b.

October 2, 2020

Event Forwarding: Kafka and Webhook Added

Two new supported integrations have been added to the Sysdig Secure Event Forwarder:

The Kafka topic integration includes support for:

  • Multiple Kafka brokers

  • Partitioner/Balancer algorithms: Murmur2, Round robin, Least bytes, Hash, CRC32

  • Compression algorithms: LZ4, Snappy, Gzip, Zstandard

The Webhook integration includes support for:

  • Authentication methods: Basic authentication, Bearer Token, and Signature Header

  • Custom headers defined by the user to accommodate any additional parameter required on the receiving end

September 29, 2020

Vulnerability Exceptions Handling Enhanced

The Vulnerability Exceptions feature in Sysdig Secure has been redesigned and enhanced.

It now offers:

  • Additional vulnerability and feed context

  • Precise mapping between images and their associated exceptions

  • A better exception management lifecycle

  • Multiple vulnerability lists, which can be flexibly assigned to different image sets (or just a particular image), using the scanning policy assignments

  • Additional information displayed to improve team awareness and security context

    • Vulnerability description

    • User-defined notes

    • Vulnerability feed info, with severities and links as provided per feed

  • Configurable expiration dates:

    • An exception is automatically disabled when the expiration date is met

    • Day resolution, all times relative to 0:00 UTC

  • Enhanced workflow integration with the “Scan results” page for an individual image, with the ability to quickly append a flagged vulnerability to a list.

Migration: The exception and evaluation behavior in the current environment will be maintained after the feature upgrade. In particular:

  • Pre-existing vulnerability exceptions will be migrated to the “Default exceptions list”

  • The “Default exceptions list” will be assigned to every pre-existing policy assignment

  • All the pre-existing vulnerability exceptions expiration date will be set to “Never.”

See also: Manage Vulnerability Exceptions and Global Lists.

AWS Threat Detection using CloudTrail and Sysdig Secure

Sysdig is happy to announce the general availability of a CloudFormation Template that will deploy a cloud-native operational security engine. By leveraging AWS CloudTrail and the Falco language, you can detect any unexpected or unwanted behavior in your AWS accounts.

Sysdig Cloud Connector leverages AWS CloudTrail as the source of truth for enabling governance, compliance, operational auditing, and risk auditing for your AWS account.

Every API action over your infrastructure resources is recorded as a set of CloudTrail entries. Once the integration is deployed in your infrastructure, the Sysdig Cloud Connector can analyze these entries in real-time and provide AWS threat detection by filtering them against a flexible set of security rules.

Example detection rules included in this release:

  • Attach a user to an Administrator Policy

  • Create an HTTP Target Group without SSL

  • Deactivate MFA for user access

  • Delete S3 bucket encryption

Sysdig Cloud Connector provides several notification options, including sending security findings to AWS CloudWatch and AWS Security Hub. When configured, you can consume the security events without leaving your cloud console.

See also: https://sysdiglabs.github.io/cloud-connector/.

September 28, 2020

Automated Fargate Image Scanning

Sysdig is pleased to announce the general availability of a new integration leveraging the Sysdig Inline Scanning capabilities to automatically analyze the base images used for any task created using AWS Elastic Container Service (ECS or Fargate).

  • Straightforward deployment using a CloudFormation template

    The only mandatory parameter is the Sysdig API token.

  • Inline scanning living inside your AWS account means improved security:

    • No need to expose or configure private AWS registries

    • Only image metadata is sent to Sysdig Secure, not the actual image contents

    • No sensitive information ever leaves your AWS account

    • An ephemeral task will be spawned to analyze each discovered images, in parallel

  • Each time you deploy a new task in AWS ECS/Fargate, an EventBridge event will be triggered and a lambda function will parse which images need to be analyzed by the CodeBuild pipeline job.

    • Fully automated

    • Scan results and scanning policies are still controlled from a single security governance point using Sysdig Secure

Node Image Analyzer Version 0.1.3

This version adds support for running the node image analyzer in Kubernetes environments with containerd, such as Google Kubernetes Engine configured with cos_containerd. See also: Scan Running Images.

July 29, 2020

Replacing RHSA Advisories with CVE Advisories

In new images scanned, RHSA advisories will be replaced with CVE advisories. The results for existing images will be updated in the background over the next week.

This change provides better matches for CVEs that are not yet fixed or will not be fixed since those do not yet have RHSAs. It also makes the CVE the match key rather than RHSA for more consistent whitelisting and policy handling compared to other distros.

Scanning Adapter Available for Harbor

The Sysdig Secure Harbor Scanner Adapter enables Harbor to use Sysdig Secure scanning engine to analyze the container images managed by the platform.

This adapter also provides a service that translates the Harbor scanning API requests into Sysdig Secure API calls, allowing Harbor to retrieve vulnerability reports and additional information from the scanning adapter. This will be presented transparently in the Harbor UI to the user.

The scanning adapter supports two operation modes:

  • Backend Scanning: Image scanning happens in the Sysdig Secure backend

  • Inline Scanning: Image scanning happens in the infrastructure where Harbor is hosted

To learn more about this integration, read the documentation.

July 28, 2020

Captures Filter on the Policies Page

Policies can now be filtered to display if a capture is associated with an active or inactive policy.

Image Exclusion on Policy Events

Users often want to tune policy events. We’ve added a button on the event detail that will add an exclusion to a specific container.image.repo for the policy that triggered the event. Once that exclusion is applied to the scope, policies will no longer fire for that container.image.repo.

July 26, 2020

Sysdig Essentials

We have introduced a new product tier, Sysdig Essentials. This tier includes everything required to achieve the five essential requirements for practicing Secure DevOps:

  • Image Scanning

  • Runtime Security

  • Compliance

  • Kubernetes and container monitoring

  • Application and cloud service monitoring

To learn more about Essentials, register for our webinar, Deploy Faster by Automating Container Security, Monitoring and Compliance.

With the introduction of Essentials, It’s also easier to get started with a trial program and manage your Sysdig subscription.

Learn the difference between Essentials and Enterprise, including pricing and features, at Pricing.

Sysdig Platform Enhancements

SAML Single Sign-On

The initial email to the following types of users will take them directly to the Single-Sign-On URL, and not the registration page.

  • SAML SSO Users

  • The users that are invited to the platform (as opposed to having them automatically created via Sysdig on-demand provisioning for SSO)

Earlier, landing on the registration page was confusing to users because they had to set up their initial password.

Rebranded Login Page

The login page has been updated with the Sysdig Kraken and the new logo.

June 29, 2020

New Sysdig Secure Overview Page

The Sysdig Secure Overview provides an at-a-glance view of the critical areas of your security posture.

Scoping

Panels can be scoped by Cluster or Namespace. The scope will update all panels that are displaying run-time data and the corresponding drill-down views.

Panels

  • Build Time - Images Scanned: Image scan results for all static image scans

    Drill-down - To Image Scanning Reports page.

  • Build Time - CVEs Found by Severity: The total number of CVEs present in each image scanned.

    Drill-down - Available in a future release

  • Run-time - Images Scanned: The pass/fail status of images running now and their trend over time.

    Drill-down - To Runtime Scanning Image page.

  • Run-time - CVEs by Severity: The total number of CVEs present in each running image

    Drill-down - Available in a future release

  • Run-time - Policy Events by Severity: The total number of policy events by severity.

    Drill-down - Secure Events page.

  • Benchmarks Tests Failing: The total number of benchmark tests that have failed.

    Drill-down - Benchmarks Results page.

See also: Secure Overview .

June 26, 2020

Sysdig Secure’s Event Forwarder Now Supports IBM Cloud Pak for Multicloud Management and IBM QRadar

IBM Cloud Pak for Multicloud Management centralizes visibility, governance, and automation for containerized workloads across clusters and clouds into a single dashboard.

You can now forward security events to an IBM MCM instance by accessing the Settings > Event Forwarding menu and selecting IBM MCM from the dropdown:

IBM QRadar Security Information and Event Management (SIEM) helps security teams accurately detect and prioritize threats across the enterprise and provides intelligent insights that enable teams to respond quickly to reduce the impact of incidents.

You can now forward security events to an IBM QRadar instance by accessing the Settings > Event Forwarding menu and selecting IBM QRadar from the dropdown.

See also: Event Forwarding.

June 23, 2020

Secure Events Feed Overhaul

The Events feed in Sysdig Secure (formerly called Policy Events) has been redesigned, both visually and functionally.

Apart from the styling and user experience improvements, these are the major new features and use cases

Advanced Filtering

We are deprecating the grouping/clustering of events present in the old version in favor of a much more powerful set of filtering capabilities:

  • Severity filters: Presented as quick buttons at the top, supporting multi-select

  • Attribute filters: Provide a simplified syntax to filter events by the attributes they contain. For example ruleType="Falco - Syscall" or image.repo!="sysdig/agent"

    • Open the event details side panel to find quick filtering widgets to include or exclude the attribute values associated with the displayed event
  • Event type selector: Supports runtime scanning alerts on top of policy runtime events (see section below), with an easy multi-selector in the UI.

  • Free text search: Allows you to search the event titles and scope label values. I.e. Terminal shell in or my-k8s-cluster.

  • New scope selector: Allows for additional selector logic (in, not in, contains, startswith, etc), improving the scoping flexibility over earlier versions. This scope selector also provides scope variables, allowing you to quickly switch between, for example, Kubernetes namespaces without having to edit the panel scope.

All these filters can be combined additively to further refine your search.

Multiple Event Types

The new event feed displays not only the policy runtime events, but also runtime image scanning alerts.

The backend architecture, filtering, and UX have been designed to accommodate additional types of security events that will be pushed to the Event Feed in the future, upgrading the interface from a policy-runtime-centric experience to a full security center control panel.

Additional Event Details

Policy runtime events: These now display the rule that was fired together with the rule labels. You can use the quick filters mentioned above to further refine the search.

Richer scope: Every security event now displays all the scope labels retrieved for the event, not just those configured in the scope selector.

See also: Secure Events.

Additional Considerations/Limitations

Events in the old and new format will be stored separately:

  • No event or event data will be lost during the transition

  • Events that were registered before the new feed is deployed can be browsed using the old feed interface, which is available from the burger menu in the top-right corner

  • Events that happen after the new feed is deployed will appear in the new event feed

  • Eventually, all events within the retention period will be present in the new interface, at which point the version switcher will disappear

June 17, 2020

The ordering of the side menu has been changed.

Image Scanning Updates

The image scanning navigation bar has changed.

  • The side menu is reorganized into Analyze and Configure sections

    • Analyze: Different areas of scanning that allow users to view scan results

    • Configure: The areas of scanning that involve the setup of the application

  • Whitelist terminology with CVEs has been removed.

    “CVE whitelist” is now CVE Exceptions.

Team, Role, and Channel Updates

A variety of enhancements have been added to the team, role, and notification channel options.

Service Manager Role Added to Sysdig Secure

RBAC capability was previously added to Sysdig Secure. (See also January 27, 2020 and User and Team Administration.)

Now a new role, Service Manager, is also available in Secure. It has the same permissions as the Standard User, plus the ability to invite existing users to the team and manage the notifications channels assigned to the team. See Team-Based Roles and Privileges

Configurable Default Team Role

You can now define the default user role to apply when a new member is added to the team. The Admin can change this default on a per-team basis. See also: Create a Team.

RBAC and Team Assignment for Notification Channels

Previously, notification channels in Sysdig Secure and Monitor were treated as global entities, visible and editable for most users of the platform regardless of team configurations.

We are enhancing the management and RBAC controls in the following ways:

  • Notification channels can now be “global” or limited to a particular team

  • Global channels can be managed by admins and can be viewed/used by other roles, while team-limited channels are available only to team members

  • Team Manager , Advanced User, and Service Manager (Secure) roles can create/update/delete team-scoped notification channels, they can also read and use the global ones

  • Standard and View Only roles can read team-limited and global notification channels

  • Admins will be able to create global notification channels and migrate channels from “global” to “team-limited”, and also from one team to another.

See also: Set Up Notification Channels and the Share With field in each individual channel setting page.

June 12, 2020

CLI-Based Admission Controller for Image Scanning

An additional tool for evaluating and admitting images is now available.

Sysdig Admission Controller

Sysdig’s Admission Controller (UI-based) combines the Sysdig Secure image scanner with a policy language to evaluate scan results and the admission context, providing great flexibility in the admission decision. It also provides the first line of defense against image-based security threats.

By using Kubernetes API extensions to perform image scanning and other security checks on admission, we cover a major threat-prevention and hardening use case: “Only the images that are explicitly approved will be allowed to run on my cluster”.

The admission decision relies not only on the image name and tag but also on additional context from the admission review, including namespace, pod metadata, etc.

Features

  • Registry and repository whitelist / blacklist

  • Global and per-namespace admission configuration

  • Configurable pre-scan and post-scan behavior, i.e.:

    • Accept only the images that pass the scan (default)

    • Directly reject non-whitelisted registries / repos, without scanning

    • Accept the image even if it doesn’t pass the scan

    • Do not accept any image that hasn’t been scanned already

  • Pod mutation: image tag is replaced by digest to prevent TOCTOU (Time of Check, Time of Use) issue if the tag is updated between the scan and the pod scheduling

Requirements

  • Helm 3

  • Kubernetes 1.15 or higher

For more information, see Admission Controller .

June 4, 2020

New Vulnerability Feed Available: VulnDB

We’ve added VulnDB as an additional 3rd-party vulnerability source to improve Sysdig’s coverage in non-OS package vulnerabilities.

In addition, a new page is available for each VULNDB-linked advisory. It lists the CVEs and details about the Common Vulnerability Scoring System (CVSS) scores and external references.

See also: Vulnerability Databases Used.

May 11, 2020

Optimized Runtime Page

We’ve released a new Runtime page for the Image Scanning module within Sysdig Secure. Improvements include:

  • Filtering based on pass/fail/unscanned

  • The ability to search results for a specific image

  • Optimized queries to improve response times

For more information, see Review Scan Results.

April 20, 2020

Added Automatic Image Scanning using Node Analyzer

The (node) image analyzer (NIA) provides the capability to scan images as soon as they start running on hosts where the analyzer is installed. It is typically installed alongside the Sysdig agent container.

This component was introduced to reduce dependencies on analyzing images within the Sysdig backend (SaaS or On-prem). Some advantages include:

  • Sharing credentials with the Sysdig backend in order to pull images is not required

  • Sharing the image content and potentially code with the Sysdig backend is not required; only metadata will be sent out

  • Opening a network route to allow the Sysdig backend to reach the user’s registries is not required

If you have run the single line agent install with the --image-analyzer flag, then this component is already running in your infrastructure.


The feature is available for Kubernetes environments.

For more information, see Scan Running Images.

April 13, 2020

Added Image Scanning Integration Options

Two new scanning integrations are available for CI/CD pipelines. Sysdig provides:

  • A reference implementation with Tekton Pipelines (prototype)

  • A fully supported integration with Amazon Elastic Container Registry (ECR) for triggering auto-scans from the registry

Integrating Secure Image Scanning with Tekton Pipelines

Tekton Pipelines allow you to implement CI/CD workflows using a highly modular, cloud-native approach that:

  • Uses containers as the building blocks for individual tasks

  • Runs directly on Kubernetes/OpenShift without requiring a dedicated infrastructure

  • Uses tasks that are purely declarative and described using their own CRD, making them easily composable and reusable

Sysdig’s reference implementation details the prototype task to invoke Sysdig Secure image scanning as a pluggable step in your CI/CD pipeline with just a YAML file:

Leveraging Tekton integration with the orchestration layer, you can retrieve the image scanning policy evaluation and state (pass/fail) directly from the logs of the task pod.

Read the “Securing Tekton pipelines in OpenShift with Sysdig” blog post for additional details

Integrating Secure Image Scanning with Amazon ECR

Automatically scan images pushed to your Amazon Elastic Container Registry (ECR) using AWS-native technologies and Sysdig Secure.

Sysdig image scanner integration is deployed as a CloudFormation template that listens to ECR registry events and uses AWS resources to streamline the image scanning process.

  • ECR itself will trigger the scan, no need for your CI/CD pipelines to actively pull from the registry

  • Deployed in a few clicks, you just provide basic configuration parameters such as the Sysdig API token or the Sysdig backend URL

  • No need to configure registry scanning credentials on the Sysdig Secure side

This integration offers two different operation modes

Inline scanning:

  • Scanning will be performed inside an AWS CodeBuild pipeline allocating ephemeral resources

  • No need to configure any registry credentials for Sysdig Secure

  • No need to expose your ECR registry to the Sysdig Secure backend

  • Sysdig Secure will not retrieve the image contents, only the metadata that is required to perform the policy evaluation

Backend scanning:

  • Sysdig Secure will retrieve the full image contents in order to perform the scan

  • Your ECR registry must be reachable by the Sysdig Secure backend

  • Registry credentials are required, but they are pushed automatically by a lambda function, no need for manual configuration

April 9, 2020

Updates to Default Rules and Policies

The following changes have been made to default Policies in Sysdig Secure, and to default Falco rules:

  • New rule tags added that map Falco rules to PCI and NIST controls

  • New default policies added specifically for PIC/NIST compliance

  • Tuning modifications for:

    • Write below etc

    • Write below root

    • Change thread namespace

    • Run shell untrusted

    • Detect outbound connections to common miner pool ports

For more information, see also Falco Rules Changelog.

April 7, 2020

Updated Inline Scan Script

  • Added header values for import API for better supportability.

  • Upgraded to Anchore engine v0.6.1.

  • Use docker:dind instead of ubuntu for the base image. This reduces the image size and speeds up downloading.

The latest version of the inline script will always be available at https://download.sysdig.com/stable/inline_scan.sh

Link to repo for script source code: https://github.com/sysdiglabs/secure-inline-scan

March 12, 2020

New Get Started Page

The Get Started page provides the key steps to ensure users are getting the most value out of Sysdig Secure. We’ll update this page with new steps as we add new features to Sysdig Secure. 

The Get Started page also serves as a linking page for:

  • Documentation

  • Release Notes

  • The Sysdig Blog

  • Self-Paced Training

  • Support

Users can access the page at any time by clicking the rocketship in the side menu.

See also: Getting Started with Sysdig Secure.

Linux CIS Benchmark Test Added

Sysdig Agents can run the Independent Linux benchmark against the underlying host where the agent is installed. The Linux benchmark can be scheduled to run at a chosen interval in your environment and emits results and metrics about the status of the tests.

Openshift Hardening Guide

The Openshift hardening guide implements configuration checks run by the agent against Openshift environments.

See https://access.redhat.com/documentation/en-us/openshift_container_platform/3.11/html/container_security_guide/index

Note: This is supported for 3.x versions of Openshift. When Openshift releases a hardening guide for 4.x versions, we will update the configuration checks.

Captures can be Routed to Specific Storage Locations

As a user, you may have different S3 buckets where you’d like to store Sysdig captures, based on the environment where the policy event was triggered. New options are available for deciding what storage option you’d like to use for each policy event.

Feeds Status Page Added

It’s useful to understand the last time the feeds were updated, especially in self-hosted environments. The Feeds Status page shows the different vulnerability feeds we integrate with, their feed group (often the distro version), the time of the last sync, and how many CVE records are present in the feed group.

See also: Feeds Status.

March 5, 2020

Inline Scanning Reporting Improvements and Documentation

This script from SysdigLabs is useful for performing image analysis on locally built container images and posts. The only dependency for this script is access to docker-engine, Sysdig Secure endpoint (with the API token) and network connectivity to post image analysis results.

Here are examples of using the inline scanner in different pipelines:

PDF Reports from the Inline Scanner

A new option

-R  [optional] Download scan result pdf report

will generate a PDF artifact that is available for developers to consume in the pipeline.

February 6, 2020

Data Retention Limits for Scan Results

Use this feature to set limits on how long image scan metadata is stored, either by tags or days. This removes stale data and helps keep scan results easy to read.

See Data Retention for details.Data Retention

January 29, 2020

Enhanced Kubernetes Audit Log Integration

We’ve extended our test and documentation coverage for various Kubernetes audit log integrations. This integration enables Sysdig Secure to use Kubernetes audit log data for Falco rules, activity audit, and to test the impact of Pod Security Policies.

We now have examples for:

  • OpenShift

  • Minishift

  • Kops

  • GKE

  • EKS

  • RKE

  • IKS

  • Minikube

Read more here: Kubernetes Audit Logging.

Vulnerability Scan Results Comparison

In image scanning reports, the vulnerability comparison feature allows users to compare two different tags within the same repo to see which vulnerabilities are new or have been fixed in version X compared to version Y.

This allows developers easily to compare the latest image to a previous version to easily report on which vulnerabilities have been addressed and which are new.

See Review Vulnerability Summaries for details.Review Vulnerability Summaries

January 28, 2020

File Data Source Support for Activity Audit

Sysdig Secure’s Activity Audit now supports a new data source element: File activity.

Sysdig agent version 9.5.0+ is required to enable this new data source.

  • You can now filter the audit trail by file type or specific file attributes:

    • File name

    • Directory

    • Command (used to access the file)

    • Access mode

  • File activity is also visible in the time-series graph at the top (pink color):

  • Activity Audit will capture non-read file operations executed by interactive commands

January 27, 2020

RBAC Capability Available in Sysdig Secure

The new role-based access control (RBAC) model available in Sysdig Secure allows you to define the access privileges granted to each user in a Sysdig Secure team.

Besides the Admin role, which has full access and belongs to every team, there are four roles that can be assigned when adding a user to a team. (Note that the role names are the same in Monitor and Secure, but the privileges differ slightly. Users must be assigned Monitor team roles and Secure team roles separately.)

  • View Only: Read access to every Secure feature within the team scope. A View Only user cannot modify runtime policies, image scanning policies, or any other content.

  • Standard User: Can push container images to the scanning queue and view the image scanning reports. Standard Users can also display the runtime security events within the team scope. They cannot access the Benchmarks, Activity Audit. or Policy definition sections of the product.

  • Advanced User: Can access every Sysdig Secure feature within the team scope in read and write mode. Advanced Users can create, delete, or update runtime policies, image scanning policies or any other content. The Advanced User cannot manage other users.

  • Team Manager: Same permissions as the Advanced User + ability to add/delete team members or change team member permissions.

    Team Managers only have user administration rights within the specific team(s) for which they are designated Managers.

See User and Team Administration for details.

January 16, 2020

Redesigned Captures Page

The Captures function in Sysdig Secure has a new look and the following usability improvements:

  • Bulk deletion of capture files

  • Ability to see whether a capture was triggered manually or by a policy

  • Search across all capture files

November 13, 2019

Activity Audit (Beta)

The Activity Audit in Sysdig Secure allows you to browse a live stream of activity from your Kubernetes containers and nodes. Audit takes the highly detailed data from syscalls and Kubernetes audit logs captured at the agent level, and makes it always-on, searchable, and indexed against your cloud-native assets.

This stream includes executed commands, network activity, and kubectl exec requests to the Kubernetes API. The Activity Audit allows users to view different data sources in-depth for monitoring, troubleshooting, diagnostics, or to meet regulatory controls (SOC2, NIST, PCI, etc).

Flexible filtering and scoping to help you focus on what’s relevant: Filters allow you to search, sort, and surface meaningful data and connections as they are needed. You can filter by data source type, data source attributes (like command name or Kubernetes user) and dynamic Kubernetes scope

Automatically trace a kubectl exec session : The built-in trace functionality allows you to isolate and trace a kubectl exec access to a pod, automatically correlating the original Kubernetes user and IP that accessed the pod with the activity that was performed during the interactive session, including commands and network connections.

Kubernetes Policy Advisor (Beta)

With the Kubernetes Policy Advisor, Sysdig Secure auto-generates Pod Security Policies (PSPs) to significantly decrease the time spent configuring Kubernetes Policies. Strict security policies reduce risk, but can also break applications. Sysdig tests the impact of pod security policies through simulations, enabling teams to adjust misconfigurations before shifting to production. There are three main features that comprise the Kubernetes Policy Advisor:

Auto generation: Sysdig Secure can parse any Kubernetes yaml file that includes a pod spec to generate a tailor-made PSP based on the configuration.

Simulations: Start a simulation of the auto-generated PSP or any user-inputted PSP to see what pods would have been blocked from running if this PSP had been actively applied to the cluster.

Events and tuning: Each pod/activity that would have violated the PSP will generate an event. Within the event details, users can see information about potential modifications they may need to make to the policy or the pod configuration.

Image Scanning Improvement

Support for images based on Google distro-less OS, including detection of base OS/version and installed OS dpkg packages.

November 4, 2019

Scanning Improvements

New Scanning Rules

File attributes can now be verified as part of the image scan analysis. A specific file can be validated against a node or sha256 hash.

Scale Improvements to Scanning Reporting

No query conditions are required as part of the Package and Policy Queries.

October 10, 2019

In-Line Scanning

Images can now be analyzed locally before they are pushed to a registry. This has a couple key benefits to users.

  • Images can be analyzed before they’re pushed to a registry and reduce registry cost

  • Customers using the Sysdig Secure SaaS offering don’t need to expose their registry to our SaaS for images to be scanned

  • For openshift customers the in-lince scan option can be integrated into the S2I process to scan images without needing to expose a local cluster registry via a route

Learn more and access the script here: https://github.com/sysdiglabs/secure-inline-scan

Sysdig CLI

The Sysdig CLI provides an easy way to interact with the cli via the command line. Read more here.

Usage

Run it without parameters to get a list of all the commands.

$ sdc-cli
Usage: sdc-cli [OPTIONS] COMMAND [ARGS]...

  You can provide the monitor/secure tokens by the SDC_MONITOR_TOKEN and
  SDC_SECURE_TOKEN environment variables.

Options:
  -c, --config TEXT  Uses the provided file as a config file. If the config
                     file is not provided, it will be searched at
                     ~/.config/sdc-cli/config.yml and /etc/sdc-cli/config.yml.
  -e, --env TEXT     Uses a preconfigured environment in the config file. If
                     it's not provided, it will use the 'main' environment or
                     retrieve it from the env var SDC_ENV.
  --json             Output raw API JSON
  --version          Show the version and exit.
  --help             Show this message and exit.

Commands:
  alert       Sysdig Monitor alert operations
  backup      Backup operations
  capture     Sysdig capture operations
  command     Sysdig Secure commands audit operations
  compliance  Sysdig Secure compliance operations
  dashboard   Sysdig Monitor dashboard operations
  event       Sysdig Monitor events operations
  policy      Sysdig Secure policy operations
  scanning    Scanning operations
  settings    Settings operations
  profile     Profile operations

New Package Reports

Package name/version are now grouped together to provide easy parsing of all CVE’s associated with a package and the images using that package.

Sept 24, 2019

New Trigger Parameters for CVSS Score

Image Vulnerabilities can now be evaluated against their CVSS (Common Vulnerabilities Scoring System) score. If a vulnerability is =, <;>, <=, or >= to a specific score, then the rule can trigger a warn/stop action.

Sept 18, 2019

Time Ranges Updated

The default time range options have been updated in Sysdig Secure.

The default time ranges are now set to:

  • 10 Minutes 

  • 30 Minutes

  • 1 HR

  • 6 HRs

  • 1 Day

  • 3 Days

To look at a custom window of time, use the manual time window.

Sysdig Secure Summary Dashboard in Sysdig Monitor

Sysdig Monitor includes default dashboards that provide metrics about number of agents installed, active policies, events that have occurred, and the policies that have triggered them. Use these dashboards to identify trends, report on coverage, or facilitate the tuning process.

Aug 12, 2019

Policy Editor

*Please upgrade to an agent version 0.92.0 or greater

This UX overhaul brings three major improvements for every Sysdig Secure user:

  • Runtime policies can import any number of security rules. You can scope the security policy using container, cloud and Kubernetes metadata.

  • Tighter Falco integration, directly from the web UI. You will be able to define a new trigger condition or append to the list of forbidden external IPs just clicking on the rule.

  • A more structured way to group, classify and lookup rules, following the standard Cloud native procedure: tags and labels.

Rules Library

Visualize your runtime rules properties in just a glance:

  • Where this rule comes from (Published By). The security team can instantly recognize whether a rule came from a specific Sysdig update, from a custom rules file created within the organization or from an external rules source (like the Falco community rules).

  • When was the last time it was updated (Last Updated). You can use this information to audit your rules or if you schedule periodic updates, to confirm when last happened.

  • Rule tags: An effective method for organizing your rules. You can use these tags to describe the targeted entity (host, k8s, process), the compliance standard it belongs to (MITRE, PCI, CIS Kubernetes) or any other criteria you want to use to annotate your rules.

Falco Lists

Easily browse, append, and re-use lists to create new rules. Lists can also be updated directly via API if users want to add existing feeds of malicious domains, or IPs.

Falco Macros

Easily browse, append, and re-use macros to create new rules.

Image Scanning - View Scan Results

Scan Results Page - The existing repositories page has been renamed “Scan Results” this page also includes new capabilities to filter based on where the images are deployed, and to easily browse/expand the different repositories to see the image:tag’s that were evaluated and their results

Whitelist labels available in vulnerabilities view - If a vulnerability has been added to a whitelist then that status is reflected in the Vulnerability report within the scan results.

Image Scanning Reports

Please contact Sysdig Support to enable this feature

The reports feature allows users to query the contents of a scan against a static or run-time scope to generate a report that shows the risk, exposure, or components of an image.

Use cases could include:

  • A new CVE has been announced, let me find all the running images in my US East Cluster that are exposed to that CVE

  • Show me all images within my Google Container registry that have the tag prod and have a vulnerability with a fix that’s more than 30 days old

  • Show me all images with a high severity vulnerability with a fix that are running in my billing namespace

Types of Scanning Reports

There are three types of queries in the image scanning Reports:

Vulnerability Query Type

This report returns rows of vulnerabilities mapped to packages within images in a static or run-time scope. In the example above we can see the two images that are actively running in my environment now that have the CVE - CVE-2017-8831

Package Query Type

This report shows all images actively running in my environment that have a version of the bash package. It also shows if multiple images are running the same package name & version and if there are any CVE’s associated.

Policy Reports

Policy reports show all the policy evaluations that have occured, whether or not they passed or failed, and the reason why an image may have passed or failed. Reasons for passing or failing could be because of, whitelists, blacklists, or just a standard policy evaluation.

July 12, 2019

Minor Improvements

  • Compliance Dashboards in MonitorLink from Sysdig Secure now defaults to a 90-day view, to give users better visibility into how their posture is changing over time.

  • Image ScanningNegligible vulnerabilities are now also shown as part of the scan results summary.

June 27, 2019

Image Scanning: New Trigger Options

  • New Image Analyzed - Send notifications to different channels when images with a particular registry, repo, tag are scanned.

    • Some users implement these type of alerts for implementing workflows for image promotion, i.e.

      “Push an image from staging to prod registry after a webhook is sent that the image was scanned and it passed.”

  • CVE Update - Be notified whenever a vulnerability is added, updated, or removed from an image within a registry.

Repository Alerts

Receive alerts about activity and changes that occur within your registry. See Manage Scanning Alerts.

Slack Notifications

Sample output of a CVE alert:

Sample output of an image-analyzed alert:

5 - Sysdig On-Premises Release Notes

You may also want to review the update log for Falco rules used in the Sysdig Secure Policy Editor. See Falco Rules Changelog.

Oversight Services Now Offered for All Installs and Upgrades

As part of our continued focus on our customers, we are now offering oversight services for all on-premise installs and upgrades. Your Technical Account Manager (TAM), in conjunction with our support organization and Professional Services \[where applicable\], will work with you to:
  • Assess your environment to ensure it is configured correctly

  • Review your infrastructure to validate the appropriate storage capacities are available

  • Review and provide recommendations for backing up your Sysdig data

  • Work with you to ensure our teams are ready to assist you during the install and upgrade process

  • Provide the software for the install

  • Be available during the process to ensure a successful deployment

You can always review the process in the documentation on GitHub (v. 3.6.0+) or the standard docs site (for older versions).

If you are a new customer looking to explore Sysdig, please head over here to sign up for a trial on our SaaS Platform. Alternatively, you can contact us here.

Review the Sysdig On-Premises Release Support statement.

Supported Web Browsers

Sysdig supports, tests, and verifies the latest versions of Chrome and Firefox.

Other browsers may also work, but are not tested in the same way.

4.0.8 Hotfix July 2022

Supported Upgrades From: 3.6.X

Bug Fixes

  • Fixed an issue with PVC metrics not displaying properly in the UI
  • Fixed a filtering issue when RDS metrics would not populate in the RDS Overview Dashboard

5.1.2-2 Hotfix July 2022

Upgrade Process

Supported Upgrades From: 4.0.x, 5.0.x

Sysdig Platform

  • Added support for Openshift 4.10

5.1.2 Hotfix May 2022

Upgrade Process

Supported Upgrades From: 4.0.x, 5.0.x

Secure Feature: Reporting

  • Added the Run Now and Download(s) menu items

Bugs

  • Fixed an “Unable to load latest task result” bug when accessing compliance benchmarks results

5.1.1 Hotfix May 2022

Upgrade Process

Supported Upgrades From: 4.0.x, 5.0.x

Sysdig Platform

  • Added the RelayState parameter optional for SAML configuration
  • Upgraded the Spring Framework to version 5.2.20 in the sysdig-backend container

Monitor

  • Added the ability to choose regions with Capture Storage.

Installer Improvements

  • Fixed an issue with MultiAZ GCP/GKE platforms that would prevent Elasticsearch from starting.
  • Fixed an ingress permissions issue when upgrading from 5.0.4 to 5.1.0 that would result in the Sysdig UI generating a 404 Not Found error.
  • Fixed an installer bug when cloudProvider.name was set and cloudProvider.region was not set.
  • Fixed a Kafka/Zookeeper statefulset naming issue when installing or upgrading Sysdig on-premise

Bugs

  • Monitor Alert re-notification messages now provide the latest metric value instead of the metric value at time of triggering.
  • Fixed a Runtime scan page issue not displaying image results based on specific Team scopes.

Release 5.0.5 Hotfix for CVE-2022-22965

This hotfix upgrades the Spring Framework to version 5.2.20 in the sysdig-backend container.

Release 5.1.0 March 2022

Upgrade Process

Supported Upgrades From: 4.0.x, 5.0.x

For the full supportability matrix, see the Release Notes on Github. There you will also find important Install instructions.

Sysdig Platform

Installer Improvements

  • Kubernetes versions 1.22 and 1.23 are now supported.
  • An optional cronjob for the falco-rules-installer, which runs once a month, can now be created through the Installer values file.
  • Users operating their own ingress controller, such as Rancher, are no longer need to manually create Ingress Objects Go HTTP APIs. Note that the Collector uses TCP and will need external configuration.
  • The Installer now has a pre-flight check to verify the kubectl and Kubernetes versions of the cluster with the context provided by the user.

Secure

API Docs

  • API documentation for Sysdig Secure are now enabled by default.

Bugs

  • Fixed an issue with Secure Events not displaying the correct number of events in the dashboard.
  • Fixed an issue that prevented Rapid Response being enabled with a Secure Team created with LDAP.
  • Fixed a network issue that would sometimes occur during an upgrade which would cause PostgreSQL to timeout.
  • Fixed an issue when the nats-streaming-init container failed to start due to permission problem when storageClassProvisioner is set to hostPath.
  • Fixed a Compliance Database Password issue during upgrades from on-prem 4.0.x to on-prem 5.0.x
  • Fixed an issue with the StatefulSet definition when upgrading from 4.0.x to 5.0.x on a Kubernetes cluster prior to 1.18.x

Release 4.0.7/5.0.4 Hotfix for CVE-2021-44228 in Apache’s log4j (3.6.4, 4.0.7, 5.0.4)

The patch relese upgrades all components that compose Sysdig’s Platform running Apache’s vulnerable Log4j library to 2.16.

Note on ElasticSearch: This is using Log4j v2.11.1. An additional JVM parameter has been added through the Installer in accordance with the recommendations from Elastic. In addition, the impacted class from the Log4j library has been removed completely. Security scanners may still list this as vulnerable but in this case it will be a false positive. Elastic currently does not offer a way to fully remove or upgrade this component.

Release 4.0.6/5.0.3 Hotfix for CVE-2021-44228 in Apache’s log4j (3.6.3, 4.0.6, 5.0.3)

Security researchers recently disclosed the vulnerability CVE-2021-44228 in Apache’s log4j, which is a common Java-based library used for logging purposes Sysdig is using an alternative framework for logging called Logback. The logback framework isn’t vulnerable to this issue.

Sysdig components include a log4j library in our standard distribution that was vulnerable. This library is included for compatibility reasons only, is not used for primary logging, and our security team has determined we are not vulnerable based on our application architecture and existing mitigating controls.

We have released a patch version of our self hosted-software which upgrades the vulnerable version of log4j or adds additional mitigating controls suggested by vendors.

  • 3.6.3
  • 4.0.6
  • 5.0.3

Please reach out to support or the customer success team for assistance with your upgrade.

Release 5.0.2 Hotfix December 2021

Upgrade Process

Supported Upgrades From: 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.0.5, 5.0.0, 5.0.1

For the full supportability matrix, see the Release Notes on Github. There you will also find important Install instructions.

Fixes

  • Fixed a version-comparison bug in RedHat rpm packages.
  • Enabled a retention manager for Secure-only on-prem installations to handle data retention.

Release 5.0.1 Hotfix November 2021

Upgrade Process

Supported Upgrades From: 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.0.5, 5.0.0

For the full supportability matrix, see the Release Notes on Github. There you will also find important Install instructions.

Fixes

  • Fixed missing field “Last Evaluation Date” in the scanning policy evaluation results and Scheduled Reports
  • Kubernetes environment / labels are no longer mandatory to generate a scanning Scheduled Report
  • Fixed CVSS filters in scanning Scheduled Reports
  • Fixed an issue in scanning Scheduled Reports when scanning Red Hat images that caused related Red-Hat advisories (RHSA) to not be displayed
  • Fixed priority sorting for ‘Unknown’ severity vulnerabilities that are now considered less severe than ‘Negligible’ in scanning Scheduled Reports

Release 4.0.5 Hotfix October 28, 2021

Upgrade Process

Supported Upgrades From: 3.6.2, 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4

For the full supportability matrix, see the Release Notes on Github. There you will also find important Install instructions.

Fixes

  • Fixed Scheduled Reports not displaying last evaluation date field
  • Fixed an issue in 4.0.x Scheduled Reports when scanning Red Hat images, causing vulnerabilities missing related Red-Hat advisory (RHSA) to not be displayed

Release 4.0.4 Hotfix September 29, 2021

Upgrade Process

Supported Upgrades From: 3.6.2, 4.0.0, 4.0.1, 4.0.2, 4.0.3

For the full supportability matrix, see the Release Notes on Github. There you will also find important Install instructions.

Fixes

  • Fixed a timeout issue for policy advisor and scanning database init containers occuring in some environments
  • Fixed a certificate handling issue at network security component

Release 5.0 September 7, 2021

Known limitations:

Upgrade Process

**Supported Upgrades From: **4.0.x

For the full supportability matrix, see the Release Notes on Github. There you will also find important Install and Upgrade instructions.

Sysdig Platform

Define S3 Bucket Path for Storing Captures

Sysdig Platform users can now define a custom path in the S3 bucket they are using for storing captures. This is useful to those who want to reuse a certain bucket used for other purposes or send captures from different installations to the same S3 bucket. For more information, see (On-Prem) Configure Custom S3 Endpoint.(On-Prem) Configure Custom S3 Endpoint

Webhook Channel Enhancements

Sysdig supports the following on a Webhook channel integration:

  • Insecure connections: You now have the ability to skip the TLS verification.

  • Custom headers: If your Webhook integrations require additional headers or data you can append to the alert format by using a custom header on the UI. This option is in addition to the existing API facility to add custom headers programmatically.

S3-Compatible Storage for Capture Files

Configuring S3-compatible storage, such as Minio or IBM Cloud Object Storage, for your Sysdig captures is now supported on Sysdig Monitor. The capability can be turned on by configuring the system appropriately, as given in (SaaS) Configure Custom S3 Storage Endpoint.

Microsoft Team Channel

You can now use Microsoft Team s as a notification channel in Sysdig Monitor. See Configure a Microsoft Teams Channel for more details.

Dark Mode

The dark appearance, known as Dark Mode, is available in Sysdig applications.

Sysdig can now automatically match your OS preferences. Available in Sysdig platform on-premises, or in SaaS in the US East and rolling out globally. For more information, see Configure Theme Preference.

Customized Session Expiration

Session expiration is the amount of time a user can remain idle before the session is automatically ended or expired. After the session expires, the user must log in to the Sysdig application again.

Sysdig now gives you the ability to make a shorter or longer idle session expiration for Sysdig applications. When a user browser is idle for a certain period of time, they will get automatically logged out. For more information, see Configure Customized Session Expiration.

Sysdig Monitor

Workload Label

Sysdig Monitor now supports two new labels, kubernetes.workload.name and kubernetes.workload.type which can be used for scoping Dashboards and configuring Gropings.

Earlier, each type of object (deployment, replicaset, statefulset, etc.) was unique, and in turn, you needed to use different types of Kubernetes Dashboards and a different Grouping resulting in n/a , where distinct types of Kubernetes objects are listed.

For more information, see Unified Workload Labels.

Silencing Alert Notifications

Sysdig Monitor allows you to silence alert notifications for a given scope for a predefined amount of time, and schedule silence in advance. When silenced, the alert will still be triggered and posted on the Events feed and in the graph overlays but will indicate it has been silenced. The types of notification channels you can use are Email, Slack, and Amazon SNS.

You will be notified 30 minutes before the start time and 30 minutes before the end time of a silence window. You will also be able to easily extend or end an active silence. To access the feature, navigate to Alerts > Silence on the Monitor UI.

For more information, see Silence Alert Notifications.

Sysdig Secure

Sysdig Secure for cloud

Sysdig Secure for cloud is available with Cloud Risk Insights for AWS, Cloud Security Posture Management based on Cloud Custodian for AWS and multi-cloud threat detection for AWS using Falco.

What’s Included in this release:

  • Insights: a powerful new visualization tool for threat detection, investigation, and risk prioritization, to help identify compliance anomalies and ongoing threats to your environment. With Insights, all findings generated by Sysdig across both workload and cloud environments are aggregated into a visual platform that streamlines threat detection and forensic analysis.

  • Threat Detection based on AWS CloudTrail: To detect threats, anomalies and suspicious activities with the flexible Falco engine. See also: Sept 29, 2020.

  • Cloud Security Posture Management with AWS Benchmarks: The AWS CIS Benchmarks assessment evaluates your AWS services  against the benchmark requirements and  returns the results and remediation activities you need to fix misconfigurations in your cloud environment.

    We’ve included several UI improvements to provide additional details such as:  control descriptions, affected resources, failing assets, and guided remediation steps, both manual and CLI-based.

  • Image Scanning for ECR and Fargate: one-click deployment– see also ECR April 13, 2020 and Fargate Sept. 28, 2020.

Falco Policy Tuner

Sysdig is now releasing a managed version of the standalone Falco Tuner.

Previously, you had to run the tuner in your local environment, print suggestions, and manually update a rule with those suggestions. The new feature runs in the background and automatically tunes noisy rules and false positives. To streamline the creation of these exceptions, we’ve created a new object within Falco called exceptions.

Note: To enable the tuner, Admin access rights to Sysdig Secure are required.

Feature Enhancement: Falco Exceptions

Previously, exceptions were created using and not conditions inside a Falco rule, e.g.

- rule: Write below binary dir
  ...
  condition: >
    bin_dir and evt.dir = < and open_write
    and not package_mgmt_procs
    and not exe_running_docker_save
    and not python_running_get_pip
    and not python_running_ms_oms
    and not user_known_write_below_binary_dir_activities    
  ....

However, this process can be unwieldy and can result in unintended behavior. The new format, using exceptions, looks like this:

- rule: Write below binary dir
  ...
  condition: bin_dir and evt.dir = < and open_write
  ....
  exceptions:
    - name: package_mgmt_procs
      fields: proc.name
      comps: in
      values: package_mgmt_binaries # list of known binaries
  ...

See the full documentation here.

Tunable Exclusions Available in Insights Details

We’ve added the ability to identify and add exceptions using the Policy Tuner in the Insights module. Now you can receive policy tuning recommendations directly within the Insights view, enhancing usability, ease, and refinement of results.

See also: Insights and Runtime Policy Tuning .

New Scan Results Page Layout

We have reorganized the visual layout of the Scan Results summaries to clearly distinguish policy evaluation from vulnerability matching and to better summarize the information.

Improvements include:

  • Vulnerabilities and Policies are now two different sections in the UI

  • Vulnerability match update time is displayed to further distinguish from the Policy Evaluation time

  • Policy breakdown is collapsed by default to reduce cognitive load

  • Re-evaluate policies button is now located in the impacted section only, as opposed to whole page

  • Apart from the vulnerability update time, the data remains unchanged from previous versions

See also: Review Scan Results.

New and Improved Host OS and Container Scanning Tools

We at Sysdig are working hard to improve your security posture and compliance experience. As part of this commitment we are implementing a new framework to generate host benchmark results, introducing host scanning, and making backend improvements to the image scanning mechanism.

Installation Steps

The new features require a new component to be installed called the Node Analyzer. We’ve provided an installation script to automate the installation or to upgrade an existing Node Image Analyzer daemonset, if applicable.

Once you’ve installed or updated the components, the UI will automatically show Host Scanning and new Benchmarks functionality (Legacy Benchmarks can still be accessed.)

Host Scanning: New

In addition to Sysdig Secure’s rich array of tools for scanning container images, you can now scan the hosts as well.

  • Scan hosts for vulnerabilities, and detailed Software Bill of Materials (SBoM)

  • Support for OS (e.g. rpm) and non-OS (e.g. Java, Ruby, Python) packages

  • Compare and diff scan results

Host Benchmarks: Updated
  • More checks

  • Better results

  • Clustered aggregations - understand the posture of your environments, not just a single entity

Image Scanning: Updated
  • Automatically scan images if they have not been scanned

Kubernetes Network Security: New Configuration and Improved User Experience

Sysdig’s Kubernetes Network Policy tool has been updated to include additional fine-tuning configurations and an improved user experience.

Additional Configuration Panel
  • Workload Labels: Depending on your workload labelling policy, some labels may not be relevant for generating a KNP policy. Use the additional config to include/exclude a particular set of labels per cluster/namespace to declutter your UI and the resulting policy.

  • Unresolved IP Configuration: Now it is possible to label raw IPs that are not mapping to your Kubernetes/OpenShift entities, i.e. external cloud provider services, so these labels will be automatically applied to the topology and ingress / egress tables.

  • Cluster CIDR configuration: If the CIDR configuration is not automatically detected by the agent, you can now directly configure internal subnets per cluster using the Sysdig interface.

Improved UX
  • Topology map: Additional information pop-up when hovering over a network connection or a network node, such as server process, source, destination, and more.

  • Unresolved IP filtering: In the ingress and egress tables, by type or using free text search.

Additionally, Network is now presented as a top-level item in the Sysdig Secure navigation.

Activity Audit Improved

The Activity Audit user interface was enhanced as follows:

  • Activity Audit entry point moved under the Investigate menu

  • Trace feature, used for kube exec, is now also available for parent commands

  • The filter selector is also available in-line, with no need to open the detail view

  • Lateral Tree view removed and replaced with the Scope menu above, in alignment with the Event panel

Alert Notification Channel for Microsoft Teams

Microsoft Teams is now available as an Alert Notification Channel in Sysdig Secure for Runtime Policies. See also: Manage Policies

Internal Scanning Date Improvements

Scanning policies have improved the reliability of the Max days since creation and Max days since fix rule gate parameters. The information is now included in the inline-scan JSON report and available in the Jenkins plugin.

Reporting Improved with Multi-Select Option

Added the option to select multiple policies and multiple package types as part of a scheduled scanning report.

Release 4.0.3 August 27, 2021

This release is a hot-fix only release.

Upgrade Process

Supported Upgrades from: 3.6.2, 4.0.0, 4.0.1, 4.0.2

For the full supportability matrix, see the Release Notes on Github. Other upgrade notes are maintained in the GitHub upgrade folder.

Installation Instructions

Full installation instructions for Kubernetes environments: here.

Defect Fixes

Inline Scanning Fix for Sysdig Secure

Fixed an issue when scanning long Java manifest files that caused the scan to fail.

LDAP Improvements for Sysdig Platform

Fixed an issue with the LDAP sync Job running out of shared memory. The LDAP sync will no longer stop if it encounters an intermittent issue or error, but will allow the sync to complete.

4.0.2 June 29, 2021

This release is a hot-fix only release for Sysdig Secure features.

Upgrade Process

Supported Upgrades From: 3.6.2, 4.0.0, 4.0.1.

For the full supportability matrix, see the Release Notes on Github.

Improvements

CSV Runtime Reports

  • The runtime labels that were described in a single CSV column (JSON encoded) will now be represented using one column per label.

  • If the same vulnerability, same package, same image is found in several runtime contexts, the CSV will separate each runtime context in a separate row, instead of building a JSON array with several objects nested.

See also: Scheduled Reports.

Defect Fixes

Fixed Incorrect Fingerprinting Causing False Positives in Scanning

Fixed incorrect version detection for Apache Struts 2 packages leading to false positives.

Fixed Metadata Retrieval Issue in Scanning

Fixed incorrect metadata retrieval for corner cases when imageIDs are associated with several digests.

Improved Memory Usage

Reduced Redis memory consumed by scanning by optimizing the usage of the scanning API cache.

Fixed Subscription Alert Entries

Fixed scanning alerts triggers for images discovered via the Node Image Analyzer or Inline Scan container.

Readable Filenames for Scanning Reports

The scheduled scanning reports now generate report files named after the report name i.e. my-daily-critical-vulns-2021-05-04.zip

Release 4.0.1 May 05, 2021

This release is a hotfix-only release for Sysdig Secure features.

Upgrade Process

Supported Upgrades From: 3.6.2

For the full supportability matrix, see the Release Notes on Github.

Improvements

Improved RHEL Vulnerability Matching

The RedHat OVAL source feed interpretation and the matching algorithm have been improved to handle special RedHat packages versioning rules. This should effectively translate into fewer false positives and more accurate fix versions for RH-based packages.

Defect Fixes

Security Fix

A SQL injection vulnerability discovered in 4.0.0 has been fixed in 4.0.1.

Scan Results

The vulnerability list on the UI shows a different number of vulnerabilities as compared to the summary PDF report for the same image. This issue has been fixed as part of Improved RHEL Vulnerability Matching.

Secure Audit Reporting Errors

Secure Audit Reporting displayed intermittent errors for custom agent versions. Fixed the agent version parsing to correctly assess feature support.

Release 4.0.0 April 06, 2021

Upgrade Process

Supported Upgrades From: 3.6.2

For the full supportability matrix, see the Release Notes on Github.

Migrating MySQL to PostgreSQL

For consolidation and to meet higher performance requirements, upgrading to v4.0.0 from v3.x.x involves migrating MySQL to the PostgreSQL database. The migration process is seamless and no user intervention is expected. For more information, see Migration Documentation on Github.

InstallationAdditional Docs
KubernetesREADMEReview the Upgrade and other files within the version-specific GitHub folder for additional information.
ReplicatedNot supported on 4.0.0

Deprecations

Deprecating “Scan Image” Reaction in Alerts

When setting up runtime alerts in previous versions, there was an option to trigger “scan image” when an unscanned image was detected. This has been deprecated in the UI in favor of the Node Image Analyzer, which is bundled by default with the Sysdig agent as an additional container per node.

See also: Manage Scanning Alerts.

Defect Fixes

Large SAML Metadata

An issue was detected in an earlier version where large SAML metadata could not be saved due to limits in the database field size. This issue is now fixed and Sysdig now supports large SAML metadata.

Single Sign-On for Monitor and Secure

When a user logs in to Sysdig products successively, a confusing error message related to SAML was displayed if:

  • If both Secure and Monitor have been configured with SSO.

  • The Create User on login feature has been turned on for both products.

This issue is fixed with this release.

When a user created in one product logs in to another, and if the Create user on login feature is turned on, no error message is thrown. The user is added to the appropriate team in the product and can log in to the other.

Sysdig Platform

Monitor UI Displays On-Prem License Information

The on-prem license information is now displayed on the Monitor UI. Additionally, users will be warned of imminent license expiration on the UI.

Changes to Auditing Sysdig Platform Activities

Due to the changes in the underlying database (PostgreSQL instead of MySQL), the existing Sysdig auditing data will be dropped when performing the upgrade from 3.x to 4.0 on-premise version. The audit data is not migrated due to the potentially large size of the table, which could prolong the upgrade process. The data remains available in the MySQL database. If you require the data, do the following:

  1. Before upgrading, dump the audit_events table from MySQL.

  2. When the upgrade is completed, import the data back to the new database if you desire.

    Contact your Sysdig contact for details on how to perform this operation.

Sysdig Monitor

Improved Alerts

The Alert interface has been improved to allow faster browsing and easier management. For more information, see Alerts.

Explore Workflow Enhancements

The Explore interface has been improved to allow faster troubleshooting.

You are now launched directly into the drill-down view when you navigate to Explore. You will still be able to group and navigate your infrastructure by using the hierarchical scope tree.

The new Grouping editor helps you create and manage your infrastructure groupings.

For more information, see Explore.

Visualizing Missing Data on Dashboards

Dashboards now show null or missing data values as gaps instead of zero. Optionally, missing data can be displayed as a dotted or solid line in both Form-based and PromQL panels. StatsD metrics will continue to show null values as zero unless overridden by the settings. For more information, see Display Missing Data.

Host Overview

To complement Sysdig Kubernetes Overviews, Hosts Overview has been released. Host Overview provides a unified view of the performance and health of physical hosts in your infrastructure.

Sysdig Secure

Serverless Agent Preview Feature

The 1.0.x serverless agent is supported as a preview feature with Sysdig Platform 4.0. Note that there is no guarantee of forward or backwards compatibility with this preview release.

Sysdig Serverless Agent 1.0.0 for Fargate ECS

The “container-as-a-service” serverless environment calls for new agent models, and Sysdig provides them. Whereas in ECS, users still manage the underlying instances, with AWS Fargate the host is never visible and users simply run their workloads. And while this model is convenient, it can introduce risk as many people leave the containers unattended, without monitoring security events within that can exfiltrate secrets, compromise business data, impact performance, and increase their AWS costs. In addition, it is not possible to install a standard agent in an environment where you do not have access to a host.

For these reasons, Sysdig has introduced a new “serverless agent” model that can be deployed in these container-based cloud environments. The first implementation is for Fargate (ECS).

Sysdig will be rolling out security features on the serverless agent over time. In v1.0.0, users will see:

  • Runtime Policies and Rules

  • Secure Events

To obtain secure event information and the associated Falco policies and rules in the Sysdig Secure UI from a Fargate environment, users install the serverless agent using a CloudFormation Template. Then log in to Sysdig Secure and review the events in the UI.

See also: AWS Fargate Serverless Agents and Serverless Agent Release Notes (for future updates).

Kubernetes-Native Network Security with Sysdig Secure (Beta)

A new feature has been added to Sysdig Secure for authoring and refining Kubernetes network policies (KNPs) that:

  • Automatically extracts the connection information, by observing the cluster networks and microservices communications

  • Offers a visual flow to fine-tune the Kubernetes network policies, incorporating the user’s adjustments

  • Automatically generates the KNP YAML to be applied, without requiring previous Kubernetes policy knowledge from the user.

As soon as the feature is enabled, the Sysdig agent starts collecting and processing application communications, which are then enriched using Kubernetes metadata and presented in two different ways:

  • Topology maps: a visual representation of the network flow between the Kubernetes entities (Services, Deployments, StatefulSets, DaemonSets, Jobs)

  • Ingress / Egress tables: for additional detail on each inbound/outbound communication and policy tuning.

Once the user has finished editing the desired policy, Sysdig will automatically compute the associated KNP YAML:

  • Enforcement is delegated to the Kubernetes control plane, favoring policy-as-code and avoiding direct tampering with cluster communications

  • Allow-only approach ensures that any communication which is not explicitly allowed by the policy will be forbidden

Prerequisites

Sysdig agent version 10.7+

Supported Orchestrator Distributions and CNI Plugins:

  • Vanilla Kubernetes (kops, kube-admin) using Calico

  • OpenShift 4.x using OVS

  • Google GKE using Calico

  • Amazon EKS using Calico

  • Rancher Kubernetes using Calico

Please contact us to enable this feature for your Sysdig Secure accounts.

See also: Network Security Policy Tool .

Network Micro-Segmentation: Support for CronJobs, Weave, & Cilium CNIs

The Sysdig Network Security Policy Tool has been upgraded to add support for CronJob pod Owners.

With the addition of CronJob support, communication is aggregated to the CronJob (scheduler) level, rather than the Job. Therefore, when administrators review the activity in the Network Security Policy menu, they will see the higher-level CronJobs listed, and not an excess number of individual Job entries.

This update also adds support for Weave and Cilium CNIs on top of Calico support.

New Product: Rapid Response

Rapid Response is an Endpoint Detection and Response (EDR) solution built for cloud-native workloads, which gives security engineers the ability to respond to incidents directly via a remote shell. The shell uses the underlying host tooling already installed, such as kubectl, Docker commands, cloud CLIs, etc. Users can also mount their own scripts to use any familiar tooling.

Rapid Response requires a component installed on the host machine. This component provides end-to-end encrypted communication using a passphrase only your team knows. The Rapid Response feature is disabled by default and can only be accessed to teams that have the feature enabled. Admins can see all user activity, including access to audit logs, and can initiate a rapid response session. Advanced users can view only their own user activity, including their audit logs, and can initiate a rapid response session.

See also: Rapid Response: Installation and Rapid Response

Image Scanning Reports v3 [BETA]

The Image Scanning Reports feature has been thoroughly updated and has moved from a synchronous model to an asynchronous mode, in which you schedule the reports you need and then receive them through your normal notification channels (email, Slack, webhook.). The new version also includes:

  • A preview function to check report structure in the UI

  • A more advanced query builder

  • Extended set of data columns (i.e. CVSS base score and vector) and extended set of available filters (i.e. package type)

Reporting v3 supports two different types or reports:

  • Vulnerability report: Containing vulnerability, package and image data

    I.e. Vulnerabilities in my runtime with Severity ≥ High, a Fix available and not included in a vuln exception list.

  • Policy report: Containing scanning policies and evaluated images data

    I.e. Images in my internal registry failing the “NIST” scanning policy.

You need to enable this feature from the Sysdig Labs setting on the User Profile page.

See Scheduled Reports for more detail.

UI-Based Admission Controller Released

Kubernetes’ admission controllers help you define and customize which requests are allowed on your cluster. An admission controller intercepts and processes requests to the Kubernetes API prior to persistence of the object, but after the request is authenticated and authorized.

Sysdig’s Admission Controller (UI-based) builds upon Kubernetes and enhances the capacity of the image scanner to check images for Common Vulnerabilities and Exposures (CVEs), misconfigurations, outdated images, etc., elevating the scan policies from detection to actual prevention. Container images that do not fulfill the configured admission policies will be rejected from the cluster before being assigned to a node and allowed to run.

See also: Admission Controller.

Main Features
  • Granular admission policies: Defining a global policy per cluster, but also at the level of particular namespaces or image paths (i.e. registries) Registry and repository whitelist

  • Only allow images that pass the scanning evaluation criteria

  • Only allow images that have been evaluated recently

  • Only allow images that have been scanned before creation is requested to Kubernetes

  • Registry and repository whitelist

  • Scan unscanned requested images immediately (optional)

CIS AWS Cloud Benchmark Released

A new cloud compliance standard has been added to the Sysdig compliance feature -  CIS AWS Benchmark. This assessment is based on an  open-source engine - Cloud Custodian - and is an initial release of Sysdig Cloud Security Posture Management (CSPM) engine. This first Sysdig cloud compliance standard will be followed by additional security compliance and regulatory standards for GCP, IBM Cloud and Azure.

The CIS AWS Benchmarks assessment evaluates your AWS services  against the benchmark requirements and  returns the results and remediation activities you need to fix misconfigurations in your cloud environment. We’ve also included several UI improvements to provide additional details such as:  control descriptions, affected resources, failing assets, and guided remediation steps, both manual and CLI-based when available.

See also: AWS Foundations Benchmarks.

New Runtime Policy Events JSON Format

The JSON format for the runtime policy events has been upgraded to include full scope information, rule labels, and a single-line representation for the event field’s keys and values.

To preserve backwards compatibility with existing integrations, the former JSON format is still available (and used by default on migration).

From the Event Forwarder page, under “Data to Send,” the old JSON format is labeled “Policy Events (Legacy)” and the new one as “Runtime Policy Events.”

See also: Event Forwarding.

Scan Results List Updated

The UI for the list of scanned images has been updated to include several functionality and design improvements:

  • Status column (Passed or Failed) is now filterable

  • Image Origin (Inline Scanner, Node image analyzer, etc.) is now visible, filterable, and has multi-select option

  • Image registry is now visible on the table

  • Ability to sort by date-added (default) or image name

  • Flexible free-text search: filter by registry/repo:tag, repo:tag, repo, etc.

See also: Review Scan Results.

Improved UI for New Users

We have added introductory splash screens throughout the product to help you get started when using a feature for the first time.

UI Improvement on Rules Library and Rule Details

Usability improvement so you can see in which policies a rule is used, from both the Rules Library list and the Rule Detail view. See Manage Rules for details.

Deprecation Notice: Legacy Commands Audit & Legacy Policy events

  • The “Commands Audit” feature was deprecated in favor of Activity Audit in November 2019. This feature will be completely removed from the On-prem distribution in version 4.1.

    Sysdig agent version 9.5.0+, released in January 2020, is required by the Activity Audit feature.

  • The “Policy Events” feature was deprecated in favor of the new Events feed in June 2020. This feature will be completely removed from the On-prem distribution in version 4.1.

    Sysdig agent version 10.3.0+ is recommended.

Windows Scanning Released

A beta version of the Windows Scanning Inspector has been released. This is a new feature from Sysdig for scanning Windows containers.

This is a standalone scanning engine. There is no centralized UI, management, or historical data. These features are planned for a future release.

See also: Windows Container Image Scanning [BETA].

Features
  • Identify Windows container image vulnerabilities from:

    • Windows OS CVEs
  • Windows or Linux hosts

  • Reports in JSON and PDF

  • Policy support

    • Severity

    • Fix available

    • Days since fixed

Malware Detection during Inline Image Analysis

As part of the inline scanner version 2.3.1 release, malware scanning was added as a configurable detection that can be performed during inline analysis.

The default behavior if this feature is enabled and malware is found is to consider the scanning failed, report malware details, and abort analysis:

See Perform Inline Malware Scanning for recommended parameters and output options.

Release 3.6.2 December 14, 2020

This release contains bug fixes and minor improvements.

Upgrade Process

Supported Upgrade From: 3.2.2, 3.5.1, (3.6.0 or 3.6.1 if it was installed)

For the full supportability matrix see the GitHub documentation.

Bug Fixes

  • Fixed email notifications error

    In some cases, including alerts with very large scopes and some others, email notifications were not sent due to a bug in the email renderer. This issue has been fixed.

  • Fixed Kubernetes metadata display delay

    In 3.6.0 and 3.6.1 releases, upon connecting an agent, it would take 1h for Kubernetes metadata to appear. With this bug fixed, the metadata is displayed a couple of minutes after connecting the agent.

  • Fixed dashboard display error when switching teams

    When the user switched teams, the dashboard menu was not displayed and required the user to reload the application. This has been fixed.

  • Improvements to the security setup of our Intercom integrations

    We have improved the security of the Sysdig Intercom integration, as in some cases, the conversations could leak between different users.

  • Fix to Activity Audit Janitor

    Fixed an Activity Audit Janitor error that stopped the AA clean-up process when a particular set of Sysdig Secure features were not enabled.

Improvements

Increased Decimal Precision from 4 to 6

With this release, we increased the decimal precision from 4 to 6 decimal places. This feature is mostly useful for customers using Prometheus metrics, as by convention, the metrics for time are given in seconds in Prometheus exporters, which does not work well for low numbers (for example - latencies in microseconds).

New Runtime Policy Events JSON Format

The JSON format for the runtime policy events has been upgraded to include full scope information, rule labels, and a single-line representation for the event field’s keys and values.

To preserve backwards compatibility with existing integrations, the former JSON format is still available (and used by default on migration).

From the Event Forwarder page, under “Data to Send,” the old JSON format is labeled “Policy Events (Legacy)” and the new one as “Runtime Policy Events.”

See also: Event Forwarding.

Release 3.5.3 December 14, 2020 (Replicated Only)

This release is a bug fix only release.

Upgrade Process

Sysdig Platform v 3.5.3 has been tested and qualified against the same components as in v. 3.5.1.

Supported Upgrade from: 3.5.1, 3.2.x, 3.0

Bug Fixes

Sysdig Platform

  • Fixed email notifications error

    In some cases, including alerts with very large scopes and some others, email notifications were not sent due to a bug in the email renderer. This issue has been fixed.

  • Improvements to the security setup of our Intercom integrations

    We have improved the security of the Sysdig Intercom integration, as in some cases, the conversations could leak between different users.

Sysdig Secure

  • Events Forwarder improvement

    Fixed a crash condition in the Events Forwarder service stemming from a microservices connectivity issue.

Release 3.6.1 November 23, 2020

Oversight Services Now Offered for All Installs and Upgrades

As part of our continued focus on our customers, we are now offering oversight services for all on-premise installs and upgrades. Your Technical Account Manager (TAM), in conjunction with our support organization and Professional Services [where applicable], will work with you to:

  • Assess your environment to ensure it is configured correctly

  • Review your infrastructure to validate the appropriate storage capacities are available

  • Review and provide recommendations for backing up your Sysdig data

  • Work with you to ensure our teams are ready to assist you during the install and upgrade process

  • Provide the software for the install

  • Be available during the process to ensure a successful deployment

You can always review the process in the documentation on GitHub (v. 3.6.0+) or the standard docs site (for older versions).

If you are a new customer looking to explore Sysdig, please head over here to sign up for a trial on our SaaS Platform. Alternatively, you can contact us here.

Upgrade Process

Supportability Matrix

Sysdig Platform has been tested and qualified against the following.

* Note that as of this release, there are no upgrades for Replicated installations.

Supported Upgrade From3.2.2, 3.5.1, 3.6.0
PlatformVersion
Vanilla Kubernetes1.13.4, 14.10, 1.15.12, 1.16.13, 1.17.9, 1.18.6
OpenShift3.11, 4.4
GKE1.14.10-gke.36
EKSv1.17.7-eks-bffbac
Rancherv2.3.3
IBMUnqualified
PKSUnqualified
AgentVersion
sysdig/agentQualified with agent release 10.5.2
ComponentsKubernetes with Statefulsets
Redis4.0.12
MySQL5.6.44, 8.0.16
MySQL HA8.0.16
ElasticSearch6.8.6
Cassandrarelease_version: 2.1.21, cql_version: 3.2.1
RDS8.0.16
Postgres (image scanning)12.4
Anchore (image scanning)0.8.1
NATS Exporter0.6.0.1
NATS Streaming0.17.0
HA-Proxy0.6.2
InstallationAdditional Docs
KubernetesREADMEReview the Upgrade and other files within the version-specific GitHub folder for additional information.
ReplicatedNo Replicated release from 3.6.0 forward.

Sysdig Secure

The following improvements were introduced in release 3.6.1:

Node Image Analyzer: Scan “Repo-less” Images

Added support to scan images that lack a Repo tag, such as OpenShift 4.x distribution images.

Audit Tap Forwarding: Fixed Splunk Event Timestamp Metadata

The format of the “time” field included in the Splunk event metadata for forwarded Audit Tap events is now increased to millisecond granularity.

Fixed an issue that resulted in log4j-jboss-logmanager and log4j-1.2-api being incorrectly detected as log4j, possibly generating vulnerability false positives.

NOTE: Inline Scanner v2.1

Inline Scanner v2.1 has been released.

This component is independent of the Sysdig Platform version you are running–it can be used with Sysdig On-Prem version 3.6.1 and with earlier versions.

Inline Scanner 2.1 includes the following enhancements:

  • NEW

    Added ability to analyze scratch-based images

  • FIXES

    Fixed a bug retrieving the PDF output for previously- scanned images

    Addressed several vulnerabilities found in the inline scanner container

See also: Integrate with CI/CD Tools.

Release 3.6.0 November 10, 2020

Oversight Services Now Offered for All Installs and Upgrades

As part of our continued focus on our customers, we are now offering oversight services for all on-premise installs and upgrades. Your Technical Account Manager (TAM), in conjunction with our support organization and Professional Services [where applicable], will work with you to:

  • Assess your environment to ensure it is configured correctly

  • Review your infrastructure to validate the appropriate storage capacities are available

  • Review and provide recommendations for backing up your Sysdig data

  • Work with you to ensure our teams are ready to assist you during the install and upgrade process

  • Provide the software for the install

  • Be available during the process to ensure a successful deployment

You can always review the process in the documentation on GitHub (v. 3.6.0+) or the standard docs site (for older versions).

If you are a new customer looking to explore Sysdig, please head over here to sign up for a trial on our SaaS Platform. Alternatively, you can contact us here.

Upgrade Process

Supportability Matrix

Sysdig Platform has been tested and qualified against the following.

* Note that as of this release, there are no upgrades for Replicated installations.

Supported Upgrade From3.2.2, 3.5.1
PlatformVersion
Vanilla Kubernetes1.13.4, 14.10, 1.15.12, 1.16.13, 1.17.9, 1.18.6
OpenShift3.11, 4.4
GKE1.14.10-gke.36
EKSv1.17.7-eks-bffbac
Rancherv2.3.3
IBMUnqualified
PKSUnqualified
AgentVersion
sysdig/agentQualified with agent release 10.5.2
ComponentsKubernetes with Statefulsets
Redis4.0.12
MySQL5.6.44, 8.0.16
MySQL HA8.0.16
ElasticSearch6.8.6
Cassandrarelease_version: 2.1.21, cql_version: 3.2.1
RDS8.0.16
Postgres (image scanning)12.4
Anchore (image scanning)0.8.1
NATS Exporter0.6.0.1
NATS Streaming0.17.0
HA-Proxy0.6.2
InstallationAdditional Docs
KubernetesREADMEUpgrade notes, parameters, and more
ReplicatedNo Replicated release of 3.6.0

Sysdig Platform

Interactive Session Expiration Installation-Wide

With this release, you can define a period of interactive-session expiration, so that when a user is idle for a defined period of time, the session terminates. This helps enterprises with strict security and compliance requirements comply with relevant security controls, such as NIST or PCI-DSS 8.1.8 .

Currently, this feature is available for on-premises only and is configured per installation.

See also: Configure Interactive Session Expiration.

Minor Enhancements and Fixes around Users and Teams

  • Team Search Available when Switching Teams

    You can now search for Teams on the Team Switcher. This feature is especially handy for Admins who are members of many teams.

    See also: Switching Teams in the UI.

  • User search now supports many more users

    With this release, we have enhanced the performance for listing and search for users on both Settings>Users and Settings>Teams pages. We now support tens of thousands of users comfortably.

  • LDAP: Search for users by both username and email address

    For enterprises using LDAP, this release enables search on both username and user email address in the Settings > Users and Settings > Teams pages. Users are listed by name but can be searched by email as well.

  • LDAP: Default team role respected

    This fix ensures that when LDAP users are created upon login, the default user role for the team is respected.

Inline Scanner 2.0

A new version of the Sysdig inline scanner script has been released.

Major improvements:

  • The inline analysis container doesn’t need to spawn any additional containers

    • This removes the requirement for the Docker client, docker-in-docker, etc.

    • This enables usage in environments where docker-in-docker is not feasible or hard to instrument (e.g., Tekton).

  • Additional analysis workflows and formats:

    • Added support to analyze a docker archive

      • A .tar.gz file containing the image, i.e. the output from a “docker save”

      • Example execution

    • Added support to analyze OCI images (both and directory and archive)

    • Added support to retrieve an image from the container storage (CRI-O and others)

Additional improvements:

  • Faster image ingestion

  • More verbose logs available for troubleshooting and diagnosis

  • Machine-readable JSON output via --format JSON command

To upgrade an earlier Sysdig Inline Scanning version to 2.0, you need to take into account the new invocation parameters, which are not backward compatible.

Sysdig Inline scanner can be used stand-alone or as a step inside a CI/CD pipeline (Jenkins, Tekton, CircleCI, etc). In the upcoming weeks, we will update the different integrations to provide out-of-the-box support for the 2.0 version.

Sysdig Secure

Regulatory Compliance Control Validation & PCI Checks

A new feature has been added to Sysdig Secure for checking controls from various compliance standards. For the first release, we provide checks against specific controls in PCI 3.2. Future releases will include SOC2, NIST-800-53, and more. See also: Compliance in Sysdig documentation.

Compliance Validator and Reports

The validator checks many Sysdig Secure features, including: image scanning policies, Falco runtime policies and rules, scheduled benchmark testing, Admission Controller, Network Security Policies, Node Image Analyzer, and more. Over time we will add new compliance coverage.

Disclaimer: Sysdig cannot check all controls within a framework, such as those related to physical security.

This feature is a beta release. A Sysdig Secure admin must enable it from the Sysdig Labs interface under Settings.

PCI Control Details

The PCI Quick Reference describes the full range of controls required to pass a PCI 3.2 audit. In this release, Sysdig Secure will check the following subset:

Controls 1.1.2, 1.1.3, 1.1.6.b, 2.2, 2.2.1, 2.2.2, 2.2.a, 2.4, 2.6, 4.1, 6.1, 6.2, 6.4.2, 6.5.1, 6.5.6, 6.5.8, 7.1.2, 7.2.3, 10.1, 10.2, 10.2.1, 10.2.2, 10.2.3, 10.2.6, 10.2.7, 10.3, 10.5.5, 10.6.1, 11.4, 11.5.a, 11.5.b.

Replacing RHSA Advisories with CVE Advisories

In new images scanned, RHSA advisories will be replaced with CVE advisories.

Benchmarks support for Kubernetes Benchmark 1.6

  • Kubernetes Bench upgraded to version 1.6

  • Using the Kubernetes benchmark, we now provide customer-selected benchmark checks for GKE and EKS (rather than just the Kubernetes default).

Vulnerability Exceptions Handling Enhanced

The Vulnerability Exceptions feature in Sysdig Secure has been redesigned and enhanced.

It now offers:

  • Additional vulnerability and feed context

  • Precise mapping between images and their associated exceptions

  • A better exception management lifecycle

  • Multiple vulnerability lists, which can be flexibly assigned to different image sets (or just a particular image), using the scanning policy assignments

  • Additional information displayed to improve team awareness and security context

    • Vulnerability description

    • User-defined notes

    • Vulnerability feed info, with severities and links as provided per feed

  • Configurable expiration dates:

    • An exception is automatically disabled when the expiration date is met

    • Day resolution, all times relative to 0:00 UTC

  • Enhanced workflow integration with the “Scan results” page for an individual image, with the ability to quickly append a flagged vulnerability to a list.

Migration: The exception and evaluation behavior in the current environment will be maintained after the feature upgrade. In particular:

  • Pre-existing vulnerability exceptions will be migrated to the “Default exceptions list”

  • The “Default exceptions list” will be assigned to every pre-existing policy assignment

  • All the pre-existing vulnerability exceptions expiration date will be set to “Never.”

See also: Manage Vulnerability Exceptions and Global Lists.

Event Forwarding: Kafka and Webhook Added

Two new supported integrations have been added to the Sysdig Secure Event Forwarder:

The Kafka topic integration includes support for:

  • Multiple Kafka brokers

  • Partitioner/Balancer algorithms: Murmur2, Round robin, Least bytes, Hash, CRC32

  • Compression algorithms: LZ4, Snappy, Gzip, Zstandard

The Webhook integration includes support for:

  • Authentication methods: Basic authentication, Bearer Token, and Signature Header

  • Custom headers defined by the user to accommodate any additional parameter required on the receiving end

Image Exclusion on Policy Events

Users often want to tune policy events. We’ve added a button on the event detail that will add an exclusion to a specific container.image.repo for the policy that triggered the event. Once that exclusion is applied to the scope, policies will no longer fire for that container.image.repo.

Captures Filter on the Policies Page

Policies can now be filtered to display if a capture is associated with an active or inactive policy.

Quick Menu to Captures from Runtime Events

For runtime policy events that have an associated capture, we now offer a contextual menu for performing quick actions over the event capture, rather than a simple link to the Captures interface. You can:

  • View the capture directly in Sysdig Inspect

  • Directly download or delete the capture

Additionally, if the event is scoped to a particular container, Sysdig Inspect will automatically filter the displayed information to the scope of that Container ID.

Image Scan Results Page Redesigned to Improve Load Times & User Experience

The user interface is cleaned up, reorganized, and provides the following functional improvements:

  • Load times are significantly decreased because the last known evaluation for the image is automatically fetched

    • View the latest evaluation time directly in the scan summary Evaluated at

    • Use the new Re-evaluate button to fetch current data if desired

  • View the image origin/reporting mechanism in the new “Added By” field.

    Possible values are: Sysdig Secure UI, Node Image Analyzer, API, Sysdig Inline Scanner, or Scanning alert.

  • Copy the Image Digest and Image ID to the clipboard using a quick pop-up panel.

Forwarding the Activity Audit Information

The Sysdig Secure Event Forwarder has added support to forward Activity Audit data to external platforms.

Sysdig Monitor

Time Navigation in Events Feed

You can now browse and find historic events easily by using time navigation.

Zooming Out Dashboards

You now have the ability to zoom out Dashboards. This feature doubles the selected timeframe for a better context surrounding a problem when troubleshooting an incident.

Release 3.5.1 August 24, 2020

NOTE: Version 3.5.1 includes a fix for vulnerabilities that were detected in version 3.5.0. It is recommended to skip version 3.5.0 and install version 3.5.1 instead. As of this release, all on-premises installs and upgrades include oversight services from Sysdig support.

Oversight Services Now Offered for All Installs and Upgrades

As part of our continued focus on our customers, we are now offering oversight services for all on-premise installs and upgrades. Your Technical Account Manager (TAM), in conjunction with our support organization and Professional Services [where applicable], will work with you to:

  • Assess your environment to ensure it is configured correctly

  • Review your infrastructure to validate the appropriate storage capacities are available

  • Review and provide recommendations for backing up your Sysdig data

  • Work with you to ensure our teams are ready to assist you during the install and upgrade process

  • Provide the software for the install

  • Be available during the process to ensure a successful deployment

You can always review the process in the documentation on GitHub (v. 3.6.0+) or the standard docs site (for older versions).

If you are a new customer looking to explore Sysdig, please head over here to sign up for a trial on our SaaS Platform. Alternatively, you can contact us here.

Upgrade Process

Sysdig Platform has been tested and qualified against the following:

Supported Upgrade From3.5.0, 3.2.x, 3.0
PlatformVersion
Vanilla Kubernetes1.13.4, 14.10, 1.15.12, 1.16.13, 1.17.9, 1.18.6
OpenShift4.4 –> 1.17.1+1aa1c48
GKE1.14.10-gke.36
EKSv1.17.7-eks-bffbac
Rancherv2.3.3
IBMUnqualified
PKSUnqualified
AgentVersion
sysdig/agent10.2.0

Components

Replicated TBD

Kubernetes with Statefulsets

Redis

n/a

4.0.12

MySQL

n/a

5.6.44

MySQL HA*

n/a

8.0.16 (see note)

ElasticSearch

n/a

5.6.16

Cassandra

n/a

release_version: 2.1.21

cql_version: 3.2.1

RDS

n/a

8.0.16

Postgres (image scanning)*

n/a

12.3 (see note)

Anchore (image scanning)

n/a

0.6.1

NATS Exporter

n/a

0.6.0.1

NATS Streaming

n/a

0.17.0.1

HA-Proxy

n/a

1.9.15

  • MySQL8: You can use MySQL8 for non-HA setups using the flag useMySQL8: true
  • Postgres: Upgrading to 3.5.0 will also involve an automatic Postgres version upgrade from 10.6.x to 12.x. Depending on your database size, the upgrade could take some time. See Postgres Version Update v10.x to 12.x for details.
InstallationUpgrade
KubernetesInstaller (Kubernetes | OpenShift)Installer Upgrade (3.5.0-3.5.1)
ReplicatedInstall with ReplicatedBasic Upgrade (Replicated)

Sysdig Platform

Endpoint for Feeds Update Has Changed

We no longer point to ancho.re for feeds update but to[ https://api.sysdigcloud.com/api/scanning-feeds/v1/feeds](http:// https://api.sysdigcloud.com/api/scanning-feeds/v1/feeds). This could require a change to your firewall rules, as an exception to your proxy for ancho.re would impact the feeds update.

Sysdig Secure

Note that the Secure Overview is not available with Replicated installations.

New Sysdig Secure Overview Page

The Sysdig Secure Overview provides an at-a-glance view of the critical areas of your security posture.

Scoping

Panels can be scoped by Cluster or Namespace. The scope will update all panels that are displaying run-time data and the corresponding drill-down views.

Panels
  • Build Time - Images Scanned: Image scan results for all static image scans

    Drill-down - To Image Scanning Reports page.

  • Build Time - CVEs Found by Severity: The total number of CVEs present in each image scanned.

    Drill-down - Available in a future release

  • Run-time - Images Scanned: The pass/fail status of images running now and their trend over time.

    Drill-down - To Runtime Scanning Image page.

  • Run-time - CVEs by Severity: The total number of CVEs present in each running image

    Drill-down - Available in a future release

  • Run-time - Policy Events by Severity: The total number of policy events by severity.

    Drill-down - Secure Events page.

  • Benchmarks Tests Failing: The total number of benchmark tests that have failed.

    Drill-down - Benchmarks Results page.

See also: Secure Overview .

New Get Started Page

The Get Started page provides the key steps to ensure users are getting the most value out of Sysdig Secure. We’ll update this page with new steps as we add new features to Sysdig Secure. 

The Get Started page also serves as a linking page for:

  • Documentation

  • Release Notes

  • The Sysdig Blog

  • Self-Paced Training

  • Support

Users can access the page at any time by clicking the rocketship in the side menu.

See also: Getting Started with Sysdig Secure.

Feeds Status Page Added

It’s useful to understand the last time the feeds were updated, especially in self-hosted environments. The Feeds Status page shows the different vulnerability feeds we integrate with, their feed group (often the distro version), the time of the last sync, and how many CVE records are present in the feed group.

See also: Feeds Status.

Secure Events Feed Overhaul

The Events feed in Sysdig Secure (formerly called Policy Events) has been redesigned, both visually and functionally.

Apart from the styling and user experience improvements, these are the major new features and use cases

Advanced Filtering

We are deprecating the grouping/clustering of events present in the old version in favor of a much more powerful set of filtering capabilities:

  • Severity filters: Presented as quick buttons at the top, supporting multi-select

  • Attribute filters: Provide a simplified syntax to filter events by the attributes they contain. For example ruleType="Falco - Syscall" or image.repo!="sysdig/agent"

    • Open the event details side panel to find quick filtering widgets to include or exclude the attribute values associated with the displayed event
  • Event type selector: Supports runtime scanning alerts on top of policy runtime events (see section below), with an easy multi-selector in the UI.

  • Free text search: Allows you to search the event titles and scope label values. I.e. Terminal shell in or my-k8s-cluster.

  • New scope selector: Allows for additional selector logic (in, not in, contains, startswith, etc), improving the scoping flexibility over earlier versions. This scope selector also provides scope variables, allowing you to quickly switch between, for example, Kubernetes namespaces without having to edit the panel scope.

All these filters can be combined additively to further refine your search.

Multiple Event Types

The new event feed displays not only the policy runtime events, but also runtime image scanning alerts.

The backend architecture, filtering, and UX have been designed to accommodate additional types of security events that will be pushed to the Event Feed in the future, upgrading the interface from a policy-runtime-centric experience to a full security center control panel.

Additional Event Details

Policy runtime events: These now display the rule that was fired together with the rule labels. You can use the quick filters mentioned above to further refine the search.

Richer scope: Every security event now displays all the scope labels retrieved for the event, not just those configured in the scope selector.

See also: Secure Events.

Additional Considerations/Limitations

Events in the old and new format will be stored separately:

  • No event or event data will be lost during the transition

  • Events that were registered before the new feed is deployed can be browsed using the old feed interface, which is available from the burger menu in the top-right corner

  • Events that happen after the new feed is deployed will appear in the new event feed

  • Eventually, all events within the retention period will be present in the new interface, at which point the version switcher will disappear

Team, Role, and Channel Updates

A variety of enhancements have been added to the team, role, and notification channel options.

Service Manager Role Added to Sysdig Secure

RBAC capability was previously added to Sysdig Secure. (See also January 27, 2020 and User and Team Administration.)

Now a new role, Service Manager, is also available in Secure. It has the same permissions as the Standard User, plus the ability to invite existing users to the team and manage the notifications channels assigned to the team. See Team-Based Roles and Privileges

Configurable Default Team Role

You can now define the default user role to apply when a new member is added to the team. The Admin can change this default on a per-team basis. See also: Create a Team.

RBAC and Team Assignment for Notification Channels

Previously, notification channels in Sysdig Secure and Monitor were treated as global entities, visible and editable for most users of the platform regardless of team configurations.

We are enhancing the management and RBAC controls in the following ways:

  • Notification channels can now be “global” or limited to a particular team

  • Global channels can be managed by admins and can be viewed/used by other roles, while team-limited channels are available only to team members

  • Team Manager , Advanced User, and Service Manager (Secure) roles can create/update/delete team-scoped notification channels, they can also read and use the global ones

  • Standard and View Only roles can read team-limited and global notification channels

  • Admins will be able to create global notification channels and migrate channels from “global” to “team-limited”, and also from one team to another.

See also: Set Up Notification Channels and the Share With field in each individual channel setting page.

Optimized Runtime Page

We’ve released a new Runtime page for the Image Scanning module within Sysdig Secure. Improvements include:

  • Filtering based on pass/fail/unscanned

  • The ability to search results for a specific image

  • Optimized queries to improve response times

For more information, see Review Scan Results.

The ordering of the side menu has been changed.

Image Scanning Updates

The image scanning navigation bar has changed.

  • The side menu is reorganized into Analyze and Configure sections

    • Analyze: Different areas of scanning that allow users to view scan results

    • Configure: The areas of scanning that involve the setup of the application

  • Whitelist terminology with CVEs has been removed.

    “CVE whitelist” is now CVE Exceptions.

CLI-Based Admission Controller for Image Scanning

An additional tool for evaluating and admitting images is now available.

Sysdig Admission Controller

Sysdig’s Admission Controller (UI-based) combines the Sysdig Secure image scanner with a policy language to evaluate scan results and the admission context, providing great flexibility in the admission decision. It also provides the first line of defense against image-based security threats.

By using Kubernetes API extensions to perform image scanning and other security checks on admission, we cover a major threat-prevention and hardening use case: “Only the images that are explicitly approved will be allowed to run on my cluster”.

The admission decision relies not only on the image name and tag but also on additional context from the admission review, including namespace, pod metadata, etc.

Features

  • Registry and repository whitelist / blacklist

  • Global and per-namespace admission configuration

  • Configurable pre-scan and post-scan behavior, i.e.:

    • Accept only the images that pass the scan (default)

    • Directly reject non-whitelisted registries / repos, without scanning

    • Accept the image even if it doesn’t pass the scan

    • Do not accept any image that hasn’t been scanned already

  • Pod mutation: image tag is replaced by digest to prevent TOCTOU (Time of Check, Time of Use) issue if the tag is updated between the scan and the pod scheduling

Requirements

  • Helm 3

  • Kubernetes 1.15 or higher

For more information, see Admission Controller .

Added Automatic Image Scanning using Node Analyzer

The (node) image analyzer (NIA) provides the capability to scan images as soon as they start running on hosts where the analyzer is installed. It is typically installed alongside the Sysdig agent container.

This component was introduced to reduce dependencies on analyzing images within the Sysdig backend (SaaS or On-prem). Some advantages include:

  • Sharing credentials with the Sysdig backend in order to pull images is not required

  • Sharing the image content and potentially code with the Sysdig backend is not required; only metadata will be sent out

  • Opening a network route to allow the Sysdig backend to reach the user’s registries is not required

If you have run the single line agent install with the --image-analyzer flag, then this component is already running in your infrastructure.


The feature is available for Kubernetes environments.

For more information, see Scan Running Images.

Added Image Scanning Integration Options

Two new scanning integrations are available for CI/CD pipelines. Sysdig provides:

  • A reference implementation with Tekton Pipelines (prototype)

  • A fully supported integration with Amazon Elastic Container Registry (ECR) for triggering auto-scans from the registry

Integrating Secure Image Scanning with Tekton Pipelines

Tekton Pipelines allow you to implement CI/CD workflows using a highly modular, cloud-native approach that:

  • Uses containers as the building blocks for individual tasks

  • Runs directly on Kubernetes/OpenShift without requiring a dedicated infrastructure

  • Uses tasks that are purely declarative and described using their own CRD, making them easily composable and reusable

Sysdig’s reference implementation details the prototype task to invoke Sysdig Secure image scanning as a pluggable step in your CI/CD pipeline with just a YAML file:

Leveraging Tekton integration with the orchestration layer, you can retrieve the image scanning policy evaluation and state (pass/fail) directly from the logs of the task pod.

Read the “Securing Tekton pipelines in OpenShift with Sysdig” blog post for additional details

Integrating Secure Image Scanning with Amazon ECR

Automatically scan images pushed to your Amazon Elastic Container Registry (ECR) using AWS-native technologies and Sysdig Secure.

Sysdig image scanner integration is deployed as a CloudFormation template that listens to ECR registry events and uses AWS resources to streamline the image scanning process.

  • ECR itself will trigger the scan, no need for your CI/CD pipelines to actively pull from the registry

  • Deployed in a few clicks, you just provide basic configuration parameters such as the Sysdig API token or the Sysdig backend URL

  • No need to configure registry scanning credentials on the Sysdig Secure side

This integration offers two different operation modes

Inline scanning:

  • Scanning will be performed inside an AWS CodeBuild pipeline allocating ephemeral resources

  • No need to configure any registry credentials for Sysdig Secure

  • No need to expose your ECR registry to the Sysdig Secure backend

  • Sysdig Secure will not retrieve the image contents, only the metadata that is required to perform the policy evaluation

Backend scanning:

  • Sysdig Secure will retrieve the full image contents in order to perform the scan

  • Your ECR registry must be reachable by the Sysdig Secure backend

  • Registry credentials are required, but they are pushed automatically by a lambda function, no need for manual configuration

Updated Inline Scan Script

  • Added header values for import API for better supportability.

  • Upgraded to Anchore engine v0.6.1.

  • Use docker:dind instead of ubuntu for the base image. This reduces the image size and speeds up downloading.

The latest version of the inline script will always be available at https://download.sysdig.com/stable/inline_scan.sh

Link to repo for script source code: https://github.com/sysdiglabs/secure-inline-scan

Inline Scanning Reporting Improvements and Documentation

This script from SysdigLabs is useful for performing image analysis on locally built container images and posts. The only dependency for this script is access to docker-engine, Sysdig Secure endpoint (with the API token) and network connectivity to post image analysis results.

Here are examples of using the inline scanner in different pipelines:

PDF Reports from the Inline Scanner

A new option

-R  [optional] Download scan result pdf report

will generate a PDF artifact that is available for developers to consume in the pipeline.

Updates to Default Rules and Policies

The following changes have been made to default Policies in Sysdig Secure, and to default Falco rules:

  • New rule tags added that map Falco rules to PCI and NIST controls

  • New default policies added specifically for PIC/NIST compliance

  • Tuning modifications for:

    • Write below etc

    • Write below root

    • Change thread namespace

    • Run shell untrusted

    • Detect outbound connections to common miner pool ports

For more information, see also Falco Rules Changelog.

New Vulnerability Feed Available: VulnDB

We’ve added VulnDB as an additional 3rd-party vulnerability source to improve Sysdig’s coverage in non-OS package vulnerabilities.

In addition, a new page is available for each VULNDB-linked advisory. It lists the CVEs and details about the Common Vulnerability Scoring System (CVSS) scores and external references.

See also: Vulnerability Databases Used.

Linux CIS Benchmark Test Added

Sysdig Agents can run the Independent Linux benchmark against the underlying host where the agent is installed. The Linux benchmark can be scheduled to run at a chosen interval in your environment and emits results and metrics about the status of the tests.

Openshift Hardening Guide

The Openshift hardening guide implements configuration checks run by the agent against Openshift environments.

See https://access.redhat.com/documentation/en-us/openshift_container_platform/3.11/html/container_security_guide/index

Note: This is supported for 3.x versions of Openshift. When Openshift releases a hardening guide for 4.x versions, we will update the configuration checks.

Captures can be Routed to Specific Storage Locations

As a user, you may have different S3 buckets where you’d like to store Sysdig captures, based on the environment where the policy event was triggered. New options are available for deciding what storage option you’d like to use for each policy event.

Sysdig Monitor

New Dashboards is GA

Sysdig Monitor offers a new version of dashboards. Its improved editing experience provides you with more flexibility and the new set of functionalities offers additional ways to visualize and consume your Sysdig data.

Features and Enhancements
Improved User Experience

The New Dashboard offers a more fluid, natural dashboard building experience. For more information, see About the Dashboard UI.

Dashboard Sharing

You can now share your dashboard with members within your Sysdig team or share it across teams with fine-grained access controls. Define who should be able to see the dashboards and what level of access they should be granted: view only or collaborator with edit privileges. For more information, see Sharing New Dashboards

Time Series Name Templating

Customize the time series names on the legend on the panel editor by using the labels associated with Prometheus metrics and segments to gain context faster. For more information, see Create a New Panel.

Multi-Metric, Multi-Segmentation Options

Configure multiple queries within a single panel, and configure each query with multiple segmentation and scoping options. Individual queries can be customized to render as a line or stacked area. For more information, see Create a New Panel.

Event Overlay

Contextualize metrics and understand the “why” faster with a unified view of both metrics and events. Configure event overlay to display events from Kubernetes environments as well as alert events, and any other events ingested using Sysdig’s open REST API. For more information, see Display Dashboard Specific Events.

Dashboard Library

Formerly, Dashboard Templates.

You can quickly view your infrastructure through the lens of one of Sysdig’s curated dashboards, or use it as a base to start building your own. You can find dashboards in the Library for managing Kubernetes capacity and health, hosts and server performance, applications and services telemetry, and the security posture of your infrastructure with data fed from Sysdig Secure. See Dashboard Library to learn more.

Mapping Values to Text

Instantly understand what’s going on by mapping number panel values to text. If you have a metric that returns 1 for up, and 0 for down, map those values to “UP” and “DOWN” respectively. By defining thresholds and mapping to text, you don’t need to be concerned about the values. This is critically valuable when dashboards are shared between team members. For more information, see Text.

Granular Axes and Legend Controls

You have more flexibility when customizing the axes, as well as better support for time series with long names. You can now configure the legend by toggling its visibility and moving it to the bottom of the panel. See About the Dashboard UI.

Major Changes

Significant changes have been introduced to enhance the usability of the existing functionalities. Review the changes before you explore the functionalities.

Topology Maps

Topology maps are no longer available in Dashboard. Access Topology maps through Explore, as you explore your microservices and Kubernetes applications.

Dashboard Wizard

My Dashboards are no longer accessible in Explore. Additionally, Dashboard Wizard has been removed. Instead, the concept of Templates has been introduced in Dashboards to help you get started with a library of templates addressing key use cases.

Histogram and Summary Metric Type

Histogram and summary metrics are no longer supported in the Histogram panel type. You can continue to use them within Explore.

APIs and Integrations

API endpoints for the legacy dashboards (v2) will soon be deprecated. If you are directly integrating into the API, please contact Sysdig for guidance. Additionally, our Python SDK and CLI have been updated to support the new dashboards APIs.

Sysdig Monitor Rebranding

The Monitor app has been refreshed with new logos and icons. The navigation pane has been re-organized. The Explore tab is moved below Dashboards.

The New Get Started Page

The Get Started page provides the key steps to ensure that you are getting the most value out of Sysdig Monitor. We’ll update this page with new steps as we add new features to Sysdig Monitor.

The Get Started page also serves as a linking page for:

  • Documentation

  • Release Notes

  • The Sysdig Blog

  • Self-Paced Training

  • Support

You can access the page at any time by clicking the rocketship icon in the left navigation bar. See Getting Started with Sysdig Monitor.Getting Started with Sysdig Monitor

RBAC and Team Assignment for Notification Channels

Previously, notification channels in Sysdig Secure and Monitor were treated as global entities, visible and editable for most users of the platform regardless of team configurations.

We are enhancing the management and RBAC controls in the following ways:

  • Notification channels can now be “global” or limited to a particular team

  • Global channels can be managed by admins and can be viewed/used by other roles, while team-limited channels are available only to team members

  • Team Manager , Advanced User, and Service Manager (Secure) roles can create/update/delete team-scoped notification channels, they can also read and use the global ones

  • Standard and View Only roles can read team-limited and global notification channels

  • Admins will be able to create global notification channels and migrate channels from “global” to “team-limited”, and also from one team to another.

See also: Set Up Notification Channels and the Share With field in each individual channel setting page.

AWS Role Delegation

Sysdig Monitor can now utilize the Amazon Web Service (AWS) AssumeRole functionality and discover cloud assets, grab CloudWatch metrics from your AWS account, and use custom S3 bucket for storing captures. Upon integrating with an AWS role, you can delegate access to AWS resources that are not associated with your Sysdig AWS account.

Role delegation is an alternative to the existing integration method using the access keys. This method is considered secure as sharing developer access keys with third-parties is not recommended by Amazon.

For more information, see Integrate with AWS Role Delegation.

Configurable Default Team Role

You can now define the default user role to apply when a new member is added to the team. The Admin can change this default on a per-team basis. See also: Create a Team.

Default Dashboards for Istio 1.5

Default dashboards (Overview and Services dashboards) are now available for Istio v1.5 in addition to the existing ones for Istio v1.0.

Release 3.2.2, June 11, 2020

This is a hotfix release for Benchmarks. See Defect Fixes for details.

Upgrade Process

Sysdig Platform has been tested and qualified against the following:

Supported Upgrade From2.5.0, 3.0
PlatformVersion
Vanilla Kubernetes1.13.4, 1.15.3 and 1.16.0
OpenShift3.11, 4.2 and 4.3
GKEv1.14.6-gke.13
EKSEKS .7, Kubernetes 1.14
Rancherv2.3.3
IBMUnqualified
PKSUnqualified
AgentVersion
sysdig/agent10.1.1
ComponentsReplicatedKubernetes with Statefulsets
Redis4.0.12.74.0.12.7
MySQL5.6.44.08.0.16.2
ElasticSearch5.6.16.155.6.16.15
Cassandra2.1.21.162.1.21.16
RDSn/a8.0.16
Postgres (image scanning)n/a10.6.11
Anchore (image scanning)n/a0.5.1.2
NATS Exportern/a0.6.0.1
NATS Streamingn/a0.16.2.1

Installation

Upgrade

Replicated

Install with Replicated

Basic Upgrade (Replicated)

Kubernetes

Installer-based:

Installer (Kubernetes | OpenShift) 2.5.0-3.2.2

Installer-based:

Installer Upgrade (2.5.0+)

Manual:

Manual Install 3.0.0+ (Kubernetes)

Manual:

Manual Upgrade (3.0.0+)

Sysdig Secure

Defect Fixes

Problem: On a cluster running Kubernetes v1.12 or later versions with Sysdig agent v9.7.0 or later versions, the CIS Kubernetes benchmark result could not be interpreted, resulting in an infinite spinner displayed in the UI.

Resolution: Sysdig agents v9.7.0 or later versions can now be used with Kubernetes v1.12 or later versions. The CIS Kubernetes versions included are 1.3, 1.4, and 1.5.

Sysdig Monitor

This release contains no new features or defect fixes.

Sysdig Platform

This release contains no new features or defect fixes.

Release 3.2.1-Onprem (Replicated Only), March 23, 2020

This is a hotfix release that enforces a minimum Replicated Console version to include a necessary security patch. This release contains no new Sysdig functionality and is not a required upgrade.

Use of release 3.2.1-onprem requires first upgrading your Replicated Console to version 2.42.4 or newer.

Release 3.2.0, March 04, 2020

Upgrade Process

Sysdig Platform has been tested and qualified against the following:

Supported Upgrade From2.5.0, 3.0
PlatformVersion
Vanilla Kubernetes1.13.4, 1.15.3 and 1.16.0
OpenShift3.11, 4.2 and 4.3
GKEv1.14.6-gke.13
EKSEKS .7, Kubernetes 1.14
Rancherv2.3.3
IBMUnqualified
PKSUnqualified
AgentVersion
sysdig/agent9.6.1
ComponentsReplicatedKubernetes with Statefulsets
Redis4.0.12.74.0.12.7
MySQL5.6.44.08.0.16.2
ElasticSearch5.6.16.155.6.16.15
Cassandra2.1.21.162.1.21.16
RDSn/a8.0.16
Postgres (image scanning)n/a10.6.11
Anchore (image scanning)n/a0.5.1.2
NATS Exportern/a0.6.0.1
NATS Streamingn/a0.16.2.1

Installation

Upgrade

Replicated

Install with Replicated

Basic Upgrade (Replicated)

Kubernetes

Installer-based:

Installer (Kubernetes | OpenShift) 2.5.0-3.2.2

Installer-based:

Installer Upgrade (2.5.0+)

Manual:

Manual Install 3.0.0+ (Kubernetes)

Manual:

Manual Upgrade (3.0.0+)

Sysdig Secure

Data Retention Limits for Scan Results

Use this feature to set limits on how long image scan metadata is stored, either by tags or days. This removes stale data and helps keep scan results easy to read.

See Data Retention for details.Data Retention

RBAC Capability Available in Sysdig Secure

The new role-based access control (RBAC) model available in Sysdig Secure allows you to define the access privileges granted to each user in a Sysdig Secure team.

Besides the Admin role, which has full access and belongs to every team, there are four roles that can be assigned when adding a user to a team. (Note that the role names are the same in Monitor and Secure, but the privileges differ slightly. Users must be assigned Monitor team roles and Secure team roles separately.)

  • View Only: Read access to every Secure feature within the team scope. A View Only user cannot modify runtime policies, image scanning policies, or any other content.

  • Standard User: Can push container images to the scanning queue and view the image scanning reports. Standard Users can also display the runtime security events within the team scope. They cannot access the Benchmarks, Activity Audit. or Policy definition sections of the product.

  • Advanced User: Can access every Sysdig Secure feature within the team scope in read and write mode. Advanced Users can create, delete, or update runtime policies, image scanning policies or any other content. The Advanced User cannot manage other users.

  • Team Manager: Same permissions as the Advanced User + ability to add/delete team members or change team member permissions.

    Team Managers only have user administration rights within the specific team(s) for which they are designated Managers.

See User and Team Administration for details.

Vulnerability Scan Results Comparison

In image scanning reports, the vulnerability comparison feature allows users to compare two different tags within the same repo to see which vulnerabilities are new or have been fixed in version X compared to version Y.

This allows developers easily to compare the latest image to a previous version to easily report on which vulnerabilities have been addressed and which are new.

See Review Vulnerability Summaries for details.Review Vulnerability Summaries

Redesigned Captures Page

The Captures function in Sysdig Secure has a new look and the following usability improvements:

  • Bulk deletion of capture files

  • Ability to see whether a capture was triggered manually or by a policy

  • Search across all capture files

File Data Source Support for Activity Audit

Sysdig Secure’s Activity Audit now supports a new data source element: File activity.

Sysdig agent version 9.5.0+ is required to enable this new data source.

  • You can now filter the activity by file type or specific file attributes:

    • File name

    • Directory

    • Command (used to access the file)

    • Access mode

  • File activity is also visible in the time-series graph at the top (pink color):

  • Activity Audit will capture non-read file operations executed by interactive commands

Sysdig Monitor

This release contains various bug fixes and improvements. There are no new features in v3.2.0.

Sysdig Platform

S3-Compatible Storage for Capture Files

Configuring S3-compatible storage (such as Minio or IBM Cloud Object Storage) for your Sysdig captures is now supported on Sysdig Platform on-prem deployments. The capability can be turned on by configuring the system appropriately, as given in (On-Prem) Configure Custom S3 Endpoint.

Release 3.0.0, December 19, 2019

Upgrade Process

Sysdig Platform has been tested and qualified against the following:

Supported Upgrade From2.4.1, 2.5.0
PlatformVersion
Vanilla Kubernetes1.13.4, 1.15.3 and 1.16.0
OpenShift3.11, 4.1 and 4.2
GKEv1.14.6-gke.13
EKSv1.14-eks.7
Rancherv2.3.3
IBMUnqualified
PKSUnqualified
AgentVersion
sysdig/agent0.93.1
ComponentsReplicatedKubernetes with Statefulsets
Redis4.0.12.74.0.12.7
MySQL5.6.44.08.0.16.2
ElasticSearch5.6.16.155.6.16.15
Cassandra2.1.21.162.1.21.16
RDSn/a8.0.16
Postgres (image scanning)n/a10.6.11
Anchore (image scanning)n/a0.5.1.
NATS Exportern/a0.6.0.1
NATS Streamingn/a0.16.2.1

Installation

Upgrade

Replicated

Install with Replicated

Basic Upgrade (Replicated)

Kubernetes

Installer-based:

Installer (Kubernetes | OpenShift) 2.5.0-3.2.2

Installer-based:

Installer Upgrade (2.5.0+)

Manual:

Manual Install 3.0.0+ (Kubernetes)

Manual:

Manual Upgrade (3.0.0+)

Sysdig Secure

Activity Audit (Beta)

The Activity Audit in Sysdig Secure allows you to browse a live stream of activity from your Kubernetes containers and nodes. Audit takes the highly detailed data from syscalls and Kubernetes audit logs captured at the agent level, and makes it always-on, searchable, and indexed against your cloud-native assets.

This stream includes executed commands, network activity, and kubectl exec requests to the Kubernetes API. The Activity Audit allows users to view different data sources in-depth for monitoring, troubleshooting, diagnostics, or to meet regulatory controls (SOC2, NIST, PCI, etc).

Flexible filtering and scoping to help you focus on what’s relevant: Filters allow you to search, sort, and surface meaningful data and connections as they are needed. You can filter by data source type, data source attributes (like command name or Kubernetes user) and dynamic Kubernetes scope

Automatically trace a kubectl exec session : The built-in trace functionality allows you to isolate and trace a kubectl exec access to a pod, automatically correlating the original Kubernetes user and IP that accessed the pod with the activity that was performed during the interactive session, including commands and network connections.

Activity Audit is a Preview Beta feature. Contact your customer success manager to learn more about rolling out this feature.

Kubernetes Policy Advisor (Beta)

With the Kubernetes Policy Advisor, Sysdig Secure auto-generates Pod Security Policies (PSPs) to significantly decrease the time spent configuring Kubernetes Policies. Strict security policies reduce risk, but can also break applications. Sysdig tests the impact of pod security policies through simulations, enabling teams to adjust misconfigurations before shifting to production. There are three main features that comprise the Kubernetes Policy Advisor:

Auto generation: Sysdig Secure can parse any Kubernetes yaml file that includes a pod spec to generate a tailor-made PSP based on the configuration.

Simulations: Start a simulation of the auto-generated PSP or any user-inputted PSP to see what pods would have been blocked from running if this PSP had been actively applied to the cluster.

Events and tuning: Each pod/activity that would have violated the PSP will generate an event. Within the event details, users can see information about potential modifications they may need to make to the policy or the pod configuration.

Scanning Improvements

New Scanning Rules

File attributes can now be verified as part of the image scan analysis. A specific file can be validated against a node or sha256 hash.

Scale Improvements to Scanning Reporting

No query conditions are required as part of the Package and Policy Queries.

Google Distro-less OS

Support for images based on Google distro-less OS, including detection of base OS/version and installed OS dpkg packages.

Sysdig Monitor

Overview Is GA

Overview is now generally available. Overview leverages Sysdig’s unified Kubernetes data platform to monitor, secure, and troubleshoot your Kubernetes clusters and workloads.

Please contact your Sysdig Technical Account Manager or email support to enable Overview for on-premises environments.

Cluster Overview

Major highlights of Overview GA include but are not limited to:

  • Multi-cloud view of the health, risk, and capacity of your Kubernetes infrastructure— a single pane of glass for Kubernetes Clusters, Nodes, Namespaces, and Workloads across a multi- and hybrid-cloud environment. You can easily filter by any of these entities and view associated events and health data. View the infrastructure organized by Clusters, Nodes, Workloads

  • Shows metrics prioritized by event count and severity, allowing you to get to the root cause of the problem faster.

  • Drill down to Dashboards for instant insights.

To learn about the capabilities of the Overview feature, see Overview.

Enhanced Out-of-the-box Dashboards

In an attempt to improve the Dashboards experience, the following changes have been introduced:

The following Dashboards are added:

  • Kubernetes Cluster Overview: Provides nodes and workloads availability and highlights the high-level health of your Clusters. It also summarizes resources consumption (CPU, memory) across Nodes and Namespaces to pinpoint possible anomalies and node disk utilization

  • Kubernetes Node Overview: Provides availability of the Nodes, indicating potential issues reported by Kubernetes; a summary of resource (CPU and Memory) allocation and utilization, as well as Network and Disk utilization.

  • Kubernetes Namespace Overview: Provides a high-level summary of availability, and resource allocation and utilization across all the Workloads in the selected Namespace.

  • Kubernetes Deployment Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods for each Workload.

  • Kubernetes StatefulSet Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods for each StatefulSet.

  • Kubernetes DaemonSet Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods.

  • Kubernetes Job Overview: Provides a detailed summary of job status, completion trend, pod restarts, as well as resource allocation and utilization across pods.

  • Kubernetes ReplicaSet Overview: Provides a detailed summary of pod status, pod restarts, as well as resource allocation and utilization across pods for each ReplicaSet.

  • Kubernetes Pod Overview: Provides a detailed summary of pod status, pod restarts, and resource allocation and utilization in a selected pod.

  • Kubernetes Workloads CPU Usage and Allocation: Helps you verify that CPU requests are properly configured and actual utilization is expected.

  • Kubernetes Workloads Memory Usage and Allocation: Helps you verify that memory requests are properly configured and actual utilization is expected.

  • Kubernetes CPU Allocation Optimization: Helps you verify that infrastructure resources are available for future needs and are not wasted.

  • Kubernetes Memory Allocation Optimization: Helps you verify that infrastructure resources are available for future needs and are not wasted.

The following Dashboards are retained:

  • Health Overview (applicable to all the objects in the environment)

  • Horizontal Pod Autoscaler (the default Dashboard when selecting an HPA)

  • Resource Quota

  • Service Health (the default dashboard when selecting a service)

  • Cluster and Node Capacity

The following Dashboards are removed:

  • State Overview

  • Daemonset State

  • Namespace State

  • Stateful State

  • Nodes State

  • Deployment State

  • Deployment Health

  • Nodes Health

  • Namespace Health

  • Pod State

  • Pod Health

  • Replica Set Health

For more information, see Pre-Defined Dashboards

Filtering Events by Scope

Events are now filtered by Scope to show the most relevant Events in Explore and Dashboards. This is an extension of the existing Event Scope functionality. You can toggle between showing Event feed from the entire infrastructure and only from the particular scope you are interested in within the infrastructure. Event scoping for Dashboards and Explore is enabled by default.

By default, Events are filtered to show only the relevant ones. However, you can turn the filtering off and see Events from the complete scope. To do so:

  1. Click the Dashboard Settings (three dots) icon and select Events.

  2. Use the toggle button to turn off Filter events by dashboard Scope.

  3. Click Save.

Similarly, you can filter Events by Scope in Explore.

What’s n/a?

The Sysdig Monitor UI displays n/a in several scenarios associated with labeling. The Explore UI has now been enhanced to add a tooltip for n/a to help you understand the scenario. See The Meaning of n/a for more information.

Release 2.5.0, October 29, 2019

Upgrade Process

Kubernetes and OpenShift environments upgrade to 2.5.0 using the new installer tool (see below).

Supported Upgrade Path: 2.3.0, 2.4.1

Sysdig Platform

New Installer Tool for Kubernetes/OpenShift Environments

With this release, Sysdig platforms can be installed and upgraded using a semi-automated installer tool that greatly simplifies the installation process. Available for Kubernetes and OpenShift environments.

SeeInstaller (Kubernetes | OpenShift) 2.5.0-3.2.2 and Installer Upgrade (2.5.0+) for details.

Enhancement: New Documentation Site at docs.sysdig.com

Sysdig’s documentation platform has been upgraded and moved to docs.sysdig.com.

Improvements include:

  • Look and feel: Updated to match the rest of the Sysdig branding

  • Search: Enhanced search speed, accuracy, and ease

  • Structure and content: Enhancements to content have been added and are being continuously updated

  • Feedback: Buttons on each page enable users to communicate directly with the documentation team.

Sysdig CLI

The Sysdig CLI provides an easy way to interact with the cli via the command line. Read more here.

Usage:

Run it without parameters to get a list of all the commands.

$ sdc-cli
Usage: sdc-cli [OPTIONS] COMMAND [ARGS]...

  You can provide the monitor/secure tokens by the SDC_MONITOR_TOKEN and
  SDC_SECURE_TOKEN environment variables.

Options:
  -c, --config TEXT  Uses the provided file as a config file. If the config
                     file is not provided, it will be searched at
                     ~/.config/sdc-cli/config.yml and /etc/sdc-cli/config.yml.
  -e, --env TEXT     Uses a preconfigured environment in the config file. If
                     it's not provided, it will use the 'main' environment or
                     retrieve it from the env var SDC_ENV.
  --json             Output raw API JSON
  --version          Show the version and exit.
  --help             Show this message and exit.

Commands:
  alert       Sysdig Monitor alert operations
  backup      Backup operations
  capture     Sysdig capture operations
  command     Sysdig Secure commands audit operations
  compliance  Sysdig Secure compliance operations
  dashboard   Sysdig Monitor dashboard operations
  event       Sysdig Monitor events operations
  policy      Sysdig Secure policy operations
  scanning    Scanning operations
  settings    Settings operations
  profile     Profile operations

Sysdig Monitor

Ability to “Favorite” a Dashboard

Users can click the star icon to mark a “Favorite” dashboard, which will then be listed under “My Favorites” in the Dashboard view.

Sysdig Secure

In-Line Scanning

Images can now be analyzed locally before they are pushed to a registry. This has a few key benefits to users.

  • Images can be analyzed before they’re pushed to a registry and reduce registry cost

  • Customers using the Sysdig Secure SaaS offering don’t need to expose their registry to our SaaS for images to be scanned

  • For OpenShift users, the in-lince scan option can be integrated into the S2I process to scan images without needing to expose a local cluster registry via a route

Learn more and access the script here: https://github.com/sysdiglabs/secure-inline-scan

SSO Configuration Pages Available in Secure

A UI for configuring Single Sign-On for Sysdig Secure is now available from the Settings menu. See Authentication and Authorization (On-Prem Options).

New Package Reports

Package name/version are now grouped together to provide easy parsing of all CVE’s associated with a package and the images using that package.

New Trigger Parameters for CVSS Score

Image Vulnerabilities can now be evaluated against their CVSS (Common Vulnerabilities Scoring System) score. If a vulnerability is =, <;>, <=, or >= to a specific score, then the rule can trigger a warn/stop action.

Time Ranges Updated

The default time range options have been updated in Sysdig Secure.

The default time ranges are now set to:

  • 10 Minutes 

  • 30 Minutes

  • 1 HR

  • 6 HRs

  • 1 Day

  • 3 Days

To look at a custom window of time, use the manual time window.

Sysdig Secure Summary Dashboard in Sysdig Monitor

Sysdig Monitor includes default dashboards that provide metrics about number of agents installed, active policies, events that have occurred, and the policies that have triggered them. Use these dashboards to identify trends, report on coverage, or facilitate the tuning process.

Release 2.4.1, September 18, 2019

Upgrade Process

Review the Migration Path tables in On-Premises Upgrades

Supported upgrade path: 2.3.0

Sysdig Platform

Secure Authentication for Cassandra and Elasticsearch on Replicated

Cassandra and Elasticsearch datastores now have an extra layer of security on Replicated. Sysdig Replicated install allows you to enable authentication and secure communication between Sysdig backend components and the Elasticsearch or Cassandra datastores. For more information, see Install with Replicated.

[BETA] Audit Logging

The following APIs have been introduced to support administrators to view a log of user activities and modifications to the components in the system:

  • AppAttributes

  • AuditEvents

Audit logs stand for chronologically cataloged events to provide a history of operational actions and to mitigate challenges. The ability to trace an event back to its origin provides proof of compliance, operational integrity, and protection from unsolicited use. For more information, see [BETA] Auditing Sysdig Platform Activities.

Known Issues

If you want to use Audit logging and have MySQL in your Kubernetes HA environment, run kubectl -n sysdigcloud delete pod -l role=worker to ensure Audit logging works as expected. This issue is observed only in Kubernetes HA environments.

Sysdig Monitor

New Default Kubernetes Grouping

Groupings for Kubernetes have been modified. This updated Grouping is available to new teams. Default groupings are immutable–-they cannot be modified or deleted other than by copying. Modifying a copy is allowed.

New Groupings:

  • Clusters and Nodes (cluster.name > node.name > pod.name > container.name)

  • Deployments (cluster.name > namespace.name > deployment.name > pod.name > container.name)

  • Services ( cluster.name > namespace.name > service.name > pod.name > container.name)

  • Statefulsets (cluster.name > namespace.name > statefulset.name > pod.name > container.name)

  • Daemonsets (cluster.name > namespace.name > daemonset.name > pod.name > container.name)

  • ReplicaSets (cluster.name > namespace.name > deployment.name > replicaset.name > pod.name)

  • HPAs (cluster.name > namespace.name > hpa.name > pod.name > container.name)

For more information, see Grouping, Scoping, and Segmenting Metrics.

Units for Metrics

The format of metric units are the same for the following:

  • The CPU and Memory metrics for Host and Container.

  • Kube-state CPU and Memory metrics.

Introducing the same format now makes the comparison of those metrics easier on a chart.

Container Segmentation

Sysdig now supports segmenting all net.* metrics at container or pod level by low level net.* dimensions, such as net.http.url or net.http.status.code. Container-based teams now display segmentations for net.http.* metrics as expected. The net.http.url and net.http.status.codes are displayed if you select a container-based team as it does for a host-based team for the same cluster.

Enhanced Event Notification

The ability to customize the subject and body of alert notifications with variables has been extended to Event notifications. Event titles and notification messages are in sync in the following cases:

  • Event feed on the Events page

  • Event overlay on Dashboards page

For more information, see Events.

Default Dashboard for Cluster and Node Capacity

Kubernetes Cluster and Node Capacity Dashboard has been refreshed to add actual usage of CPU and Memory compared to Requests, Limits and Allocatable capacity.

Aggregation for Kubernetes Nodes Health

Aggregation method has been refreshed for Kubernetes Node metrics. The Kubernetes Node Health dashboard has been updated with metric aggregations that are ‘summed’ across all containers running on the node to reflect accurate node level data.

Bug Fixes

  • Export CSV/JSON was missing columns, not all data was exported as expected. All columns from the dashboard should exist in the exported output.

  • All data and columns are is now exported as expected.

Sysdig Secure

Policy Editor

*Please upgrade to an agent version 0.92.0 or greater

This UX overhaul brings three major improvements for every Sysdig Secure user:

  • Runtime policies can import any number of security rules. You can scope the security policy using container, cloud and Kubernetes metadata.

  • Tighter Falco integration, directly from the web UI. You will be able to define a new trigger condition or append to the list of forbidden external IPs just clicking on the rule.

  • A more structured way to group, classify and lookup rules, following the standard Cloud native procedure: tags and labels.

Rules Library

Visualize your runtime rules properties in just a glance:

  • Where this rule comes from (Published By). The security team can instantly recognize whether a rule came from a specific Sysdig update, from a custom rules file created within the organization or from an external rules source (like the Falco community rules).

  • When was the last time it was updated (Last Updated). You can use this information to audit your rules or if you schedule periodic updates, to confirm when last happened.

  • Rule tags: An effective method for organizing your rules. You can use these tags to describe the targeted entity (host, k8s, process), the compliance standard it belongs to (MITRE, PCI, CIS Kubernetes) or any other criteria you want to use to annotate your rules.

Falco Lists

Easily browse, append, and re-use lists to create new rules. Lists can also be updated directly via API if users want to add existing feeds of malicious domains, or IPs.

Falco Macros

Easily browse, append, and re-use macros to create new rules.

Image Scanning Reports

Please contact Sysdig Support to enable this feature

The reports feature allows users to query the contents of a scan against a static or run-time scope to generate a report that shows the risk, exposure, or components of an image.

Use cases could include:

  • A new CVE has been announced, let me find all the running images in my US East Cluster that are exposed to that CVE

  • Show me all images within my Google Container registry that have the tag prod and have a vulnerability with a fix that’s more than 30 days old

  • Show me all images with a high severity vulnerability with a fix that are running in my billing namespace

Image Scanning - View Scan Results

Scan Results Page - The existing repositories page has been renamed “Scan Results” this page also includes new capabilities to filter based on where the images are deployed, and to easily browse/expand the different repositories to see the image:tag’s that were evaluated and their results

Whitelist labels available in vulnerabilities view - If a vulnerability has been added to a whitelist then that status is reflected in the Vulnerability report within the scan results.

Event Forwarding

Sysdig Secure can forward policy events to tools like Splunk or events can be forwarded via syslog as an easy way to send policy events to any downstream SIEM.

Release 2.3.0, July 29, 2019

Upgrade Process

Review the Migration Path tables in On-premise Upgrades.

Supported upgrade paths: 1929, 2435.

Important Note for Kubernetes Upgrades

Due to the new Secure Elasticsearch and Cassandra feature, Kubernetes installations must follow an Expanded Upgrade process.

This version of Sysdig On-Premise requires Elasticsearch to be at 5.6.x, which is done automatically when you follow the Expanded Upgrade process.

If you are running your own instance of ES, you will need to update it to 5.6.x.

Replicated Upgrades

For Replicated installations, the upgrade instructions are here: Upgrade Replicated Installations.

Sysdig Platform

Option to Secure Elasticsearch and Cassandra (Kubernetes only)

It is now possible to secure Elasticsearch and the Cassandra DB with password authentication and/or SSL/TLS protection.

Sysdig Monitor

Enhanced Dashboard Menu

The Dashboard menu features a drawer-style popover that displays on-demand to provide maximum real estate for your Dashboards. The menu displays an alphabetical list of Dashboards you own and those shared by your team. With the popover menu, you can add new Dashboards and search for existing ones. Click a Dashboard name to access the relevant Dashboard page where you can continue with the regular Dashboard settings.

Customize Alert Notification Template

Sysdig Monitor alerts now provide an option to customize the messages that are sent with alert notifications in email and other channels, such as Pagerduty and Webhook.

Use the Alert Editor to input dynamic variables, such as hostname, or a hyperlink, and to add custom messages in plain text to the notifications for intended recipients. You can modify both the subject and the body of the alert notification with a hyperlink or a variable. For example, you can add an agent id or a link to a Dashboard to the message. This can help provide context for troubleshooting the errors that triggered the alert.

For more information, see Customizing Alert Notification.

Prometheus Remote Scraping

Sysdig Monitor can now collect Prometheus metrics from remote endpoints with minimal configuration.

Remote endpoints (remote hosts) refer to hosts where the Sysdig agent cannot be deployed, e.g., a Kubernetes master node on managed Kubernetes services such as GKE and EKS, where user workload cannot be deployed. To enable remote scraping on such hosts, simply identify an agent to perform the scraping and declare the endpoint configurations in the agent configuration file.

The collected Prometheus metrics are reported under and associated with the agent that performed the scraping, rather than with a process. See Collecting Prometheus Metrics from Remote Hosts for details.

Enhancements to Kafka App Check

Kafka integrations can now support authentication and SSL/TLS. If the authentication or SSL/TLS are enabled in Kafka, see Apache Kafka Example 5 for how to enable configuration details on the Sysdig side.

Two New Metrics for Accurate Pod Counts

Two new Kubernetes metrics, kubernetes.namespace.pod.desired.count and kubernetes.namespace.pod.available.count, have been added at the Namespace level to track desired and available pod counts.

Sysdig Secure

Image Scanning: New Trigger Options

  • New Image Analyzed - Send notifications to different channels when images with a particular registry, repo, tag are scanned.

    • Some users implement these type of alerts for implementing workflows for image promotion, i.e.

      “Push an image from staging to prod registry after a webhook is sent that the image was scanned and it passed.”

  • CVE Update - Be notified whenever a vulnerability is added, updated, or removed from an image within a registry.

Repository Alerts

Receive alerts about activity and changes that occur within your registry. See Manage Scanning Alerts.

Slack Notifications

Sample output of a CVE alert:

Sample output of an image-analyzed alert:

Image Scanning: Policies - New rule parameter available

A new field: Max days since creation is now available. This allows users to only take Stop or Warn actions if a vulnerability has been in the feed for a certain number of days.

For example: Only stop a build if an image has a high-severity CVE with a fix, and the CVE is more than 30 days old.

Image Scanning: Policy Assignments - New compliance audits available

Policy assignments now support the ability to add audit policies to provide a second step of validation of container images. Additional audit policies evaluate images against Dockerfile Best Practices, PCI, and NIST 800-190. These Audit policies have “Warn” actions set by default and are intended to validate compliance/audit use cases and not cause CI/CD builds to fail.

Updated Menu Navigation in Sysdig Secure

The top-menu navigation has been replaced by a context-sensitive drawer-style side navigation bar.

Image Scanning: Scan Results Redesign

Scan results have been expanded to help users get a better idea about the policy evaluation status and vulnerabilities present in an image. This new version of scan results allows the user to

  • Get a breakdown of the different OS/Non-OS Critical, High, Medium, Low CVEs present in the image

  • See the different policies the image has been evaluated against

  • See which specific rules have triggered the most stop/warn actions and identify areas needing attention

A breakdown of the evaluation result has been added to give users a better idea about what has triggered warn/stop actions as part of the evaluation.

In this case, we can look at the Dockerfile Best Practice policy to see the image

  • Has an effective user of root

  • Doesn’t include a Healthcheck

  • Uses apt-get upgrade as part of a Run instruction

  • Includes an ADD instruction

The Vulnerabilities section also now supports enhanced sorting and filtering by severity level and whether or not a fix is available.

Image Scanning: PDF Reports

PDF reports, which include a summary of the policy evaluation and all vulnerabilities present in the image, can be downloaded from the console.

Bug Fixes

  • Explore display fix

    Fixed an issue where, when the Explore Table had no columns configured, the Explore view showed an error.

  • Enable/disable alerts fix

    Fixed a problem where users were unable to toggle alerts.

  • Event posting fix

    Fixed an issue where events posted in Slack did not appear in the event stream. Now they do.

  • Monitor Spotlight fix

    Fixed issue where Monitor Spotlight incorrectly alerted to update On-Premise releases all the time. Update alert now turns on only when an update is actually available.

  • Improved access to kube-state metrics

    Teams based on ‘hosts’ (e.g., scoped by agent.tag.* ) will now have access to all host and container data, including kube-state metrics and dashboards. In previous versions, kube-state metrics were not available for host-based teams.

Release 2435, July 24, 2019

Release 2435 replaces version 2172, 2266 and 2304 which were released on May 28, 2019, June 17, 2019 and June 21, 2019. If you installed 2172, 2266 or 2304, upgrade to 2435.

Upgrade Process

Review the Migration Path tables in On-premise Upgrades.

Supported upgrade paths: 1765, 1929.

(Note that if you installed 2172, 2266 or 2304, please upgrade to 2435. Otherwise, skip 2172, 2266 and 2304.)

Important Note Regarding Dashboard Migration V1 > V2

If you are upgrading from a previous version, the Dashboards will be upgraded from V1 to V2. The process requires 20-30 minutes on large systems, and the environment remains live throughout the rolling upgrade.

DO NOT create or delete dashboards during the upgrade. After upgrading, if you have saved v1 dashboards previously and need to upload them to the v2 environment, see Migrate Saved Dashboards from V1 to V2.

Sysdig Platform Fix

Custom certificates fix

Fixed an install issue caused when using custom certificates.

Release 2304, June 21, 2019

Release 2304 replaces version 2172 and 2266 which were released on May 28, 2019 and June 17, 2019. If you installed 2172 or 2266, upgrade to 2304.

Upgrade Process

Review the Migration Path tables in On-Premises Upgrades.

Supported upgrade paths: 1765, 1929.

(Note that if you installed 2172 or 2266, please upgrade to 2304. Otherwise, skip 2172 and 2266.)

Important Note Regarding Dashboard Migration V1 > V2

If you are upgrading from a previous version, the Dashboards will be upgraded from V1 to V2. The process requires 20-30 minutes on large systems, and the environment remains live throughout the rolling upgrade.

DO NOT create or delete dashboards during the upgrade. After upgrading, if you have saved v1 dashboards previously and need to upload them to the v2 environment, see Migrate Saved Dashboards from V1 to V2.

Architecture Change in the Containers

In previous releases, there was a single backend container which ran several processes.

As of version 2266, the processes have been divided into unique containers, following container best practices.

Previous:

New:

Sysdig Platform Fix

Redis Client Fix

Updated an underlying tool (Jedis 2.9.1) to Jedis 2.9.3, to address a bug in the connection pool.

Sysdig Monitor

Manage Notification Frequency for Alerts

Users now have the ability to specify how often they want to be reminded about an alert if the event is unresolved. Available under ‘Notify’ section of the alert configuration screen. See Alerts.

Advanced Scope Selection

The scope editor (for dashboards, alerts, teams, etc.) has added improved granularity, intelligent scope restriction, and the ability to add custom values on-the-fly. The editor now restricts the scope of the selection for subsequent filters by rendering values that are specific to the selected label. The values that are only relevant to the previous selection are displayed. For more information, see Dashboard Scope.

Ability to Choose Unit of Metric

Sysdig Monitor now automatically detects the type of input and scale for custom metrics. Earlier, custom metrics were marked as numbers on both Explore and Dashboard UI. The UI now supports custom unit scale for custom metrics. The supported units are byte, percent, and time. This enhancement simplifies the mapping of units of measurement with that of integrated application metrics, such as Prometheus. For more information, see Editing the Unit Scale.

Kubernetes Horizontal Pod Autoscaling (HPA) metrics

Support for the following HPA metrics has been introduced: kubernetes.hpa.replicas.min, kubernetes.hpa.replicas.max, k ubernetes.hpa.replicas.current, and k ubernetes.hpa.replicas.desired. For more information, see Resource Usage.

Expose Dashboard Scope in URL

The Dashboard URL can include scope parameters, including scope variables. Users can now share the URL with non-Sysdig Monitor users and allow them to collaborate on dashboard scope. Collaborators with a valid link can change the scope parameters without having to sign in. They can edit either on the UI or in the URL. For more information, see Share a Dashboard.

Sysdig Secure

Image Scanning: Policy Assignments

Policy assignments allow you to specify where your image scanning policies are applied. A policy assignment can include a Registry, Repository, Tag combination and has full wildcard support for each of those fields.

Policy assignments are evaluated in descending order, so be sure to specify the most important policies first.

Examples

  • To evaluate all images with a “Prod” tag with your Example Prod Image Policy, use the assignment: */*/Prod

  • To evaluate all images from gcr.io with an Example Google Policy, use the assignment: gcr.io/*/*

See Manage Scanning Policies.

Image Scanning: Map Internal Registries (for OpenShift environments)

The recommended way to run an image registry for an OpenShift cluster is to run it locally. The Sysdig agent will detect the internal registry names, but for the Anchore engine to pull and scan the image it needs access to the internal registry itself. There can now set this path in the Registries UI. See Manage Registry Credentials.

Compliance: Custom Report Filters

When running CIS benchmark tests, you can filter your view of the results to show only high-priority items or selected controls.

See Understanding Report Filters and Filter Report Results.

Bug Fixes

  • Improved metric aggregation defaults in Explore window

    When a metric is first selected on the Explore page, the time and group aggregation will be pre-populated with the most reasonable choice, rather than average/average.

  • Topology view fixes: Implemented fixes for proper loading of Topology panels in public dashboards, and proper “group by” and ‘scope" Topology Views.

    See Visualizing Metrics using Topology View.

  • Non-root user security enhancements

    Added changes to permit running Sysdig applications as non-root user.

  • Image scanning fix in Sysdig Secure

    Bug fix in the Jenkins plugin used to scan images in Sysdig Secure.

Release 1929, April 12, 2019

This release supports upgrades from

1149, 1245, 1402 (1511), 1586 (1630), 1765

New Features

Sysdig Platform

CRI-O Support

Sysdig on Kubernetes now provides support for CRI-O, an implementation of the Kubernetes Container Runtime Interface (CRI).

See Sysdig documentation here.

CRI-O container runtimes can be identified by the symbol beside the entry in the Explore table:

Customize Data Retention Times using Sysdig REST API

The Sysdig platform has predefined data retention settings determined by license plan. Using the Sysdig REST API, it is possible to configure separate retention times (up to plan limit).

See Customize Data Retention for details.

Sysdig Secure

Global Whitelists

Sysdig Secure allows users to manage CVEs and images that may impact builds by defining them as globally trusted or blacklisted. See Manage Vulnerability Exceptions and Global Lists for more information.

Kubernetes Audit Logging

Sysdig Secure allows users to create Falco security rules based on a stream of Kubernetes audit events, integrating Kubernetes audit logging with the Sysdig Agent. This allows users to track changes made to the cluster, and send alerts where necessary. See Kubernetes Audit Logging for more information.

Enhancements

Manual PagerDuty Notification Channel Setup

Sysdig has expanded the PagerDuty notification channel configuration process to allow users that have a team role of Manager, but a user role of Team Responder or lower, to manually configure the channel settings in order to add new channels. See PagerDuty Notifications for more details.

Agent Installation Changes

The default agent installation instructions in the UI have been updated to ensure all agents use SSL. If SSL is not required, the following JVM parameter will need to be set in the backend:

-Ddraios.agents.installParams.sslEnabled=false

See Integrate JMX Metrics from Java Virtual Machines.

Bug Fixes

Anchore issue that caused scanning to hang when adding a registry

An issue occurred where scanning stopped functioning when adding a new image scanning registry to an environment. This was caused by a bug found in the Anchore open-source engine. This on-premises release includes the approved workaround patch that corrects the issue. The next release of the Anchore open-source engine will include the full fix.

Scanning service degradation due to orphaned services

An issue occurred in systems with substantial churn where the event system became overloaded/flooded with orphaned service events, resulting in service and performance degradation. This was caused by the Anchore engine emitting an event each time it found a service that was down/orphaned. This issue has been resolved.

Images with host/port component weren’t flagged with the correct analysis

An issue occurred where images with a host/port component were not flagged correctly, resulting in them showing as unscanned. This was caused by a bug in the scanning backend and has now been resolved.

Scan alert e-mail

An issue occurred in on-premises version 1765, where email alerts for scanning results directed users to an internal Sysdig environment, rather than their own. This has been corrected.

Some panels in self-monitored dashboards not working

An issue occurred where some panels in the Self-Monitored default dashboards were not displaying data correctly, because of an error in the default dashboard configuration file. This error has been corrected.

Relocated “Control Plane” from Default Dashboard in Explore

Kubernetes Control Plane Health dashboard has relocated to the Dashboards module. This dashboard allows users to monitor the health of Kubernetes master components (kube-apiserver, etcd, kube-scheduler, kube-controller-manager). The Kubernetes Control Plane health dashboard has been removed from the list of default dashboards available under Resource Usage.

ElasticSearch on Replicated Restarts into Split Brain

When a customer restarted their Replicated environment, ElasticSearch sometimes came up in a split-brain scenario (generally 2 + 1). This issue has now been addressed.

Install code lines for Sysdig Agent corrected

On the Agent Installation page of the Sysdig UI, the supplied install strings for Docker and Linux were incorrect and would not work “out of the box” for a Replicated deployment. This issue has been addressed.

Release 1765, March 13, 2019

This release supports upgrades from: 987, 1149, 1245, 1402 (1511), 1586

Upgrade Process for Sysdig in Kubernetes Environments

If you are running Sysdig Secure in OpenShift OR if you are running more than 400 agents, please contact Sysdig Support before upgrading.

If you are running Sysdig in Kubernetes, then the upgrade process for this release is comprised of two parts:

  1. Run the migration script:

    This accommodates the backend transition to a different library for communicating with the database.

  2. Perform the Upgrade:

    For Sysdig Monitor Only: If you have not licensed Sysdig Secure and run only Sysdig Monitor, use the Basic Upgrade instructions.

    For Sysdig Platform (including Secure): If you have licensed both Sysdig Monitor and Sysdig Secure, you must follow the v1765 Upgrade (Kubernetes) instructions. These steps add the components necessary to run the Scanning feature.

New Features

Sysdig Platform

Containerd Support

The Sysdig agent will automatically detect containerd metadata, as well as any Docker metadata, in your environment. Note that you must have agent version 0.88.1 or higher. See the agent install instructions for details.

If you are upgrading from an earlier version of the agent, note that you must also download the latest sysdig-agent-daemonset-v2.yamlfrom GitHub for containerd functionality.

Sysdig Monitor

Improved Notification Channels Configuration

A newly redesigned notification channels page under settings has been implemented. For more information, see Set Up Notification Channels.

New Kubernetes Dashboards

Added two new default Kubernetes dashboards to help users monitor Cluster / Node health and Namespace health. The dashboards are available under the default dashboard list in Explore.

Sysdig Secure

Improved Registry Credential UI

The user interface for adding registry credentials has been redesigned to improve user experience and add new configuration functionality. See Registries.

Event Forwarding

Sysdig Secure policy events can now be forwarded to Splunk. See Event Forwarding.

New Scanning Policies

New scanning policies have been added for compliance use cases and best practices, interpreting NIST 800-190 and PCI controls to detect misconfigured images.

Remediation Information

Remediation information has been added to assist in solving non-passing test results, in order to bring an environment into compliance. See Remediation Information.

Identify the Kubernetes Master Node

A new label has been added to the Compliance task results page to assist in identifying the Kubernetes master node. See Identify the Kubernetes Master Node.

Run a Compliance Task Manually

Users can now choose to run a compliance task immediately, rather than scheduling a task for later. See Run a Benchmark Test Manually.

Jenkins Plugin Available in Jenkins Community

The Sysdig Secure Jenkins plugin is now available here: https://wiki.jenkins.io/display/JENKINS/Sysdig+Secure+Jenkins+Plugin

Enhancements

Sysdig Monitor

User Interface Changes

The Intercom button has been moved from the bottom right corner of the Sysdig Monitor UI to the bottom left to facilitate a better user experience, as the previous location interfered with other UI elements. It can now be found below the Help, Spotlight, and User menus.

Bug Fixes

The following issues have been fixed in this release:

Dashboard data display issue

An issue occurred when users in a team scoped by container tried to access a dashboard. While building the read requests, the correct team filters were used, but the write request incorrectly set the domain to host instead of container, resulting in the backend not reading the data correctly. This issue has been resolved.

AWS data display issue

For some AWS queries, data displayed incorrectly because the backend could not determine the AWS resource type being queried, so the aws.resource.type metadata was added to the request scope.

Assign User to Team in Secure

In some cases, users could not be added to Sysdig Secure teams, because of a backend issue that occurred when loading the list of available users to add to a team. This has been resolved.

Release 1630 Hotfix, January 31, 2019

This release supports upgrades from: 1149. 1245, 1402, 1511, and 1586.

Performance Issues

A performance issue was found when creating snapshots for large number of teams and large number of custom metrics. This issue has been fixed.

Release 1586, January 21, 2019

This release supports upgrades from: 1149. 1245, 1402, and 1511.

New Features

Sysdig Monitor

New Events Feed

A redesigned Events Feed is now available. The new design unifies all of your infrastructure-related events, alerts, and other activity in a single view to help you quickly identify critical issues that need your attention. For more information, refer to the Events documentation.

New Topology is now GA

The new topology map functionality in Sysdig Monitor has moved from a labs feature to full general availability. It features a redesigned layout and enhanced interaction model to provide insight into dependencies with drill-down to the container-process level.

Authentication UI

Administrators can now configure single sign-on authentication methods (LDAP, SAML, OpenID, Google OAuth) via the Sysdig Monitor UI. For more information, refer to the Authentication and Authorization (On-Prem Options) documentation.

Enhancements

New Metrics

An additional metric (kubernetes.pod.restart.rate) has been added to show the number of pod restarts since the last check.

Kubernetes Groupings

In previous releases, the default Kubernetes groupings used kubernetes.cluster.id. This has been changed to kubernetes.cluster.name to improve user experience.

Java Virtual Machine (JVM)

The JVM flag -UseContainerSupport has been disabled for performance reasons.

Alert Delay at Startup

Sysdig alert jobs begin immediately at start-up. However, in instances where Sysdig goes down unexpectedly, or without proper shutdown/startup procedures implemented, data can be missing, triggering alert notifications.

A start-up delay in alert jobs can be configured in on-premises environments, by setting the draios.alerts.startupDelay parameter during the installation process. The parameter requires a duration value; the example below shows a duration of 10 minutes:

draios.alerts.startupDelay=10m

This parameter can be configured for either Replicated or Kubernetes environments:

  • For Replicated environments, add the parameter to the Sysdig application JVM options list. For more information, refer to the Install Using the Replicated GUI documentation.

  • For Kubernetes environments, add the parameter to the sysdigcloud.jvm.worker.options parameter in the configmap. For more information, refer to the Sysdig Install with Kubernetes 1.9+ documentation.

Sysdig Secure

Compliance (Benchmarks)

  • CIS compliance benchmarks now support customizable schedules, using a selection of intervals, days, and times, for different compliance tasks to execute on.

  • Users can now download individual compliance results as a CSV file. For more information, refer to the Download Task Results documentation.

  • The Compliance scheduling page now displays when the next compliance test will run.

  • An error log is now displayed when a compliance test fails.

  • Users can now search the list of compliance tests by hostname.

Bug Fixes

Mesos.*percent metrics do not currently have ‘%’ as a selectable unit scale

Mesos.*percent metrics did not include percentage as an option for the metric unit scale. This has been corrected in the backend.

Split brain in Elasticsearch when launching Kubernetes HA env

A bug in the Elasticsearch container configuration created the potential for the nodes to fail to discover all of the members of their cluster at start-up. This resulted in a “split-brain” in the Elasticsearch cluster, where nodes created multiple separate clusters, instead of a single cohesive cluster.

The configuration of the container was re-tooled to allow the Kubernetes cluster to expose the existence of the pods to their peers before they finish starting up, and the cluster pods will now be aware of all of the cluster members at start-up.

Release 1511 Hotfix, January 8, 2019

Issue: Better Handle Unknown Container Runtimes

In previous releases, snapshot jobs would fail if data for computing aggregations for Kubernetes pods from unsupported container runtimes was present. Containers in unknown runtimes are now skipped when computing these aggregations to circumvent the error.

These containers are still present, and the metrics can be seen in non-kubernetes contexts, as well as some Kubernetes contexts. (For Kubernetes contexts, they are listed as null).

Issue: JVM Settings Fix

Prior to JVM update 191, the JVM was not container-aware, and used system-level resources for auto-configuration. Update 191 changed this behavior to use container values instead. Sysdig has now updated the default settings in order to use system-level resources for auto-configuration.

Users who want to fix the issue, but do not want to upgrade to the new Sysdig hotfix, need to update the JVM settings in either the config.yaml or the Replicated console, by adding the -XX:-UseContainerSupport flag.

Release 1472, December 13, 2018

Tuned the configuration of metrics rollups to handle high-scale environments

Release 1402 December 3, 2018

Sysdig Monitor

Global silence alerts for scheduled downtime

Administrators can now temporarily disable alert events to mute notifications during planned downtime or maintenance. The new feature also supports sending a downtime notification to selected channels. Access the new capability via Settings > Notification Channels. See Disable or Delete a Notification Channel.

Dashboard Templating

New dashboard templating enables users to create and configure a fixed dashboard that enables alternating between multiple scope variables. Users can assign custom names for labels and choose to set fixed or variable label selection values.

Integration with AWS IAM role to grant permissions

New support for Amazon Web Services IAM roles grants permissions via IAM to applications running on Amazon.

See the Integrate AWS Account Using the Implicit Key (On-Prem Only)in the AWS integration documentation.

Updated Users and Teams Settings Pages

The Users and Teams settings pages have been updated to improve performance and now feature a streamlined full-page edit layout. See Manage Teams and Roles.

Sysdig Secure

CIS Compliance Checks

The ability to schedule CIS compliance tasks for the agent to run on your infrastructure is now available.

These tasks will generate metrics that are available in Sysdig Monitor and reports that are available in Sysdig Secure.

Bug Fixes

Several minor enhancements to improve performance and usability.

Release 1245 November 05, 2018

Please skip this release and install 1402 instead.

Enhanced connection tracking features

Security updates

  • Backend updates to address security vulnerabilities.

  • Teams functionality is now available in Sysdig Secure.

  • Caching on image scanning run-time page for performance improvements.

Various bug fixes and improvements

Release 1149 September 14, 2018

Prerequisites

Your on-premises Sysdig installation MUST be running release v1091 before you can upgrade to this release v1149. Please upgrade to v1091 before proceeding.

Unified Events table and migration tool (Required before upgrade)

A change was introduced in how events are indexed and stored in the Sysdig platform. In prior versions, the three types of events were stored in three separate indexes based on their different sources. After migration and upgrade are complete, they will be combined in one index. Before upgrading to v1149 it is necessary to run a Unified Events migration tool.

Sysdig Agent Crash custom event

Generates a custom event if a Sysdig agent crash is experienced.

Node Ready alert reset

Enables transition of a notification from active => ok for a down node (NodeNotReady) when the node with the same scope becomes ready again (NodeReady).

Improved Mesos/Marathon label handling

Improved handling of Mesos/Marathon labeling to ensure proper display of containers within the Sysdig UI.

Various bug fixes and improvements.

Release 1091 August 16, 2018

Component updates and CVE patches

Delivers minor-minor upgrades and CVE patches for all 3rd party components in Replicated install. The Kubernetes install includes a major upgrade for MySQL from 5.6.34 to 8.0.11. Please see product README for upgrade guidance and details.

StatefulSets for Kubernetes deployment

Provides StatefulSet option for select Redis and MySQL with Kubernetes. Please see product README for usage eligibility and further details.

New ‘Standard User’ role and RBAC changes

Introduces new ‘Standard User’ role for developers that includes edit access to dashboards, alerts, events but NO access to Explore. Renames ‘Edit user’ role to ‘Advanced user’ and ‘Read only’ role to ‘View only’. See Manage Teams and Roles for details.

Team scoping performance improvement

When creating or editing teams, the first 30 labels and tags are displayed with the ability to search for additional options.

Multi-select alerts and bulk actions

New checkboxes on the alerts page enable selection of multiple alerts for bulk actions.

Kubernetes Node Ready alert

A new alert provides notification when a Kubernetes node is not ready. Default alert level is ‘warning’ (user-configurable).

Release 987 July 11, 2018

Solr dashboards update

Modifications to default Solr dashboard

Metrics aggregation fix

Fixed an issue with metrics aggregation

Release 963 June 26, 2018

LDAP enhancements

  • Enabling and disabling of LDAP authentication is now performed via API configuration rather than Replicated console or K8S ConfigMap. See LDAP for details.

  • An option has been added to allow chasing of referrals during LDAP authentication. See the documentation for details.

HTTPS enforcement

Sysdig is now enforcing HTTPS connectivity and using secure cookies. With this change, we have disabled TLS v1.0. Users should modify any scripts and/or applications to use HTTPS and TLS v1.2 for uninterrupted operation.

Text Panels

You can now add text panels to your dashboards to provide additional information. Text panels can be used as title headers or to provide additional context that you would like to communicate. Features limited markdown support .

Multiple segments for a single metric

You can now add up to five different segments for a given metric in time-series and stacked area panels.

Default entry point

Admins can now set a default entry point for a team to simplify the onboarding process. This determines the first page users see when they start the application (e.g., a specific dashboard, settings, etc.).

Default Istio dashboards

Sysdig provides out of the box dashboards for monitoring Istio using Prometheus exporters.

Test notification channels

New test function lets you pre-test your notification channels such as email, Slack, PagerDuty, etc.

Copy and share groupings

Copy and share unique groupings with all of your teams.

Icon labels

New icon labels appear on hover to clarify underlying function for users.

Alert on rate of change

Introducing a new ‘rate of change’ math function for metrics. Now you can alert by the rate at which a metric changes vs. a static threshold. For example, a default alert: Rate of change of disk usage alerts you if your disk usage increases more than x% in a day.

Release 925 June 10, 2018

Solr dashboards improvement

Increased number of segments for Solr default dashboard panels

Public dashboards fix

Fixed an issue that caused errors when loading public dashboards due to missing metrics

Release 917 June 7, 2018

Google OAuth fix

Fixed an issue with Google OAuth (On-Prem) login.

Upgrades in LDAP environments

Fixed an issue in upgrades with LDAP Authentication Configuration (for Platform v.1149 - 1511).

Release 914 June 6, 2018

Solr dashboards

Added application dashboards for Solr metrics.

Release 904 May 31, 2018

Performance improvements

Enhancements to improve Sysdig Monitor response time during login.

Release 893 May 9, 2018

Daily metric rollup fix

Fixed an issue caused during daily metric rollup due to Cassandra-14092.

Release 892 May 2, 2018

Various bug fixes and improvements.

Release 890 April 30, 2018

New default ports for API/Collector containers (Replicated)

New default TCP ports are exposed from Sysdig backend API/collector containers to the host level in Replicated-based installs. Read this support article for info on avoiding possible port conflicts.

‘SSO CA certificate in PEM format’ option

Replicated-based installs using SSO that access their IDP via SSL/TLS and need to import a CA certificate for Sysdig to trust the connection can now do using the SSO CA certificate in PEM formatoption. This is available under the ‘Advanced’ section of the ‘Settings’ tab in the Admin console. Kubernetes-based installs can do the equivalent as described in this README.

LDAP settings changes

LDAP authentication settings are now configured via the Sysdig Platform Admin API. Environments running releases pre-890 will have their LDAP settings automatically migrated to the new API endpoints automatically when upgrading to 890.

New UI design

Our new user interface provides a more modern framework for interacting with the product. Navigation is re-oriented from a top-of-screen menu to an icon-driven left side panel, providing more space for viewing your metrics and dashboards. Click here for a quick video introduction!.

Alert on rate of change

Introducing a new ‘rate of change’ math function for metrics. Now you can alert by the rate at which a metric changes vs. a static threshold. For example, a default alert: Rate of change of disk usage alerts you if your disk usage increases more than x% in a day.

Support for Prometheus histogram metrics

Sysdig Monitor can now ingest a Prometheus histogram metric type and visualize them in a chart to show the distribution of specific metrics.

Did you know you can add Sysdig as a Grafana data source? To help you get started visualizing Sysdig-collected metrics in Grafana, we’ve added a Grafana Plugin link to the help menu that takes you to the setup instructions.

Revised alerting with Kubernetes metrics

Alert configuration settings for Kubernetes metrics now limit scope and segmentation based on the metric that is selected to allow for more accurate alerting. Check out our support page for more details.

Compare-to for timeseries

In your time series line charts you can now compare time-shifting metrics to easily spot trends and anomalies. With compare-to for time series you can configure and observe how one or more metrics have changed since a previous time (e.g., 1 hour ago or 2 days ago).

‘Compare to’ for number panels

Metric number panels now feature a configurable ‘Compare to’ function to display the change in measurement since a previous time frame. Provides insight into the increase or decrease of metrics over time.

New Metrics for CPU Core Usage

We’ve added cpu.cores.used and cpu.cores.used.percent that align with the way Kubernetes exposes CPU usage. Now you can compare values using kube-state-metrics such as kubernetes.node.capacity.cpuCores, kubernetes.pod.resourceLimits.cpuCores in order to determine if resources are oversubscribed. These metrics are also key for capacity planning and chargeback calculations.

Improved documentation for CPU metrics

The Sysdig Monitor Metrics Dictionary now features updated CPU metrics descriptions to provide more insight into each available metric.

Resizable columns

The UI now allows columns to be resized for all tables in the application including alerts, events, teams, and users.

Suggest Mode

Suggest mode auto-selects only the relevant dashboards and metrics, hiding any inapplicable views. This is now the normal mode of operation. The turn on/off option is no longer available.

Redesigned login screen

We’ve put a new, more modern face on the Sysdig Monitor login screen.

Release 858 April 12, 2018

Captures and Sysdig Inspect fix

Upgrades the open source sysdig version in on-prem build to resolve sysdig capture and Sysdig Inspect compatibility issue.

Customers running version 693 and above can upgrade directly to release 858.

Release 800 March 13, 2018

New Explore design

We’ve redesigned Sysdig Monitor’s Explore page to give you extra screen space to view your killer dashboards and metrics. The new vertical layout helps you see more and get to what you need faster.

Golden Signals dashboards

New Service Golden Signal dashboards provide out-of-the-box metrics that developers need when launching and monitoring a service or app. Includes slowest transactions, latency, request volume, error rates, and most requested URLs.

Spotlight

Want a simple way to quickly see what matters most in your environment? Spotlight helps you quickly discover, detect, and optimize your infrastructure and services. A Spotlight health check shows you new integrations, infrastructure, app, and agent status, and more at-a-glance.

Export table data as JSON/CSV

You can now download table data in JSON or CSV format for offline viewing and analysis.

UI updates

We’ve simplified the dashboard panel copy function and added a duplicate panel option in menu. We’ve also redesigned the dropdowns in the top-right header including making it easier to quickly see and select your teams.

Additional items

Various bug fixes and improvements including:

  • Performance and stability fixes for metrics

  • Fix for issue with ElasticSearch migration

  • Configurable program retention by customer (default limit 12)

  • Fix for migrations using BE mapper – now use dedicated customer mapper.

Release 760 February 23, 2018

Explore grouping and scoping enhancements

We’ve massively simplified grouping and scopes. Our new approach gives you better, more precise data - with less chance of invalid groupings (e.g. Kubernetes deployment > hostname). Have questions? Watch this video, read this article, or contact Customer Success and we’ll analyze your account for you!

kube-state-metrics

Sysdig Monitor now collects kube-state-metrics for monitoring and alerting on the state of Kubernetes objects. New dashboards provide visibility of metrics for nodes, namespaces, services, daemonSets, jobs, replicaSets and pods. Requires update to the Sysdig agent version 0.77.0 or higher.

Public URL dashboards

Ever want to share a killer dashboard with a colleague who is not a Sysdig Monitor user? Now you can! Just pick, click, and send your URL.

Team Manager role

We’ve introduced a new ‘Team Manager’ role that provides the privilege to add, delete, and modify team users as well as grant read or edit access.

Proxy support for outgoing HTTP/HTTPS connections

You can now configure outgoing HTTP/HTTPS connections to be made via proxy. Supports outgoing web connections to support notification channels, PagerDuty, Slack, Amazon SNS, VictorOps, OpsGenie, WebHooks, AWS CloudWatch data gathering. Read more here.

Suggest mode enabled by default

Last year we introduced suggest mode – available in ‘Settings>Sysdig Labs’ – as a way to boost your efficiency by showing only the views, metrics, and grouping presets applicable to your environment. This option has proven so popular that it is now enabled by default.

Custom headers for webhooks

When using webhooks, typically used to pass authentication credentials, you can now add custom headers to pass along additional details with an outgoing request.

Rename of Admin team to Monitor Operations

As part of the broader Sysdig Platform initiative, ‘Admin Team’ within Sysdig Monitor is now renamed to ‘Monitor Operations.’ The Monitor Operations team will continue to behave the same as the previous Admin team:

  • The Monitor Operations team cannot be deleted.

  • Monitor Operations users have full visibility to all resources.

  • To change settings for any team, admins must switch to the Monitor Operations team.

Support for JMX metrics from Java 9

Sysdig Monitor now supports JMX monitoring for Java 9 applications. To enable collection of Java 9 metrics, update to the latest Sysdig Agent. For more details, review the Sysdig Agent changelogs.

Introducing read-only users

Users can have different roles for each of the teams they belong to, either ‘Read user’ or ‘Edit user’. A read user can only use the app in read-only mode, with no permission to create/edit/delete dashboards, alerts, etc while the edit user is allowed to make those changes. This is a per team role defined by Admin users.

Memcached default dashboard

A new default dashboard has been added to the Explore page where you can see the most important Memcached performance monitoring metrics: connections, commands, get hits/misses, evictions, etc.

Python client changes: Team/User configs

Changes to support Role Based Access Control (RBAC) modify how ‘Teams’ and ‘User’ configurations are stored and modified via the API. This affects the functionality of the Python client. If you currently have scripts that use these methods, click here for details on how to upgrade your Python client and make the necessary changes to your scripts.

Release 722 January 8, 2018

CPU usage host-level segmentation

CPU usage at host level can now be segmented by CPU core.

AWS and Cloudwatch improvements

Enabled more reliable AWS metadata by separating AWS metadata from Cloudwatch metrics

Additional items

Various bug fixes and improvements.

It is recommended to follow upgrade best practices

  • Keep upgrades current
  • Test upgrades in a non-mission-critical or staging environment before rolling into production.

5.1 - Sysdig On-Premises Release Support

Sysdig On-Premises releases are versioned and labeled as [Major Version].[Minor Version].[Patch Number] and a build number. On-Premises releases are categorized as:

  • Major
  • Minor
  • Patch

A major release is defined as having significant changes to the application, such as changes in architecture, addition of a component or service, features, or infrastructure components.

A minor release typically includes functionality or feature enhancements, new features, UX improvements, etc.

A patch release is created when an issue is identified, either in the field or internally, that requires an immediate fix. These typically include bugs, or patches for newly discovered vulnerabilities.

Supported Releases

Customers who run Sysdig on premise are encouraged to stay up to date with our latest releases. This ensures the most hardened code base, infrastructure components, vulnerability patches, and new features.

Sysdig provides support (in accordance with our on prem Support Services Policy) for the most recent major version (n) and one version prior (n -1). Once a release has become unsupported, Sysdig will continue to support that release for a period of three months. This allows our customers a window of planning for an upgrade to a supported version (n or n-1).

Additionally, Sysdig will work to patch the known and impacting vulnerabilities for the most recent version (n) as of the on-premise build date. This will include critical, high, medium, and low CVE severities.

For critical and high vulnerabilities that are not fixed with the latest build, Sysdig will provide a Security Advisory document that details the impact exposure. This can include false-positives or benign vulnerabilities. For vulnerabilities of grave consequence (such as Log4J), Sysdig may provide hot-fixes for the most recent version (n) and the version prior (n -1). All customers are encouraged to keep their environments to n and n-1 versions.

VersionSupported UntilEOS Date
5.x (Latest Version)7.x3 Months after 7.x release date
4.x6.x3 Months after 6.x release date
3.xObsolete
2.xObsolete

If you have questions regarding Sysdig’s product supportability policy, please Contact Support .

6 - Falco Rules Changelog

Falco rules are used in the Sysdig Secure Policy Editor.

Commit Date

Rule Notes

Version of the Falco Rules Installer (On-Prem)

September 23, 2022

Rule Changes

  • Increased IoCs and added additional exceptions.

  • Added exclusions to reduce false Positives.

  • Adding additional parameters to sensitive_kernel_parameter_files list.

0.87.0

September 08, 2022

Rule Changes

  • Added additional exceptions to aid in addressing false positives: Suspicious Kernel Parameter Modification.

  • Updated IoCs Ruleset with new findings.

Default Policy Changes

Removed the following rules from default policies:Scripting Language Execution below dev.

0.85.0

August 24, 2022

Rule Changes

  • New rules:Share RDS Snapshot with Foreign Account

  • Rule tuning for the following:

    • PTRACE anti-debug attempt

    • Suspicious Cron Modification

    • Suspicious Java Child Processes

    • Create Symlink Over Sensitive Files

    • Netcat Remote Code Execution in Container

    • eBPF Program Loaded into Kernel

  • Updated IoCs Ruleset with new findings.

0.83.0

August 19, 2022

Rule Changes

  • Fixed the output for two PTRACE rules.

  • Added additional conditions to improve detections for Delete/rename Bash History.

  • Enable the do_unexpected_udp_checkmacro.

  • Added the new rule: GCP Firewall Remote Access from Internet. It detects remote access ports allowed through the firewall from the public internet (0.0.0.0/0).

Auto-Tuner Exception Updates

  • Added additional exceptions for Privileged Shell Inside Container.

  • Added Azure core image to the exception, Suspicious Cron Modification.

0.82.0

Aug 11, 2022

Rule Changes

  • Added Azure rule: Azure RDP Access Is Allowed from The Internet

  • Updated auto-tuner exceptions to reduce excessive noise:

    • Change Resource Record Sets (AWS)

    • Create Hidden Files or Directories

    • Describe Instances (AWS)

    • GCP Delete Compute VM Instance

    • GCP Operation by a Non-corporate Account

    • List Buckets (AWS)

    • Non sudo setuid

    • Root User Executing AWS Command

    • Run shell untrusted

    • The docker client is executed in a container

    • User mgmt binaries

  • Updated IoCs Ruleset with new findings.

  • Default Policy Changes

    Added new rules: Azure RDP Access Is Allowed from The Internet

0.81.2

Aug 05, 2022

Rule Changes

  • Added additional exceptions to older agent versions to aid in addressing false positives:

    • Linux Kernel Module Injection Detected

    • eBPF Program Loaded into Kernel

    • Privileged Shell Spawned Inside Container

  • Added the following new rules:

    • GPG Key Reconnaissance

    • Create Access Key for User

  • Extended the condition of the following rules:

    • Base64-encoded Python Script Execution

    • nsenter Container Escape

  • Updated IoCs Ruleset with new findings.

  • Default Policy Changes

    Added new rules to default policies.

    • nsenter Container Escape

    • GPG Key Reconnaissance

    • Create Access Key for User

0.80.1

July 26, 2022

Rule Changes

  • Added additional exceptions to older agent versions to aid in addressing false positives:

    • Non sudo setuid

    • Set Setuid or Setgid bit

    • eBPF Program Loaded into Kernel

  • Added the following new rules:

    • PTRACE anti-debug attempt

    • PTRACE attached to process

    • Detect reconnaissance scripts

    • Detect malicious cmdlines

    • GCP Create DNS Record

    • GCP Create DNS Zone

    • GCP Delete DNS Record

    • GCP Update DNS Record

    • GCP Update DNS Zone

    • GCP Cloud Armor Blocked Connection

    • GCP Cloud IDS Alert

    • Delete AWS user (SSO)

  • Updated the following rule: Reconnaissance attempt to find SUID binaries

  • Updated the following lists: falco_privileged_images

  • Updated IoCs Ruleset with new findings.

  • Default Policy Changes

    Added new rules to default policies.

0.79.2

July 08, 2022

Rule Changes

  • Restored the following missing rule: nsenter Container Escape

  • Cleaned up the following duplicate macro: falco_sensitive_mount_containers

  • Adjusted the following eBPF rule: eBPF Program Loaded into Kernel

  • Updated IoCs Ruleset with new findings.

  • Updated all the Cloudtrail rules to add ARNs to output.

Default Policy Changes

Modified to work with both old default_policies and managed default_policies.

0.77.0

July 01, 2022

Rule Changes

Added Miner IP Pool Threat Intelligence: Detect outbound connections to common miner pool ports

0.76.1

June 30, 2022

Rule Changes

  • Added additional exceptions : Linux Kernel Module Injection Detected

  • Created the following new rules:

    • GCP App Engine Firewall Rule Deleted

    • GCP App Engine Firewall Rule Updated

    • GCP Create Cloud Function v2 Not Using Latest Runtime

    • GCP Create Cloud Function v2

    • GCP Compute Firewall Rule Deleted

    • GCP Compute Firewall Rule Updated

    • GCP Delete Compute VM Instance

    • GCP Update Cloud Function v2

    • Malicious Environment Variable in Spawned Process

    • nsenter Container Escape

  • Updated the following GCP rules:

    • GCP Create Cloud Function Not Using Latest Runtime

    • GCP Create Cloud Function

    • GCP Create DLP Job

    • GCP Delete DLP Job

    • GCP Paused DLP Job

    • GCP Suspicious IP Inbound Request

    • GCP Update Cloud Function

    • GCP Updated DLP Job

  • Added CIS tag to rules related to CIS Docker Security Benchmark controls:

    • Container Run as Root User

    • Disallowed SSH Connection

    • Launch Privileged Container

    • Launch Root User Container

    • Launch Sensitive Mount Container

    • Mount Launched in Privileged Container

    • Privileged Shell Spawned Inside Container

    • Reconnaissance attempt to find SUID binaries

    • The docker client is executed in a container

    • Write below root

  • Updated IoCs Ruleset with new findings.

Default Policy Changes

Added the following rules to the default policy:

  • GCP App Engine Firewall Rule Deleted

  • GCP Compute Firewall Rule Deleted

  • Malicious Environment Variable in Spawned Process

  • nsenter Container Escape

0.76.0

June 24, 2022

Rule Changes

  • Added additional exceptions to older agent versions to aid in addressing false positives:

    • Create Symlink Over Sensitive Files

    • Execution of binary using ld-linux

    • Run shell untrusted

  • Modified the following macros:

    • truncate_shell_history

    • modify_shell_history

  • Extended the condition of the rule, Detect crypto miners using the Stratum protocol , to improve detection capabilites

  • New rules created:

    • Launch malicious container image

    • GCP Suspicious IP Inbound Request

    • GCP Allow Public Access to Bucket

    • GCP KMS Schedule Key Deletion

    • GCP Create DLP Job

    • GCP Delete DLP Job

    • GCP Update DLP Job

    • GCP Paused DLP Job

  • Updated IoCs Ruleset with new findings.

Default Policy Changes

  • Added the following rule to the default policy, IoCs Malware Activity: Launch malicious container image

  • Added the following rules to the default policy, Sysdig GCP Best Practices:

    • GCP Suspicious IP Inbound Request

    • GCP Allow Public Access to Bucket

    • GCP KMS Schedule Key Deletion

    • GCP Delete DLP Job

    • GCP Paused DLP Job

0.75.0

June 17, 2022

Rule Changes

  • Added the following new rules:

    • AWS Suspicious IP Inbound Request

    • eBPF Program Loaded into Kernel

  • Modified the following rules:

    • Symlink over Sensitive Files

    • Container Drift rules (with new exceptions)

  • Updated the macro: sysdig_commercial_images. It now contains two new KSPM images.

  • Add the new macro ti_anon_ips  for Tor source IPs.

  • Updated IoCs Ruleset with new findings.

Default Policy Changes

  • Added the new rule,  AWS Suspicious IP Inbound Request to the Sysdig AWS Best Practices policy.

  • Added the new rule, eBPF Program Loaded into Kernel  to the Suspicious Container Activity policy.

0.74.3

June 03, 2022

Rule Changes

  • Added a new rule: Suspicious Java Child Processes

  • Updated the package_mgmt_procs macro to detect package management process with python

  • Updated some exceptions in the rule,Change thread namespace

  • Updated IoCs Ruleset with new findings.

Default Policy Changes

Added the new rule, Suspicious Java Child Processes,to the IoCs Malware Activity

0.72.0

May 26, 2022

Rule Changes

  • Added the following new rules:

    • Reconnaissance attempt to find SUID binaries

    • Suspicious Home Directory Creation

  • Modified exceptions to reduce noise:

    • Change thread namespace

    • Contact cloud metadata service from container

    • DB program spawned process

    • K8s ConfigMap Created

    • K8s ConfigMap Deleted

    • K8s Serviceaccount Created

    • Netcat Remote Code Execution in Container

    • Privileged Shell Spawned Inside Container

    • Set Setuid or Setgid bit

    • System ClusterRole Modified/Deleted

    • Write below monitored dir

    • Write below root

  • Updated IoCs Ruleset with new findings.

Default Policy Changes

  • Added the following new policies:

    • Reconnaissance attempt to find SUID binaries

    • Suspicious Home Directory Creation

0.70.3

May 20, 2022

Rule Changes

  • Added additional exceptions to the following rules to aid in addressing false positives:

    • Set Setuid or Setgid bit

    • Execution from /tmp

  • Fixed the condition of the following rules:

    • Execution from /tmp

    • Execution from /dev/shm

  • Updated IoCs Ruleset with new findings.

0.69.0

May 13, 2022

Rule Changes

  • Added additional exceptions to older agent versions to aid in addressing false positives:

    • Run shell untrusted

    • Launch Privileged Container

    • Container Run as Root User

    • Write below root

    • Write below rpm database

    • DB program spawned process

    • Privileged Shell Spawned Inside Container

    • Launch Suspicious Network Tool in Container

    • Remove Bulk Data from Disk

    • Set Setuid or Setgid bit

    • Packet socket created in container

    • Execution from /tmp

  • Created the new rule, Possible Backdoor using BPF. This rule triggers if process was seen attaching a BPF filter on a network socket, this could indicate packet sniffing for use in a backdoor such as BPFDoor. Network diagnostic tools may also trigger this rule.

  • Created the new rule, Execution of binary using ld-linux. This method can be used to execute programs that do not have the exec bit set and may possibly evade detection measures.

  • Fixed the condition of the following rules:

    • Write below binary dir

    • Set Setuid or Setgid bit

  • Updated IoCs Ruleset with new findings

Default Policy Changes

    • Added the new rule,  Possible Backdoor using BPF to the Notable Network Activity policy. Write below binary dir

    • Added the new rule,  Execution of binary using ld-linux to the IoCs Malware Activity policy.

0.68.1

May 6, 2022

Rule Changes

  • Added additional exceptions to older agent versions to aid in addressing false positives:

    • Modify binary dirs

    • Redirect STDOUT/STDIN to Network Connection in Container

    • Container Run as Root User

    • Execution from /tmp

  • Created the new rule Tampering with Security Software in Container. This rule detects common techniques by threat actors to disable runtime security software.

  • Created the new rule Detect outbound connections to TOR Entry Nodes. This rule detects when clients reach the TOR network through its entry nodes. NOTE: This is an EXPERIMENTAL rule and only contains a subset of TOR entry nodes. It will be improved upon in the future.

  • Fixed the condition of the following rule: Execution from /tmp

  • Updated IoCs Ruleset with new findings.

Default Policy Changes

  • Moved the rule Redirect STDOUT/STDIN to Network Connection in Container to the Notable Container Activity default policy

  • Added the new rule Tampering with Security Software in Container to the Suspicious Container Activity default policy

  • Added the new rule Detect outbound connections to TOR Entry Nodes to the IoCs Malware Activity default policy

0.67.1

April 28, 2022

Rule Changes

  • Added new rule file, threat_intelligence_feed.yaml , with lists and macros directly updated by Sysdig Threat Research Team.

  • Updated the following list: sysdig_commercial_images

  • Updated IoCs Ruleset with new findings.

  • Updated Falco rules conditions:

    • Execution from /tmp

    • Execution from /dev/shm

    • Network Connection outside Local Subnet

  • Added additional exceptions to aid in addressing false positives:

    • Execution from /tmp

    • Create Symlink Over Sensitive Files

    • Change thread namespace

    • DB program spawned process  

    • Suspicious Cron Modification

0.66.1

April 21, 2022

Rule Changes

  • Added a new AWS Cloudtrail rule: Create RDS DB Instance with Public Access

  • Added the following Falco rules:

    • Base64-encoded Shell Script Execution
    • Execution from /dev/shm
  • Added additional exceptions to aid in addressing false positives:

    • Service Account Created in Kube Namespace
    • K8s Serviceaccount Created
  • Modified to add a list of malicious IPs: Outbound Connection to C2 Servers

  • Updated IoCs Ruleset with new findings

Default Policy Changes

  • Added the following:

    • Base64-encoded Shell Script Execution
    • Execution from /dev/shm
  • Moved to enabled policy: Outbound Connection to C2 Servers

0. 65.1

April 18, 2022

Rule Changes

Added additional exceptions to the following rules to aid in addressing false positives:

  • Change thread namespace

  • Create Symlink Over Sensitive Files

  • Container Run as Root User

  • DB program spawned process

  • Privileged Shell Spawned Inside Container

  • Run shell untrusted

  • Set Setuid or Setgid bit

  • Write below etc

0.65.0

April 17, 2022

Rule Changes

Added additional exceptions to the following rules to aid in addressing false positives: Privileged Shell Spawned Inside Container

0.64.1

April 15, 2022

Rule Changes

  • Added additional exceptions to the following rules to aid in addressing false positives:

    • Packet socket created in container

    • Change thread namespace

    • Run shell untrusted

    • Container Run as Root User

  • Created the new rule Base64-encoded Python Script Execution. This rule detects base64-encoded Python scripts on command line arguments. Base64 can be used to encode binary data for transfer to ASCII-only command lines. Attackers can leverage this technique in various exploits to load shellcode and evade detection.

  • Fixed the output of the following rules:

    • K8s Serviceaccount Created

    • K8s Serviceaccount Deleted

    • Updated IoCs Ruleset with new findings

Rule Changes

  • Added the Base64-encoded Python Script Execution  rule to the IoCs Malware Activity default policy

  • Added the Launch Ingress Remote File Copy Tools in Container rule to the Notable Container Activity default policy

  • Created the new default policy,  Known Exploit Detection. This policy embedes the rules that can identify potential exploits of well-known CVEs.

0.64.0

April 12, 2022

Rule Changes

  • Added additional exceptions to the following rules to aid in addressing false positives:

    • Schedule Cron Jobs

    • Set Setuid or Setgid bit

    • Create Symlink Over Sensitive Files

  • Disable the Unprivileged Delegation of Page Faults Handling to a Userspace Process rule removing its condition.

0.63.0

April 09, 2022

Rule Changes

  • Updated the following rules:

    • Simple output changes to the Detect outbound connections to common miner pool portscode> rule.

    • Updated priority and included additional cron paths for the Create Symlink Over Sensitive Files rule.

    • Updated IoCs Ruleset with new findings

  • The following new rules have been introduced.

    • Privileged Shell Spawned Inside Container. This rule detects a root shell being opened by a compromised process for interaction by the attack

    • Debugfs Launched in Privileged Container. This rule detects file system debugger, debugfs, launched inside a privileged container which might lead to container escape.

    • Mount Launched in Privileged Container. This rule detects file system mount occurrence inside a privileged container which might lead to container escape.

    • Unprivileged Delegation of Page Faults Handling to a Userspace Process. This rule detects a successful unprivileged userfaultfd syscall which might act as an attack primitive to exploit other bugs

    • Launch Ingress Remote File Copy Tools in Container. This rule detects ingress remote file copy tools launched in a container. For example, curl and wget.

    • Suspicious Cron Modification. This rule detects direct writes to cron job files.

Default Policy Changes

  • Policy: Notable Filesystem Changes

    • added the Suspicious Cron Modification rule.

    Policy: Suspicious Container Activity

    • Added the Debugfs Launched in Privileged Container rule.

    • Added the Unprivileged Delegation of Page Faults Handling to a Userspace Process rule.

    Policy: Suspicious Lateral Movement Activity to Cloud

    • Added the Mount Launched in Privileged Container rule.

    Policy: Unexpected Spawned Processes

    • Added the Privileged Shell Spawned Inside Container rule.

0.62.1

April 06, 2022

Rule Changes

Reduce noise for the rulesWrite below monitored dir and write below etc by adding additional exceptions.

0.62.0

March 18, 2022

Rule Changes

  • Updated the Launch Root User Container condition rule.

  • Updated the following lists to address false positive :

    • miner_domains

    • allowed_k8s_users

  • Updated some exceptions in the  Schedule Cron Jobs rule.

  • Created the sssd_writing_krb  macro from the new release of OSS Falco.

  • Updated IoCs Ruleset with new findings.

  • Updated the following macros based on the changes in Falco OS:

    • modify_shell_history

    • truncate_shell_history

    • write_etc_common

Default Policy Changes

  • TheIoCs Malware Activity policy has been updated.

    • Malicious filenames writtenadded.

    • Malicious process detected removed.

  • Removed some rules from Notable Filesystem Changes policy:

    • Write below etc

    • Write below root

    • Write below rpm database

    • Write below binary dir

  • Removed one rule from the Notable Container Activity policy: Change thread namespace

0.59.2

March 10, 2022

Rule Changes

  • Exclude ptp and dp from the Change thread namespacerule.

  • Exclude self from the K8s Serviceaccount Created rule.

  • Exclude known cron writers from the Schedule Cron Jobs rule.

  • Updated the IoCs Ruleset with new findings.

0.58.1

March 06, 2022

Rule Changes

  • Add additional exceptions to aid in addressing false positive for rules:

    • Schedule Cron Jobs

    • Non sudo setuid

    • Launch Privileged Container

    • K8s Serviceaccount Created

  • Updated the following macros baed on the changes in Falco OS:aws_eks_core_images

  • Updated IoCs Ruleset with new findings.

0.57.2

March 03, 2022

Rule Changes

Fixed exception to aid in addressing false positives for rules: Contact K8S API Server From Container

0.56.5

March 01, 2022

Rule Changes

  • Update rule: DB program spawned process

  • Create macro:pgbackrest_info_childs

0.56.4

February 18, 2022

Rule Changes

  • Add additional exceptions to older agent versions to aid in addressing false positive for rules:

    • Modify Shell Configuration File

    • Modify Shell Configuration File

    • Write below etc

    • Write below rpm database

    • DB program spawned process

    • Clear Log Activities

    • Launch Root User Container

  • Updated the following macros based on the changes in Falco OS:

    • containerd_shell_modify

    • tanium_client_running_python

    • postgres_running_pgbackrest

    • proc_file_suffix

    • known_redirect_procs

  • Updated the following lists to address false positives:

    • known_setuid_binaries

    • known_k8s_api_programs

    • gke_trusted_images_launch_root_list

  • Updated IoCs Ruleset with new findings.

0.55.2

February 10, 2022

Rule Changes

  • Add additional exceptions to older agent versions to aid in addressing false positive for rules:

    • Change thread namespace

    • Write below rpm database

    • Write below root

    • Clear Log Activities

    • Launch Root User Container

  • Updated the following macros based on the changes in Falco OS:

    • parent_python_running_sdchecks

    • python_running_sdchecks

    • exe_sysdig

    • tanium_client_running_python

    • sysdig_dragent

    • trusted_logging_images

  • Updated the following lists based on the changes in Falco OS:

    • sysdig_commercial_images

    • allowed_dev_files

    • user_known_chmod_applications

    • miner_domains

  • Updated IoCs Ruleset with new findings.

0.54.3

February 07, 2022

Rule Changes

    Add additional exceptions to older agent versions to aid in addressing false positive for rules: Launch Root User Container

0.53.4

February 04, 2022

Rule Changes

  • Add additional exceptions to older agent versions to aid in addressing false positive for rules:

    • Modify Shell Configuration File

    • Write below etc

    • Write below root

    • Read sensitive file trusted after startup

    • Change thread namespace

    • Launch Suspicious Network Tool in Container

    • Redirect STDOUT/STDIN to Network Connection in Container

  • Updated the following macros based on the changes in Falco OS:

    • spawned_process

    • sensitive_mount

  • Updated the following lists based on the changes in Falco OS:

    • falco_hostnetwork_images

    • deb_binaries

    • known_sa_list

    • falco_sensitive_mount_images

  • Updated the following lists to address false positives:

    • db_server_binaries

    • user_known_chmod_applications

  • Updated IoCs Ruleset with new findings.

0.53.3

January 29, 2022

Rule Changes

  • Add additional exceptions to older agent versions to aid in addressing false positives for rules:Write below etc.

  • Updated IoCs Ruleset with new findings.

  • Add new rules:

    • Modify ld.so.preload

    • Polkit Local Privilege Escalation Vulnerability(CVE-2021-4034)

0.52.0

January 21, 2022

Rule Changes

Updated IoCs Ruleset with new findings.

0.51.1

January 14, 2022

Rule Changes

  • Created a new AWS Rule: Read Object through AWSSupportServiceRolePolicy Assumed Role.

  • Updated tags for AWS Rule:AWS Command Executed on Unused Region.

  • Updated tags for the following GCP Rules:

    • GCP Invitation Sent to Non-corporate Account

    • GCP Create User-managed Service Account Key

    • GCP Create GCP-managed Service Account Key

    • GCP Create Cloud Function Not Using Latest Runtime

    • GCP Set Bucket IAM Policy

    • GCP Create Bucket

0.50.5

December 16, 2021

Rule Changes

  • Add a new rule:Malicious C2 IPs or domains exploiting log4j: detect connections with malicious IPs involved in log4j exploitation.

  • Updated IoCs Ruleset with new findings

0.49.2

January 03, 2022

Rule Changes

  • Created a new AWS Rule: Read Object through AWSSupportServiceRolePolicy Assumed Role.

  • Updated tags for AWS Rule:AWS Command Executed on Unused Region.

  • Updated tags for the following GCP Rules:

    • GCP Invitation Sent to Non-corporate Account

    • GCP Create User-managed Service Account Key

    • GCP Create GCP-managed Service Account Key

    • GCP Create Cloud Function Not Using Latest Runtime

    • GCP Set Bucket IAM Policy

    • GCP Create Bucket

0.48.0

December 06, 2021

Rule Changes

  • Add a new rule:Find AWS Credentials: Find or grep AWS credentials in host or container.

  • Add additional exceptions formats to aid in addressing false positives for rules: K8s ConfigMap Deleted.

  • Updated IoCs Ruleset with new findings

0.46.2

November 30, 2021

Rule Changes

  • Add additional exceptions formats to aid in addressing false positives for rules: Create Sensitive Mount Pod.

  • Updated IoCs Ruleset with new findings

0.46.0

November 22, 2021

Rule Changes

  • Created a new GCP Rule: GCP Create Cloud Function

  • Create following Azure Rules:

    • Azure Remember MFA for User Access on Devices

    • Azure Users Can Consent to Apps Accessing Company Data on Their Behalf

    • Azure Deactivate MFA for User Access

    • Azure Container ACL Modified

  • Add additional exceptions formats to aid in addressing false positives for rules:

    • Modify Shell Configuration File

    • Launch Privileged Container

    • Container Run as Root Users

  • Updated IoCs Ruleset with new findings

  • Updated AWS, Azure,and GCP tags

0.45.1

November 16, 2021

Rule Changes

Updated IoCs Ruleset with new findings.

0.44.1

November 15, 2021

Rule Changes

  • Added new rule for AWS Cloudtrail: Create Lambda Function Using Unsupported Runtime

  • Modified rule for AWS Cloudtrail:Run Instances with Non-standard Imagenow checks the image ID from aws.ec2.imageID instead of getting this value from respondeElements/instanceSet/items using jevt

0.44.0

November 11, 2021

Rule Changes

Added new tags to the following rules:

  • GCP Delete Resources from the PCI Blueprint Environment

  • GCP Create KMS Key Without Rotation

  • GCP Remove KMS Key Rotation

  • GCP Delete DNS Zone

  • GCP Delete GKE Node Pool

  • GCP Delete Router

  • GCP Delete GKE IAM Role

  • GCP Delete VPC Network

  • GCP Delete GKE Subnetwork

0.43.2

November 5, 2021

Rule Changes

Added new tags to existent rules for MITRE and NIST categories.

0.43.1

October 29, 2021

Rule Changes

  • Added new tags to the following rules:

    • Modify RDS Snapshot Attribute

    • Modify Image Attribute

    • Modify Snapshot Attribute

    • Detect outbound connections to common miner pool ports

    • Detect crypto miners using the Stratum protocol

  • Updated Malware IoCs with the new findings.

0.42.0

October 20, 2021

Rule Changes

Add additional exceptions formats to aid in addressing false positives for rules:

  • Modify Shell Configuration File

  • Run shell untrusted

  • Launch Sensitive Mount Container

  • Outbound or Inbound Traffic not to Authorized Server Process and Port

  • Create Sensitive Mount Pod

  • Create NodePort Service

  • Attach/Exec Pod

  • Service Account Created in Kube Namespace

  • System ClusterRole Modified/Deleted

Default Policy Changes

Lowered Severity to INFO for the following policies:

  • All K8s User Modifications

  • All K8s Object Modifications

0.41.0

October 11, 2021

Rule Changes

  • Add additional exceptions formats to aid in addressing false positives for rules:

    • Modify binary dirs

    • Clear Log Activities

    • Remove Bulk Data from Disk

    • Create HostNetwork Pod

    • Launch Suspicious Network Tool in Container

  • Added three new Falco rules to detect Malware:

    • Malicious IPs or domains detected on command line

    • Malicious binary detected

    • Malicious process detected

Default Policy Changes

Added New Policy IoCs Malware Activity

0.40.0

October 07, 2021

Rule Changes

  • Changed inbound_outbound macro condition.

  • Add additional exceptions formats to aid in addressing false positives for rules:

    • Write below etc

    • Read sensitive file untrusted

    • Search Private Keys or Passwords

    • Disallowed K8s User

    • K8s Deployment Created

    • K8s Deployment Deleted

    • K8s Service Created

    • K8s Service Deleted

    • K8s ConfigMap Created

    • K8s ConfigMap Deleted

    • K8s Namespace Created

    • K8s Namespace Deleted

    • K8s Serviceaccount Created

    • K8s Serviceaccount Deleted

    • K8s Role/Clusterrole Created

    • K8s Role/Clusterrole Deleted

    • K8s Role/Clusterrolebinding Created

    • K8s Role/Clusterrolebinding Deleted

0.39.0

September 23, 2021

Rule Changes

Changed net_miner_pool macro used in the Detect outbound connections to common miner pool ports rule.

0.37.1

September 21, 2021

Rule Changes

Add additional exceptions formats to aid in addressing false positives for rules: Non sudo setuid rule.

0.37.0

August 26, 2021

Rule Changes

  • Added the following rules:

    • Console Login Through Assume Role

    • AWS Command Executed by Untrusted User

    • Console Login Success

    • Console Login Success From Untrusted IP

    • Delete AWS user

    • Remove AWS User from Group

    • Put Object in Watched Bucket

    • Read Object in Watched Bucket

  • Added new lists:

    • trusted_aws_users

    • watched_buckets

  • Updated rules:

    • Console Login Without MFA now does not fire on assumed role

    • Console Root Login Without MFA now does not fire on assumed role

    • Add AWS User to Group now outputs the user added to the group

0.36.0

POSTPONED August 20, 2021

POSTPONED Rule Changes

  • Added a new rule: Unprivileged Delegation of Page Faults Handling to a Userspace Process

  • Update the list:

    • sysdig_commercial_images

    • falco_hostnetwork_images

  • Updated the macro: interactive macro updated checking the tty state. The interactive session is with proc.tty != 0

POSTPONED 0.35.0

August 13, 2021

Rule Changes

Added additional exceptions formats to aid in addressing false positive for the rules:

  • Launch Package Management Process in Container

  • Terminal shell in container

  • The docker client is executed in a container

Updated the list: sysdig_commercial_images

Updated the macro: interactive macro updated checking the tty state. The interactive session is with proc.tty != 0

0.34.0

August 02, 2021

Rules Changes

Add additional exceptions formats to aid in addressing false positive for rules:

  • DB program spawned process Rule

  • Change thread namespace

  • The docker client is executed in a container

  • Launch Suspicious Network Tool in Container Rule

0.33.0

July 27, 2021

Default Policy Changes

Enable the Sysdig GCP Best Practices policy by default.

0.32.0

July 25, 2021

Rule Changes

  • GCP events were consumed directly from the protoPayload, which removed some fields that are used and are not part of the protoPayload itself. All the rules that use jevt.value are updated now to reference protoPayload in the root path. It is a breaking change for GCP rules, and you are required to use cloud-connector versions above v0.8.0.

  • Updated GCP rules to use protoPayload JSON path. Affected rules:

    • GCP Create API Keys for a Project

    • GCP Delete Bucket

    • GCP Create Bucket

    • GCP List Buckets

    • GCP List Bucket Objects

    • GCP Put Bucket ACL

    • GCP Set Bucket IAM Policy

    • GCP Update Bucket

    • GCP Create Cloud Function Not Using Latest Runtime

    • GCP Create Cloud Function

    • CloudRun Create Service

    • CloudRun Replace Service

    • GCP Create a Default VPC Network

    • GCP Disable Subnet Flow Logs

    • GCP Enable Connecting to Serial Ports for a VM Instance

    • GCP Creation of a VM Instance with IP Forwarding Enabled

    • GCP Suspected Disable of OS Login in a VM Instance

    • GCP Enable Project-wide SSH keys for a VM Instance

    • GCP Shield Disabled for a VM Instance

    • GCP Create or Patch DNS Zone without DNSSEC

    • GCP Describe Instance

    • GCP Command Executed on Unused Region

    • GCP Create GCP-managed Service Account Key

    • GCP Create User-managed Service Account Key

    • GCP Invitation Sent to Non-corporate Account

    • GCP Operation by a Non-corporate Account

    • GCP Super Admin Executing Command

    • GCP Update, Disable or Delete Sink

    • GCP Monitoring Alert Deleted

    • GCP Monitoring Alert Updated

    • GCP Disable Automatic Backups for a Cloud SQL Instance

    • GCP Disable the Requirement for All Incoming Connections to Use SSL for a Cloud SQL Instance

  • Added a new rule: GCP Set a Public IP for a Cloud SQL Instance

0.31.0

July 22, 2021

No rule changes. No default policy changes.

Fix a defect related to installing rules for older backend versions (Sysdig 4.0.*).

0.30.0

July 20, 2021

Default Policy Changes

  • Sysdig AWS Best Practices severity is now set to 'medium'

  • Sysdig GCP Best Practices severity is now set to 'medium'

0.29.0

July 19, 2021

Rule Changes

Add additional exceptions formats to aid in addressing false positive for rules:

  • DB program spawned process Rule

  • Change thread namespace

  • The docker client is executed in a container

0.28.0

July 16, 2021

Default Policy Changes

Disabled Access Cryptomining Network Policy by default

0.27.0

July 15, 2021

Rule Changes

Add additional exceptions formats to aid in addressing false positive for rules:

  • Run shell untrusted

  • DB program spawned process

  • Change thread namespace

0.26.0

July 11, 2021

Default Policy Changes

Rule changes have been applied in the following default policies:

  • Suspicious Package Management Changes

  • Notable Filesystem Changes

  • Suspicious Filesystem Reads Policy

  • Suspicious Filesystem Changes

  • User Management Changes

  • Disallowed Network Activity

  • Inadvised Container Activity

  • Disallowed Container Activity

  • Suspicious Container Activity

New default policies created:

  • Suspicious Lateral Movement Activity to Cloud

  • Notable Network Activity

Default policies removed:

  • Suspicious Package Management Changes

  • Suspicious Filesystem Reads Policy

  • User Management Changes

  • Disallowed Network Activity

  • Disallowed Container Activity

  • Inadvised Container Activity

Existent policies status changes:

Access AcceCryptomining Network enabled by Default

0.25.0

July 01, 2021

Rule Changes

Add additional exceptions formats to aid in addressing false proofs for rules:

  • Netcat Remote Code Execution in Container

  • Launch Sensitive Mount Container

  • Redirect STDOUT/STDIN to Network Connection in Container

0.24.0

June 25, 2021

Rule Changes

Add additional exceptions formats to aid in addressing false proofs for rules:

  • Write below root

  • Change thread namespace

0.23.0

June 22, 2021

Rule Changes

Add additional exceptions formats for rules:

  • Change thread namespace

  • Create Privileged Pod

  • Modify Shell Configuration File

  • Write below binary dir

  • Launch Privileged Container

  • The docker client is executed in a container

  • ClusterRole With Wildcard Created

  • Create HostNetwork Pod

  • Service Account Created in Kube Namespace

  • K8s Role/Clusterrole Created

  • K8s Role/Clusterrole Deleted

  • K8s Role/Clusterrolebinding Created

  • Netcat Remote Code Execution in Container

  • Delete Bash History

  • ClusterRole With Write Privileges Created

  • Clear Log Activities

  • Modify binary dirs

  • Unexpected outbound connection destination

  • Unexpected UDP Traffic

0.22.0

June 19, 2021

A new policy, Sysdig GCP Best Practices, has been added.

Rule Changes

New GCP Rules have been added for AuditLog:

  • GCP Create API Keys for a Project

  • GCP Create Bucket

  • GCP Delete Bucket

  • GCP List Buckets

  • GCP List Bucket Objects

  • GCP Put Bucket ACL

  • GCP Set Bucket IAM Policy

  • GCP Update Bucket

  • GCP Create Cloud Function Not Using Latest Runtime

  • GCP Create Cloud Function

  • GCP Update Cloud Function

  • CloudRun Create Service

  • CloudRun Replace Service

  • GCP Create a Default VPC Network

  • GCP Disable Subnet Flow Logs

  • GCP Enable Connecting to Serial Ports for a VM Instance

  • GCP Creation of a VM Instance with IP Forwarding Enabled

  • GCP Suspected Disable of OS Login in a VM Instance

  • GCP Enable Project-wide SSH keys for a VM InstanceGCP Shield Disabled for a VM Instance

  • GCP Create or Patch DNS Zone without DNSSEC

  • GCP Describe Instance

  • GCP Command Executed on Unused Region

  • GCP Create GCP-managed Service Account Key

  • GCP Create User-managed Service Account Key

  • GCP Invitation Sent to Non-corporate Account

  • GCP Operation by a Non-corporate Account

  • GCP Super Admin Executing Command

  • GCP Update, Disable or Delete SinkGCP Monitoring Alert Deleted

  • GCP Monitoring Alert Updated

  • GCP Disable Automatic Backups for a Cloud SQL Instance

  • GCP Disable the Requirement for All Incoming Connections to Use SSL for a Cloud SQL Instance

0.21.0

June 17, 2021

Fixed a defect in v0.20.3. The fix is for the detection of older backend versions when looking for accounts scheduled for deletion.

0.20.4

June 17, 2021

Skip accounts scheduled for deletion when verifying Falco rules compatibility.

0.20.3

June 16, 2021

Rule Changes

Add additional exceptions formats to allow addressing false positives for rules:

  • Launch Package Management Process in Container

  • Set Setuid or Setgid bit

  • Terminal shell in container

0.20.2

June 11, 2021

Rules Changes

Add additional exceptions formats to help address false positives for rules:

  • Run shell untrusted

  • Set Setuid or Setgid bit

0.20.1

June 03, 2021

Rule Changes

  • The Non sudo setuid rule: Add macmnsvc (mcafee service host) to set of programs that are allowed to setuid.

  • The Launch Suspicious Network Tool in Container rule: Add another zookeeper image pattern that's allowed to run network tools.

  • The Clear Log Activities rule: Add another fluentd image as allowed to clear log files.

  • Add additional exceptions formats to aid in addressing false positives for rules:

    • System procs network activity

    • K8s Serviceaccount Created

    • K8s Serviceaccount Deleted

    • K8s Role/Clusterrole Created

    • K8s Role/Clusterrole Deleted

0.20.0

June 01, 2021

Rule Changes

  • The Read Sensitive File Untrusted rule:

    • Allow clamscan to read sensitive files

    • Allow db2ckpw (IBM DB2 Credential Checker) to read sensitive files

  • The Launch Suspicious Network Tool in Container rule: Add another zookeeper image that is allowed to run nc inside a container.

  • Add additional exception patterns for the following rules:

    • Launch Package Management Process in Container

    • K8s Serviceaccount Created

    • K8s Serviceaccount Deleted

    • K8s Role/Clusterrole Created

    • K8s Role/Clusterrole Deleted

0.19.0

May 26, 2021

Rule Changes

  • Add additional Qualys binaries as exceptions for rules:

    • Read sensitive file untrusted

    • User mgmt binaries

    • Write below etc

  • The Write below etc rule:

    • Allow newrelic to write below /root/newrelic instead of specific files

    • Allow nessuscli write state file

    • Allow masvc to write below /etc/ma.d/

    • Allow grafana to write state

  • The Write below root rule : Add an additional cmdline writing to exec.fifo.

  • The DB program spawned process rule: Allow sqlplus spawn oracle.

  • Add additional sets of exception fields for rules:

    • Write below monitored dir

    • The docker client is executed in a container

0.18.0

May 25, 2021

The Sysdig AWS Best Practices policy no longer includes the Logged in without Using MFA rule.

Rule Changes

  • Add five new rules for AWS Cloudtrail events.

  • Disable the AWS Cloudtrail rule, Logged in without Using MFA.

  • The Read Sensitive File Untrusted rule: Let the TaniumEndpoint agent read additional sensitive files.

  • The Write below root rule, docker_writing_state macro: Allow for paths that simply specify a path below an implied / or /root of current working directory.

  • The DB program spawned process rule: Add additional allowed Postgres backup utilities.

  • The Write below root rule:

    • Use a more flexible string match against the /exec.fifo paths.

    • Allow newrelic CLI to write to CLI log file.

    • Allow the docker cleanup image utility to write state files below /.

  • The Write below rpm database rule: Allow tanium endpoint script to write to the rpm database.

  • The Contact K8S API Server From Container rule: Add another fluent-bit program that is allowed to contact the API Server.

0.17.0

May 20, 2021

Rule Changes

Added exception to the following to address false positives:

  • The Non sudo setuid rule: Let swiagent read setuid.

  • The Read sensitive file untrusted rule:

    • Let refresh-mcollec (tive-metadata), part of puppet, read sensitive files.

    • Let puppet directly read sensitive files.

    • Let Tanium endpoint read sensitive files.

    • Let ir_agent (rapid7 agent) read sensitive files.

  • The Write below root rule:

    • Add an additional command line pattern for Cassandra to allow writes to /root/.cassandra.

    • Add additional exec.fifo path below root for runc.

    • Let docker write to certain files below /. It is part of some docker-in-docker setups.

    • Let Tanium joval write to /root/.jOVAL/.

  • The Change thread namespace rule:

    • Add an additional weaveworks/kured process name.

    • Let avinetworks/se images run programs that can change thread namespaces.

  • The System procs network activity rule : Add an additional exception pattern.

  • The User mgmt binaries: Let refresh-mcollec (tive-metadata), part of puppet, run user management binaries.

  • The Contact K8S API Server From Container rule: Let fluent-bit images run programs to contact the API server.

  • The Launch Suspicious Network Tool in Container rule: Let certain Openshift images run dig to perform DNS lookups.

  • The Clear Log Activities rule: Let certain Workinggrafana-related images clear log files in the container.

0.16.0

May 19, 2021

Rule Changes

Additional exception fields are added to the following rules to aid in customization:

  • K8s Secret Created

  • K8s Secret Deleted

0.15.1

May 18, 2021

Rule Changes

  • The Detect outbound connections to common miner pool ports rule: Add additional known miner domains.

  • Add additional exception fields to the following rules to aid customization:

    • Modify Shell Configuration File

    • Write below monitored dir

    • Write below etc

    • Write below root

    • Write below rpm database

    • Launch Privileged Container

    • Launch Sensitive Mount Container

    • Terminal shell in container

    • System procs network activity

    • Launch Suspicious Network Tool in Container

    • Set Setuid or Setgid bit

    • Launch Remote File Copy Tools in Container

    • The docker client is executed in a container

    • Disallowed K8s User

    • Create Privileged Pod

    • Create Sensitive Mount Pod

    • Create HostNetwork Pod

    • Attach/Exec Pod

    • Pod Created in Kube Namespace

    • Service Account Created in Kube Namespace

    • ClusterRole With Wildcard Created

    • K8s Secret Created

    • K8s Secret Deleted

  • The Change thread namespace rule: Add an additional exception for the Sysdig agent.

  • The Pod created in the Kube Namespace rule: Allow users starting with "system:" to create pods in the kube-system/kube-public namespaces.

  • The Read sensitive file untrusted rule: Allow puppet to run scripts that might read sensitive files.

  • The Write below root rule: Add an additional way to detect Cassandra to allow writes to /root/.cassandra.

  • The Change thread namespace rule: Allow Weaveworks Kured (Kubernetes Reboot Daemon) to change thread namespaces.

0.15.0

May 17, 2021

Rule Changes

  • Add rpmdb_verify as an RPM Package Management program. This affects the following rules:

    • Update Package Repository

    • Write below binary dir

    • Write below monitored dir

    • Write below etc

    • Read sensitive file untrusted

    • Modify binary dirs

    • Mkdir binary dirs

    • Run shell untrusted

    • Package management process ran inside container

  • Write below etc: Add haproxy-ingress as a program that can write below /etc/haproxy.

  • Change thread namespace: Allow images ending with /ext-cilium-startup-script to change namespaces.

  • Launch Suspicious Network Tool in Container: Allow images ending with sysdig/cassandra and bitnami/zookeeper to run network tools inside containers.

  • Set setuid or setgid bit: Allow the images in the sysdig_commercial_images list to include applications with setuid/setgid binaries.

0.14.0

May 05, 2021

Rule Changes

Add a macro to allow backward compatibility for using older pre-exceptions rules content.

0.13.2

May 05, 2021

Rule Changes

Remove the aws_cloudtrail rule named Create Internet-facing AWS Public Facing Load Balancer without Required Tags from the previous release that uses features yet to be released.

0.13.1

May 04, 2021

Added the Launch Root User Container rule to the Notable Container Activity policy.

Rule Changes

  • All Rules with the source, aws_cloudtrail: Switch from using jevt.value[/path] to aws.xxx to extract information out of aws_cloudtrail events.

  • A new rule, Launch Root User Container , has been added. It matches when a container is started and is configured to run as root. This works for Docker and CRI-O container runtimes, but not for Openshift 4.x, which does not make the necessary information available.

  • Macro spawned_process: Consider only successful executables. For example, where the return value is 0. This affects the following rules:

    • Schedule Cron Jobs

    • DB program spawned process

    • Run shell untrusted

    • System user interactive

    • Terminal shell in container

    • Program run with disallowed http proxy env

    • User mgmt binaries

    • Launch Package Management Process in Container

    • Netcat Remote Code Execution in Container

    • Launch Suspicious Network Tool in Container

    • Launch Suspicious Network Tool on Host

    • Search Private Keys or Passwords

    • Remove Bulk Data from Disk

    • Delete Bash History

    • Launch Remote File Copy Tools in Container

    • Detect crypto miners using the Stratum protocol

    • The docker client is executed in a container

    • Linux Kernel Module Injection Detected

    • Container Run as Root User

      This could affect the following rules if they are triggered based on an exec() process rather than a container-started event.

    • Launch Privileged Container

    • Launch Sensitive Mount Container

    • Launch Disallowed Container

    • Launch Root User Container

0.13.0

April 09, 2021

Rule Changes

Restore several old macros and lists that are no longer used by any of the default rules, but might be used by some users' local rules.

0.12.2

April 05, 2021

Fixed a defect that could prevent deploying rules to several older Sysdig backend versions.

0.12.1

March 31, 2021

Rule Changes

Added new versions of falco_rules.yaml/k8s_audit_rules.yaml that uses exceptions instead of collections of macros and long condition strings. The rules coverage should be identical to older versions.

0.12.0

March 19, 2021

Fixed minor problems with the rules installation script.

0.11.1

March 11, 2021

Rule Changes

Added 164 rules that detect suspicious/anomalous/notable behavior from a stream of AWS CloudTrail events. This requires a Sysdig backend that supports policy types and running the Cloud Connector for Secure for cloud..

For a full list of rules for different AWS services, see CloudTrail Rules for Secure for Cloud.

Default Policy Changes

The new policy, Sysdig AWS Best Practices, includes 41 of the above rules that Sysdig recommends using for the AWS environments.

0.11.0

February 9, 2021

Rule Changes

  • rule Change thread namespace: Let cilium nsenter

  • rule Change thread namespace: Let dynatrace setns

  • rule Change thread namespace: Let sysdig agent setns (the process name was changed recently)

  • rule Clear Log Activities: Allow fluentd to write/access log files in a container

  • macro exe_running_docker_save: Added support for Crio setting up containers. This affects several rules including:

    • Modify Shell Configuration File

    • Update Package Repository

    • Write below binary dir

    • Write below monitored dir

    • Write below etc

    • Write below root

    • Write below rpm database

    • Modify binary dirs

    • mkdir binary dirs

    • Set Setuid or Setgid bit

    • Create Hidden Files or Directories

  • rule Launch Package Management Process in Container: Let sysdig node-image-analyzer run rpm

0.10.5

December 14, 2020

Rule Changes

  • Add a new rule, Container Run as Root User ,to the Inadvised Container Activity policy.

  • Add crio and multus to the user_known_change_thread_namespace_binaries list

0.10.4

December 1, 2020

Rule Changes

  • Ensure that falco_rules_local.yaml is evaluated against all the default files.

  • Ensure that the logs clearly show which files are being evaluated.

0.10.3

November 16, 2020

Rule Changes

  • Add the new rule, Linux Kernel Module Injection Detected,  to the  Notable Filesystem Changes policy.

  • Add the  multipath_writing_conf macro as an exception in the Write below etc rule.

  • Add the chage_list macro as exception in the User mgmt binaries rule

  • Update compliance tags.

0.10.2

October 14, 2020

Add CSRF token protection.

Rule Changes

Add a new rule, Outbound Connection to C2 Servers, to the Disallowed Network Activity policy.

0.10.1

September 30, 2020

Rule Changes

  • Write below root: Similar to the rules that rely on a process name for exceptions, events will not be triggered if the process name is missing. For example, "".

  • Delete or rename shell history. Ignore docker programs that would prevent modifying shell history, when the path is expressed within the container filesystem (/.bash_history) and host filesystem (/var/lib/docker/overlay/.../.bash_history).

  • All Rules: Changes to the tags to add NIST 800-53 and SOC2 tags:

    • Renamed previous NIST 800-190 tags to use the prefix NIST_800-190_.

    • Fixed rule names for some Kubernetes rules.

0.10.0

September 23, 2020

Rule Changes

  • Launch Sensitive Mount Container: Change image matching to correctly identify Sysdig images as compared to names starting with "sysdig..."

  • Detect shell history deletion: Ignore paths below /var/lib/docker. For example, the container filesystem overlay images that are removed when a container is removed.

  • The Packet socket created in container rule is now enabled by default.

0.9.1

September 10, 2020

Rule Changes

  • All Rules: Add user.loginuid as an output field. This uid is generally unchanging across sudo/su commands, and can more reliably identify users.

  • Launch Privileged Container: Add additional images that can run with privileged=true.

  • Launch Sensitive Mount Container: Fix a typo that allows docker.io/sysdig/agent-slim to perform sensitive mounts.

  • Read sensitive file untrusted: Allow linux-bench to read sensitive files containing user information.

  • Update Package Repository: Restrict checks to files below known package management directories.

  • Write below etc: Add exceptions related to calico within containers.

  • Write below root: Allow mysqlsh write to /root/.mysqlsh .

  • Read sensitive file untrusted: Allow google_oslogin_{control} read sensitive files.

  • Change thread namespace: Trigger only when the process name is known.

  • Create HostNetwork Pod: Allow several images related to GKE + default metrics/routing services run with hostnetwork=true.

  • Disallowed Kubernetes User: Add several known Kubernetes users to allowed list.

  • Pod Created in Kube Namespace: Allow several images related to GKE + default metrics/routing services run in kube-system/kube-public namespaces.

  • System ClusterRole Modified/Deleted: Allow modifications to the role system:managed-certificate-controller.

0.9.0

September 08, 2020

Added support for updating Falco rules across multiple accounts in an on-prem setup.

0.8.3

August 17, 2020

Rule Changes

  • Created a new rule, EphemeralContainers Created for the Suspicious K8s Activity policy.

  • Replace the endswith operator when checking with an image repository.

  • Whitelisted sysdig/agent and sysdig/agent-slim . They are not available with the open-source Falco Rules.

  • Whitelisted dockerd-current and docker-current in the exe_running_docker_save macro.

0.8.2

August 03, 2020

Rule Changes

Add the k8s_image_list  list to the  trusted_pod macro

0.8.1

July 27, 2020

Rule Changes

  • Move the Write below root rule from the Suspicious Filesystem Changes policy to the Notable Filesystem Changes policy

  • Delete the NIST 800-190 Application Container Security Guide policy

  • Delete the Payment Card Industry Data Security Standard (PCI DSS) policy

  • Add a new macro, user_read_sensitive_file_containers for the Read sensitive file untrusted rule

  • Add docker.io/falcosecurity/falco to the falco_privileged_images list

  • Add kubernetes-admin to the allowed_k8s_users list

0.8.0

July 20, 2020

Rule Changes

  • Disable Disallowed K8s Activity policy

  • Add placeholder macros for multiple rules

  • Fix the root_dir macro

  • Add snapd to the package_mgmt_binaries list

  • Add zmap to the network_tool_binaries list

  • Whitelist protokube, dockerd, tini, and aws in the change thread namespace rule

  • Add sysdig/agent-slim and sysdig/node-image-analyzer images to the user_trusted_containers macro

  • Add kube-apiserver-healthcheck to the allowed_k8s_users list

0.7.9

July 7, 2020

  • Remove unnecessary logging.

  • Add a new flag, --saas

0.7.8

July 1, 2020

Handle an improper error.

0.7.7

June 25, 2020

Disable rule Container Drift Detected (chmod) by default

0.7.6

June 23, 2020

Update rule Container Drift Detected (open+create) to avoid warning

0.7.5

June 22, 2020

Rule Changes

Added two new rules: Container Drift Detected (chmod) and Container Drift Detected (open+create) to policy Suspicious Container Activity

The Container Drift Detected (open+create)  rule is disabled until an agent is released that supports the new evt.is_open_exec filter.

Updated macros bin_dir_mkdir and bin_dir_rename using evt.arg.path instead of evt.arg

Added placeholder macro user_known_write_below_binary_dir_activities to rule Write below binary dir

Fixed rule Anonymous Request Allowed to update the auth decision with ka.auth.decision=allow instead of ka.auth.decision!=reject

0.7.4

May 28, 2020

Rule Changes

Write below etc: Added lvs as a logical volume writing program that can write below /etc/lvm.

Clear Log Activities: Allowed additional Fluentd images to write to log file directories.

Set Setuid or Setgid bit: Added macro user_known_set_setuid_or_setgid_bit_conditionsthat makes it easier to add locally provided exceptions.

Launch Remote File Copy Tools in Container: Fixed the use of the list remote_file_copy_binaries so the list items are included.

The docker client is executed in a container: Now allow hcp-tunnelfront to run kubectl in containers.

Disallowed K8s User: Added vertical pod autoscaler programs as known Kubernetes users.

0.7.3

May 5, 2020

Rule Changes

For a brief time, Falco rules/macros had fields with k8s.* in them. These fields do not work in Sysdig Secure, so the relevant macros have been rewritten to omit them:

  • calico_writing_state

  • user_known_metadata_access

  • k8s_containers

  • user_known_k8s_client_container

0.7.2

May 1, 2020

Rule Changes

  • Add new rule Redirect stdout/stdin to network connection in container to policy Suspicious Container Activity

  • Add new rules Network Connection outside Local Subnet and Outbound or Inbound Traffic not to Authorized Server Process and Port to policy Suspicious Network Activity

  • Add new rules K8s Secret Created and K8s Secret Deleted to policy All K8s Object Modifications

  • Add rules Untrusted Node Successfully Joined the Cluster and Untrusted Node Unsuccessfully Tried to Join the Cluster to policy Suspicious K8s Activit

  • Add rule Full K8s Administrative Access to policy Suspicious K8s User Activity

  • Add rule Ingress Object without TLS Certificate Created to policy Inadvised K8s Activity

  • Check dsc_host in macro ms_oms_writing_conf

  • Add macros mcafee_writing_cma_d and avinetworks_supervisor_writing_ssh as exceptions in rule Write below etc

  • Add macro runc_writing_exec_fifo as exception in rule Write below root

  • Use "pmatch" instead of "in" operator to check known files under root directory

  • Update rule Change thread namespace to check exit event only

  • Add macro known_system_procs_network_activity_binaries for rule System procs network activity

0.7.1

April 9, 2020

Rule Changes

  • Add PCI/NIST tags to the following rules:

    • Disallowed SSH Connection

    • Unexpected outbound connection destination

    • Unexpected inbound connection source

    • Write below binary dir

    • Write below monitored dir

    • Write below etc

    • Write below root

    • Read sensitive file untrusted

    • DB program spawned process

    • Modify binary dirs

    • Mkdir binary dirs

    • Change thread namespace

    • Launch Privileged Container

    • Launch Sensitive Mount Container

    • Launch Disallowed Container

    • Terminal shell in container

    • Unexpected UDP Traffic

    • Create files below dev

    • Contact K8S API Server From Container

    • Unexpected K8s NodePort Connection

    • Search Private Keys or Passwords

    • Clear Log Activities

    • Create Symlink Over Sensitive Files

    • Detect crypto miners using the Stratum protocol

  • Write below etc:

    • Add "dsc_host" as a MS OMS program

    • Let McAfee write to /etc/cma.d

    • Let AVI Networks supervisor write somessh cfg files

    • Allow writes to /etc/pki from OpenShift secrets dir

  • Write below root:

    • Let runc write to /exec.fifo

  • Change thread namespace

    • Only allow Kubernetes/Docker programs to use setns directly on the host

    • Let children of kubelet/hyperkube use setns

  • Run shell untrusted

    • Let Puma reactor spawn shells

  • Detect outbound connections to common miner pool ports

    • When attempting to resolve crypto mining hostnames, exclude hosts that resolve to localhost/rfc1918 ips

Default Policy Changes

  • Remove the default Policy Launch Privileged Container.

    The rule it used is also in the existing default policy Inadvised Container Activity, so there's no change in rule coverage.

  • New default policies Payment Card Industry Data Security Standard (PCI DSS) and NIST 800-190 Application Container Security Guide, which are disabled by default, contain rules specifically related to PCI and NIST standards.

0.7.0

Dec 9, 2019

Expand allowed_k8s_users list with default users created by Kops

Add macro calico_writing_envvars to whitelist of rule Write below etc

Update operators with intersect

Add calico/node in the falco_privlieged_image list

Add amazon/amazon-ecs-agent in falco_sensitive_mounts_image list

Add hyperkube to the whitelist of rule

Set Setuid or Setgit bit

Add docker-runc-cur to container_entrypoint macro

Add a rule to detect Kubernetes client tool in container

Add rules Contact cloud metadata service from container and Packet socket created in container to policy Suspicious Container Activity

Update macro exe_running_docker_save

Add exe_running_docker_save as exception to rules Modify Shell Configuration File, and Update Package Repository

Create macro automount_using_mtab and add it as exception to rule Write below etc

Update macro k8s_api_server with Kubernetes headless service name

Add placeholder macro user_known_package_manager_in_container to rule Launch Package Management Process in Container

Add kubelet to list user_known_chmod_applications

Create macro user_known_k8s_client_container and add it as exception to rule The docker client is executed in a container

Add more directories to Sensitive mounts rules

0.6.0

Oct 9, 2019

Add rule Delete or rename shell history (a better version of Delete Bash History) to policy Suspicious Filesystem Changes

Add rule Detect crypto miners using the Stratum protocol to policy Suspicious Container Activity

Add a new policy, Access Cryptomining Network ,with a new rule Detect outbound connections to common miner pool ports associated (disabled by default)

Add new macros chmod and modify_repositories

Enhance rules Update Package Repository, Set Setuid or Setgid bit, and Create Hidden Files or Directories

Add imagefluent/fluentd-kubernetes-daemonset to macro trusted_logging_images

0.5.0

Aug 21, 2019

Update rule Update Package Repository with modify action

Update rule Delete Bash History with more bash history files

Update rule Set Setuid or Setgid bit using system calls instead of process name

Update rule Create Hidden Files or Directories with modify action

0.4.9

Aug 1, 2019

Add /exec.fifo to known_root_files macro (GKE)

Add macro amazon_linux_running_python_yum as exception in rule Write below rpm database (Amazon Linux 2)

Add docker.io/google/cadvisor and docker.io/prom/node-exporter to list falco_sensitive_mount_images

0.4.8

July 23, 2019

Add image k8s.gcr.io/kube-proxy to list falco_privileged_images

Add runc to macro container_entrypoint

Add macro trusted_logging_images for rule Clear Log Activities

Add image docker.io/netdata/netdata to list falco_sensitive_mount_images

0.4.7

July 1, 2019

Add placeholder for user macro

Add rfc 1918 addresses

Add image prometheus-node-exporter to macro openshift_image

Add weaveworks_scope macro used by rule Change thread namespace

0.4.6

June 20, 2019

Add whitelist to rules Change thread namespace and Non sudo setuid

0.4.5

June 17, 2019

Add trusted_container macro back

0.4.4

June 13, 2019

Extend macro mkdir with syscall mkdirat

Add placeholder for whitelist in rule Clear Log Activities

Add docker.io/ to the trusted images list

Add container.id and image in the rule output, except those rules with "not container" in condition

0.4.3

June 6, 2019

Remove image check from rancher_write_conf macro

Remove healthcheck from rancher_writing_conf

Update nginx_writing_conf macro

0.3.7

June 5, 2019

Updated macro container_started

IBM Cloud Kubernetes Service is a hosted Kubernetes from IBM

Allow Ansible to run using Python 3

Fix egrep rule and ncat rule

Add Sematext Monitoring & Logging agents to trusted Kubernetes containers

0.3.6

May 30, 2019

Add rules: remote file copy in container, create symlink over sensitive files

In macro prometheus_conf_writing_conf, use startswith instead of =

0.3.5

Apr 18, 2019

Add MITRE tags to existing rules

Add new MITRE rules mainly for persistence category

0.3.4

7 - Enhanced Metric Store

Sysdig has launched our next generation metric store, introducing a number of new features, as well as changes to and removal of some features in Sysdig Monitor. This document covers the major enhancements and changes introduced by the metric store.

New Features and Enhancements

Prometheus-Compatible Naming Conventions for Metrics & Labels

In prior versions of Sysdig Monitor, metrics were inconsistent between PromQL and Form querying. This behavior has been changed. Metrics are now unified — all the metrics are now presented in a Prometheus compatible naming convention, as opposed to the previous statsd compatible naming convention. For example, underscore is used instead of dot notation as given below:

kubernetes.node.allocatable.cpuCores will be mapped to kube_node_status_allocatable_cpu_cores and kubernetes.namespace.name to kube_namespace_name.

Your existing dashboards, alerts and notifications will be automatically migrated to the new naming convention. Sysdig APIs support metrics and labels in both old and new naming conventions. Note that for the initial release, Labels will not be migrated to the new naming convention in the old explore, events, and team settings.

Notifications sent via alerts (webhooks, PagerDuty, etc) will use the new label/metric convention. If you are performing further processing to parse the metric or label names within these notification messages please update your scripts as appropriate.

If you have any concerns or questions regarding this mapping or you feel like you need more time to adjust you notification tools, please contact Sysdig Support.

For metrics mapping, see Metrics and Label Mapping.

Context-Specific Metrics

Metrics such as cpu.used.percent previously would either be showing values from a process, container, or host depending on your query segmentation or scope. This has been improved by creating new sets of context-specific metrics and resource specific semantics of Prometheus naming convention. For example:

Classic MetricsNew Metrics
cpu.used.percentsysdig_program_cpu_used_percent
sysdig_container_cpu_used_percent
sysdig_host_cpu_used_percent
uptimesysdig_program_up
sysdig_container_up
sysdig_host_up

Network metrics previously would either be showing values from a host, container, program, or connection depending on your query segmentation or scope. This has been improved by creating also a new sets of context-explicit metrics, in this case also per connection metrics:

Classic MetricsNew Metrics
net.bytes.insysdig_connection_net_in_bytes
sysdig_container_net_in_bytes
sysdig_host_net_in_bytes
sysdig_program_net_in_bytes

Your existing dashboards, alerts and notifications will be automatically migrated to the new naming convention. Sysdig APIs support metrics in both old and new naming conventions.

For the complete list of context-specific metrics, see Mapping Classic Metrics with Context-Specific PromQL Metrics.

Faster Query Performance

Queries now perform faster and handle larger volumes of data. You can expect queries executed in Sysdig Monitor to be noticeably faster.

Single Stat Panels Displays Latest Value

Number panels, tables, histograms, and toplist panels can now show the latest value for an entity. This can be done without having to aggregate multiple values over the time selection.

Overview Displays Latest Data

Overview pages now shows the latest data as opposed to an aggregated value for widgets over the time window selected. Time navigation has been removed to focus this view on the live (latest) status of your infrastructure.

Scope Variable in PromQL Dashboard

You can easily reference a dashboard scope in PromQL queries. To do so, use the reserved $__scope variable as shown below:

Under the hood $__scope will be substituted with the expression specified in the dashboard scope. This is achieved by leveraging Sysdig ServiceVision technology which allows for automatically enriching metrics with Kubernetes and application context. Learn more about ServiceVision.

Mixed-Metric Granularity

Sysdig Monitor can now display metrics scraped at different intervals, for example 10s and 1m, on the same graph.

Improved Granularity for PromQL panels

Granularity of graphs has been improved for promQL panels. For example, a 1 hour selection now shows metrics with 10 second intervals. In prior versions, 1-hour selection in Dashboards showed metrics in 1-minute interval.

Removed Re-Alignment

Previously, Sysdig Monitor would re-align time selections in graphs due to certain performance limitations. This time re-alignment has been removed to show more up-to-date metrics.

Troubleshooting Metrics

Troubleshooting metrics (program metrics, connection-level network metrics, and Kubernetes troubleshooting metrics) are being reported on a granular level at 10s and will be stored for 4 days. For the list of troubleshooting metrics and the labels that you can use to segment them, see Troubleshooting Metrics.

Discontinued Features

Discontinued Metrics and Labels

Below is the list of metrics and labels that are going to be discontinued. We made an effort to not deprecate any metrics or labels used in existing alerts, but in case you encounter any issues please contact us.

It is important to note that we have applied automatic mapping of all net.*.request.time.worst metrics to net.*.request.time, as max aggregation gives equivalent results and it was almost exclusively used in combination with these metrics.

Discontinued Metrics

The following metrics are no longer supported:

  • net.request.time.file
  • net.request.time.file.percent
  • net.request.time.local
  • net.request.time.local.percent
  • net.request.time.net
  • net.request.time.net.percent
  • net.request.time.nextTiers
  • net.request.time.nextTiers.percent
  • net.request.time.processing
  • net.request.time.processing.percent
  • net.request.time.worst.in
  • net.request.time.worst.out
  • net.incomplete.connection.count.total
  • net.http.request.time.worst
  • net.mongodb.request.time.worst
  • net.sql.request.time.worst
  • net.link.clientServer.bytes
  • net.link.delay.perRequest
  • net.link.serverClient.bytes

Discontinued Labels

The following labels are no longer supported:

  • net.connection.client
  • net.connection.client.pid
  • net.connection.direction
  • net.connection.endpoint.tcp
  • net.connection.udp.inverted
  • net.connection.errorCode
  • net.connection.l4proto
  • net.connection.server
  • net.connection.server.pid
  • net.connection.state
  • net.role
  • cloudProvider.resource.endPoint
  • host.container.mappings
  • host.ip.all
  • host.ip.private
  • host.ip.public
  • host.server.port
  • host.isClientServer
  • host.isInstrumented
  • host.isInternal
  • host.procList.main
  • proc.id
  • proc.name.client
  • proc.name.server
  • program.environment
  • program.usernames
  • mesos_cluster
  • mesos_node
  • mesos_pid

In addition to this, composite labels ending with the ‘.label’ string will no longer be supported. For example kubernetes.service.label will be deprecated, but kubernetes.service.label.* labels will continue to be supported.

Removed Featurees

Topology Maps

Topology Maps will be deprecated due to their incompatibility with the new data store and had limitations at scale for certain users.

Agent Percentiles

Agent derived percentiles will be deprecated. If you have been using these, your query will stop working and you will have to manually migrate your queries to leverage Prometheus histograms or PromQL functions such as histogram_quantile to achieve more precise results.

Change in Functionality

Usage of Labels in Table Panel

Querying labels as metrics is limited to Infrastructure labels. For example, you can use all the host level labels (for example, agent tags), aws tags (for example, region) and the Kubernetes labels (for example, workload) to build table panels.

Aggregated Data for Non Timecharts

Due to the underlying changes we made to our core metric ingestion engine, charts that are not Timecharts (e.g. Number panels) will sometimes not display aggregated data for the full requested time range. In this case, we will

  1. aggregate a portion of data. This will in all cases be no less than across the span of 2 weeks
  2. clearly define the time range for which we were able to aggregate in the warning message. It is important to note that this is a transient side effect, and will be less likely to happen over time.

Contact Us

If you have any questions or comments about these changes, feel free to Sysdig Support or contact your Sysdig representative.