Vulnerability Management

Sysdig Secure’s Vulnerabilities module pulls together the scan results from your runtime, pipeline, and container registries to provide highly accurate views of vulnerability risk, access to public exploits, and risk management. Vulnerabilities replace the Scanning module from earlier versions of Sysdig Secure.

This document applies only to the Vulnerability Management engine, released April 20, 2022. Make sure you are using the correct documentation: Which Scanning Engine to Use

Vulnerabilities Management in Sysdig Secure

Sysdig Secure’s Vulnerabilities module addresses the top requirements for effective vulnerability management:

  • Provides highly accurate views of vulnerability risk at scale
  • Rich details provide precision about vulnerability risk (ex. CVSS vector, score, fix age) and insights from multiple expert feeds (ex. VulnDB)
  • Access to public exploits allows you to verify security controls and patch efficiently
  • Prioritized risk data focused on the vulns that are tied to the packages loaded at runtime
  • Accepting risks when the vulnerability matching does not apply for your particular deployment, or you want to postpone remediation

At this time, the Vulnerability Management engine supports: CI/CD pipeline and runtime image scanning, policies, notifications, and reporting for runtime. Registry scanning does not yet support policies.

Understanding Vuln Management Stages

To create an effective vulnerability management plan, it’s important to recognize the various stages of the lifecycle that need to be addressed.

Key Concepts

  • During the build phase, software vulnerabilities may be introduced when container images are defined and assembled. Container images are, by definition, immutable. Altering the contents of an image will update the ImageID and, thus, will be considered a different image by Sysdig Secure.

  • Even though unique container images (ImageIDs) cannot be modified, Sysdig can identify new vulnerabilities in running containers (for example, in Kubernetes workloads) as security feeds are continuously updated. For instance, a container image with no known vulnerabilities during its build phase might be affected by a critical vulnerability discovered ten days after being deployed into the runtime. The image remains unchanged, but new security information related to the software it contains has been found.

  • Sysdig uses the same fundamental concepts to analyze the contents of an image (SBOM) and match vulnerabilities but treats images differently based on their location. Sysdig can analyze the vulnerabilities of images in a development pipeline, stored in a container image registry, or used as the template for a running container, known as runtime workloads.

Stages: Pipeline - Registry - Runtime

Pipeline

Any analysis conducted before the registry phase is considered a pipeline. A clear example is CI/CD builds (Jenkins, Github, etc.), but also the execution of the sysdig-cli-scanner binary performed on a developer laptop or using a custom scanning script.

  • Pipeline images do not have runtime context.
  • The scan happens outside of the execution nodes where the agent is installed:
    • CI/CD
    • External instrumentation
    • Custom scripts or image scanning plugins
  • Pipeline scans are one-off vulnerability reports; the information is a static snapshot with its corresponding execution date.
    • If you want to evaluate a newer version of the image or reevaluate the same image with newer feed information, the analysis needs to be triggered again.
  • Images analyzed using the sysdig-cli-scanner will show up in the Pipeline section of the vulnerability management interface.

Registry

Container Registry scanning allows you to integrate Sysdig with image registries from a range of vendors. Registry scanning provides an extra layer of defense between pipeline and runtime, where:

  • Software that sits in the registry before being deployed is checked for newly discovered vulnerabilities
  • Third-party software that may have been installed without going through pipeline scanning will be checked

Registry scans occur as scheduled through a cron job in the installation Helm chart once per week (Saturdays at 6:00 AM) by default.

  • Batch scanning is done asynchronously and separately from the development pipeline; regardless of the time it takes to scan the batch, the pipeline is unaffected.

See also: Install Registry scanner(s).

Runtime

Runtime workloads are executed using a container image. Accessing the Runtime section of the Vulnerabilities menu, you will be able to see those images and their vulnerability and policy evaluation.

  • Runtime workloads are located in an execution node and are being monitored by a Sysdig agent/node analyzer, for example a Kubernetes node that is instrumented using the Sysdig agent bundle.
  • Runtime workloads will offer a live, auto-refreshing state. This means:
    • Workloads that are no longer running will be removed from the runtime view
    • Vulnerabilities and policies evaluations will automatically refresh without any user interaction, offering always the most up-to-date information known.
      • At least once per day
  • Runtime workload have a runtime context associated with them, i.e. Kubernetes cluster and namespace.
  • Workloads analyzed during runtime will show up in the Runtime section of the vulnerability management interface.

See also: Runtime.

Asset Types: Workload, Host, and Container

The way things are scanned and how results are filtered relies on how Sysdig defines and uses the concepts of “image,” “workload,” “host,” and “container.”

Image

Image is not officially an asset type, but is the primary unit underlying all types of scanning.

  • In Pipeline and Registry scanning, Sysdig scans images used in container-based environments.

  • In Runtime, images have the context of either Kubernetes or the container host environment.

These images are strictly evaluated based on the image definition without reference to the deployed context (neither runtime nor orchestrator).

All images are extracted into a Software Bill of Materials (SBOM) format that is compatible with the CycloneDX standard. See CycloneDX for details.

Workload

When scanning images that are deployed in Kubernetes environments, the scanner evaluates the image in the context of the container running in a Kubernetes workload. This context makes it contextually different from images and containers.

Host

A host is a full-stack Operating System (OS) running on virtual infrastructure such as Cloud IaaS or Virtual Machines. The host will also have an SBOM, without the layers that are seen in images. All the context data for a host is relative to a full-stack OS with network connectivity (OS, Host Name, IPs, etc.).

Container

When Sysdig scans a container (either agentlessly or with the agent) it extracts a container from a Host image via the filesystem. In container scans, only the parent host context is available, not the runtime and orchestration context.

To be scanned by Sysdig, containers must all be compliant with the Open Container Initiative (OCI). See OCI for details.

Usage

When defining a Vulnerability policy or filtering Runtime vulnerability results, you can define the asset type to use.

Getting Started with Vulnerabilities

  1. For full vulnerability coverage, ensure you have completed the Sysdig Secure steps, so you have:

  2. Log in to Sysdig Secure with Advanced User+ permissions and select Vulnerabilities.

    The out-of-the-box policies for Pipeline and Runtime vulnerabilities will work without further setup.

  3. Choose Pipeline, Registry, or Runtime to see the scanning results.

  4. Choose Reporting to configure schedules for creating downloadable reports on runtime vulnerability results.

  5. To create or edit Pipeline or Runtime Vuln Policies and Rule Bundles, select the relevant links from the Policies tab in the navigation bar.

  6. To accept the risk of detected vulnerabilities, configure an acceptance based on scope, justification, and length of time. See Understanding and Usage steps.

Understanding Risk Acceptance for Vulnerabilities

As of November, 2022, users can choose to accept the risk of a detected vulnerability or asset. Accept Risk is available for both Runtime and Pipeline, and for specific CVEs or specified hosts or images.

Enablement Prerequisites

Accept Risk requires Sysdig Secure SaaS to be installed with:

  • sysdig-deploy Helm chart version 1.5.0+

    • vuln-runtime-scanner version 1.4.0+
  • sysdig-cli-scanner version 1.3.0+

Because Accept Risk is applied to both pipeline and runtime vuln results impartially, the required versions of both components are required.

If the minimum enablement requirements are not met, the Accept Risk button and panel will show in your interface, but will not activate. The created Acceptance will appear in Pending status for 20 minutes, then disappear as if you had never created it.

Check Your Versions

Check sysdig-deploy Helm Chart: Must be 1.5.0+

helm list -n <namespace> (default namespace is sysdig-agent)

Example:

$ helm list -n sysdig-agent
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
sysdig-agent    sysdig-agent    5               2022-11-11 17:57:54.109917081 +0100 CET deployed        sysdig-deploy-1.5.0

Upgrade Helm Chart Instructions here.

Check Cli Scanner: must be 1.3.0+.

./sysdig-cli-scanner --version

Upgrade Cli Scanner: Instructions here.

When to Use

When faced with a large number of reported vulnerabilities, organizations need to know which are the most relevant for their security posture. Sysdig already highlights critical vulns with a fix available], and vulns that occur in images actually in use.

An additional feature is the targeted ability to accept the risk of a vuln and not count it towards a policy violation, for example, when:

  • An internal security team has analyzed the vuln and declared it a false positive
  • The preconditions of the vuln don’t apply
  • Deployment in production is required and it is reasonable to postpone the fix
  • etc.

What Types of Risk

You can accept risk for different entities:

  • Individual CVE IDs
  • Assets
    • Container images
    • Hosts

Accepting Risk in the context of vulnerability management applies an exception to the Vulnerability policy. Adding an accept to a CVE doesn’t make the CVE disappear. It still shows in the list, but voids the policy violation associated with that CVE.

When accepting risks it is important to:

  • Be careful with the accept scope or context; overly broad exceptions can create false negatives.
    • Sysdig offers several scoping options for the accepts created.
  • Remain aware of what is accepted so it doesn’t become a visibility gap.
  • The Sysdig UI presents clear indications of what is accepted and why.

Usage

See:

Remediate with Jira

Sysdig has an API-enabled remediation workflow with Jira (see Jira Ticketing).

To create a Jira ticket from the Sysdig Secure Vulnerabilities module:

  1. Navigate to any results page.

  2. Open the Vulnerabilities tab.

  3. Click on a vulnerability

    The details panel will open on right.

  4. Click Create a Jira ticket.

  5. Fill in the details, such as Project, Ticket type, assignee, and customer labels.

  6. Submit the ticket.

Once submitted the CVE drawer will note that there is an existing Jira ticket for that remediation.

Supported Operating Systems

The Vulnerability Management scanners can scan any OS, but for the following they will also report vendor-specific information.

  • Alibaba Linux

  • Alpine

  • Amazon Linux 2022/2023

  • CentOS 7

  • CentOS 8-stream

  • CentOS 9-stream

  • Debian

  • RHEL: The Sysdig Secure UI displays Red Hat Security Updates in the UI and in CLI version > 1.6.1 using the option --output-json. Note that this information is not included in Reports.

  • Ubuntu

  • Ubuntu kinetic (v22.10)

  • Ubuntu Lunar (v23.04)

Runtime

  • Kubernetes Runtime and Hosts
  • Supported container runtimes:
    • Docker daemon
    • ContainerD
    • CRI-O

Installation Options

  • Helm chart
  • Plain daemonset
    • Runtime scanner
    • Runtime scanner + benchmark runner

Agentless Host Scanning

Cloud Hosts (AWS EC2): Discovery of running hosts

Supported Operating Systems

  • Debian 10 and Debian 11
  • Ubuntu 20.04 LTS and Ubuntu 22.04 LTS
  • Amazon Linux 2 and Amazon Linux 2023
  • RedHat 8 and RedHat 9
  • OSes that the agent supports

CI/CD Pipeline

Supported Container Image Formats

  • Docker Registry V2 - compatible
  • Docker Daemon
  • Podman
  • Docker Archive (tar)
  • OCI Archive

Supported Package Types

  • Debian

  • Alpine

  • RHEL: The Sysdig Secure UI displays Red Hat Security Updates in the UI and in CLI version > 1.6.1 using the option --output-json. Note that this information is not included in Reports.

  • Ubuntu

  • Java Maven

  • Golang (built with go 1.13+)

  • Pypi (Python)

  • NPM (JS)

  • Ruby Gems

  • NuGet (.Net)

  • Cargo (Rust)

  • Composer (PHP)

Supported Container Image CPU Architectures

  • linux/amd64
  • linux/arm64
  • (others coming soon)