Vulnerability Management

About Vulnerability Management

Sysdig Secure Vulnerabilities module addresses the top requirements for effective vulnerability management:

• Provides highly accurate views of vulnerability risk at scale
• Rich details provide precision about vulnerability risk (ex. CVSS vector, score, fix age) and insights from multiple expert feeds (ex. VulnDB)
• Access to public exploits allows you to verify security controls and patch efficiently
• Prioritized risk data focused on the vulns that are tied to the packages loaded at runtime
• Accepting risks when the vulnerability matching does not apply for your particular deployment, or you want to postpone remediation

Understand Vulnerability Management Stages

To create an effective vulnerability management plan, it’s important to recognize the various stages of the lifecycle that need to be addressed.

Key Concepts

• During the build phase, software vulnerabilities may be introduced when container images are defined and assembled. Container images are, by definition, immutable. Altering the contents of an image will update the ImageID and, thus, will be considered a different image by Sysdig Secure.

• Even though unique container images (ImageIDs) cannot be modified, Sysdig can identify new vulnerabilities in running containers (for example, in Kubernetes workloads) as security feeds are continuously updated. For instance, a container image with no known vulnerabilities during its build phase might be affected by a critical vulnerability discovered ten days after being deployed into the runtime. The image remains unchanged, but new security information related to the software it contains has been found.

• Sysdig uses the same fundamental concepts to analyze the contents of an image (SBOM) and match vulnerabilities but treats images differently based on their location. Sysdig can analyze the vulnerabilities of images in a development pipeline, stored in a container image registry, or used as the template for a running container, known as runtime workloads.

Stages: Pipeline - Registry - Runtime

Pipeline

Any analysis conducted before the registry phase is considered a pipeline. A clear example is CI/CD builds (Jenkins, Github, etc.), but also the execution of the sysdig-cli-scanner binary performed on a developer laptop or using a custom scanning script.

• Pipeline images do not have runtime context.
• The scan happens outside of the execution nodes where the agent is installed:
  • CI/CD
  • External instrumentation
  • Custom scripts or image scanning plugins
• Pipeline scans are one-off vulnerability reports; the information is a static snapshot with its corresponding execution date.
  • If you want to evaluate a newer version of the image or reevaluate the same image with newer feed information, the analysis needs to be triggered again.
• Images analyzed using the sysdig-cli-scanner will show up in the Pipeline section of the vulnerability management interface.

Registry

Container Registry scanning allows you to integrate Sysdig with image registries from a range of vendors. Registry scanning provides an extra layer of defense between pipeline and runtime, where:

• Software that sits in the registry before being deployed is checked for newly discovered vulnerabilities
• Third-party software that may have been installed without going through pipeline scanning will be checked

Registry scans occur as scheduled through a cron job in the installation Helm chart once per week (Saturdays at 6:00 AM) by default.

• Batch scanning is done asynchronously and separately from the development pipeline; regardless of the time it takes to scan the batch, the pipeline is unaffected.

See Install Registry scanner(s).

Runtime

Runtime workloads are executed using a container image. Accessing the Runtime section of the Vulnerabilities menu, you will be able to see those images and their vulnerability and policy evaluation.

• Runtime workloads are located in an execution node and are being monitored by a Sysdig agent/node analyzer, for example a Kubernetes node that is instrumented using the Sysdig agent bundle.
• Runtime workloads will offer a live, auto-refreshing state. This means:
  • Workloads that are no longer running will be removed from the runtime view
  • Vulnerabilities and policies evaluations will automatically refresh without any user interaction, offering always the most up-to-date information known.
    • At least once per day
• Runtime workload have a runtime context associated with them, i.e. Kubernetes cluster and namespace.
• Workloads analyzed during runtime will show up in the Runtime section of the vulnerability management interface.

See Runtime.

Asset Types: Workload, Host, and Container

The way things are scanned and how results are filtered relies on how Sysdig defines and uses the concepts of “image,” “workload,” “host,” and “container.”

Image

Image is not officially an asset type, but is the primary unit underlying all types of scanning.

• In Pipeline and Registry scanning, Sysdig scans images used in container-based environments.

• In Runtime, images have the context of either Kubernetes or the container host environment.

These images are strictly evaluated based on the image definition without reference to the deployed context (neither runtime nor orchestrator).

All images are extracted into a Software Bill of Materials (SBOM) format that is compatible with the CycloneDX standard. See CycloneDX for details.

Workload

When scanning images that are deployed in Kubernetes environments, the scanner evaluates the image in the context of the container running in a Kubernetes workload. This context makes it contextually different from images and containers.

Host

A host is a full-stack Operating System (OS) running on virtual infrastructure such as Cloud IaaS or Virtual Machines. The host will also have an SBOM, without the layers that are seen in images. All the context data for a host is relative to a full-stack OS with network connectivity (OS, Host Name, IPs, etc.).

Container

When Sysdig scans a container (either agentlessly or with the agent) it extracts a container from a Host image via the filesystem. In container scans, only the parent host context is available, not the runtime and orchestration context.

To be scanned by Sysdig, containers must all be compliant with the Open Container Initiative (OCI). See OCI for details.

Usage

When defining a Vulnerability policy or filtering Runtime vulnerability results, you can define the asset type to use.

Key Vulnerability Management Terminology

A cybersecurity vulnerability is a flaw in a computer system that enables an adversary to gain direct access to a network or system to compromise security and inflict harm.

CVE

Common Vulnerabilities and Exposures (CVE) is the database of such known cybersecurity vulnerabilities and exposures.

You can view the following information for all the asset types in all the vulnerability management stages:

• The severity level reported by National Vulnerability Database
• The package where vulnerability was found
• The directory path to the package
• The version in which the vulnerability was fixed

CVSS

The Common Vulnerability Scoring System (CVSS) is used to assess the severity of information security vulnerabilities. Each entry in the CVE database is assigned a corresponding CVSS score, which quantifies the severity of the vulnerability.

You can view the following information for all the asset types in all the vulnerability management stages:

• The sources selected for score calculation. Fo rexample, National Vulnerability Database

• The CVSS Score and version

• The date when the vulnerability was discovered.

CISA KEV

The Known Exploited Vulnerabilities (KEV) Catalog maintained by CISA (Cybersecurity and infrastructure Security Agency). It is the list of software flaws that have already been exploited in the real-world.

You can view the following information for all the asset types in all the vulnerability management stages:

• The date when the vulnerability was added to the KEV catalog
• The deadline for when the vulnerability should have a fix
• If the vulnerability has been used in any Ransomware campaigns

Exploit

An exploit is a software program created to identify and exploit security flaws or vulnerabilities present in a system.

You can view the following information for all the asset types in all the vulnerability management stages:

• Link to the existing exploit PoC or source code
• The date when the exploit was disclosed.

Vulnerability Databases

Sysdig Secure continuously checks against a wide range of vulnerability databases, updating the Runtime scan results with any newly detected CVEs.

The current database list includes:

• CVE
• NIST
• NPM
• Alpine Linux
• NVD
• VulnDB
• Centos
• Debian
• Ruby
• Red Hat
• Ubuntu
• Python

Getting Started with Vulnerability Management

1. For full vulnerability coverage, ensure you have completed the prerequisites:

   • A current Sysdig agent that includes the vulnerability management engine and runtime scanner.

   • The installed host scanner, whether agent-based (for Kubernetes, as a container, or as a package), or agentless (for AWS cloud accounts)

   • The downloaded cli-scanner for pipeline.

   • The installed container registry scanner(s) for registries.

2. Log in to Sysdig Secure with Advanced User+ permissions and select Vulnerabilities.

   

   The out-of-the-box policies for Pipeline and Runtime vulnerabilities will work without further setup.

3. Choose Pipeline, Registry, or Runtime to see the scanning results.

4. Choose Reporting to configure schedules for creating downloadable reports on runtime vulnerability results.

5. To create or edit Pipeline or Runtime Vuln Policies and Rule Bundles, select the relevant links from the Policies tab in the navigation bar.

6. To accept the risk of detected vulnerabilities, configure an acceptance based on scope, justification, and length of time. See Understanding and Usage steps.

   

Understand Image Scanning

Image scanning enables you to examine container images for vulnerabilities, misconfigurations, and license violations. You can integrate image scanning into your development build process to verify images added to your container registry, as well as scan images used by active containers in your infrastructure.

Image scanning provides registry insights into where your images are stored, initiates scans, and allows you to review the scan results.

Behind the scenes:

• SBOM is analyzed

• SBOM is compared against multiple vulnerability databases

• SBOM is then evaluated against default or user-defined policies.

• Results are reported, both in Sysdig Secure and (if applicable) in a developer’s external CI tool.

Platform-Based Scanning

Sysdig Vulnerability Management module offers platform-based scanning by default. The scanning tools analyze images and host filesystems to extract the Software Bill of Materials (SBOM) and send them to the Sysdig backend for vulnerability matching and policy evaluation. You can retrieve the scanning result by using an API.

Prerequisites

• VM Tools installed

  The following version of VM tools support SBOM scanning:

  • Host Scanner v1.0 and above for non-Kubernetes containers

    Host scanner for host filesystem is not currently supported

  • Cluster Scanner v1.23 and above

  • CLI Scanner v1.8.0 and above

  • Registry Scanner v1.1.26 and above

  • Agentless scanning

• For on-prem environment, use v6.8.0

View SBOM

To retrieve the SBOM of your asset, issue a curl GET request against the Sysdig Secure endpoint:

curl -XGET -H 'Authorization: Bearer <API_TOKEN>' 'https://<HOSTNAME>/secure/vulnerability/v1beta1/sboms?assetId=<sha256:xxxxxx>&assetType=container-image'

Query Parameters

• assetId: The ID of the asset for which you want to retrieve the SBOM. It’s the imageId for container-image and the hostId for hosts. For example, assetId=sha256:xxxxxx'. You can provide bomIdentifieror bothassetIdandassetType`
• assetType: The type of the asset for which you want to retrieve the SBOM. For example, container-image. Provide this with the assetId if you are not providing bomIdentifier.
• bomIdentifier: The ID of a single SBOM. Either you can provide bomIdentifier or both assetId and assetType to retrieve the SBOM result. For example, bomIdentifier=urn:uuid:45667rrrr-b8f2-42345-b996-dkffllflp.

Instant Scanning with Scan Now

You can skip deploying CLI-based scanning tools such as Registry Scanner and directly scanning the images in your running environment using the Scan Now button. This initiates platform-based scanning, making it easier to get started with Vulnerability Management. You can effortlessly scan and view image results without needing to deploy any extra components. For more information, see Manual Image Scanning.

Report Scanning Results

The analysis generates a detailed report of the image contents, including:

• Official OS packages

• Unofficial OS packages

• Configuration files

• Credentials files

• Localization modules and software-specific installers:

  • Javascript with NPM

  • Python PiP

  • Ruby with GEM

  • Java/JVM with .jar archives

• Image metadata and configuration attributes

Understanding Risk Acceptance for Vulnerabilities

As of November, 2022, users can choose to accept the risk of a detected vulnerability or asset. Accept Risk is available for both Runtime and Pipeline, and for specific CVEs or specified hosts or images.

Prerequisites

Accept Risk requires Sysdig Secure SaaS to be installed with:

• sysdig-deploy Helm chart version 1.5.0+
• vuln-runtime-scanner version 1.4.0+
• sysdig-cli-scanner version 1.3.0+

Because Accept Risk is applied to both pipeline and runtime vulnerability results impartially, the required versions of both components are required.

Check Your Versions

Check sysdig-deploy Helm Chart: Must be 1.5.0+

helm list -n <namespace> (default namespace is sysdig-agent)

Example:

$ helm list -n sysdig-agent
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
sysdig-agent    sysdig-agent    5               2022-11-11 17:57:54.109917081 +0100 CET deployed        sysdig-deploy-1.5.0

Upgrade Helm Chart Instructions here.

Check Cli Scanner: must be 1.3.0+.

./sysdig-cli-scanner --version

Upgrade Cli Scanner: Instructions here.

When to Use

When faced with a large number of reported vulnerabilities, organizations need to know which are the most relevant for their security posture. Sysdig already highlights critical vulns with a fix available, and vulnerabilities that occur in images actually in use.

An additional feature is the targeted ability to accept the risk of a vulnerability and not count it towards a policy violation, for example, when:

• An internal security team has analyzed the vulnerability and declared it a false positive
• The preconditions of the vulnerability don’t apply
• Deployment in production is required and it is reasonable to postpone the fix

What Types of Risk

You can accept risk for different entities:

• Individual CVE IDs
• Assets
  • Container images
  • Hosts

Accepting Risk in the context of vulnerability management applies an exception to the Vulnerability policy. Adding an accept to a CVE doesn’t make the CVE disappear. It still shows in the list, but voids the policy violation associated with that CVE.

When accepting risks it is important to:

• Be careful with the accept scope or context; overly broad exceptions can create false negatives.
  • Sysdig offers several scoping options for the accepts created.
• Remain aware of what is accepted so it doesn’t become a visibility gap.
• The Sysdig UI presents clear indications of what is accepted and why.

Usage

See:

• Accept Risk: Runtime VM
• Accept Risk: Pipeline VM
• Risk Acceptance Management

Remediate with Jira

Sysdig has an API-enabled remediation workflow with Jira (see Jira Ticketing).

To create a Jira ticket from the Sysdig Secure Vulnerabilities module:

1. Navigate to any results page.

2. Open the Vulnerabilities tab.

3. Click on a vulnerability

   The details panel will open on right.

4. Click Create a Jira ticket.

5. Fill in the details, such as Project, Ticket type, assignee, and customer labels.

6. Submit the ticket.

Once submitted the CVE drawer will note that there is an existing Jira ticket for that remediation.

Supported Operating Systems

The Vulnerability Management scanners can scan any OS, but for the following they will also report vendor-specific information.

• Alibaba Linux

• Alpine

• Amazon Linux 2022/2023

• CentOS 7

• CentOS 8-stream

• CentOS 9-stream

• Debian

• RHEL

  The Sysdig Secure UI displays Red Hat Security Updates in the UI and in CLI version > 1.6.1 using the option --output-json. Note that this information is not included in Reports.

• Ubuntu

• Ubuntu kinetic v22.10

• Ubuntu Lunar v23.04

Runtime

• Kubernetes Runtime and Hosts
• Supported container runtimes:
  • Docker daemon
  • ContainerD
  • CRI-O

Installation Options

• Helm chart
• Plain daemonset
  • Runtime scanner
  • Runtime scanner + benchmark runner

Agentless Host Scanning

Cloud Hosts (AWS EC2): Discovery of running hosts

Supported Operating Systems

• Debian 10 and Debian 11
• Ubuntu 20.04 LTS and Ubuntu 22.04 LTS
• Amazon Linux 2 and Amazon Linux 2023
• RedHat 8 and RedHat 9
• OSes that the agent supports

CI/CD Pipeline

Supported Container Image Formats

• Docker Registry V2 - compatible
• Docker Daemon
• Podman
• Docker Archive (tar)
• OCI Archive

Supported Package Types

• Debian

• Alpine

• RHEL: The Sysdig Secure UI displays Red Hat Security Updates in the UI and in CLI version > 1.6.1 using the option --output-json. Note that this information is not included in Reports.

• Ubuntu

• Java JAR, WAR, EAR

• Golang (built with go 1.13+)

• Pypi (Python)

• NPM (JS)

• Ruby Gems

• NuGet (.Net)

• Cargo (Rust)

• Composer (PHP)

Supported Container Image CPU Architectures

• linux/amd64
• linux/arm64