Container Drift Policy
Drift Control applies to containers only and does not work on hosts.
Overview
Drift Control helps you:
- Prevent attacks by blocking container drift in production: Drift Control automatically flags and denies deviations from the original container, blocking malicious executables before damage is done.
- Enforce immutability best practice: Drift Control ensures that container software is not modified during its lifetime, driving good practices, consistency from source to run, and preventing actions that could be part of an attack.
- Enable easy and effective security: Teams are often overwhelmed by cloud-native complexity and blind to container drift, especially at scale. Now, security teams and IT can enable Drift Control for the entire container environment and immediately start protecting it.
The Container Drift policy has the following unique attributes:
- It includes only one pre-configured rule, which cannot be edited, and no other rules can be added to Drift Control policies.
- It is a custom policy, not a managed policy.
Event notifications are generally limited to a frequency of once every five minutes. For details, see Message Throttling in Sysdig Secure.
Prerequisites
Agent version 12.16 and above for Drift policy
Agent version 13.0 and above for captures and container stop/pause/kill actions
On agent version v13.1 and above:
ignore_container_action
: Ignores kill, stop, pause container operationignore_action
: Ignores all the actions including kill, prevent malware, prevent drift and container actions.
Kernel version 5.0 and above (see Prevent action)
Agent version 13.1.0 and above to Detect Volume Binaries
In v13.1.1, it is enabled by default.
In v13.1.0, add the following configuration to the
dragent.yaml
file:drift_deny_execution_from_volumes: true
See Configure Drift Control in Agent Configuration.
Create a Container Drift Policy
To configure a Container Drift Policy in the Sysdig Secure UI:
Select Policies > Threat Detection > Runtime Policies to display the Runtime Policies page.
Click + Add Policy in the top right of the page.
Select Container Drift policy type.
Next, you can configure the policy.
Configure a Container Drift Policy
Name: Enter a policy name.
Description: Provide a meaningful and searchable description.
Enabled: Toggle to enable or disable the policy. When the policy is enabled, it generates events.
Severity: Choose the appropriate severity level as you would like to see it in the Runtime Policies UI: High, Medium, Low, or Info.
- Policy severity is subjective and is used to group policies within a Sysdig Secure instance.
- There is no inheritance between the underlying rule priorities and the severity you assign to the policy.
Scope: Define the scope to which the policy will apply, based on the type-dependent options listed.
- Auto-tuning is not used with Drift policies. If you have too many false positives, use Scope to tune them.
Link to Runbook: (Optional) Enter the URL of a company procedure that should be followed for events resulting from this policy. For example, https://www.mycompany.com/our-runbook-link.
- If you enter a value here, a View Runbook option will be displayed in any corresponding Event.
Detect
Drifted Binaries: Toggle on to dynamically detect the execution of drifted binaries.
A drifted binary is any binary that was not part of the original image of the container. It is typically downloaded or compiled into a running container.
If a detected binary attempts to run when this toggle is enabled, Sysdig will create an alert. To prevent the binary from running, use Prevent.
Volume Binaries: Toggle this on to treat all binaries from mounted volumes as drifted. This will generate events and prevent the binaries from being executed.
Exceptions
Create exceptions to a policy to prevent execution based on either the attributes of a file itself or the process attempting to execute the file.
Define binaries with their full path, separated by commas, using string or regex (if enabled).
String is very specific, while regex is pattern matching and more flexible.
Use Regex for exceptions: Toggle this on to use regex (regular expression) when you configure exceptions.
- Regex allows you to create patterns for matching file names or process paths. For example,
.*\.log$
matches any file ending with “.log”. You can use regex to identify drifted files or processes that should be allowed or blocked by matching their names or paths the the regex pattern defined. - To learn more about regex syntax, see Regular Expression Syntax.
- Agent version 13.2.0 or above is required for this feature.
File-based Exceptions
Exceptions: Specify which drifted files can execute within a container. This means that even if a file has been modified or added post-deployment, it can still be executed if it is listed as an exception. This is useful for scenarios where certain files need to be updated or added and executed as part of legitimate operations, such as configuration scripts or updates.
Prohibited Binaries: Specify which drifted files are not allowed to execute.
Process-based Exceptions
Exceptions: Specify which processes can execute drifted files. This means that specified processes can interact with modified or newly added files without triggering security alerts, facilitating legitimate operations that require such interactions.
Prohibited Binaries: Specify binaries of processes whose execution is blocked even if they are built with the image.
The Prohibited Binaries option was formerly called always deny.
Actions
Determine what should be done if a Policy is violated.
Prevent
Prevent Execution: Toggle this to stop detected executables from running.
Depending on the kernel version, the agent will either:
Kill the process once it is detected as drifted (kernel version <5.0)
Prevent drifted processes from running (kernel version 5.0+)
The latter method is preferable because it stops the drifted process from running for any period. The agent determines which mode to use at agent startup. One of the above-mentioned types of prevention is always available with 12.15+ agents. For example, Bottlerocket prevents by using the kill action.
Containers
Select what should happen to affected containers if the policy rules are breached. The appropriate action depends on your use case:
- No container action: Do not change the container behavior; send a notification according to Notification Channel settings.
- Kill: Kills one or more running containers immediately. This is recommended to make an attack harder for the attacker.
- Stop: Allows a graceful shutdown (10-seconds) before killing the container. This is recommended if stopping an attacker is your priority. However, this might affect services that rely on the container.
- Pause: Suspends all processes in the specified containers.
For a detailed breakdown and use cases, see Container Actions.
You can configure the agent to prevent kill/pause/stop
actions, regardless of the policy.
Capture
Toggle Capture ON if you want to create a capture in case of an event, and define the number of seconds before and after the event that should be in the snapshot.
See also: Captures.
Notify
Select a notification channel from the drop-down to send notifications of events to appropriate personnel.
This will ensure you know there’s a serious security incident. You can then save the notification as a record of what happened for later analysis.
See also: Set Up Notification Channels.
Container Actions
In Threat Detection Policies, Kill, Stop and Pause are different response actions, each with distinct impacts on workloads and use cases:
Kill: Killing the container immediately terminates it, removing it from the cluster. Kubernetes will typically attempt to restart the container based on its configuration. This can stop a malicious activity quickly, but can result in data loss (in stateful applications) and service disruption. It is the best choice for severe and urgent threats.
Stop: Stopping a container shuts down its processes and removes it from the scheduling pool, but unlike killing, it allows for a more graceful shutdown of services. Stopping a container might require manual intervention to bring it back online. It is best for stateful applications.
Pause: Pausing a container suspends its execution, freezing its processes and preserving its runtime state. The container remains in the cluster but is temporarily halted. It’s ideal for forensic analysis and limiting disruption, but may result in higher resource consumption and prolonged risk due to lack of resolution.
Summary
Action | Impact on Container | Use Case | Drawbacks |
---|---|---|---|
Kill | Immediate termination | High risk threat. Rapid mitigation. | Loss of forensic data. Possible service disruption. |
Stop | Graceful shutdown | Controlled threat removal. Preserving service dependencies. | Residual risk. Manual restart of container needed. |
Pause | Execution halted, state preserved. | Stealthy containment. Preserves data for analysis. | Continued resource use. May prolong risk. |
Check Events
When the policy is enabled, you can check for any detected events:
Log in to Sysdig Secure and select Events.
Type
Drift
in the filter bar to find where the Drift policy was triggered and drill down to examine the event details.
Caveats
There are certain conditions that the Drift policy will not catch.
Script Execution using Binaries from the Base Image
Drift control tracks the execution of binaries that were not part of the original image. It does not cover cases where a script is executed leveraging binaries from the original image. For example, if an attacker downloads a Python script and leverages an existing Python binary from the base image, drift detection will not detect it at runtime.
Persistent Volumes
As the Drift policy uses overlays for detection, execution from persistent volumes is not considered “drifted.” Therefore, if a malicious binary is downloaded to a persistent volume and executed, it will not be caught.
Suppose you are using persistent volumes exclusively for data. In that case, you can set the following option in the agent config file to treat the execution of all binaries from persistent volumes as drifted:
drift_deny_execution_from_volumes: true
Use with caution. When this option is used with the Prevent action, the execution of any binary from any mounted volumes will be stopped.
Container Limits
For kernel versions below v5.13, Drift Control can monitor up to 128 containers per node.
For kernel versions v5.13 or above, you can modify the container limit as described in Configure Container Limits.
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.