Runtime Threat Detection Policy Tuning
When auto-tune is enabled, exceptions are automatically added to Falco policy rules to remove particularly noisy sets of events and leave lower-volume events for later analysis.
Additionally, event details (accessed either in Events feed or the Insights panels) may include Tunable Exception suggestions which can be ignored, applied, or edited and applied.
The tuner can be especially helpful when deploying Sysdig Secure runtime policies in a new environment. Your environment may include applications that legitimately perform actions such as running Docker clients in containers, changing namespaces, or writing below binary directories, but which trigger unwanted floods of related policy events in the default policies and rules provided by Sysdig.
Only users with the Admin user role are permitted to perform the actions in this section, directly on the Tuner page.
Use the Runtime Policy Tuner
There are two ways to use the policy tuner:
- Auto-Tune: enable/disable
- Tunable Exceptions: Accessed in the Secure Events feed and Insights, these features do not require the auto-tuning feature to be enabled.
Enable/Disable Auto Tuning
The tuner is enabled and disabled as needed to tame false positives and optimize the use of the Events feed.
Log in to Sysdig Secure as Admin and choose Policies > Threat Detection | Runtime Policy Tuning.
Enable/disable the feature with the
It may take up to 24 hours to see the initial Applied Tuning Exceptions listed in the left panel.
In the background, the tuner will evaluate policy events as they are received by the Sysdig backend, find applicable exceptions values, and add them. The Applied Tuning Exceptions file is passed along to all Sysdig agents, along with the rules and policies.
Toggle the Tuning Engine
offwhen you feel the feature has addressed the most commonly occurring (unnecessary) policy events.
NOTE: Turning the auto tuner off does not disable the exceptions that were added by the auto tuner, nor any exceptions manually added from the tuner suggestions.
To start over from scratch, clear the Applied Tuning Exceptions text and re-enable the Tuning Engine toggle.
Edit & Review Exceptions
Edit in Runtime Policy Tuning Page: After an exception is added by either the auto tuning feature or tunable exceptions, admins can directly edit or review the exceptions in the editor file in the Runtime Policy Tuning page.
Review in the Rules Library: View rules that have been tuned by selecting those that have “Published By: Tuner 0.0.0”
Understand How the Tuning Engine Works
When Does the Tuner Add Exceptions?
The Policy Tuning feature is conservative, only adding exceptions for commonly occurring events for a single rule with similar attributes.
All the conditions must be met:
The rule has generated at least 25 policy events in the past hour
A candidate set of exception values is applicable to at least 25% of the events in the past hour
This ensures the tuning feature only adds exceptions for high-volume sets of events that can be easily addressed with a single set of exception values.
Exceptions Behind the Scenes
To understand the process of exception insertion by the tuner, consider a sample rule:
- rule: Write below root desc: an attempt to write to any file directly below / or /root condition: root_dir and evt.dir = < and open_write exceptions: - name: proc_writer fields: [proc.name, fd.filename]
And a stream of policy events with outputs such as:
File below / or /root opened for writing (user=root user_loginuid=-1 command=/usr/local/bin/my-app-server parent=java file=/state.txt program=my-app-server container_id=a97d44bbe437 image=my-registry/app-server:latest File below / or /root opened for writing (user=root user_loginuid=-1 command=/usr/local/bin/my-app-server parent=java file=/state.txt program=my-app-server container_id=a97d44bbe437 image=my-registry/app-server:latest File below / or /root opened for writing (user=root user_loginuid=-1 command=/usr/local/bin/my-app-server parent=java file=/state.txt program=my-app-server container_id=a97d44bbe437 image=my-registry/app-server:latest File below / or /root opened for writing (user=root user_loginuid=-1 command=/usr/local/bin/my-app-server parent=java file=/state.txt program=my-app-server container_id=a97d44bbe437 image=my-registry/app-server:latest
Then the tuner would add the following exception values to address the false positives:
- rule: Write below root exceptions: - name: proc_writer values: - [my-app-server, /state.txt] append: true
See the Falco proposal for more background information on using exceptions.
Tuning Best Practices
Learning/Onboarding to Sysdig Secure
The auto-tuner is best used when you are onboarding to Sysdig Secure with environments that have a lot of tooling that may look like some nefarious actions, but are benign. For example, a database backup tool can trigger certain rules that are monitoring for data exfiltration.
Note: In many instances, we have already added exceptions out of the box, and auto tuning is not needed.
In most cases, you may only get a handful of alerts from the same rule for the same workloads. See the best practices for tunable exceptions
- Use Insights and review a grouped set of rules, using the Tunable Exceptions to add exceptions for your workloads or applications
- Use the Events feed and review an event within a grouped policy, using the Tunable Exceptions to add exceptions for your workloads or applications
Review the auto-tuned exceptions after a learning/onboarding period
After or during the auto-tuning period, you can review rules that have been modified in the Rules Library by viewing the rules that have “Published By: Tuner 0.0.0”
How long should I leave auto-tuning on to learn about my environment?
In many cases, a week should suffice, as most workloads would have likely taken any actions that were required
Should I apply all suggestions?
No, in most cases the first suggestion will create an exception that will encompass all of the other suggestions. If the number of events shown on the top right of each suggestion are the same, it’s likely that you are just duplicating exceptions, which is mostly okay.
Use the first suggestion
Do not blindly accept all exceptions. The top suggestion targets a combination of the most events with the most targeted set of fields and values. Take, for example, the following suggestions for the rule
Read sensitive file trusted after startup.
The first suggestion successfully stops alerting for all fifteen events by specifying an exception,
known_sensitive_reader with a specified process,
httpd, when reading a specific type of password file,
The second suggestion targets the same 15 events, but specifies only the process
httpd. If implemented, this would prevent
Read sensitive file trusted after startup rule from triggering for any
httpd process, including if the process opened other sensitive files such as
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.