Silence Alert Notifications

Sysdig Monitor helps to reduce noise and alert fatigue with Silence Rules. Silences Rules temporarily mute alert notifications that might result from maintenance activities, scheduled downtime, or known issues. Alerts that are silenced by scope or by alert will still trigger but alert notifications to the notification channels will not be sent.

See Working with Alert APIs for programmatically configuring Silence Rules.

Configure Silence Rule

You can silence alert notifications in two ways:

  • by Scope: For instance, configuring a silence on the region=us-east-1 scope will silence all alerts in the us-east-1 region.

  • by Alert: A silence can be configured for one or more alerts. Silencing the Datacenter Down alert instead of the region=us-east-1 scope allows other alerts within the scope to continue notifying.

Combining silences by both scope and by alert allows for maximum specificity.

Silence Rules can be triggered immediately or scheduled in the future. This provides the flexibility to plan ahead and avoid unnecessary disruptions.

Additionally, you can send an optional info message to a notification channel to keep teams updated on when a Silence starts and ends.

To configure a Silence Rule:

  1. Click Alerts on the left navigation on the Monitor UI.

  2. Click the Silence tab.

    The page shows the list of all the existing silences.

  3. Click Set a Silence.

  1. Specify the following:

    • Name: Specify a name to identify the silence.

    • Silence Criteria: Create a silence by alert, scope, or both.

      • Scope: Specify the entity you want to apply the scope as.

      For example, a particular workload or namespace.

      • Alert: Specify the alert you want to mute.
    • Begins: Specify one of the following: Today, Tomorrow, Pick Another Day. Select the time from the drop-down.

    • Duration: Specify how long notifications should be muted.

    • Notify: Select a channel you want to notify about the silence.

  2. Click Save.

Silence Alert Notifications from Alerts Page

To configure a Silence Rule for a specific alert, navigate to the Alerts page and select the alert you want to silence. You can then configure a Silence Rule specifically for that alert.

Silence Alert Notifications from Events Feed

To configure a Silence Rule, access the Events feed. On the Events feed, find the Alert Event created when an alert is triggered. From the Alert Event, you can configure the Silence Rule to include both the scope and the alert that generated the event.

Alert Events from Silenced Alerts

When an alert is silenced, it will not send notifications to your configured notification channels, but it will still generate Alert Events. These events will include information indicating that the alert was triggered during an active silence.

By generating Alert Events, you can review the event history to see when the alert was silenced and when it resumed normal notifications. This helps provide visibility into what happened during the silence and can help with troubleshooting any issues that may arise.

Alerts without a notification channel won’t be marked as silenced, and won’t have the crossed bell icon or option to silence events in the events feed.

Manage Silence Rules

Silences can be managed individually, or as a group, by using the checkboxes on the left side of the Silence UI and the customization bar. Select a group of silences and perform batch delete operations. Select individual silences to perform tasks such as enabling, disabling, duplicating, and editing.

Change States

Enable or disable a silence by sliding the state bar next to it. Active Silences include both running silences and scheduled silences.

Completed silences cannot be re-enabled, but can be duplicated with a new silence period.

Duplicate a Silence Rule

To duplicate a Silence Rule, click the Duplicate button on the menu.

Filter a Silence Rule

Filter silences by name using the search bar. Silence Rules can also be filtered by the following categories: Active, Scheduled, Completed.

Edit a Silence Rule

Scheduled Silence Rules can be edited before they begin.

Extend the Time Duration

Active Silences can only be extended or disabled. If a notification channel was configured for the Silence Rule, the channel will be updated accordingly.