Event Alerts
Alerts on events support one or more segmentation labels. An alert is generated for each segment.
Defining an Event Alert
Guidelines
Specify a meaningful filter text to count the number of related events.
Set a severity level for your alert. The order of priority of High, Medium, Low and Info are reflected in the Alert list, where you can sort by the severity by using the top navigation pane. You can use severity as a criterion when creating events and alerts. For example, you can receive a notification when there are more than 10 high severity events.
Filter by one or more Event Sources that should be considered by the alert. Predefined options are included for infrastructure event sources (kubernetes, docker, and containerd), but you can freely specify other values to match custom event sources. You can also view custom tags on the Event overlay.
Set the threshold for the total number of events within a specified time range that will trigger the alert rule.
Set a meaningful name and descriptio to help recipients easily identify the alert.
Configure Scope
Filter the environment on which this alert will apply. Use advanced operators to include, exclude, or pattern-match groups, tags, and entities. You can also create alerts directly from Explore and Dashboards for automatically populating this scope.
In this example, failing to schedule a pod in a default namespace triggers an alert.
Frequency of Alert Rule Evaluation
The Alert Editor automatically displays the time window that works best with your alert rule. Every data point in the alert preview corresponds with an evaluation of an alert rule.
The frequency at which an alert rule is evaluated depends on the Count Over Last specified in its query. For example:
- If you set up an alert query with a Count Over Last of 40 minutes, the rule evaluates every 1 minutes
- If you set up a query with a Count Over Last of 4 hours, the alert evaluates every 10 minutes.
Re-notifications for an alert cannot be sent more frequently than the alert rule’s evaluation interval and must be a multiple of this interval. For example, if an alert rule is evaluated every 10 minutes, re-notifications can only occur at multiples of the evaluation frequency, such as 20 minutes, 30 minutes, and so forth.
| Count Over Last | Frequency of Alert Rule Evaluation |
|---|---|
| up to 2h | 1m |
| up to 1d | 10m |
| up to 14d | 1h |
| up to 60d | 1d |
| 60d+ | Not Supported |
To view time series data older than the recommended window, click Explore Historical Data in the top right corner of Alert Editor. This will populate a PromQL Query in the Explore module with your current settings.
Configure Trigger
Define the threshold and time window for assessing the alert condition. Single alert fires an alert for your entire scope, while multiple alert fires if any or every segment breach the threshold at once.
If the number of events triggered in the monitored entity is greater than 5 for the last 10 minutes, recipients will be notified through the selected channel.
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.