Alert Grouping
Learn how to create an alert group.
Alert grouping reduces noise and alert fatigue by consolidating related alerts into a single notification, making it easier for incident responders to focus on critical issues. When alerts are grouped together, responders will only be paged for the initial alert.
This improves response efficiency, enhances prioritization, and simplifies communication, ultimately leading to faster incident resolution and better overall system reliability.
Alerts can be grouped by:
- Destinations: Alerts are grouped based on their route. For example, alerts routed to the same team will be grouped.
- Time Window (also known as time-based): Alerts are grouped on a rolling window of time. For example, all new alerts triggered within 10 minutes of each other will be grouped.
- Content Matching: Alerts are grouped based on the value of specific fields like title, urgency, and payload. For example, all alerts with the same alert title will be grouped.
When to enable alert grouping
You might want to enable alert grouping if your organization has multiple monitors for the service.
For example, you might have a monitor for error rate, another for latency, another for CPU, and then maybe a monitor for something on the database. With a lot of monitors, if something goes wrong with that particular service, it will trigger all related monitors to start sending alerts - this is where alert grouping comes in.
When using alert grouping, the responder only gets paged for the first alert that comes in and not for each monitor that gets triggered.
Configuring an Alert Group
To create a new alert group in the web app:
Go to Alerts > Grouping Tab and click + New Alert Group.
Enter a Name (required) and a Description (optional).
Conditions
Destination condition
Select the route that should be used to consider an alert for a group.
- All services, teams, and escalation policies
- All services
- All teams
- All escalation policies
- Select routes will consider alerts routed to a specific service, team, or escalation policy. For example, only group alerts that are routed to a specific team.
- Select ‘Select routes’ in the first dropdown under Destinations.
- Select the target service, team, or escalation policy that you would like to group alerts by.
Select your group’s route logic
Define how the alerts should be grouped based on the alert’s route.
- Groups should only contain alerts for the same route: This ensures that alerts will only be grouped if they’re routing to the same service, team, or escalation policy defined in the destination condition. For example, alerts routing to Service A will be grouped together, and alerts routing to Service B will be grouped together.
- Groups can contain alerts for any selected route: This will group alerts regardless of the destination service, team, or escalation policy defined in the destination condition. For example, any alert routed to any team will be grouped together.
Time Window (required)
Define the group’s time window in minutes. This dictates how long alerts should be grouped before creating a new group for new incoming alerts.
The time window is set on a rolling basis and will start based on when the last alert was added to the group.
Example
A 10-minute time window will result in a group continuing to accept new alerts until a 10-minute downtime, where no new alerts have been added to the group.
Content Matching
Content matching allows for more granularity to define the conditions under which alerts get grouped.
-
Alert Title can be used to group alerts that come in with the same title.
-
Alert Urgency can be used to group alerts by different urgencies (high, medium, low).
-
Payload can be used to group alerts based on any specific field from your payload.
Example
When you want to group alerts based on a specific alert feature in your payload, the payload may look like “$.alert.feature”
Working with Alert Groups
The first alert in a group is considered the group’s leader. The leader is the alert that initially paged the responder. Any matching grouped alerts will become members of the leader’s group.
When a subsequent alert is grouped with a leader, the leader will act as the source of truth for all grouped alerts.
- Any new alerts that match the group will be automatically grouped under the leader. They will not page the responder.
- Any status changes to the group’s leader will also update all of the alert members’ statuses.
- You can review any individual alert’s group from the alert in the Rootly dashboard under the ‘Alert Group’ tab.