Alerts

​

Overview

Grafana can be configured to send events into Rootly as alerts. The alerts received on Rootly can then be routed to a Slack channel, used to initiate incidents, or used to page Rootly On-Call targets.

​

Configure Webhook in Grafana

Log into your Grafana account and navigate to the Contact points page (under Alerting). Select + Add contact point. You’ll be prompted to enter the following details for the contact point.

​

Name

Give your contact point a representative name.

​

Integration

Select Webhook.

​

URL

​

For General Alert

Grafana requires the webhook URL and secret to be included in the URL field. This will result in a regular (non-paging) alert. The webhook URL and secret can be obtained from your Grafana integration page in Rootly (Integrations > Grafana > Configure). Enter the webhook URL and secret in the following format in the URL field: https://webhooks.rootly.com/webhooks/incoming/grafana_webhooks/?secret=<enter_sercret_here>

​

To Page Rootly On-Call

Paging through Rootly On-Call also relies on webhook alerts. The main difference being the inclusion of a notification target in the query parameter. The notification target consists of the following:

• resource_type - this defines the Rootly resource type that will be used for paging.
  • The following are the available values: User | Group (Team) | EscalationPolicy | Service
• resource_id - this specifies the exact resource that will be targeted for the page.
  • The id of the resource can be found when editing each resource.

Enter the webhook URL and secret in the following format in the URL field: https://webhooks.rootly.com/webhooks/incoming/grafana_webhooks/notify/<resource_type>/<resource_id>?secret=<enter_sercret_here>

​

HTTP Method

Select POST. Once ready, you can select the Test button to test out your contact point. You should see a test alert appear from Grafana on your Rootly Alerts page. Finally, save contact point.

​

Attach Contact Point to Alert Rules

Once your contact point is created, you need to attach it to a Grafana alert rule. An alert rule contains the firing logic that determines when alerts will be sent out. Navigate to Alert rules and select + New Alert rule. You’ll be prompted to define a rule set. The key is to set the Contact point to the one you created above. Note: You can also add the contact point to your existing alert rules.