​

Why

Prometheus Alertmanager will let you send a webhook to Rootly as an incoming alert. The incoming alert can then be used to either create an incident, notify channels, or page on-call targets.

​

Installation

Locate Alertmanager on the Integrations catalogue and select Setup. You will be presented with the following pop-up.

​

Receiving General Alerts

In order to send general (non-paging) alerts to Rootly, you’ll need to modify the alert-manager.yml configuration file as shown below:

route:
  receiver: default
  group_by:
  - job
  routes:
  - receiver: rootly
    match:
      alertname: Rootly
    repeat_interval: 1m 

receivers:
 - name: rootly
   webhook_configs:
   - url: 'https://webhooks.rootly.com/webhooks/incoming/alertmanager_webhooks'
     send_resolved: true
     http_config:
       authorization:
         type: Bearer
         credentials: a0b9fcad1aae0689cfa05c17df497b2bc5c56d26d3e253503438864dbd6697ee

Copy the Webhook URL field and secret from the pop-up above, and set it as the url and credentials parameters, respectively.

​

To Page Rootly On-Call

Prometheus (Alertmanager) can be configured in two different ways to page Rootly On-Call.

​

Via Receiver URL in Alertmanager

Similarly to the non-paging alert setup above, you’ll need to modify the alertmanager.yml configuration file as shown below. The main difference here being the notification target must be specified as part of the receivers url.

route:
  receiver: default
  group_by:
  - job
  routes:
  - receiver: rootly
    match:
      alertname: Rootly
    repeat_interval: 1m 

receivers:
 - name: rootly
   webhook_configs:
   - url: 'https://webhooks.rootly.com/webhooks/incoming/alertmanager_webhooks/notify/User/27854'
     send_resolved: true
     http_config:
       authorization:
         type: Bearer
         credentials: a0b9fcad1aae0689cfa05c17df497b2bc5c56d26d3e253503438864dbd6697ee

Copy the Webhook URL field and secret from the Prometheus (Alertmanager) configuration modal in Rootly, and set it as the url and credentials parameters, respectively.

Then set the notification target by appending the following to the end of the url.

notify/<resource\_type>/<resource\_id>

The notification target consists of the following:

• resource_type - this defines the Rootly resource type that will be used for paging.
  • The following are the available values: User | Group (Team) | EscalationPolicy | Service
• resource_id - this specifies the exact resource that will be targeted for the page.
  • The id of the resource can be found when editing each resource.

​

Via Prometheus Rules Annotation

If you are using Prometheus’s alerting rules, you can set the notification target through the annotations in your prometheus.rules.yml file.

groups:
- name: ./rules.conf
  rules:

  # heartbeat alert
  - alert: Heartbeat
    expr: vector(1)
    labels:
      event: "Heartbeat"
      instance: "prometheus"
      monitor: "prometheus"
      severity: "major"
      timeout: "120"
    annotations:
      summary: "Heartbeat from prometheus"
      description: "Heartbeat from from prometheus"
      rootly: "{\"notification_target\":{\"type\":\"User\",\"id\":\"27854\"}}"

The notification target values can be set to the following under the rootly field:

• Type - this defines the Rootly resource type that will be used for paging.
  • The following are the available values: User | Group (Team) | EscalationPolicy | Service
• id - this specifies the exact resource that will be targeted for the page.
  • The id of the resource can be found when editing each resource.

​

Support

Please visit https://prometheus.io/docs/alerting/latest/configuration/ for more information on Prometheus Alertmanager.

If you need help or more information about this integration, please contact support@rootly.com or start a chat by navigating to Help > Chat with Us.