Prometheus WebHook to SNMP-trap forwarder

This is a quick (and dirty) way to get Prometheus to send SNMP traps, by mapping AlertManager "Annotations" and "Labels" to generic SNMP OIDs.

Integration with Prometheus

1. Prometheus gathers metrics
2. Prometheus appraises metrics against rules
3. If rules are triggered then alerts are raised through the AlertManager
4. The AlertManager triggers notifications to the webhook_snmptrapper
5. The webhook_snmptrapper forwards alerts as SNMP traps to the configured trap-address

SNMP integration

The provided MIB (PROMETHEUS-TRAPPER-MIB.txt) defines two notifications:

• prometheusTrapperFiringNotification: Notification for an alert that has occured
• prometheusTrapperRecoveryNotification: Notification for an alert that has recovered

The MIB can be loaded into whatever SNMP Trap-server you're using. See Dockerfile for a working demo using net-snmp on Alpine Linux.

SNMP variables

Both of these traps contain the following variables:

• prometheusTrapperNotificationInstance: The instance or hostname
• prometheusTrapperNotificationService: A name for the service affected
• prometheusTrapperNotificationLocation: The physical location where the alert was generated
• prometheusTrapperNotificationSeverity: The severity of the alert
• prometheusTrapperNotificationDescription: Text description of the alert
• prometheusTrapperNotificationTimestamp: When the alert was first generated

AlertManager configuration

AlertManager needs to be configured to fire webhooks as notifications, with a pre-defined assortment of labels and annotations (these map to the SNMP MIB provided). Each alert should have the following parameters:

Annotations:

• description: A string describing the alert (prometheusTrapperNotificationDescription)

Labels:

• instance: A string containing a unique host-identifier / hostname / instance-id / IP-address etc (prometheusTrapperNotificationInstance)
• severity: A string describing the severity of the alert (prometheusTrapperNotificationSeverity)
• location: A string describing the location of the instance(s) / system(s) generating the alert (prometheusTrapperNotificationLocation)
• service: A string describing the service affected (prometheusTrapperNotificationService)

Command-line flags

• -snmpcommunity: The SNMP community string (default = public)
• -snmpretries: The number of times to retry sending traps (default = 1)
• -snmptrapaddress: The address to send traps to (default = 127.0.0.1:162)
• -webhookaddress: The address to listen for incoming webhooks on (default = 0.0.0.0:9099)
• -snmptrapbinary: Path to the net-snmp 'snmptrap' binary (default /usr/bin/snmptrap)