TrapperKeeper

Description

TrapperKeeper is a suite of tools for ingesting and displaying SNMP traps. This is designed as a replacement for snmptrapd and to supplement existing stateful monitoring solutions.

Normally traps are stateless in nature which makes it difficult to monitor with a system like nagios which requires polling a source. TrapperKeeper will store traps in an active state for a configured amount of time before expiring. This makes it possible to poll the service for active traps and alert off of those traps.

One example might be a humidity alert. If you cross over the humidity threshold and it clears immediately you might not want to be paged at 3am. But if it continues to send a trap every 5 minutes while it's over that threshold the combination of (host, oid, severity) will remain in an active state as long as that trap's expiration duration is longer than 5 minutes. This allows something like nagios to alarm when a single trap remains active for greater than some period of time.

Another benefit is allowing aggregation of pages. Previously we'd just had an e-mail to a pager per trap but now we're only paged based on the alert interval regardless of how many traps we receive. This also allows us to schedule downtime for a device during scheduled maintenance to avoid trap storms.

Requirements

Ubuntu

$ sudo apt-get install libmysqlclient-dev libsnmp-dev

Installation

New versions will be updated to PyPI pretty regularly so it should be as easy as:

$ pip install trapperkeeper

Once you've created a configuration file with your database information you can run the following to create the database schema.

$ python -m trapperkeeper.cmds.sync_db -c /path/to/trapperkeeper.yaml

Tools

trapperkeeper

The trapperkeeper command receives SNMP traps and handles e-mailing and writing to the database. An example configuration file with documentation is available here.

trapdoor

trapdoor is a webserver that provides a view into the existing traps as well as an API for viewing the state of traps. An example configuration file with documentation is available here.

API

/api/activetraps

Optional Parameters:

• host
• oid
• severity

Returns:

[
    (<host>, <oid>, <severity>)
]

/api/varbinds/<notification_id>

Returns:

[
    {
        "notification_id": <notification_id>,
        "name": <varbind_name>,
        "pretty_value": <pretty_value>,
        "oid": <oid>,
        "value": <value>,
        "value_type": <value_type>
    }
]

MIB Configuration

trapperkeeper and trapdoor use the default mibs via netsnmp. You can see the default path for your system by running net-snmp-config --default-mibdirs. You can use the following environment variables usually documented in the snmpcmd man page

MIBS - The list of MIBs to load. Defaults to SNMPv2-TC:SNMPv2-MIB:IF-MIB:IP-MIB:TCP-MIB:UDP-MIB:SNMP-VACM-MIB.

MIBDIRS - The list of directories to search for MIBs. Defaults to /usr/share/snmp/mibs.

For example I run both the trapperkeeper and trapdoor commands with the following environment to add a directory to the path and load all mibs.

MIBS=ALL MIBDIRS=+/usr/share/mibs/local/

TODO

• Allow Custom E-mail templates for TrapperKeeper
• cdnjs prefix for local cdnjs mirrors
• User ACLs for resolution
• Logging resolving user

Known Issues

• Doesn't currently support SNMPv3
• Doesn't currently support inform
• Certain devices have been known to send negative TimeTicks. pyasn1 fails to handle this.