SNMP Monitoring with Prometheus

A fine way to monitor our infrastructure, is trough SNMP, a connectionless protocol,
supported by almost all, if not every, device with a network interface.
We make use of an agent, the snmp_exporter, which in our case, will be a docker process,
running on the same machine we use for Prometheus.

The following requirements to successfully operate :

UDP ports 161 in and 162 out, opened on every device we plan to add
SNMP deamon must be configured to accept connections from our IP, software side.
Must be generated a working configuration, introducing the correct MIB files, basically text files defining the interpretation of the variables extracted by SNMP (o.i.d)

Nothing left, but configuring Prometheus to pull the metrics from the exporter.

To test SNMP connectivity, the following commnand :

snmpwalk -v 2c -c $COMMUNITY $IP_ADDRESS
Clone the repository where the snmp_exporter is hosted :
git clone https://github.com/prometheus/snmp_exporter.git
Now, the configuration for the exporter, has to be generated.
Go to snmp_exporter/generator/mibs.
It is now necessary to retrieve the specifics MIB files, to the monitored device.
Such files, are usually found either on the device itself, or the vendor support page.
Once we have our MIBs in the folder, go to the parent directory, and edit the config.yaml.
Here, we define the o.i.d. basically variables that SNMP will retrieve from the device,
and that will be translated according to the MIB definition. Exemple :

# Cisco Switch
  cisco_2950_2960:   #modulo snmp exporter device
    walk:
      - 1.3.6.1.2.1.31.1.1.1.6 # IfHcInOctects
      - 1.3.6.1.2.1.31.1.1.1.10 # IfHcOutOctects
      - 1.3.6.1.2.1.31.1.1.1.18 # ifAlias
      - 1.3.6.1.2.1.2.2.1.8 # ifOperStatus
      - 1.3.6.1.2.1.2.2.1.7 # ifAdminStatus
      - 1.3.6.1.2.1.31.1.1.1 # ifXEntry
      - 1.3.6.1.2.1.1.3 # sysUpTime
      - 1.3.6.1.2.1.2.2.1 #ifEntry
    version: 2
    max_repetitions: 25
    retries: 3
    timeout: 30s
    auth:
      community: ********

It is now possible to generate the configuration :

docker run -ti   -v "${PWD}:/opt/"   snmp-generator generate
We can now run the exporter, passing the configuration to the container as a volume :
docker pull prom/snmp-exporter

docker run -dit -v /data/snmp/:/home --name snmp_exporter -p 9116:9116  -p 116:116 prom/snmp-exporter --config.file=/home/snmp.yml
All we have left, is to instruct Prometheus on the IP address and module of targets :
- job_name: 'snmp_cisco'
  static_configs:
    - targets:
      - 192.168.100.33
      - 192.168.100.34
  metrics_path: "/snmp"
  params:
    module: [cisco_2950_2960]   #instert module from generator.yaml
  scrape_interval: 80s
  scrape_timeout: 70s
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: localhost:9116  # The SNMP exporter's real hostname:port.
Reload the Prometheus to apply the new configuration :
curl -X POST http://localhost:9090/prometheus/-/reload

And now, we should be able to see the metrics pulled from Prometheus in real time.

Comments

Subscribe to our Newsletter