Migrating from Observium to LibreNMS

You may not have heard about LibreNMS yet, but I have a feeling you will be hearing lots about it in the near future. LibreNMS is a fork of Observium, an excellent network monitoring system. The fine folks behind LibreNMS have promised open, active development and a desire to help as many people as they can.

LibreNMS Demo Screen

I heard about LibreNMS on /r/networking and decided to give it a shot. I started with my lab and slowly started adding a few IT switches into the mix. After some time in the IRC channel (there are awesome people in there) and fiddling with things, I decided it was time to convert. Our Observium server was a couple years old and an upgrade was on the to-do list anyway. I went with a custom Ubuntu 14.04 build, however LibreNMS offers a ready-to-go VM image on their site.

My production Observium server has about two years worth of information and, like anyone, I did not want to lose all of my historical data. Additionally we were monitoring around 125 devices with Observium, all of which were going to require setup on the new LibreNMS server. Even more troublesome, the original Observium server was running on 32-bit Ubuntu, and RRD files cannot move between architectures.

With that set of constraints, I set out to see if I could automate the migration and move all of the historical data to LibreNMS. This post is the result of that attempt.

Setup:

First I had to lay out my script requirements:

  • Build the RRD directories on LibreNMS
  • Convert the RRD files on Observium to XML (x86 to x64 move)
  • Copy the RRD/XML files to LibreNMS
  • Convert the XML files back to RRD files
  • Add the device to LibreNMS

A pretty long set of requirements, but by scripting all of this I could set the script to run overnight and come in to a converted environment.

Second, I had to add a second server to my SNMP ACL. Depending on how your equipment is configured this may or may not be a requirement. If you can handle some missing data, I would actually recommend doing the conversion, shutting your Observium server down, and putting the old server IP on your new LibreNMS machine.

Script:

I have made two versions of the scripts available for you to download. One converts the RRDs to XML and then back to RRD files when they hit the destination. This is a requirement if you are moving from x86 to x64. Assuming you’re moving servers that are on the same architecture, we can skip that step and just SCP the original RRD files.

For everything to work as originally intended, you’ll need four files. Put all four files on both servers, the scripts default to /tmp/:

  • nodelist.txt – this file contains the list of hosts you would like to move. This must match exactly to the hostname Observium uses
  • mkdir.sh – this script creates the necessary directories on your LibreNMS server
  • destwork.sh – depending on the version you choose, this script will add the device to LibreNMS and possibly convert from XML to RRD
  • convert.sh – convert is the main script we’ll be calling. All of the magic happens here.

Feel free to crack open the scripts and modify them to suit you. Each file has a handful of variables you’ll need to set for your conversion. They should be self-explanatory, but please leave a comment if you have trouble. I’ve uploaded both versions of the scripts here – LibreNMS Migration Scripts

Conversion:

Now for the fun part. This section assumes the following:

  • Root access is available on both servers
  • You have SSH access to both servers
  • All four files have been placed in the tmp directory of both servers

I would strongly suggest you start with just one or two hosts and see how things work. For me, 10 standard sized devices took about 20 minutes with the RRD to XML conversion. Every environment will be different, so start slow and work your way up to full automation.

SSH Keys

First thing we will want to do is exchange SSH keys so that we can automate the login process used by the scripts. Perform these steps on your Observium server:

ssh-keygen -t rsa

Accept the defaults and enter a passphrase if you wish. Then:

ssh-copy-id librenms

Where librenms is the hostname or IP of your destination server.

Nodelist.txt

The nodelist.txt file contains a list of hosts we want to migrate from Observium. These names must match the name of the RRD folder on Observium. You can get those names by running the following –

ls /opt/observium/rrd/

Also important, the nodelist.txt file must be on both your Observium and LibreNMS server. Once you have your list, edit nodelist.txt with nano:

nano /tmp/nodelist.txt

And replace the dummy data with the hosts you are converting. CTRL+X and then Y to save your modifications. Make the same changes on the LibreNMS server.

Script Variables

Now that we have nodelist.txt setup correctly, it is time to set the variables in all three shell scripts. We are going to start with convert.sh. Edit it with nano:

nano /tmp/convert.sh

and change the variables to suit your environment. Here is a quick list of them:

  • DEST – This should be the IP or hostname of your LibreNMS server
  • L_RRDPATH – This signifies the location of the LibreNMS RRD directory. The default value is the default install location
  • O_RRDPATH – Location of the Observium RRD directory. The default value is the default install location
  • MKDIR – Location of the mkdir.sh script
  • DESTSCRIPT – Location of the destwork.sh script
  • NODELIST – Location of the nodelist.txt file

Next, edit the destwork.sh script:

nano /tmp/destwork.sh

The variables here are similar, but have a few key additions:

  • L_RRDPATH – This signifies the location of the LibreNMS RRD directory. The default value is the default install location
  • ADDHOST – Path to the addhost.php file LibreNMS uses when adding a host. The default value is the default location
  • SNMPSTRING – this is the SNMP string all of the devices you are converting use. Only enter one value here.
  • NODELIST – Location of the nodelist.txt file

Finally, lets finish up by editing the mkdir.sh script

nano /tmp/mkdir.sh

Variables will look familiar here, and there are only two to edit:

  • L_RRDPATH – This signifies the location of the LibreNMS RRD directory. The default value is the default install location
  • NODELIST – Location of the nodelist.txt file

After setting all of the variables, we’re ready to start the migration.

Migration

With the hard part done, the migration should be pretty straightforward. From your Observium server, run the convert.sh script:

/tmp/convert.sh

The script will take some time to run, especially if you are doing the RRD to XML conversion. In my experience, it takes about twenty minutes for about 480 ports.
LibreNMS Conversion

Once the script finishes, you should be able to load up the LibreNMS server and see all the devices.

LibreNMS Device List

Wrap-Up

This is a bit of an involved post, but the benefits are worth it. By creating these scripts (Download), I was able to maintain all of my historical graphs and automate the migration of about 125 switches.

We started by setting up our switches, adding the LibreNMS IP address to the SNMP ACL. Next, we exchange SSH keys between the Observium and LibreNMS server. Then we setup the nodelist.txt file and all of the variables, making sure to match the files on both servers. After all of the setup, we ran the migration script and confirmed that the devices were added to LibreNMS, verifying all of the historical graphs are there.

Historical Graphs on LibreNMS

Historical Graphs on LibreNMS after migration

If you are interested in learning more about LibreNMS, they have a great IRC Channel and a solid set of documentation. I will be working on a few more posts about LibreNMS, but their documentation and IRC is great place to get started. As always if you have any questions, leave a comment below.

9 Comments

  1. This is fantastic as I just started seeing others mention LibreNMS, since the head dev/owner of Observium is a douche.

    Reply

  2. Great post! These scripts are for migrating from Observium on one server to LibreNMS on another server. Is it possible to migrate from Observium to LibreNMS in place on the same server?

    Reply

  3. Hi, I came across your post while looking around about observium as I’m planning to use a monitoring tool for my office; since you’ve been using Obervium for years, why did you decide to migrate to LibreNMS? Appreciate your respond, thanks

    Reply

  4. Hi,

    First of all thanks a bunch for getting this done, seems to be working like a charm!

    I did run into a small little problem tho, the fact that we’re using multiple snmp communities at work.
    So i edited the workflow somewhat in destwork.sh

    First of all, a csv-file was needed with each host and it’s corresponding community:

    MYSQL: (make sure user is granted the FILE privilege on *.*)
    ===========
    use observium
    SELECT hostname, community FROM devices INTO OUTFILE ‘nodelist.csv’ FIELDS TERMINATED BY ‘,’ LINES TERMINATED BY ‘\n’;
    ===========

    once that file’s created, don’t forget to move it to /tmp.

    I then updated the destwork.sh to look like this instead:

    BASH
    ===========
    # Enter path to LibreNMS addhost module
    ADDHOST=/opt/librenms/addhost.php

    # Enter path to nodelist csv file
    NODELIST=/tmp/nodelist.csv

    # Save current value of IFS and then set new separator to ,
    OLDIFS=$IFS
    IFS=,

    while read HOST SNMPSTRING
    # Add each host from the node list file to LibreNMS
    do php $ADDHOST $HOST $SNMPSTRING v2c;
    done < $NODELIST

    # Set IFS back to original value
    IFS=$OLDIFS
    ===========

    Should work like a charm 🙂

    Reply

  5. Is it possible to use the scripts if the librenms install is on the same server as the observium install? And is it then possible to just make a symlink to the observium rrd directory (which is in ramdisk so a separate dir anyway)? Basically just want to add devices to librenms and copy the history from observium to librenms.

    Reply

  6. Fantastic set of scripts, got any tips on how to migrate Traffic Bills?

    Also got a small problem:
    running a test migration my source server (observium) seem to be stuck
    keeps repeating what is below, the device has been migrated, and all
    seems to be there in LibreNMS

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    LibreNMS Add Host Tool

    Usage (SNMPv1/2c): ./addhost.php [-g ] [-f] [-p ] [community] [v1|v2c] [port] [udp|udp6|tcp|tcp6]
    Usage (SNMPv3) : Config Defaults : ./addhost.php [-g ] [-f] [-p ] any v3 [user] [port] [udp|udp6|tcp|tcp6]
    No Auth, No Priv : ./addhost.php [-g ] [-f] [-p ] nanp v3 [user] [port] [udp|udp6|tcp|tcp6]
    Auth, No Priv : ./addhost.php [-g ] [-f] [-p ] anp v3 [md5|sha] [port] [udp|udp6|tcp|tcp6]
    Auth, Priv : ./addhost.php [-g ] [-f] [-p ] ap v3 [md5|sha] [aes|dsa] [port] [udp|udp6|tcp|tcp6]

    -g allows you to add a device to be pinned to a specific poller when using distributed polling. X can be any number associated with a poller group
    -f forces the device to be added by skipping the icmp and snmp check against the host.
    -p allow you to set a port association mode for this device. By default ports are associated by ‘ifIndex’.
    For Linux/Unix based devices ‘ifName’ or ‘ifDescr’ might be useful for a stable iface mapping.
    The default for this installation is ‘ifIndex’
    Valid port assoc modes are: ifIndex, ifName, ifDescr, ifAlias

    Remember to run discovery for the host afterwards.
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    Reply

  7. To transfer Traffic Bills

    Not sure if this will work for all, but seems to have worked for me

    Backup or snapshot both servers!!!!!!

    ON OBSERVIUM

    login as root run the following:
    mysqldump -p observium bills > bills.sql
    mysqldump -p observium bill_data > bill_data.sql
    mysqldump -p observium bill_history > bill_history.sql
    mysqldump -p observium bill_ports > bill_ports.sql

    SCP files from OBSERVIUM to LIBRENMS
    scp bills.sql root@yourlibrenmsserver.com:/root/
    scp bill_history.sql root@yourlibrenmsserver.com:/root/
    scp bluesky/bill_ports.sql root@yourlibrenmsserver.com:/root/
    scp bill_data.sql root@yourlibrenmsserver.com:/root/

    ON LIBRENMS

    login as root

    login to mysql and backup the existing tables, to give you a way out!!!!

    mysql -u root -p
    RENAME TABLE librenms.bills TO librenms.bills_backup;
    RENAME TABLE librenms.bill_data TO librenms.bill_data_backup;
    RENAME TABLE librenms.bill_history TO librenms.bill_history_backup;
    RENAME TABLE librenms.bill_ports TO librenms.bill_ports_backup;

    exit

    Import the data

    mysql -u root -p -D librenms < /root/bills.sql
    mysql -u root -p -D librenms < /root/bill_history.sql
    mysql -u root -p -D librenms < /root/bill_ports.sql
    mysql -u root -p -D librenms < /root/bill_data.sql

    You may have to quickly reconnect the ports in the LibreNMS Traffic Bills
    and run the poller just in case

    If it does not work and you renamed your old tables as suggested try this to get the database back to where you were.

    ON LIBRENMS

    login as root

    login to mysql

    mysql -u root -p
    drop table if exists librenms.bills
    drop table if exists librenms.bill_history
    drop table if exists librenms.bill_ports
    drop table if exists librenms.bill_data
    RENAME TABLE librenms.bills_backup TO librenms.bills;
    RENAME TABLE librenms.bill_data_backup TO librenms.bill_data;
    RENAME TABLE librenms.bill_history_backup TO librenms.bill_history;
    RENAME TABLE librenms.bill_ports_backup TO librenms.bill_ports;

    Hope it helps.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *