Sending nagios / centreon notifications via IRC

Lately we needed to find a convenient method of notifying service desk operators of centreon notifications. Email was not a good option since operatives tended to simply delete the messages. Besides, we wantede something a bit more realtime.

In the end, I decided on using IRC. I went with IRC rather than jabber due to it’s ease of use, maturity, and lightweight requirements.

We then installed pidgin on each of the operatives PCs, and enabled the following plugins (Tools menu > plugins)

– Join/Part hiding : does not display messages everytime a user (including the centreon notifier) joins or leaves the IRC room

– Message notifiaction : this causes the IRC room window to flash when a new message is received, in order to better grab the attention of the user. Note you need to click on “configure plugin” and manually enable the flashing feature.

Once the correct account was setup in pidgin, the last configuration we needed was to instruct pidgin to automatically join our IRC notification room (called #netops)/ This was done via the Buddies menu > add chat, and ensure that “auto-join” is enabled

There are several perl-based IRC scripts to send messages to an IRC server from a terminal. However, while these scripts worked, I found them to be too slow, as in, when the nagios / cetnreon system needed to send a large number of notifications in a short period of time (under 10 seconds), the script couldnt keep up, resulting in lost notifications.

1. Download, install and configure unrealircd on a server of your choice from their website:

http://www.unrealircd.com/

2. On the nagios / centreon monitoring server, copy / paste the following into a script called irc_msg.sh. In this case, I assume the message will be sent to all members of the IRC room “#netops”. This can of course be modified to your liking. Move the script to the normal nagios plugin location (usually /usr/local/nagios/libexec)

#!/bin/bash

echo NICK centreon_backoffice > /tmp/irc_msg
echo USER centreon_backoffice 8 \* : Centreon Notifier >> /tmp/irc_msg
#echo ‘JOIN #netops’ >> /tmp/irc_msg
echo ‘PRIVMSG #netops’ $1 >> /tmp/irc_msg
echo QUIT >> /tmp/irc_msg

#cat /tmp/irc_msg
nc 127.0.0.1 6667 < /tmp/irc_msg

The above script basically prepares a file in /tmp/irc_msg in order to send to the IRC server. The last line of the script uses netcat (nc – which comes installed by default with centos) to send the contents of the file to the IRC server (of course, change 127.0.0.1 to your IRC server IP) An example of the contents of the file would be:

NICK centreon_backoffice
USER centreon_backoffice 8 * : Centreon Backoffice Notifier
PRIVMSG #netops ‘RECOVERY’ Service: ‘check-plsql’ on RAC2 is ‘OK’ ‘Sun Oct 16 11:43:16 CEST 2011’ Output: ‘PL/SQL Check OK’
QUIT

The above IRC commands are explained here:

http://oreilly.com/pub/h/1963

This is part of the advantages of IRC. It has simple, smtp-like commands which are quite easy to understand and write.

3. Configure nagios / centreon to use the script just written. In this example, I will be concentrating on the centreon version of events

Go to configuration > commands > notifications

Click “Add” and define a command called “host-notify-by-irc”. You can send anything you need via IRC, below is what I used:

$USER1$/irc_msg.sh "’$NOTIFICATIONTYPE$’ Host : ‘$HOSTALIAS$’ IP address: ‘$HOSTADDRESS$’ ‘$HOSTSTATE$’ ‘$LONGDATETIME$’ Output: ‘$HOSTOUTPUT$’"

Important: note how the whole string passed to the script is enclosed in inverted commas (“)

Similarly, define a command called “service-notify-by-irc”. Below is what I used:

$USER1$/irc_msg.sh "’$NOTIFICATIONTYPE$’ Service: ‘$SERVICEDESC$’ Host: ‘$HOSTALIAS$’ IP address: ‘$HOSTADDRESS$’ ‘$SERVICESTATE$’ ‘$LONGDATETIME$’ Output: ‘$SERVICEOUTPUT$’"

Next, create a user which uses the above two methods as notifications. In centreon:

Configuration > Users. Click “add” and fill in the details as appropriate. The important points are that for “Host Notification Commands” you select “host-notify-by-irc”, and for “Services Notification Commands” you select “service-notify-by-irc”

The last step is that for each service you define, you select the user you just defined above as a contact.