I recently had a project where I used a combination of nagios / livecheck and Apache’s CGI to create a lightweight Nagios Dashboard for the NOC team. It was quite a simple process, much aided by the fact that the CGI gateway allows you to use any programming language, including my favorite bash.
The project took off smoothly, with apache running CGI bash scripts, which in turn piped livecheck queries to nagios via “unixcat” (a livecheck binary which pipes queries into a unix socket where the livecheck module lives). To present this in an easier way:
client request —-> apache cgi —–> bash script ——> unixcat ——> livecheck unix socket ——-> nagios.
Livecheck makes an excellent job of providing an amazingly fast, easy and simple API to the current nagios check states. It is clearly a much superior solution to using nagios + mysql. The only problem I ran across was that on one of our servers, the “unixcat” program was refusing to close, causing a very large number of unixcat processes to populate the CPU and hog resources. This meant the server went under a massive load (about 30 from “top” readings). After some hunting, I discovered that I had neglected to set the suid permission on the CGI scripts in my /var/www/cgi-bin directory. After using the chmod +s command to set the SUID attribute on the scripts, the system correctly started closing off the finished unixcat programs and freeing up resources. The load immediately fell and the dashboard was much snappier when loading.
Further Reading Links
An intro to apache CGI can be found here: http://httpd.apache.org/docs/2.0/howto/cgi.html
The livecheck homepage: http://mathias-kettner.de/checkmk_livecheck.html
David Vassallo's Blog 