I’ve been using monit on my Debian servers for a long time now.
For those that don’t know monit is a simple monitoring service which allows you to check that processes are running, automatically restarting them on failure, etc.
When I first started using monit I manually added all the checks to /etc/monit/monitrc – but that soon became unwieldy, even with only three machines to maintain.
Adding new software meant I had to keep updating the file, and because each server ran different services things couldn’t be shared. Not good.
After I while I figured I’d create /etc/monit/includes and just drop files into there – that way I could include them:
# /etc/monit/monitrc include /etc/monit/include/*.monit
Now to monitor a service I’d just create the appropriate file with a .monit suffix. All was good.
Until I needed to start monitoring different services on different systems where adding and removing the files got to be painful. Surely there must be a better way I wondered to myself?
Now there is.
I’ve come up with a nice approach (perhaps the obvious one to everybody else?):
I now have two directories:
The master.d can have all your *.monit files dropped into it, regardless of whether they apply to the given host or not.
To actually monitor a service you’d symlink master.d/foo.monit to include.d/foo.monit – In much the same way as you use symlinks to handle Apache sites and modules in Apache2.x
Now the really clever bit is that you write a (perl) script to automatically parse each .monit file and only link in the ones which apply to this host.
That means you can use cfengine to copy everything into the master.d/ file – and not care if some tests only apply to other hots. (e.g. You can crteate apache2.monit – and that will be symlinked in if apache2 is installed on the current host, but ignored otherwise.)
For the moment my parsing and detection code is very simple, but it is working.
In the future I guess I could create foo.monit and foo.package – then only link in the file foo.monit if the Debian package contained in foo.package was installed.
I will attempt to catch up on mail tomorrow…