Skip to content

Entries tagged "automation".

Dammit, Martin! This is compressed air!

So I previously mentioned I'd knocked up a simple automation tool, for deploying policies (read "scripts") from a central location to a number of distinct machines.

There seemed to be a small amount of interest, so I've written it all up:

  • slaughter - Perl System Administration & Automation tool

Why slaughter? I have no idea. Yesterday evening it made sense, somehow, on the basis it rhymed with auto - (auto as in automation). This morning it made less sense. But meh.

This list of primitives has grown a little and the brief examples probably provide a little bit of flavour.

In short you:

  • Install the package upon a client you wish to manage.
  • When "slaughter" is invoked it will fetch http://example.com/slaughter/default.policy
    • This file may include other policy files via "IncludePolicy" statements.
  • Once all the named policies have been downloaded/expanded they'll be written to a local file.
  • The local file will have Perl-fu wrapped around it such that the Slaughter::linux module is available
    • This is where the definitions for "FetchFile", "Mounts", etc are located.
  • The local file will be executed then removed.

All in all its probably more complex than it needs to be, but I've managed to get interesting things primarily with these new built-in primitives and none of it is massively Debian, or even Linux, specific.

ObSubject: Jaws

 

I don't like this ending...

I've talked before about the minimal way in which I've been using a lot of the available automation tools. I tend to use them to carry out only a few operations:

  • Fetch a file from a remote source.
    • If this has changed run some action.
  • Ensure a package is installed.
    • If this is carried out run some action.
  • Run a command on some simple criterion.
    • E.g. Every day at 11pm run a mirror.

In the pub I've had more than a few chats about how to parse a mini-language and carry these operations out, and what facilities other people use. It'd be almost trivial to come up with a mini-language, but the conclusion has always been that such mini-languages aren't expressive enough to give you the arbitrary flexibility some people would desire. (Nested conditionals and the ability to do things on a per-host, per-day, per-arch basis for example.)

It struck me last night that you could instead cheat. Why not run scripting langues directly on your client nodes? Assume you could write your automation in Ruby or Perl and all you need to do is define a few additional primitives.

For example:

#
#  /policies/default.policy - the file that all clients nodes poll.
#

#
#  Fetch the per-node policy if it exists.
#
FetchPolicy $hostname.policy ;

#
#  Ensure SSH is OK
#
FetchPolicy ssh-server.policy ;

#
#  Or explicitly specify the URL:
#
# FetchPolicy http://example.com/policies/ssh-server.policy ;

#
#  Finally a quick fetch of a remote file.
#
if ( FetchFile(
                Source => "/etc/motd",
                Dest => "/etc/motd",
                Owner => "root",
                Group => "root",
                Mode => "0644" ) )
{

    RunCommand( "id" );
}

This default policy attempts to include some other policies which are essentially perl files which have some additional "admin-esque" primitives. Such as "InstallPackage", "PurgePackage", and "FetchFile".

FetchFile is the only one I've fully implemented, but given a server it will fetch http://server/prefix/files/$FILENAME - into a local file, and will setup the owner/gid/mode. If the fetch succeeded and contents differ from the current contents of the named file (or the current file doesn't exist) it will be moved into place and the function will return true.

On the server side I just have a layout that makes sense:

.
|-- files
|   `-- etc
|       |-- motd
|       |-- motd.silver.my.flat
|       `-- motd.gold
`-- policies
    |-- default.policy
    |-- ssh-server.policy
    `-- steve.policy

Here FetchFile has been implemented to first request /files/etc/motd.gold.my.flat, then /files/etc/motd.gold, and finally the global file /files/etc/motd.

In short you don't want to be forced to write perl which would run things like this:

# install ssh
if ( -e "/etc/apt/sources.list" )
{
  # we're probably debian
  system( "apt-get update" );
  system( "apt-get install openssh-server" );
}

You just want to be able to say "Install Package foo", and rely upon the helper library / primitives being implemented correctly enough to be able to have that work.

I'll probably stop there, but it has given me a fair amount to think about. Not least of which : What are the minimum required primitives to usefully automate client nodes?

ObFilm: Moulin Rouge!