Steve Kemp's Blog

According to popcon I have just under 1000 users of xen-tools.

That was quite a suprise to discover via a random google search, although I guess there have been a lot of bugs filed against the package during its lifetime.

Funny how some things which start as random hacks (this was originally a quick and dirty hack for a Xen introduction article) become quite useful/popular, whereas other tools which were planned and designed go virtually unnoticed...

Confessor - Terry Goodkind's last novel in the Sword of Truth series.

Brilliant.

Exceptionally Brilliant.

Well worth waiting for, and the annoyance of 'Chainfire' itself which seemed to go nowhere despite its length.

I've been working on updating my online film list since Thursday evening.

I have some code which will convert static data-files containing film entries into a browsable HTML site.

The next job is to actually go through all our DVDs and make sure the lists are correct.

I've updated all our TV shows, and I've made an initial pass at making sure all our films are present but it'll take me a few more days to ensure the lists are completely correct.

In the past I used to browse my list of films via my mobile phone to make sure I didn't buy duplicate films (more than once in the past I had managed to do that!) These days I don't seem to need to, but it is nice for organizing and it appeals to my love of lists..

I'm not sure which is worse, me doing it or Megan taking one look and saying "That's so cool!".

My tool to query apache logfiles via SQL seems suprisingly popular.

Just as a recap the process goes like this:

• Start the shell.
• A temporary SQLite database is created.
• You load any number of apache logfiles into it.
• Then queries may be executed against those records until you exit.
• The temporary database is dropped.

Now it is possible to save and load the SQLite database, so that you don't need to reparse the apache logs each time, that gives a nice speed increase for non-changing files.

By tonight I'll have aliases working for queries so you can bookmark them:

alias refers SELECT distinct(referer) FROM logs

Then in the future the 'refers' command will be available and will run the named query. Neat.

Now that I'm comfortable with SQL queries it just seems so natural, easy, and right to query logfiles this way. I guess that makes me strange.

In the next week I intend to drop the search engine which archives content posted to Planet Debian.

It appears to have very little use, except for myself, and I'm significantly better at bookmarking posts of interest these days.

If you'd like to run your own copy the code is available and pretty trivial to reimplement regardless. There are only two parts:

• Poll and archive content from the planet RSS feed - taking care of duplicates.
• Scanning for /robots.txt upon the source-host, to avoid archiving content which should be "private".

Once you've done that you'll have a database populated with blog entries, and you just need to write a little search script.

ObRandom: In the time it has been running it has archived 15,464 posts!

If you're interested in working upon your CV/Resume, as Otavio Salvador was recently, then I'd highly recommend the xml-resume-library.

It allows you to write your address, previous jobs, and skills as XML then generate PDF, HTML, and plain text format documents via a simple Makefile.

It won't help with clueless agencies that mandate the use of Microsoft Word Documents for submission, so they can butcher your submission and "earn" their fee(s), but otherwise it rocks.

After recently intending to drop the Planet Debian search and recieving complaints that it was/is still useful it looks like there is a good solution.

The code will be made live and official upon the planet debian in the near future.

The DSA team promptly installed the SQLite3 package for me, and I've ported the code to work with it. Once Apache us updated to allow me to execute CGI scripts it'll be moved over, and I'll export the current data to the new database.

In other news I'm going to file an ITP bug against asql as I find myself using it more and more...

This weekend has been an interesting mix of activities. Mostly I've been tweaking my mail filtering service now that it has more users it is more interesting to do that.

The basic process of mail-scanning is pretty simple, but there are some fun things in the mix which make it slightly more fiddly than I'd like.

The basic recipe goes something like this:

• Accept mail.
• Validate the mail is addressed to a domain hosted upon the machine.
• Do the spam filtering / magic (many steps missing here)
  • If the mail should be rejected archive it to a local Maildir folder and bounce it.
  • If the mail should be accepted then forward it to the destination machine.

The archiving of all rejected messages is a big win. It means that if there is a mistake in the handling of any mail we could undo it, retraining the spam database etc. It also provides, via a web page/rss feed, a way for a user to see what a good job the filtering system is doing - by saying "Here's what you would have had ..".

Today I switched the way that the archived mail is displayed via the Web GUI. Previously I used some nasty Maildir parsing code, but now I'm running IMAP upon localhost - so the viewing of messages is a lot more straightforward. (via Net::IMAP::Simple.)

More interestingly, to most readers I'm sure, today I managed to take a new Kite out for flying. A cold and windy day, but lots of fun. There was beer, pies, and near-death!

This was also the second weekend I carried out some painting of my front room. At this rate I'll have painted all four walls of the room in less than two months! (The last time I painted a room it took approximately six months to complete. Move furnuture & paint one wall. Wait several weeks, then repeat until all walls are complete!)

If you try using some of my software, or any software come to think of it, and it doesn't work, or causes you problems then there is a simple solution.

Tell me. I might not be able to fix it immediately, I might not ever be able to fix it. But chances are I can, and if there is a record it'll help others out in the future regardless.

I've bumped into this in the past "Oh yes I tried to use that tool you wrote but it didn't work, so I ended up with something else."

Today I've been mostly unwell. Although I have managed to write some minor new code, and watch a little bit of Doctor Who on DVD.

Recently several people have been ranting about Ruby on Rails. I like it, but I wouldn't use it for personal development in a hurry. Deployment is fiddly, and upgrades are annoying.

But one thing that I utterly condemn Rails for is helping to spread bad paging throughout the online world.

So, what is "bad paging" and why is it important? Well cool URLs don't change, right? "Bad paging" is any user-interface which presents you with a limited view upon a changing list of items which is non-bookmarkable.

Consider the following "list". Assume it represents your view of a collection of items numbering 100+. You may only view ten items at a time; clicking "next", or "previous", to navigate your viewport:

1.  first item
2.  second item
..
10. tenth item

[see next: /start/1] [see prev]

Whats wrong with this picture? It is subtle, but this list is broken. The issue is that when the list grows new items are prepended to the front, yet the navigation is linked to the starting page number.

If that description wasn't clear consider what happens if you want to bookmark the page containing item 11. How can you?

Right now it is at /start/1. If a ten new items are appended to the head of the list then it will instead become /start/2 - as items that are currently numbered 1-10 will be shifted forward to become items 11-20, and and they will be on page /start/1 instead.

The solution is simple enough once you consider what you want to happen:

• Either append items to the end of the list.
  • Such that /start/1 always gives the items 11-20.
• Number the links in the reverse order.

So why does nobody do that? (As a counter example look at my website: Rather than the 'Show previous' items linking you to the changing link /start/1, it instead links you to /start/569 (for example).

My biscuits keep breaking up and falling into my coffee.

Help!

ObQuote: The Philadelphia Story

http://10.print.debian.rocks.twentygototen.org/

ObQuote: Aliens

Busy times, despite being on holiday.

Mostly this has been doing "business" work, and fiddling with self-promotion. But despite this I managed to find time to write some extremely useful new Lisp:

• Sensible (new) window placement for GNU Emacs

Anyway very little time over the coming week will be spent online. All being well. Still enjoying playing with my (loaned) Nokia 770 - maybe I'll get another one of my own eventually.

ObQuote: 30 Days Of Night

There should be a word for those silly little ways you can fool your body & brain. For example recently I've been having trouble with my boiler - so getting hot water is a challenge.

I find myself doing the crazy thing:

• Turn on hot tap(s)
• Stick my hands under them to see if the water is hot.
• Think to myself "Hey it is getting warmer..".
• Realise actually I just imagined it.

Lather, rinse, repeat.

Similarly there are times when you can imaging all kinds of bodily sensations. More than once I've been walking out, or sat at home, convinced that my mobile phone vibrated in my pocket. And it hadn't at all.

I remember, random, conversations with people who agreed they sometimes believe their phones are vibrating when they are not. Seems to be a common thing.

Which begs the question, is this a modern thing? Ten years ago if you had something vibrating against your body you damn well knew about it ... because you were doing it deliberately!

It is only recently that it was possible to have something semi-randomly vibrating against you, without your explicit control. Right?

(OK that sounds rude. It'll be our little secret.)

ObQuote: Godfather (Pt.1)

It is nice when you work for a company where you can say:

"Ice-lolly break..."

The response?

"Me too!"

Tonight has been a productive evening, I guess the ice-lolly helped!

I managed to optimize the storage of rejected SPAM mail for my commercial service. That is something I've been obsessing over recently since the volume of SPAM is currently hovering around 2.5 million messages.

Still I suspect it is only a matter of weeks before I need to expand. The current setup has me using three machines:

• Primary machine runs:
  • Web Application
  • SMTP processing/filtering/delivery
• Secondary machine runs:
  • SMTP processing/filtering/delivery
• Offsite machine:
  • Runs the blog.
  • Runs the support system.
  • Runs the service status page

Ideally I'd like to split that up further so that I have a single machine running the web application (the part the user interacts with), a pair of MX machines, and the offsite machine doing the minimal work it does.

That way the incoming mail will not affect the application at all directly.

Thankfully the split should be trivial. The only hard part is finding a fast webhost that can offer me ~1Gb of RAM, ~1000Gb of disk space, and won't charge much. Ideally around £15/$30 a month. (hahaha! hahaha! ha!)

ObQuote: Léon