Steve Kemp's Blog

For various reasons I've recently been thinking about forums.

Many technical users dislike forums, because they are things that are hard to follow. Even with RSS feeds & etc you need to keep a login and remember to return to see if your post(s) have been answered.

However non-technical users love forums, and from a community-building perspective they're very cheap and easy. Particularly if you manage to appoint moderators from within the comunity.

I currently find myself in a position where I'd kinda like to have a forum package. Something that I can integrate into existing site easily.

Unfortunately most of the "best" forum packages are PHP-based, and have their own complex login, group, and admin facilities. That makes it hard to update them to authenticate against my existing MySQL table(s). (We'll leave my PHP-allergy in the background)

So, once more, I've been contemplating the bad route; create my own forum software. I'm well aware that down that path lies badness madnesss.

Let us recap. What is a forum?

• A forum is an online site.
• With a coarse list of topics.
• Inside each topic is a list of threads.
• Each thread is comprised of a number of (threaded) messages.

Sound familiar? It should if you use email:

• ~/Maildir contains storage for a collection of mailboxes.
• Each mailbox is a course list of topic-specific discussion.
• Each topic is comprised of a number of (threaded) messages.

So, the unthinkable, could we convert (bi/uni-directinally?) from a Maildir hierarchy to an online forum?

Would that make sense? On the face of it. Yes.

There are implementation details - the forum index would be essentially a list of Maildir folders (perhaps "~/Maildir/topic1/.title" would be require to give it a pretty name).

Each thread topic would be a rendered display of the messages in the folder.

So, what are the drawbacks? Well reading Maildir folders gives us threading, and subjects, bodies, etc. But it does mean a fair bit of overhead parsing messages.

(Times like this I remember Hughe. Every time we've gotten together for beer & geekery the topic of an extensible perl-based IMAP server comes up. I'm sure it should be written ..)

I'll wrap this up now. I'm sure I've made the point. There are some details which have impact - Should the forum accept new posts online? Or only via gated email-delivery? Will it work? Should it be Maildir, or IMAP based? Still at least filtering your SPAM would be easy ;)

More questions. Some questions have no answers. Some answers we ignore because we don't like.

I need to sleep.

ObFilm: 007: Quantum of Solace

Bad film. Don't waste your pennies.

So I got bored tonight and figured I'd write a game...

I'm genuinely not sure whether I've seen this concept before, or came up with it myself. I suspect the former. I know that I sat down with the intention of coding this game and knew how it would play and what the mechanics would be.

Having said that though I cannot think of a similar game I've played - though parts are obviously derivitive.

Anyway the aim of the game:

• A (single currently) ball bounces around the screen.
• You draw lines upon the screen, using the mouse, to influence the movement of the ball.
• The level (game) is over when the ball lands in the "exit box".

Thus far the game exists only in the skelital form with the minimum required functionality. There are two modes currently: "easy" & "hard". The hard mode was primarily added to prove to myself that the "leveling" system could work in a fun way.

Feedback welcome. Especially if it can tell me where I'm going wrong with the collision detection - but even if it is to critique my hacked-up SDL coding.

(The only other SDL coding I've done was in C, and was the mousetrap game.)

Obviously the game is written in perl, and I admit nasty perl at that. To play it you'll only need:

apt-get install libsdl-perl

Code:

• http://www.steve.org.uk/Software/tmp/skx.pl

ObQuote: The Chronicles of Riddick

This has been a rather random few days.

• I bought a steam-boat (not really; but close enough.)
• I fell in love with mod_perl

For those of you that don't know it mod_perl is an Apache module which embeds perl into your webserver.

You can use this to write extensions, handlers, and all kinds of fun things in pure perl.

Me? I just changed the beefy CGI script that I use to power a couple of my sites from being plain-CGI to being mod_perl-CGI - that means:

• The same perl engine & copy of my script stays in memory.
• I don't need the fork()/exec() overhead for each incoming rquest.

The downside is that I have to "/etc/init.d/apache2 reload" if I change my script, or any of my custom modules it uses. (I suspect this is something I can fix; I just don't know how yet :)

All that was possible, with zero changes to my applications as I use the CGI::Application framework - lucky? or planned? I'll let you decide ...

In terms of speedup I can now process about 100 requests a second, compared to 10. As reported by the Apache-benchmark tool. Cool.

ObFilm: The Mummy: Tomb of the Dragon Emperor

This has been a rather random few days.

• I bought a steam-boat (not really; but close enough.)
• I fell in love with mod_perl

For those of you that don't know it mod_perl is an Apache module which embeds perl into your webserver.

You can use this to write extensions, handlers, and all kinds of fun things in pure perl.

Me? I just changed the beefy CGI script that I use to power a couple of my sites from being plain-CGI to being mod_perl-CGI - that means:

• The same perl engine & copy of my script stays in memory.
• I don't need the fork()/exec() overhead for each incoming rquest.

The downside is that I have to "/etc/init.d/apache2 reload" if I change my script, or any of my custom modules it uses. (I suspect this is something I can fix; I just don't know how yet :)

All that was possible, with zero changes to my applications as I use the CGI::Application framework - lucky? or planned? I'll let you decide ...

In terms of speedup I can now process about 100 requests a second, compared to 10. As reported by the Apache-benchmark tool. Cool.

ObFilm: The Mummy: Tomb of the Dragon Emperor

OK I'm done with this now, the sift utility has been released.

I think that is a large overlap with imapfilter; but I win because I can write simple rules, rather than any actual code, to perform jobs.

In other news I flew my kite today, and I still like eating Pies: Thank God reading Debian Planet isn't mandatory.

ObQuote: The Godfather

I've updated the IMAP utility that I mentioned previously, which has now been given the name sift. It will accept, and process, a much simpler configuration file format keeping state as it goes.

Here's my updated sample file:

username: blah.bah
password: pas.word

#
#  Comments are fine.
#
folder:livejournal status:new subject:temp mark:read exec:~/bin/notify
folder:foo status:new mark:read
folder:bar status:old exec:/usr/local/bin/record delete

Each line consists of a set of tokens, split by whitespace, which is "executed" in order.

So the first line selects the folder "livejournal", finds messages which are "new", then each message containing "temp" in the subject is marked as read, and the program "notify" is executed once for each match.

Essentially we keep a list of messages as "current" as we process each line, that list of messages is then refined as we move through the line. (When a folder is opened all messages are selected by default.)

As a simple example to delete all the messages contained in a folder we'd use this:

folder:foo delete

To refine that to only delete messages from "fred" we'd say:

folder:foo from:fred delete

(If there were no matches the "delete" action wouldn't occur.)

Consider each line of input a collection of filters each operating on the previous result. Simple to understand, simple to extend with more operations, and simple for me to code!

TODO: Add a "move:xxx" to move a message to folder "xxx", and a bit more polish, then release.

ObQuote: Tron.

So I've written a quick hack. A client-side filter/utility program for working against IMAP servers.

Consider it a general purpose system which is similar to Procmail, but applied after your remote machine has already done the sorting.

Here's a flavour:

<GMail>
  username somebody.like.me
  password yeah.right
</Gmail>

<Folders>
  <livejournal>
        unread exec /usr/local/bin/notify "Livejournal Comment"
        mark read
  </livejournal>

  <inbox>
        mark read
  </inbox>

</Folder>

What does that do? It first of all logs into GMail with the given username and password, then selects two folders:

=livejournal/

For each unread message in the folder it runs the specified command with STDIN being the message body.

Then it marks each new message as "read".

=inbox/

This simple rule just marks all messages as read.

Why? Well I have a bunch of folders on a bunch of gmail accounts and I don't pay attention to them - but some, specific, mails should result in an SMS being sent to me ... so I need to do something clever.

I'm sure with a bit of effort this could be made IMAP-server independent, and could have a more flexible matching system. The simplicity right now comes about primarily because i dont want to parse a config file.

Anyway, suggestions for potential features are welcome. It does what I need as-is, even if it isn't pretty.

ObQuote: Lost Boys

Is there an existing system which will allow me to query Apache logfiles via an SQL string? (Without importing into a database first).

I've found the perl library SQL::YASL - but that has a couple of omissions which mean it isn't ideal for my task:

• It doesn't understand DISTINCT
• It doesn't understand COUNT
• It doesn't understand SUM

Still it did allow me to write a simple shell which works nicely for simple cases:

SQL>LOAD /home/skx/hg/engaging/logs/access.log;
SQL>select path,size from requests where size > 10000;
path size 
/css/default.css 13813 
/js/prototype.js 71261 
/js/effects.js 37872 
/js/dragdrop.js 30645 
/js/controls.js 28980 
/js/slider.js 10403 
/view/messages 15447 
/view/messages 15447 
/recent/messages 25378 

It does mandate the use of a "WHERE" clause, but that was easily fixed with "WHERE 1=1". If I could just have support for count I could do near realtime interesting things...

Then again maybe I should just log directly and not worry about it. I certainly don't want to create my own SQL engine .. it just seems that Perl doesn't have a suitable library already made which is a bit of a shocker!