Skip to content

Entries tagged "bytemark".

Symbiosis is wonderful

Symbiosis

Symbiosis is the collective name given to a group of Debian GNU/Linux packages which implement simple virtual hosting. It is developed by my employers Bytemark.

Symbiosis is basically a collection of configuration snippets, code, and libraries which works to offer virtual hosting in a reliable consistent and easy to understand fashion.

You implement hosting for a new domain by merely creating a directory tree. So for example you might configure the hosting for the domain example.com by running:

mkdir -p /srv/example.com/public/htdocs
echo "hello, world" >> /srv/example.com/public/htdocs/index.html

mkdir -p /srv/example.com/mailboxes/webmaster
echo "super-secret" > /srv/example.com/mailboxes/webmaster/password

mkdir -p /srv/example.com/config
echo "3l33t" > /srv/example.com/mailboxes/config/ftp-password

There you are, now http://www.example.com/ and http://example.com/ will work, and you may login to check mail with the email address webmaster@example.com via POP3, IMAP, IMAPS, or POP3S. Finally you can FTP with username example.com and be dropped into the public directory.

The mail handling is very flexible, and the webhosting supports wonderful things.

I don't generally talk about work-stuff explicitly, but we've just made a major new release of the Symbiosis system such that it works upon Squeeze and has lots of IPv6 support out of the box. (Email, DNS, HTTP, Firewalling, FTP etc.)

All in all it is simple, well-documented, and open-source with a reasonably large user-base. More external testers, users, and developers would be a wonderful thing..

Mutt Mailboxes & Idle Hooks?

Mutt is wonderful but I'm starting to get annoyed by its lack of auto-mailbox discovery.

Assuming you use procmail you might deliver mail to ~/Maildir/.foo/ and mutt won't notice that if the directory is created once it starts.

(This is because generally mailboxes are defined via "mailboxes =one =two ..", even if you use a shell snippet it won't get updated unless you re-read configuration, or re-exec mutt).

I wish it were possible to use inotify/dnotify/something magic such that everything beneath ~/Maildir would just work.

(Re-reading mailboxes manually is one solution but it is .. nasty?)

I'm thinking that of all the possible solutions one of the most potentially interesting would be to define a new hook: "idle-hook command .."

That way "command" would be executed every time the client is idle. (This is a distinct state unrelated to IMAP IDLE times.)

Nopte: There is already "mail_check" & "timeout" options. Even running a defined command immediately following the code for mail_check would be reasonable.

Reverse Proxy

I continue to use, love, and enjoy my node.js-based reverse HTTP proxy, and pub discussions seemed to suggest it is a great idea (due to flexibility) but it will never take on because people don't trust node.

I'm almost tempted to re-code it in LUA & C. But I can't help but think that would be a waste of time which would not increase adoption - after all most people use "simple" reverse proxies, and they are well suited by Apache, nginx, or even varnish.

Still no rush I suppose.

In more personal news after living in this flat for 7 years, or so, I'm getting a new bathroom designed and deployed. Good times.

In the meantime I've been steadily watching Stargate SG-1 having recently purchased a box-set of series 1-10. I've just started series six this evening, and I'm enjoying it a lot.

ObQuote: "You have been recruited by the Star League to defend the frontier against Xur and the Ko-Dan armada. " - The Last Starfighter (1984). First film I ever saw at a cinema as a child.

 

Look are you gonna step outside or do I have to drag you?

Over the past couple of months the machine which hosts the Debian Administration website has been struggling with two distinct problems:

The dreaded scheduler bug/issue

The machine would frequently hang with the messages of the form:

Task xxx blocked for more than 120 seconds

This would usually require the application of raised elephants to recover from.

OOM-issues

The system would exhaust the generous 2Gb of memory it possessed, and start killing random tasks until the memory usage fell - at which point the server itself stopped functioning in a useful manner.

Hopefully these problems are now over:

The combination of these two changes should resolve the memory issues, and I've installed a home-made 2.6.31.4 kernel which appears to have corrected the task-blocking scheduler issue.

ObTitle: Bridget Jones: The Edge of Reason

 

There is something evil there

So I've had a hectic few days, and I'm getting close to having caught up with the things that I've been sitting on whilst I've been away.

ObRandom: Several people, independantly, have told me within the past few days that "whilst" is not a real word. it is. End of ..

Some interesting things I've been working upon recently include a fun little firewall tool. Once upon a time I wrote a firewall script which worked like this:

firewall/
`-- incoming.d
    |-- smtp
    |-- ssh
    `-- www
`-- outgoing.d
    |-- ssh
    |-- smtp
    |-- dns
    `-- icmp

When you executed the magic firewall script it would scan the incoming.d directory, and for each file it found lookup the relevant port in /etc/services. These port numbers would then be opened. And at the end you'd just have a "-j DROP".

After a long phone conversation to a colleague on Thursday/Friday of last week I've now reworked this idea anew. There is still the notion of filenames referring to what is allowed for a pair of directories (incoming.d/ + outgoing.d/) but even more flexability and no hardwired use of /etc/servvices.

I guess some ideas are just too simple to give up ..?

Anyway there are a plethora of different firewall applications of varying sophistication and complexity in the world. I don't really want to go out of my way to promote this one - but at the same time it might be a useful idea for somebody?

The next (work) job I have is determining how to make a "kernel" + "kernel-dev" RPM package based on Debian sources. Joy. Actually the more I look around the more fiddly, annoying, and troublesome I suspect this is going to be. Sigh.

ObQuote: The Grudgy

 

So here it is Merry Christmas

Lars Wirzenius recently released, and packaged for Debian, a simple script to make release tarballs. He calls it Unperish.

It makes me wonder how many other people use that kind of system?

Of the top of my head the only similar thing I can recall using is Brad Fitzpatrick's ShipIt - another moduler/plugin-based system (Perl rather than Python this time.)

For my needs I tend to just write a Makefile which has a "dist" target, and then I have a simple script called "release". This runs:

  1. make dist / make release.
  2. creates a gpg signature of the release.
  3. scp's the resulting files to a remote source.

All this is configurable via a per-project .release file.

The configuration files are very simple, the script itself is almost trivial but being able to sit in a random project directory and have a new tarball on my webserver just by typing "release" is enormously useful.

There are times when I think I should make it a mini-project of its own, with the ability to auto-build Debian packages, etc. Other times I just think .. well its a hell of a lot better than my previous ad-hoc solution.

At the very least I think I will make the cosmetic change of updating the script to run "make test" if there is a test/ or t/ directory inside the generated tarball.

In real news - tomorrow I leave for a two week holiday with my partner's parents. Yesterday I got back from a night spent with her in York. The Bytemark staff night out. Lots of fun. Over too soon, but lots of fun.