Steve Kemp's Blog

These are my personal goals for the debcamp week:

• Develop xen-tools to fix bugs reported by Henning.
• Unify debootstrap.
• Get involved in discussions of Debian init-systems.

The second item is the one that I'm most interersted in right now, but I'll leave it a day to see if there is any useful feedback or massive objections right now.

The debootstrap problem

Right now there are two versions of debootstrap - the "Debian one" and the "Ubuntu one".

The Ubuntu debootstrap package has support for installing additional suites; dapper, feisty, etc.. (My current understanding is that the Ubuntu debootstrap can still install Etch, Sarge, etc.)

Right now if you want to bootstrap an Ubuntu system on a Debian host you're out of luck unless you install the Ubuntu debootstrap package.

There is no real reason why this should be the case. Either distribution should be able to install the other.

Why I care

I wrote/maintain xen-tools. This package allows you to create Xen guests of different distributions. Most of the time the tool will install a distribution by invoking "debootstrap ...".

It would be nice if this could work upon a plain Debian system such that installing Dapper, Fiesty, etc, was supported.

What We Can't Solve

There are problems with debootstrap which I'm not going to attempt to solve. The most obvious one is that Sarge's debootstrap cannot install Etch.

There are a few open bugs in the Debian BTS which I will triage though, since some of them are safe to close.

The Approach

There are three approaches for this:

The hack

Download the source to Ubuntu's package. Test it can install Sarge, Etch & Lenny. Then upload to Debian.

What I'll do

I'll do three things:

• Triage open bugs on debootstrap.
• I'll compare the two scripts + manpages, to see if they've diverged.
• I'll move the Ubuntu scripts into the Debian package.

The neat way forward

Split debootstrap into :

• debootstrap
  • Compatability package, to pull in the next three:
• debootstrap-common
  • The tool.
• debootstrap-debian
  • Contains the scripts/support for installing Sarge, Etch & Lenny.
• debootstrap-ubuntu
  • Contains the scripts/support for installing Dapper, Feisty, etc.

This second approach should be straightforward. The first thing to do is to test that actual debootstrap script differ little between the two distributions. That should be simple enough.

The next thing is to create the additional packages.

Acceptance?

For this work to be useful it needs support from the maintainers of the Debian & Ubuntu packages. (I guess mostly the Debian maintainers actually.)

Progress

The good news is that between the version of debootstrap in Ubuntu right now, debootstrap-0.3.3.3ubuntu4, and that in Debian sid, debootstrap-0.3.3.3, the code has only one minor change. There is only a minor typo-fix in the manpage too.

So I can move the scripts into the package trivially...

Bootstrapping non-Debian distributions suck.

The only available tool appears to be rpmstrap which quite frankly fails more often than it works.

Since my xen-tools project needs to do carry out this kind of operation I've been pondering the idea of writing a tool which will install CentOS/Fedora/SuSE into a directory, in a similar fashion to debootstrap.

If there's nothing out there that you can point to, then I think that will be my next project.

I can successfully boot Fedora Core 6 & 7 - to the extent that RPM and Yum both work.

This takes in the order of 70Mb and around 2 minutes - assuming the caching mechanism works.

Now I need :

1. A cute name.
2. To abstract the common parts of the code somehow.
3. To get started on SuSE + CentOS.

Today: Hacking. Tomorrow: Security work.

I'm still in the middle of a quandry with regards to revision control.

90% of my open code is hosted via CVS at a central site.

I wish to migrate away from CVS in the very near future, and having ummed and ahhed for a while I've picked murcurial as my system of choice. There is extensive documentation, and it does everything I believe I need.

The close-runner was git, but on balance I've decided to choose mercurial as it wins in a few respects.

Now the plan. I have two options:

• Leave each project in one central site.
• Migrate project $foo to its own location.

e.g. My xen-tools could be hosted at mercurial.xen-tools.org, my blog compiler could live at mercurial.steve.org.uk.

Alternatively I could just leave the one site in place, ignoring the fact that the domain name is now inappropriate.

The problem? I can't decide which approach to go for. Both have plusses and minuses.

Suggestions or rationales welcome - but no holy wars on why any particular revision control system is best...

I guess ultimately it matters little, and short of mass-editing links its 50/50.

After recently intending to drop the Planet Debian search and recieving complaints that it was/is still useful it looks like there is a good solution.

The code will be made live and official upon the planet debian in the near future.

The DSA team promptly installed the SQLite3 package for me, and I've ported the code to work with it. Once Apache us updated to allow me to execute CGI scripts it'll be moved over, and I'll export the current data to the new database.

In other news I'm going to file an ITP bug against asql as I find myself using it more and more...

It bothers me that my Tor usage is less than I'd like because it is just so fiddly.

When it comes to privacy I want to keep things simple, I want to use tor, but I dont want to use it for things that aren't sane.

In practise that means I want to use tor for a small amount of browsing:

• When the host is a.com, b.com, & c.com
• When the traffic is not over SSL.

To do that I have to install privoxy, and use that with a configuration file like this:

# don't forward by default.
forward-socks4   /    .
# don't forward by default, even more so for HTTPS
forward-socks4   :443 .

# but we do want tor on these three sites:
forward-socks4   a.com/       127.0.0.1:9050 .
forward-socks4   b.com/       127.0.0.1:9050 .
forward-socks4   c.com/       127.0.0.1:9050 .

I'm using absolutely nothing else in my Privoxy configuration, so it seems like overkill.

I'd love to hear about a simple rule-based proxy-chaining tool - if there is one out there then I'd love to know about it lazyweb.

If not it shouldn't be too hard to write one with the Net::Proxy & Net::Socks module(s).

<global>
  listen 1234
  no-proxy
</global>

<sites>  
  hostname one.com
  port != 443
  proxy socks localhost 8050
</sites>

<sites>  
  hostname two.com
  port != 443
  proxy socks localhost 8050
</sites>

<sites>
  hostname foo.com
  port = 80
  proxy localhost 8000
</sites>

I spent a while yesterday thinking over the software projects that I'm currently interested in. It is a reasonably short list.

At the time I just looked over the packages that I've got installed and the number of bugs. I'm a little disappointed to see that the bugfixes that I applied to GNU screen have been mostly ignored.

Still I have the day off work on Thursday and Friday this week and would probbly spend it releasing the pending advisories I've got in my queue, and then fixing N bugs in a single package.

The alternative is to build a quick GPG-based mailing list manager.

I'd like a simple system which allowed users to subscribe, and only accepted GPG-signed mails. The subscriber could choose to receive their messages either signed (as-is) by the submitter or encrypted to them.

So to join you'd do something like this:

subscribe foo@example.org [encrypted]
--BEGIN PUBLIC KEY --
...
--ND PUBLIC KEY--

There is the risk, with a large enough number of users, that a list could DOS the host if it had to encrypt each message to each subscribers. But if the submissions were validated as being signed by a user with a known key it should be minimal, unless there is a lot of traffic.

The cases are simple:

• foo-subscribe => Add the user to the list, assuming valid key data found
• foo-unsubscribe => Do the reverse.
• foo:
  • If the message is signed accept and either mail to each recipient, or encrypt on a per-recipient basis.
  • If the message is not signed, or signed by a non-subscriber drop it.

There are some random hacks out there for this, including a mailman patch (did I mention how much I detest mailman yet today?) but nothing recent.

Well a brief post about what I've been up to over the past few days.

An alioth project was created for the maintainance of the bash-completion package. I spent about 40 minutes yesterday committing fixes to some of the low-lying fruit.

I suspect I'll do a little more of that, and then back off. I only started looking at the package because there was a request-for-help bug filed against it. It works well enough for me with some small local additions

The big decision for the bash-completion project is how to go forwards from the current situation where the project is basically a large monolithic script. Ideally the openssh-client package should contain the completion for ssh, scp, etc..

Making that transition will be hard. But interesting.

In other news I submitted a couple of "make-work" patches to the QPSMTPD SMTP proxy - just tidying up a minor cosmetic issues. I'm starting to get to the point where I understand the internals pretty well now, which is a good thing!

I love working on QPSMTPD. It rocks. It is basically the core of my antispam service and a real delight to code for. I cannot overemphasise that enough - some projects are just so obviously coded properly. Hard to replicate, easy to recognise...

I've been working on my own pre-connection system which is a little more specialied; making use of the Class::Pluggable library - packaged for Debian by Sarah.

(The world -> Pre-Connection/Load-Balancing Proxy -> QPSMTPD -> Exim4. No fragility there then ;)

Finally I made a tweak to the Debian Planet configuration. If you have Javascript disabled you'll no longer see the "Show Author"/"Hide Author" links. This is great for people who use Lynx, Links, or other minimal browsers.

TODO:

I'm still waiting for the creation of the javascript project to be setup so that I can work on importing my jQuery package.

I still need to sit down and work through the Apache2 bugs I identified as being simple to fix. I've got it building from SVN now though; so progress is being made!

Finally this weekend I need to sit down and find the time to answer Steve's "Team Questionnaire". Leave it any longer and it'll never get answered. Sigh.

ObQuote: Shooting Fish