Skip to content

I understand volunterering is hard

The tail end of this week was mostly spoiled by the discovery that libbeanstalkclient-ruby was not included in Wheezy.

Apparently it was removed because the maintainer had no time, and there were no reverse dependencies - #650308.

Debian maintainers really need to appreciate that no official dependencies doesn't mean a package is unused.

Last year I needed to redesign our companies monitoring software, because we ran out of options that scaled well. I came up with the (obvious) solution:

  • Have a central queue containing jobs to process.
    • e.g. Run a ping-test on host1.example.com
    • e.g. Run an SSH-probe on host99.example.com
    • e.g. Fetch a web-page from https://example3.net/ and test it has some text or a given HTTP status code.
    • (About 15 different test-types are available).
  • Have N workers each pull one job from the queue, execute it, and send the results somewhere.

I chose beanstalkd for my central queue precisely because it was packaged for Debian, had a client library I could use, and seemed to be a good fit. It was a good fit, a year on and we're still running around 5000 tests every minute with 10 workers.

The monitoring tool is called Custodian Custodian, and I think I've mentioned it before here and on the company blog.

It looks like we'll need to re-package the Ruby beanstalk client, and distribute it alongside our project now. That's not ideal, but also not a huge amount of work.

In summary? Debian you're awesome. But libraries shouldn't be removed unless it can't be helped, because you have more users than you know.

Comments On This Entry

  1. [gravitar] paolog

    popcon data for libbeanstalkclient-ruby show 9 users at its peak. Probably many people do not allow popcon stats to be collected on their systems, but that is a good way to let Debian know about its users.

  2. [author] Steve Kemp

    Yes, that's very true. I'll check on my systems later today and make sure I'm sending such data.

  3. [gravitar] Paul Wise

    You might want to reintroduce this package to Debian:

    http://www.debian.org/doc/manuals/developers-reference/pkgs.html#reintroducing-pkgs

    How about packaging Custodian and adding it to Debian? If it was in Debian that would have prevented the removal of beanstalk.

    I also wonder why you didn't notice this issue until now, beanstalk was removed a long time ago.

    Some thoughts on running testing that can also apply to running stable.

    http://bonedaddy.net/pabs3/log/2012/10/29/thoughts-on-debian-testing/

  4. [author] Steve Kemp

    We didn't notice because this is the first time we've tried to install custodian on a pristine Debian system, until now it has been on hosts that were upgraded from Squeeze where the orphaned client-package was still present.

    Custodian is already packaged as .deb releases, and early development moved so quickly there seemed little point in having it in Debian - of course if a Debian-developer wished to upload it we'd support it, happily.