Steve Kemp's Blog Writings relating to Debian & Free Software

A transient home-directory?

Wednesday, 25 November 2015

For the past few years all my important work has been stored in git repositories. Thanks to the mr tool I have a single configuration file that allows me to pull/maintain a bunch of repositories with ease.

Having recently wiped & reinstalled a pair of desktop systems I'm now wondering if I can switch to using a totally transient home-directory.

The basic intention is that:

  • Every time I login "rm -rf $HOME/*" will be executed.

I see only three problems with this:

  • Every time I login I'll have to reclone my "dotfiles", passwords, bookmarks, etc.
  • Some programs will need their configuration updated, post-login.
  • SSH key management will be a pain.

My dotfiles contain my my bookmarks, passwords, etc. But they don't contain setup for GNOME, etc.

So there might be some configuration that will become annoying - For example I like "Ctrl-Alt-t" to open a new gnome-terminal command. That's configured on each new system I login to the first time.

My images/videos/books are all stored beneath /srv and not in my home directory - so the only thing I'll be losing is program configuration, caches, and similar.

Ideally I'd be using a smartcard for my SSH keys - but I don't have one - so for the moment I might just have to rsync them into place, but that's grossly bad.

I'll be interesting to see how well this works out, but I see a potential gain in portability and discipline at the very least.



lumail2 nears another release

Monday, 16 November 2015

I'm pleased with the way that Lumail2 development is proceeding, and it is reaching a point where there will be a second source-release.

I've made a lot of changes to the repository recently, and most of them boil down to moving code from the C++ side of the application, over to the Lua side.

This morning, for example, I updated the handing of index.limit to be entirely Lua based.

When you open a Maildir folder you see the list of messages it contains, as you would expect.

The notion of the index.limit is that you can limit the messages displayed, for example:

  • See all messages: Config:set( "index.limit", "all")
  • See only new/unread messages: Config:set( "index.limit", "new")
  • See only messages which arrived today: Config:set( "index.limit", "today")
  • See only messages which contain "Steve" in their formatted version: Config:set( "index.limit", "steve")

These are just examples that are present as defaults, but they give an idea of how things can work. I guess it isn't so different to Mutt's "limit" facilities - but thanks to the dynamic Lua nature of the application you can add your own with relative ease.

One of the biggest changes, recently, was the ability to display coloured text! That was always possible before, but a single line could only be one colour. Now colours can be mixed within a line, so this works as you might imagine:

Panel:append( "$[RED]This is red, $[GREEN]green, $[WHITE]white, and $[CYAN]cyan!" )

Other changes include a persistant cache of "stuff", which is Lua-based, the inclusion of at least one luarocks library to parse Date: headers, and a simple API for all our objects.

All good stuff. Perhaps time for a break in the next few weeks, but right now I think I'm making useful updates every other evening or so.

| No comments


lumail2 approaches readiness

Thursday, 5 November 2015

So the work on lumail2 is going well, and already I can see that it is a good idea. The main reason for (re)writing it is to unify a lot of the previous ad-hoc primitives (i.e. lua functions) and to try and push as much of the code into Lua, and out of C++, as possible. This work is already paying off with the introduction of new display-modes and simpler implementation.

View modes are an important part of lumail, because it is a modal mail-client. You're always in one mode:

  • maildir-mode
    • Shows you lists of Maildir-folders.
  • index-mode
    • Shows you lists of messages inside the maildir you selected.
  • message-mode
    • Shows you a single message.

This is nothing new, but there are two new modes:

  • attachment-mode
    • Shows you the attachments associated with the current message.
  • lua-mode
    • Shows you your configuration-settings and trivia.

Each of these modes draws lines of text on the screen, and those lines consist of things that Lua generated. So there is a direct mapping:

ModeLua Function
maildirfunction maildir_view()
indexfunction index_view()
messagefunction message_view()
luafunction lua_view()

With that in mind it is possible to write a function to scroll to the next line containing a pattern like so:

function find()
   local pattern = Screen:get_line( "Search for:" )

   -- Get the global mode.
   local mode = Config:get("global.mode")

   -- Use that to get the lines we're currently displaying
   loadstring( "out = " .. mode .. "_view()" )()

   -- At this point "out" is a table containing lines that
   -- the current mode wishes to display.

    -- .. do searching here.

Thus the whole thing is dynamic and mode-agnostic.

The other big change is pushing things to lua. So to reply to an email, populating the new message, appending your ~/.signature, is handled by Lua. As is forwarding a message, or composing a new mail.

The downside is that the configuration-file is now almost 1000 lines long, thanks to the many little function definitions, and key-binding setup.

At this rate the first test-release will be out at the weekend, but API documentation, and sample configuration file might make interesting reading until then.

| No comments


It begins - a new mail client, with lua scripting

Monday, 26 October 2015

Once upon a time I wrote a mail-client, which worked in the console directly via Maildir manipulation.

My mail client was written in C++, and used Lua for scripting unlike clients such as mutt, alpine, and similar alternatives which don't have complete scripting support.

I've pondered several times whether to restart this project, but I think it is the right thing to do.

The original lumail client has a rich API, but it is very ad-hoc and random. Functions were added where they seemed like a good idea, but with no real planning, and although there are grouped functions that operate similarly there isn't a lot of consistency. The implementation is clean in places, elegant in others, and horrid in yet more parts.

This time round everything is an object, accessible to Lua, with Lua, and for Lua. This time round all the drawing-magic is will be written in Lua.

So to display a list of Maildirs I create a bunch of objects, one for each Maildir, and then the Lua function Maildir.to_string is called. That function looks like this:

-- This method returns the text which is displayed when a maildir is
-- to be show in maildir-mode.
function Maildir.to_string(self)
   local total  = self:total_messages()
   local unread = self:unread_messages()
   local path   = self:path()

   local output = string.format( "[%05d / %05d] - %s", unread, total, path );

   if ( unread > 0 ) then
      output = "$[RED]" .. output

   if ( string.find( output, "Automated." ) ) then
      output = strip_colour( output )
      output = "$[YELLOW]" .. output

   return output

The end result is something that looks like this:

[00001 / 00010 ] -
[00000 / 00023 ] - Automated.root

The formatting can thus be adjusted clearly, easily, and without hacking the core of the client. Providing I implement the apporpriate methods to the Maildir object.

It's still work in progress. You can view maildirs, view indexes, and view messages. You cannot reply, forward, or scroll properly. That said the hard part is done now, and I'm reasonably happy with it.

The sample configuration file is a bit verbose, but a good demonstration regardless.

See the code, if you wish, online here:



Robbing Peter to pay Paul, or location spoofing via DNS

Saturday, 17 October 2015

I rarely watched TV online when I was located in the UK, but now I've moved to Finland with appalling local TV choices it has become more common.

The biggest problem with trying to watch BBC's iPlayer, and similar services, is the location restrictions.

Not a huge problem though:

  • Rent a virtual machine.
  • Configure an OpenVPN server on it.
  • Connect from $current-country to it.

The next part is the harder one - making your traffic pass over the VPN. If you were simple you'd just say "Send everything over the VPN". But that would slow down local traffic, so instead you have to use trickery.

My approach was just to run a series of routing additions, similar to this (except I did it in the openvpn configuration, via pushed-routes):

ip -4 route add .... dev tun0

This works, but it is a pain as you have to add more and more routes. The simpler solution which I switched to after a while was just configuring mitmproxy on the remote OpenVPN end-point, and then configuring that in the browser. With that in use all your traffic goes over the VPN link, if you enable the proxy in your browser, but nothing else will.

I've got a network device on-order, which will let me watch netflix, etc, from my TV, and I'm lead to believe this won't let you setup proxies, or similar, to avoid region-bypass.

It occurs to me that I can configure my router to give out bogus DNS responses - if the device asks for "" it can return - which is the remote host running the proxy.

I imagine this will be nice and simple, and thought I was being clever:

  • Remote OpenVPN server.
  • MITM proxy on remote VPN-host
    • Which is basically a transparent HTTP/HTTPS proxy.
  • Route traffic to it via DNS.
    • e.g. For any DNS request, if it ends in return

Because I can handle DNS-magic on the router I can essentially spoof my location for all the devices on the internal LAN, which is a good thing.

Anyway I was reasonably pleased with the idea of using DNS to route traffic over the VPN, in combination with a transparent proxy. I was even going to blog about it, and say "Hey! This is a cool idea I've never heard of before".

Instead I did a quick google(.fi) and discovered that there are companies offering this as a service. They don't mention the proxying bit, but it's clearly what they're doing - for example OverPlay's SmartDNS.

So in conclusion I can keep my current setup, or I can use the income I receive from DNS hosting to pay for SmartDNS, or other DNS-based location-fakers.

Regardless. DNS. VPN. Good combination. Try it if you get bored.



So about that idea of using ssh-keygen on untrusted input?

Monday, 12 October 2015

My previous blog post related to using ssh-keygen to generate fingerprints from SSH public keys.

At the back of my mind was the fear that running the command against untrusted, user-supplied, keys might be a bad plan. So I figured I'd do some fuzzing to reassure myself.

The most excellent LWN recently published a piece on Fuzzing with american fuzzy lop, so with that to guide me I generated a pair of SSH public keys, and set to work.

Two days later I found an SSH public key that would make ssh-keygen segfault, and equally the SSH client (same parser), so that was a shock.

The good news is that my Perl module to fingerprint keys is used like so:

my $helper = SSHKey::Fingerprint->new( key => "ssh ...." );
if ( $helper->valid() ) {
   my $fingerprint = $helper->fingerprint();

The validity-test catches my bogus key, so in my personal use-cases this OK. That said it's a surprise to see this:

skx@shelob ~ $ ssh -i 
Segmentation fault

Similarly running "ssh-keygen -l -f ~/" results in an identical segfault.

In practice this is a low risk issue, hence mentioning it, and filing the bug-report publicly, even if code execution is possible. Because in practice how many times do people fingerprint keys from unknown sources? Except for things like githubs key management page?

Some people probably do it, but I assume they do it infrequently and only after some minimal checking.

Anyway we'll say this is my my first security issue of 2015, we'll call it #roadhouse, and we'll get right on trademarking the term, designing the logo, and selling out for all the filthy filthy lucre ;)



Generating fingerprints from SSH keys

Wednesday, 7 October 2015

I've been allowing users to upload SSH public-keys, and displaying them online in a form. Displaying an SSH public key is a pain, because they're typically long. That means you need to wrap them, or truncate them, or you introduce a horizontal scroll-bar.

So rather than displaying them I figured I'd generate a fingerprint when the key was uploaded and show that instead - This is exactly how github shows your ssh-keys.

Annoyingly there is only one reasonable way to get a fingerprint from a key:

  • Write it to a temporary file.
  • Run "ssh-keygen -lf temporary/file/name".

You can sometimes calculate the key via more direct, but less obvious methods:

awk '{print $2}' ~/.ssh/ | base64 -d | md5sum

But that won't work for all key-types.

It is interesting to look at the various key-types which are available these days:

mkdir ~/ssh/
cd ~/ssh/
for i in dsa ecdsa ed25519 rsa rsa1 ; do
  ssh-keygen -P "" -t $i -f ${i}-key

I've never seen an ed25519 key in the wild. It looks like this:

$ cat ~/ssh/
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMcT04t6UpewqQHWI4gfyBpP/ueSjbcGEze22vdlq0mW skx@shelob

Similarly curve-based keys are short too, but not as short:

ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLTJ5+  \
 rWoq5cNcjXdhzRiEK3Yq6tFSYr4DBsqkRI0ZqJdb+7RxbhJYUOq5jsBlHUzktYhOahEDlc9Lezz3ZUqXg= skx@shelob

Remember what I said about wrapping? Ha!

Anyway for the moment I've hacked up a simple perl module SSH::Key::Fingerprint which will accept a public key and return the fingerprint, as well as validating the key is well-formed and of a known-type. I might make it public in the future, but I think the name is all wrong.

The only code I could easily find to do a similar job is this node.js package, but it doesn't work on all key-types. Shame.

And that concludes this weeks super-happy fun-time TODO-list item.



All about sharing files easily

Sunday, 13 September 2015

Although I've been writing a bit recently about file-storage, this post is about something much more simple: Just making a random file or two available on an ad-hoc basis.

In the past I used to have my email and website(s) hosted on the same machine, and that machine was well connected. Making a file visible just involved running ~/bin/publish, which used scp to write a file beneath an apache document-root.

These days I use "my computer", "my work computer", and "my work laptop", amongst other hosts. The SSH-keys required to access my personal boxes are not necessarily available on all of these hosts. Add in firewall constraints and suddenly there isn't an obvious way for me to say "Publish this file online, and show me the root".

I asked on twitter but nothing useful jumped out. So I ended up writing a simple server, via sinatra which would allow:

  • Login via the site, and a browser. The login-form looks sexy via bootstrap.
  • Upload via a web-form, once logged in. The upload-form looks sexy via bootstrap.
  • Or, entirely seperately, with HTTP-basic-auth and a HTTP POST (i.e. curl)

This worked, and was even secure-enough, given that I run SSL if you import my CA file.

But using basic auth felt like cheating, and I've been learning more Go recently, and I figured I should start taking it more seriously, so I created a small repository of learning-programs. The learning programs started out simply, but I did wire up a simple TOTP authenticator.

Having TOTP available made me rethink things - suddenly even if you're not using SSL having an eavesdropper doesn't compromise future uploads.

I'd also spent a few hours working out how to make extensible commands in go, the kind of thing that lets you run:

cmd sub-command1 arg1 arg2
cmd sub-command2 arg1 .. argN

The solution I came up with wasn't perfect, but did work, and allow the seperation of different sub-command logic.

So suddenly I have the ability to run "subcommands", and the ability to authenticate against a time-based secret. What is next? Well the hard part with golang is that there are so many things to choose from - I went with gorilla/mux as my HTTP-router, then I spend several hours filling in the blanks.

The upshot is now that I have a TOTP-protected file upload site:

publishr init    - Generates the secret
publishr secret  - Shows you the secret for import to your authenticator
publishr serve   - Starts the HTTP daemon

Other than a lack of comments, and test-cases, it is complete. And stand-alone. Uploads get dropped into ./public, and short-links are generated for free.

If you want to take a peak the code is here:

The only annoyance is the handling of dependencies - which need to be "go got ..". I guess I need to look at godep or similar, for my next learning project.

I guess there's a minor gain in making this service available via golang. I've gained protection against replay attacks, assuming non-SSL environment, and I've simplified deployment. The downside is I can no longer login over the web, and I must use curl, or similar, to upload. Acceptible tradeoff.



The Jessie 8.2 point-release broke for me

Monday, 7 September 2015

I have about 18 personal hosts, all running the Jessie release of Debian GNU/Linux. To keep up with security updates I use unattended-upgrades.

The intention is that every day, via cron, the system will look for updates and apply them. Although I mostly expect it to handle security updates I also have it configured such that point-releases will be applied by magic too.

Unfortunately this weekend, with the 8.2 release, things broke in a significant way - The cron deamon was left in a broken state, such that all cronjobs failed to execute.

I was amazed that nobody had reported a bug, as several people on twitter had the same experience as me, but today I read through a lot of bug-reports and discovered that #783683 is to blame:

  • Old-cron runs.
  • Scheduled unattended-upgrades runs.
  • This causes cron to restart.
  • When cron restarts the jobs it was running are killed.
  • The system is in a broken state.

The solution:

# dpkg --configure -a
# apt-get upgrade

I guess the good news is I spotted it promptly, with the benefit of hindsight the bug report does warn of this as being a concern, but I guess there wasn't a great solution.

Anyway I hope others see this, or otherwise spot the problem themselves.


In unrelated news the seaweedfs file-store I previously introduced is looking more and more attractive to me.

I reported a documentation-related bug which was promptly handled, even though it turned out I was wrong, and I contributed CIDR support to whitelisting hosts which was merged in well.

I've got a two-node "cluster" setup at the moment, and will be expanding that shortly.

I've been doing a lot of little toy-projects in Go recently. This weekend I was mostly playing with the message-bus, and tying it together with sinatra.

| 1 comment.


Making an old android phone useful again

Thursday, 13 August 2015

I've got an HTC Desire, running Android 2.2. It is old enough that installing applications such as thsoe from my bank, etc, fails.

The process of upgrading the stock ROM/firmware seems to be:

  • Download an unsigned zip file, from a shady website/forum.
  • Boot the phone in recovery mode.
  • Wipe the phone / reset to default state.
  • Install the update, and hope it works.
  • Assume you're not running trojaned binaries.
  • Hope the thing still works.
  • Reboot into the new O/S.

All in all .. not ideal .. in any sense.

I wish there were a more "official" way to go. For the moment I guess I'll ignore the problem for another year. My nokia phone does look pretty good ..



Spiral Logo


Recent Posts

Recent Tags


RSS Feed

  • Subscribe to feed