Julius' Blog (en)

"Shouting in the Dark"

2012-02-21T15:56:00+01:00

Al Jazeera's documentary about the Bahrain protests, Shouting in the Dark, has received a prestigeous award. The documentary is really worth watching.

I found the insidious practices of the Bahraini prince – and the Arab league's suport – extremely unnerving.

C Programming Exam Fail

2012-02-17T15:26:00+01:00

I just wrote an exam for the course Technische Informatik III which was about operating systems and network communication. In the exercises throughout the semster, we had to program in C a lot. Naturally, in the exam was one task about interpreting what a C program does.

It was really simple: Listening on a UDP socket and print incoming packets along with source address and port. The program looked somewhat like this (from what I remember; also some things were done in a not so clever way on the exercise sheet, and they had obfuscated the variable names to a non-descriptive a, b, etc.):

#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <error.h>

int main(int argc, char *argv[])
{
    int sockfd;
    struct sockaddr_in listen, incoming;
    socklen_t incoming_len;

    char buf[1024];
    int len; /* of received data */

    /* listen on 0.0.0.0:5000 */
    listen.sin_family = AF_INET;
    listen.sin_addr.s_addr = INADDR_ANY;
    listen.sin_port = htons(5000);

    if((sockfd = socket(AF_INET, SOCK_DGRAM, 0)) == -1)
        perror("socket");

    if(bind(sockfd, (struct sockaddr *) &listen, sizeof(listen)) == -1)
        perror("bind");

    while(1) {
        len = recvfrom(sockfd, buf, 1024, 0, (struct sockaddr *) &incoming,
                        &incoming_len);
        buf[len] = '\0';
        printf("from %s:%d: \"%s\"\n", inet_ntoa(incoming.sin_addr),
                ntohs(incoming.sin_port), buf);
    }
}

I lol'd so hard when I saw this. It's a classic off-by-one error. (Can you spot it, too?)

If you want to store x bytes of data in a string, reserve x+1 bytes for the NULL termination character. Here, if you send a message that is exactly 1024 bytes long (or longer, as it'll get truncated), buf[len] will actually be the 1025^th byte. Which might just be anything.

And those guys want to teach network and filesystem programming – hilarious. :-D

minimizing Linux filesystem cache effects

2012-02-09T23:43:00+01:00

Last weekend I toyed around a bit and tried to write a shared object library that can be used via LD_PRELOAD to minimize the effect a program has on the Linux filesystem cache.

Basically the use case is that you have a productive system running, and you don't want your backup script to fill the filesystem cache with mostly useless information at night (files that were cached should stay cached). I didn't test whether this brings measurable improvements yet.

The coding was really fun and provided me with yet another insight how the simple concept of file descriptors in UNIX is just great. (GNU software is tough, though: I got stuck once, and found help on Stackoverflow, which I had never used before.)

shredding

2012-01-30T14:09:00+01:00

I'm currently shredding my old X41's hard drive, because I want to sell it (if you are interested, contact me). I'm overwriting it with zeros, ten passes:

$ shred -vfz -n 10 /dev/sda

Luckily, the disk was fully encrypted all the time. So it's just a precaution.

Ten Years of Vim

2012-01-30T02:17:00+01:00

About ten years ago, I began using Vim. Since about eight years ago, I have been using Vim for every email, every piece of code, literally every text I write. Today, I want to write a short text about how I came to use Vim and what I like about it.

I don't really remember when I first used Vim. It must have been around the time when I was programming PHP a lot. I had access to a "real" computer at home – running Windows XP – in 2002 for the first time; before that, I could only use older Macintoshs. It's typical for first-time Vi users to stumble into believing – by hear-say, I guess – that it is indeed a really superior editor, until they try it out the first time and can't even save, because they don't know how to. That were my first experiences too, probably.

Anyhow, at some point in time I ditched PHP Zend Studio for SciTE. Later, I got to know Vim (i.e., by reading a tutorial about it and actually understanding it) and was instantly hooked. Probably, the guys over at #html.de talked me into it. Ironically, I used Vim before I ever used a UNIX-like operating system.

In my Vim learning curve, I identify seven important advances:

Understanding the Modes Concept. – This, of course, is something everybody needs to grok. It's fairly straight-forward, once you think about it.
Understand the Visual Mode and Yank/Paste. – Line-wise selection already gives you more power than a regular editor when moving code.
Understand Mappings and Macros. – Even today I am amazed how few people automate things. If it's one line, do it manually. If it's three lines, carefully think about the task while recording a macro for it!
Unterstanding Windows. – Multiple files and stuff.
Consequently using [h], [j], [k], [l]. – This actually was a much bigger step that you might think. I went to great lengths to achive this: I configured the arrow mapping to :echoerr a message. Today, I configure all programs to use Vim key bindings, especially for horizontal and vertical navigation. It's the first thing to do. I only use the arrow keys for Mplayer seeking.
Using Text Objects. – See :help text-objects, if you don't know about them.
Switching to a US keyboard layout. – Once you do this, all the Vim commands begin to make sense. (I used a German layout before.)

Steps 1–5 happened in the first two years. The text object only came with more recent Vim development, and I'm not quite sure when I adopted them. Learning the US layout was around 2006, maybe.

When I switched to using Debian in 2004, using Vim for all tasks already felt natural. Of course, at that point I finally came to understand Vim not merely as a text editor, but as a philosophy. And that is what fascinates me to this day: The Vi way of editing text is much more than a set of clever key bindings. It's a language.

In a way, I'm really professional at using Vim. If I think of the tasks I do, I suspect there are very few superfluous keys I press during editing. I have acquired a really good intuition of how to skip to a particular line, to a particular function parameter or a certain word in a sentence. (I use [H], [M], [L] for global on-screen navigation a lot, and I heavily use the [f], [t], [F] and [T] jump commands.) Just as you don't actually think about the letters you type when you become a good typist, I don't think about what command keys I press in Normal mode. I just press them, and the cursor magically moves around to where my eyes rest. This is good.

On the other hand, I am just using core Vim features, most of which are already found in original Vi implementations. My really conservative .vimrc change history shows that I pretty much settled my editing habits. – But: I have never used a third-party plugin before. Strange as it may sound, I never felt the urge to do so. Command-T certainly looks like it could be of use; however, I usually start a new Vim instance and go with the Z Shell completion, which I suspect to be superior in more than one way, to find the file(s). – Thus I must acknowledge that there might be vast possibilities yet do discover. (Oh, and while confessing, there's another big one: I have never used Emacs. All I know about it is hear-say.)

For keyboard enthusiasts, there are two quirks with Vim: It mainly relies on Escape for mode switches, and the keys for many combinations are aligned for QWERTY layouts. There's just no way around it: while [c] and [d] are mnemonic for cut and delete, [h], [j], [k], [l] simply aren't. There's no way justify their use when switching to Dvorak, and that's why I didn't (switch). I also once tried mapping [j][j] to Escape, or using the Caps Lock key as Escape replacement; I can't really stick to using it. (I also stick to calling vim on the command line instead of a shorter alias. It is the fourth most command I type, after sudo, git, and man.)

For me, text editing is equal to using Vim. I feel like a four-year old moving a mouse when I'm forced to use another editor on other people's computers. And because text editing is really clumsy with regular text editors, I no longer wonder why people don't really bother to correct errors: the effort is just not worth it.

If I had to sum up the difference between Vim and other editors in one sentence, it is this: While other editors are great for creating text, Vim is also great at manipulating text. And text manipulation, for most programmers and authors, is what it's all about.

:wq

vlock and suspend to ram

2012-01-20T19:43:00+01:00

I've had weird race conditions when using vlock together with s2ram. It appears suspend to ram wants to switch VTs, while vlock hooks into the switch requests and explicitly disables them. So some of the time, the machine would not suspend, while at other times, vlock wouldn't be able to acquire the VT.

To solve this, I wrote a simple vlock plugin, which simply clears the lock mechanism, writes mem to /sys/power/state and later reinstates the locking mechanism. This plugin is called after all and new. Thus, the screen will be locked properly before suspending.

Here's my suspend.c:

#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

/* Include this header file to make sure the types of the dependencies
 * and hooks are correct. */
#include "vlock_plugin.h"
#include "../src/console_switch.h"

const char *succeeds[] = { "all", "new", NULL };
const char *depends[] =  { "all", "new", NULL };

bool vlock_start(void __attribute__ ((__unused__)) **ctx_ptr)
{
    int fd;

    unlock_console_switch();

    if((fd = open("/sys/power/state", O_WRONLY)) != -1) {
        if(write(fd, "mem", 3) == -1)
            perror("suspend: write");
        close(fd);
    }

    lock_console_switch();

    return true;
}

Simply paste it to the vlock modules folder, make suspend.so and copy it to /usr/lib/vlock/modules. I now invoke it like this:

env VLOCK_PLUGINS="all new suspend" vlock

Xorg, really?!

2012-01-20T13:27:00+01:00

Are you fucking kidding me? You reintroduce broken behaviour that possibly has devastating security consequences and and make it the default?! Yeah I agree the "usual" X server locking approach is not the best way to do it – but to knowingly smash the security of people's computers on a grand scale... that's priceless.

(My locking solution is env USER=feh vlock -a -n, again.)

Update: Why it happened

zsh: complete words from tmux pane

2012-01-19T20:53:00+01:00

Today I wrote a rather cool Z-Shell completion function: It will present all words that are found in the current tmux pane in a zsh completion menu. That means you can actually complete words from the output of commands that you just executed. (In a way it's a little bit like the keeper function, without the overhead of remembering to call keeper in the first place.)

The code below defines two keybindings:

Ctrl-X t to do a prefix completion: only words from the pane that share the same prefix will be presented
Ctrl-X Ctrl-X to do a "find stuff like crazy" completion. If you see the output and just enter something from the middle of the word, it'll just as well complete. For example, if you see 176.9.247.89 somewhere in the pane, try typing .9 and hitting Ctrl-X twice. It'll complete to that IP address.

Here's the code:

_tmux_pane_words() {
  local expl
  local -a w
  if [[ -z "$TMUX_PANE" ]]; then
    _message "not running inside tmux!"
    return 1
  fi
  w=( ${(u)=$(tmux capture-pane \; show-buffer \; delete-buffer)} )
  _wanted values expl 'words from current tmux pane' compadd -a w
}

zle -C tmux-pane-words-prefix   complete-word _generic
zle -C tmux-pane-words-anywhere complete-word _generic
bindkey '^Xt' tmux-pane-words-prefix
bindkey '^X^X' tmux-pane-words-anywhere
zstyle ':completion:tmux-pane-words-(prefix|anywhere):*' completer _tmux_pane_words
zstyle ':completion:tmux-pane-words-(prefix|anywhere):*' ignore-line current
zstyle ':completion:tmux-pane-words-anywhere:*' matcher-list 'b:=* m:{A-Za-z}={a-zA-Z}'

How does it work? _tmux_pane_words will just capture the current pane's contents (capture-pane), print out the buffer that contains it (show-buffer) and then delete it again (delete-buffer). – The rest of the magic happens via Zsh's excellent completion mechanisms.

See it in action (after typing spm^X^X):

trying pthreads

2012-01-17T01:37:00+01:00

Today I played around with POSIX threads a little. In an assignment, we have to implement a very, very simple webserver that does asynchronous I/O. Since it should perform well, I thought I'd not only serialize I/O, but also parallelize it.

So there's a boss that just accepts new inbound connections and appends the fds to a queue:

clientfd = accept(sockfd, (struct sockaddr *) &client, &client_len);
if(clientfd == -1)
    error("accept");
new_request(clientfd);

The new_request function in turn appends it to a queue (of size TODOS = 64), and emits a cond_new signal for possibly waiting workers:

pthread_mutex_lock(&mutex);
while((todo_end + 1) % TODOS == todo_begin) {
    fprintf(stderr, "[master] Queue is completely filled; waiting\n");
    pthread_cond_wait(&cond_ready, &mutex);
}
fprintf(stderr, "[master] adding socket %d at position %d (begin=%d)\n",
    clientfd, todo_end, todo_begin);
todo[todo_end] = clientfd;
todo_end = (todo_end + 1) % TODOS;
pthread_cond_signal(&cond_new);
pthread_mutex_unlock(&mutex);

The workers (there being 8) will just emit a cond_ready, possibly wait until a cond_new is signalled, and then extract the first client fd from the queue. After that, a simple function involving some reads and writes will handle the communication on that fd.

pthread_mutex_lock(&mutex);
pthread_cond_signal(&cond_ready);
while(todo_end == todo_begin)
    pthread_cond_wait(&cond_new, &mutex);
clientfd = todo[todo_begin];
todo_begin = (todo_begin + 1) % TODOS;
pthread_mutex_unlock(&mutex);

// handle communication on clientfd

(Full source is here: webserver.c.)

Now this works pretty well and is fairly easy. I'm not very experienced with threads, though, and run into problems when I do massive parallel requests.

If I run ab, the Apache Benchmark tool with 10,000 requests, 1,000 concurrent, on the webserver it'll go up to 9000-something requests and then lock up.

$ ab -n 10000 -c 1000 http://localhost:8080/index.html
...
Completed 8000 requests
Completed 9000 requests
apr_poll: The timeout specified has expired (70007)
Total of 9808 requests completed

The webserver is blocked; its last line of output reads like this:

[master] Queue is completely filled; waiting

If I attach strace while in this blocking state, I get this:

$ strace -fp `pidof ./webserver`
Process 21090 attached with 9 threads - interrupt to quit
[pid 21099] recvfrom(32,  <unfinished ...>
[pid 21098] recvfrom(23,  <unfinished ...>
[pid 21097] recvfrom(31,  <unfinished ...>
[pid 21095] recvfrom(35,  <unfinished ...>
[pid 21094] recvfrom(34,  <unfinished ...>
[pid 21093] recvfrom(33,  <unfinished ...>
[pid 21092] recvfrom(26,  <unfinished ...>
[pid 21091] recvfrom(24,  <unfinished ...>
[pid 21090] futex(0x6024e4, FUTEX_WAIT_PRIVATE, 55883, NULL

So the children seem to be starving on unfinished recv calls, while the master thread waits for any children to work away the queue. (With a queue size of 1024 and 200 workers I couldn't reproduce the situation.)

How can one counteract this? Specify a timeout? Spawn workers on demand? Set the listen() backlog argument to a low value? – or is it all Apache Benchmark's fault? *confused*

mutt sidebar patch improvements

2012-01-08T19:32:00+01:00

It is generally accepted as an almost universal truth that mutt sucks, but is the MUA that sucks less than all others. While people use either Vim or Emacs and fight about it, I hardly see any people fight about whether mutt is good or bad. There is, to my knowledge, no alternative worth mentioning.

Mutt dates back well into the mid-nineties. As you might imagine, with lots of contributors over the course of almost two decades, the code quality is rather messy.

When development had stalled for quite a while in the mid-2000's, a fork was attempted. While mutt-ng was quite popular for a while, most changes were incorporated back into mainline mutt at some point. (Ironically, the latest article in the mutt-ng development blog is from October 2006 and is titled "mutt-ng isn't dead!"). The development of main mutt gained some momentum again, triggered in large parts by the contributions of late Rocco Rutte.

I remember two big features that the original mutt authors just wouldn't integrate into mainline: The headercache patch and the sidebar patch. About the former I can't say anything, but lately I've been fixing the Sidebar patch in various places. (We use mutt at work and rely heavily on e-mail communication, so we'd like a bug-free user agent, naturally.)

When all the mutt forking went about five years ago, I didn't know much about it. Retrospectively, I see the people did a hell of a job. Long before mutt-ng was forked, Sven told me he and Mika met in Graz for several weeks to sift and sort through the availbale patches, intending to do a "super patch".

Mutt's code quality is arguably rather messy.

There's a wild mix of 2-, 4- or 8-space indentation, often mixed with spaces (or vice versa)
The user interface is completely tangled with application logic
It uses curses directly. Go figure

On top of that, the Sidebar patch tries to make it even worse. Imagine this: mutt draws a mail from position (line=x,char=0) to the end of the line. Now the sidebar patch will introduce a left "margin", such that the sidebar can be drawn there. Thus, all code parts where a line is started from the leftmost character has to be rewritten to check if the sidebar is active and possibly start drawing at (line=x,char=20).

The sidebar code quality is a fringe case of bad code. Really, it sucks. However, there's no real way to "do it right", since original mutt never planned for a sidebar.

Who maintains the sidebar patch? – Not sure. There's a version at thomer.com, but he says:

July 20, 2006 I quit. Sadly, there seems to be no desire to absorb the sidebar patch into the main source tree.

The most up-to-date version is found at Lunar Linux. Last update is from mid-2009.

Debian offers a mutt-patched package that includes the sidebar patch, albeit in a different version than usually found 'round the net. In short, this patch is a mess, too.

But since I made all the fixes, I decided to contact the package's maintainer, Antonio Radici. He promptly responded and said he'd happily fix all the issues, so I started by opening two bug reports. Nothing has happened since.

The patches run quite stable for my colleagues, so I think it's best to release them. Maybe someone else can use them. Please note that I have absolutely no interest in taking over any Sidebar patch maintainance. ;-)

For some of the patches I provide annotations. They all feature quite descriptive commit messages, and apply cleanly on top of the Debian mutt repository's master branch.

The first four patches are not by me, they are just the corresponding patches from the debian/patches/ directory applied to have a starting point.

The first few patches fix rather trivial bugs.

Now come the performance critical patches. They are the real reason I was assigned the task to repair the sidebar:

0009-cache-time-when-sidebar-last-counted-all-the-mails.patch

This patch fixes a huge speed penalty. Previously, the sidebar would count the mails (and thus read through the whole mbox) every time that mtime > atime! This is just an incredible oversight by the developer and must have burned hundreds of millions of CPU cycles.

This introduces a member `sb_last_checked' to the BUFFY struct. It
will be set by `mh_buffy_update', `buffy_maildir_update' and
`buffy_mbox_update' when they count all the mails.

Mboxes only: `buffy_mbox_update' will not be run unless the
condition "sb_last_checked > mtime of the file" holds. This solves
a huge performance penalty you obtain with big mailboxes. The
`mx_open_mailbox' call with the M_PEEK flag will *reset* mtime and
atime to the values from before. Thus, you cannot rely on "mtime >
atime" to check whether or not to count new mail.

Also, don't count mail if the sidebar is not active:

Then, I removed a lot of cruft and simply stupid design. Just consider one of the functions I removed:

-static int quick_log10(int n)
-{
-        char string[32];
-        sprintf(string, "%d", n);
-        return strlen(string);
-}

That is just insane.

Now, customizing the sidebar format is simple, straight-forward and mutt-like:

sidebar_format

    Format string for the sidebar. The sequences `%N', `%F' and
    `%S' will be replaced by the number of new or flagged messages
    or the total size of the mailbox. `%B' will be replaced with
    the name of the mailbox. The `%!' sequence will be expanded to
    `!' if there is one flagged message; to `!!' if there are two
    flagged messages; and to `n!' for n flagged messages, n>2.

While investigating mutt's performance, one thing struck me: To decode a mail (eg. from Base64), mutt will create a temporary file and print the contents into it, later reading them back. This also happens for evaluating filters that determine coloring. For example,

color   index  black green  '~b Julius'

will highlight mail containg my name in the body in bright green (this is tremendously useful). However, for displaying a message in the index, it will be decoded to a temporary file and later read back. This is just insane, and clearly a sign that the mutt authors wouldn't bother with dynamic memory allocation.

By chance I found a glib-only function fmemopen(), "fmemopen, open_memstream, open_wmemstream - open memory as stream".

From the commit message:

When searching the header or body for strings and the
`thorough_search' option is set, a temp file was created, parsed,
and then unlinked again. This is now done in memory using glibc's
open_memstream() and fmemopen() if they are available.

This makes mutt respond much more rapidly.

0015-keep-buffer-like-temp-file-in-memory.patch

Finally, there are some patches that fix various other issues, see commit message for details.

There you go. I appreciate any comments or further improvements.

Update 1: The original author contacted me. He told me he's written most of the code in a single sitting late at night. ;-)

Update 2: The 16^th patch will make mutt crash when you compile it with -D_FORTIFY_SOURCE=2. There's a fix: 0020-use-PATH_MAX-instead-of-_POSIX_PATH_MAX-when-realpat.patch (thanks, Jakob!)

tmux session names

2012-01-03T17:47:00+01:00

Usually, I name my tmux sessions according to what project I'm working on. To attach a specific session, I use a custom tm function, with the appropriate completion:

# 2011-10-19: tmux shortcut for creating/attaching named sessions
  tm() {
    [[ -z "$1" ]] && { echo "usage: tm <session>" >&2; return 1; }
    tmux has -t $1 && tmux attach -t $1 || tmux new -s $1
  }

# 2011-10-19
# stolen from completion function _tmux
  function __tmux-sessions() {
      local expl
      local -a sessions
      sessions=( ${${(f)"$(command tmux list-sessions)"}/:[ $'\t']##/:} )
      _describe -t sessions 'sessions' sessions "$@"
  }
  compdef __tmux-sessions tm

It looks like this:

A colleague use this, but with a modification. If [[ -z "$1" ]], he'll simply do a tmux attach, which will attach the last recently used session.

Happy New Year!

2011-12-31T17:05:00+01:00

I started this blog exactly one year ago. Over the course of this year, I wrote 105 articles. That is not really much, and I want to write more – just about the stuff that's on my mind. So stay tuned. ;-)

A Happy New Year to all my readers!

Old DVDs and mplayer

2011-12-29T22:37:00+01:00

I was just watching an old DVD of mine (from 2003), but was annoyed that the video had "hardcoded" black bars at the top and the bottom and was in interlaced format. A fast motion looks like this:

But mplayer can do everything! You just need to know how. Here it is: Use the cropdetect video filter, skip to the actual video and let it run for a while (it will detect the black areas that don't change). Then copy the output:

$ mplayer -vf cropdetect dvd://
...
[CROP] Crop area: X: 5..717  Y: 57..421  (-vf crop=704:352:10:64).

Now, play the movie in 4:3 aspect racio, and force cropping at the numbers you just found out. Additionally, use the yadif deinterlacer:

$ mplayer -aspect 4:3 -vf yadif,crop=704:352:10:64 dvd://

Now the same scene (with heavy motion) looks much nicer:

sed and jekyll tags

2011-12-27T03:18:00+01:00

Did you know the -s switch to sed? I had the strange effect that operating on several files produced different results between when I called sed repeatedly in a for loop and passing all the files as arguments just once. – -s tells sed not to consider all files to be one big stream, but to separate them from each other.

Anyway, here's a script that searches all the posts now and gives me numbers on the tags I already used.

#!/bin/sh

sed -s -n '
    /---/,/---/ {
        /tags:/ {
            s/^tags: \[//;
            s/\]$//;
            s/, */\n/g;
            p
        }
    }' ~/blog/_posts/*.markdown | sort | uniq -c | sort -n

28C3

2011-12-27T02:23:00+01:00

I'll be at the 28th Chaos Communication Congress the next few days. If you are there, too, write me an email or text me.

I haven't throughly looked at the Fahrplan yet. But there are some classics there already, for example Dan Kaminsky, the Fnord News Show or a new Neusprech talk by maha.

Some other talks seem promising: Deceiving Authorship Detection will unveil two tools for detecting and obfuscating authorship patterns; The future of cryptology: which 3 letters algorithm(s) could be our Titanic? also sounds cool.

I was surprised to see a talk about "bup". Usually, the Congress has a very un-techy focus. To outsiders this might sound strange, but it's mainly about ideas, not so much about technology of the status quo. Talk topics normally include "What if X happens?" (where X is in: internet lockdown, censorship, Aliens arrive, ...), "How can we hack X?" (X being: CCTVs, mobile devices, our government, ...), "How we hacked X" (where X might just be your occasional airport access control badges system).

Anyway, I hope I'll have some time to play around with stuff I had on my mind for some time now.

Preparation for tomorrow's Keynote: Evgeny Morozov's TED talk.

X220's UMTS card

2011-12-22T13:22:00+01:00

I've been toying around with the UMTS module in my X220 lately. I got a pre-paid SIM from blau.de, who offer 24h UMTS flatrates for 2,40 EUR. (This is probably my use case: Being somewhere without internet access for a day or two. This only happens so often, so I don't want a "real" flat.)

My UMTS card is manufactured by Sony Ericsson and connected via internal USB:

$ lsusb -v -s 004:003
    ...
    idVendor           0x0bdb Ericsson Business Mobile Networks BV
    idProduct          0x1911

The installation is easy: Just insert the SIM card behind the battery as shown here. Add yourself to the dialout group, log in again, and you're set.

You can first connect to your device using chat or picocom (which you can be terminated via C-a C-x). To ask if you can use the SIM without PIN, send the AT+CPIN? command:

$ picocom /dev/ttyACM0
...
AT+CPIN?
+CPIN: READY

If you're not ready to go, I would disable the PIN request using a regular phone. (I did.)

Dialling out is easy. I set up two profiles in the /etc/wvdial.conf that allow me to switch between "pay per megabyte" and "dayflat":

[Dialer blau]
Modem = /dev/ttyACM0
Init1 = AT+CGDCONT=1,"IP","internet.eplus.de"
Stupid mode = 1
phone= *99#
Username = blau
Password = blau

[Dialer tagesflat]
Modem = /dev/ttyACM0
Init1 = AT+CGDCONT=1,"IP","tagesflat.eplus.de"
Stupid mode = 1
phone= *99#
Username = blau
Password = blau

The rest happens automatically, once you invoke wvdial blau or wvdial tagesflat. (Note you have to execute these with root privileges because they want to modify pppd-related config files.) Most probably you want the follow-up command route add default dev ppp0 to route all traffic via the ppp0 interface.

In a test run I got a downstream speed of 190KB/s (city perimeter). Working over SSH is not painful at all.

I also played around with gammu a little bit.

$ gammu --identify
Device               : /dev/ttyACM0
Manufacturer         : Lenovo
Model                : unknown (F5521gw)
Firmware             : R2A07

The Wammu interface is nice, it can even receive SMS. But sending SMSes failed so far:

$ echo "Das ist ein Test" | gammu --debug textall --debug-file /tmp/gammu \
    sendsms TEXT +491785542342
...
1 "AT+CMGS=28"
2 "> 079194710716000011000C919471584532240000FF10C4F01C949ED341E5B41B442DCFE9^Z"
3 "+CMS ERROR: 500"

... which is somewhat of an "generic error". Maybe sending SMS is not supported at all. I'll look into that later.

Also, I'll have a look whether my Card supports GPS information retrieval. Thinkwiki claims a similar model does this. Interesting.

Update: Actually, I forgot one thing. I keep the following two entries in my /etc/wvdial.conf:

[Dialer on]
Modem = /dev/ttyACM0
Init1 = AT+CFUN=1

[Dialer off]
Modem = /dev/ttyACM0
Init1 = AT+CFUN=4

The actual sequence is now: wvdial on && wvdial blau. The AT+CFUN=1 will active the radio equipment, which is necessary. And, suddenly, also SMS delivery works! :-)

New X220

2011-12-10T14:38:00+01:00

I got a brand new Thinkpad X220 on thursday. I'm not much into hardware, I think it should mainly work. I have a model with 4 GB of RAM, an i7 at 2.7 GHz, UMTS preinstalled, SSD instead of a HDD and an IPS panel. It's a really nifty thing.

Paying the extra money for the SSD is totally worth it. Everything happens instantaneous. The bootup process is down to five seconds. The IPS panel is really worth it, too. ThinkPads have long been criticized for their bad displays – with the new panel at full brightness, my regular screen looks really dim and grey...

The Debian netinstall works smoothly. I haven't come around to testing all the stuff like the DisplayPort connectors, Bluetooth, UMTS, USB 3.0. But the usual stuff works out of the box.

However, there are major problems with the power management of both the graphics card and the whole system, the latter one being a regression in the recent 3.0 and 3.1 kernel series regarding ASPM. Currently I'm using the 3.1.0-1-amd64 kernel with the pcie_aspm=force boot parameter. I cannot really see a difference in power consumtion when varying this parameter, though.

A major thing, however, is re-enabling the RC6 mode of the graphics chip. This alone saves more than 4W when the computer is in an idle state. My /etc/modprobe.d/i915-kms.conf looks like this now:

options i915 modeset=1
options i915 i915_enable_rc6=1
options i915 i915_enable_fbc=1
options i915 lvds_downclock=1

Suspend/resume works fine, no flickering effects. I use the following command to find out the current power consumption:

while sleep 1; do
    awk '{printf"%.2f\n",$1/-1000}' < /sys/devices/platform/smapi/BAT0/power_now;
done

This requires the tp_smapi kernel module to be loaded. With full brightness (0) and while writing this blog article, the consumption is at ~12W; with medium brightness (8) it's ~8.5W; at the lowest brightness (15) it's ~8W; With the display completely turned off, it's ~6.5W. There are people who claim they only have an ~5.4 power consumption. If you have any other hints on this or if you own an X220 yourself, I'd be interested in the details.

demotivating maths

2011-12-05T21:48:00+01:00

Math is not easy. (See also) I saw this scroll by today:

GUI simplicity vs. UNIX simplicity

2011-11-24T18:59:00+01:00

I ranted about the new Unity interface some weeks ago. On several occasions thereafter, I had to help people solve problems they had using some sort of graphical user interface.

Example I: I was debugging a broken VPN connection. The connection settings were managed by the KDE network manager, which is rather easy to use. Internally, of course, the network manager just writes out some temporary configuration files and starts the PPP daemon with a lot of custom flags. That's all fine if it works – but in this case it didn't work. It just said: "connection failed", no diagnostics given. (The solution was to enable MPPE, which itself was trivial: ticking the corresponding box. How did I find this out? Tailing /var/log/dmesg while connecting. It said right there: MPPE not enabled, but server side requires it.)
Example II: The gnome network manager somehow fucked up. Even now I don't know why. It says "connecting", and then nothing happens. No diagnostics.

UNIX is simple. It really is. There is a reasonable and easy-to-follow philosophy behind it. But UNIX requires the user to know what he wants to do, and read error messages. UNIX simplicity is not the same as iPhone simplicity.

Eric S. Raymond wrote this set of rules that should guide UNIX program design. In this context, two important rules stick out (emphasis mine):

Rule of Silence: When a program has nothing surprising to say, it should say nothing.

Rule of Repair: When you must fail, fail noisily and as soon as possible.

Although this is of course mostly aimed at text user interface programs, you can get an important point here. Most GUIs adhere to the Rule of Silence quite well – in fact so well that they seldom say anything at all!

Since many UNIX GUIs invoke text-interface programs under the hood, it should be a necessity to be able to view how those program failed. Luckily, most TUI programs provide descriptive error messages. If they are hidden in the GUI there are two effects:

the "regular" user sees that something fails, and
the admin looking at the problem sees that something fails and pulls out his hair trying to find out what – so that he can repair it!

I don't use GUI programs at all, except for a Browser (Vimperator/Firefox), a PDF viewer (Zathura) and The GIMP. Mostly, this is because of usability considerations. But also, I'm afraid to use a computer where I cannot see what is happening. And that's exactly the case with GUIs that do stuff that can fail: I don't know what they are doing and why they are failing!

I the end I always go the extra mile and read up on the PPP daemon, for example. This wouldn't be necessary if GUIs had a switch to do some really verbose logging. That would help tremendously. Plus a button to display that log. Should be easy, shouldn't it?

Rap am Freitag

2011-11-11T21:16:00+01:00

Bushido hat ja den Bambi bekommen. Alle regen sie sich auf. Verständlicherweise heult die taz rum, und entblödet sich auch nicht, möglichst krasse Zitate als explizit vom Burda-Verlag als einen "wertvollen Beitrag zum gegenseitigen Verständnis sozialer Gruppen mit unterschiedlichen kulturellen Wurzeln" zu bezeichnen. Arm, das ist arm.

Selbst die Piratenpartei kritisiert die Preisvergabe. Oh. Aber für Pressemitteilungen zu wesentlichen, medial besonders wichtigen Kernthemen wie dem Bundestrojaner habt ihr Tage gebraucht? – Ja, nee, ist klar.

Den Bambi – ich habe übrigens heute das erste Mal bewusst von der Institution "Bambi-Verleihung" gehört... – den Bambi für Integration hat Bushido aber natürlich verdient. Vielleicht sollte man mal aus einem anderen Blickwinkel drauf schauen:

Ein ehemaliger Randgruppen-Rapper, der sich so weit hocharbeitet, dass er mit den Stars und Sternchen des deutschen Fernsehens koksen kann; seinen eigenen Kinofilm und eine nette Villa in Dahlem hat; ganz offensichtlich gut das Klischee des Ausländers bedienen kann, und versteht, wie er polarisieren kann. Ein Rapper, der einen fetten Major-Deal hat, und es nicht mehr nötig hat, gute Musik zu machen, sondern einfach mit dem Mainstream schwimmen kann. ("23" gehört? Puuuhh.) Einer, der offensichtlich gut genug in der schmierigen Welt zwischen Entertainment und Aufmerksamkeit heischendem Journalismus umherwandelt, um sich so einen Preis vergeben zu lassen. – Eindeutig gut integriert, würde ich mal sagen.

Boah ist das lachhaft, wie alle auf ihm rumkloppen. Staiger hat da sehr gut drüber reflektiert mit einem fiktiven "Brief an Mama".

Oh ja, Rap: Savas' Aura ist da. Ziemlich gutes Album – und zum Glück ein nicht "integrierter" Rapper, sondern fresh wie eh und je.

Don't believe the hype...?

2011-11-11T20:13:00+01:00

A while ago a link to the following chart from Debian Popcon appeared HN, claiming that "Git is exploding".

I was pretty fascinated by the steep rise of Git's curve. Of course, the statistics are not representative, but they resemble a good set of somewhat typical Debian systems.

Today, I somehow thought about the graph again – and started investigating. Take a look at this graph:

On April 1st, one of the Git package maintainers uploaded a commit that changed Git's package name to "git" from "git-core". In the graph above you can very well see the steep ascend of of the red line ("git installed"), while at the same time a sudden drop of the "git no-files" package occurs. Slowly, the purple "git-core no-files" follows, indicating that APT replaced git-core with its dummy package that only contains dependencies, no files.

This doesn't explain why the red line's ascend is so steep; however, there must be some relation the package's name change.

touch screens

2011-11-09T22:42:00+01:00

I don't own a smart phone. Neither do I own an ebook reader. It's not that I don't like the idea; I like it. It's just that the products in that area still is in a very embryoinic state.

Considering what an iPhone or the Kindle can do, that sound like a pretty strange statement. But actually, it isn't. Let the technology be really advanced – if the user interface is no good, the product is no good.

And the current user interfaces are no good. As this nice article points out really well, we are degraded to finger-swiping and tapping motions at best. With my current phone, I can type an SMS without looking at the display; control the MP3 player without taking the phone out of my pocket; switch it to silent without looking at it. – The key point here is to sense things. Or, as the article concludes:

With an entire body at your command, do you seriously think the Future Of Interaction should be a single finger?

Side show, but the same principle applies: Keyboard vs. Mouse. Can you actually feel the corner of windows, buttons, etc.? – I can't. But I can sure feel whether I pressed a key. I can type without looking at my keyboard, but I wouldn't even care to touch the mouse without looking at the screen.

The day we have smartphones that not only have "force feedback" but real, tangible, tactile buttons and the illusion of touching a non-flat, non-homogeneous surface, I'll buy one. Same goes for the Kindle as soon as I can feel the edge of the paper and roll it like it was a few pages of printout or a paperback.

rss rant

2011-10-30T14:41:00+01:00

Rant Saturday! Very good rant about sites un-implementing RSS feeds.

gnome is not aqua

2011-10-30T13:51:00+01:00

Dear GNOME team,

I'd like to point out a fact to you: You are not, and should not compete with, Apple's Aqua GUI. There are several reasons for that, but let me tell my story first.

I installed Ubuntu 11.10 on a friend's laptop yesterday. The new Unity user interface is somewhat unusable, of course. (Side rant: Unlike you might think, quite many people out there have actual monitors, not just tiny eeePC displays. Some even have computers that are more than a few months old and lack the required processing power for stupid eye-candy.) – First thing to do: Install gnome-shell and select "GNOME classic" at the login screen. Second thing: Disable startup sound.

Enter $DEVELOPER, saying: "Oh, fuck this shit. Everybody wants these drum sounds at startup. So we'll make it hard to disable it." Guess how you can do it: Edit a somewhat buried file with root privileges. Try explaining that to your parents over the phone.

At one point I realize: There is no "Settings" menu any more. There just isn't. There are vague comments in some blogs this thing is missing, but I can't find where they put it. That's what qualifies as a regression.

The "System Settings" have moved, too, and looks like its OS X counterpart. Now if there's one thing Apple is really good at, it's making people feel comfortable (or even elitist!) thinking inside the box – by ways of designing a bearable user interface that hides complicated stuff. Mostly, though, this means you can only do what some (possibly narrow-minded) developer intended.

However, You, the GNOME team, are not good at it. Part of it is the simple fact that there just happen to exist tons of configuration options. If you hide them – and by hiding I mean: making it unaccessable without using the shell and/or editing special files – you are crippling the user.

So, please, stop "making things better". Or, if you do, on your way please don't destroy the perfectly running classig GUI in order to "improve" it. You are not Apple. You will never be Apple. The Aqua design sucks, too, but they never had a lot of configuration options in the first place. The Gnome Shell had.

Thank you.

P.S.: I don't use Desktop Environments myself, so I might have got some terminology wrong. But the fact alone I cannot find ways to configure stuff in two hour's time should tell tales.

P.P.S.: Bad decision: The scrollbars. Try teaching a person over age 60 (or below five) to use the 5px-wide scrool bar to make pop up an additional small scrollbar outside the window that actually enables scrolling the window contents. Again: not all people have a scroll wheel in their mouse. At least provide an easy settings dialog to disable this behaviour.

Update: I'm not alone, even esr made that point a while ago.

great stuff

2011-10-25T21:21:00+02:00

I took a look at this animated gif and couldn't really figure out what it was and how it happened. But now I know you can create such a cool thing – called Cobra Weave Exploding Stick Bomb -- on your own, or watch professionals do that. Amazing.

twitter search

2011-10-18T21:42:00+02:00

I don't have a twitter account. But in certain cases, I follow people or events using the Twitter web site or RSS feeds. Because, luckily, Twitter provides usable RSS feeds, although I'm pretty sure they don't advocate or event document it.

Simply use a URL like this:

http://api.twitter.com/1/statuses/user_timeline.rss?screen_name=USERNAME

But, seriously, you'd want to allow people to search for stuff, right? To become interested and later immersed in Twitter.

So evey time I want to look up a hash tag, I go to twitter.com and duh – there is no search form. You can only sign in or sign up. No search form.

Not even is there a link to a search form. How hard would that be? My workaround is to go to twitter.com/a – the timeline of certain @a, of whom I don't know who he is, but "a" is comfortable letter to type – and there it is, at the top of the screen: a search box.

Seriously? – Can it be that hard?

Update: Three people approached me with different ideas/URL of how to find the search form. The most mnmonically convenient seems to be search.twitter.com, though there is a javascript-free mobile version, too. Consequently, I added a form to my general-purpose search page. (Thanks for the hints to: Skudo, Julian and Chris.)

spam ratio

2011-10-13T11:24:00+02:00

How much spam do you get? – I have a feeling I get more than I deserver. So I checked my spm ratio, i.e. the number of (recognized) spam mails divided my the number of total mails (as far as the procmail log spans, which is some 270,000 mails).

$ echo $(fgrep -c 'Subject: *****SPAM*****' Mail/from) / \
    $(fgrep -c 'Subject: ' Mail/from) | bc
.952154

Now this is gross. Only less than five percent of the mail I receive might be of interest to me. :-(

gc via cron on hosted git repos

2011-10-12T22:43:00+02:00

If you host Git repositories, you might want to implement a cron job that automatically triggers garbage collection on the server side. As a regular user you can't usually access the unreachable objects anyway, so there's no point to keep them.

However, when invoking git gc, Git will pack loose objects together. This has a huge advantage: When a user clones a whole repository, Git will compress all objects within a single packfile and transfer it via the Git protocol. If all the objects are already in one packfile, there's no overhead in creating a temporary packfile. (If you just want to get a subset of commits, it's easier for git to "thin out" the existing packfile, too.)

You can usually see if a computationally expensive temporary packfile is created if there is a message like remote: counting objects ... that keeps on counting for a while. For some hosters, this takes quite some time, because the server is under high load.

I use the following script to trigger git gc every night:

#!/bin/sh

BASE=/var/git/repositories

su - git -c "
cd $BASE
find . -name '*.git' -type d | while read repo; do
    cd $BASE/\$repo && git gc >/dev/null 2>&1
done
"

You might want to omit the su part if you create a script that's executable by the owner of your git repositories itself.

Update: If you don't use cgit's age files, you'll have all your repos displaying they were recently changed in the "idle" column. To work around this, include the following command adter the git gc call:

mkdir -p info/web &&
git for-each-ref \
    --sort=-committerdate \
    --format='%(committerdate:iso8601)' \
    --count=1 'refs/heads/*' \
    > info/web/last-modified

svn 1.7

2011-10-12T02:21:00+02:00

I feel I have a pretty good understanding of Git. Before I came accustomed to it around the end of 2009, however, I had never extensively used version control.

Sure, I used RCS for simple projects; I played around with CVS and bought a book about it; in 2007 I became somewhat accustomed to Mercurial, because the suckless and grml projects used them at the time.

All the time I happened to use Subversion, also for coordinating the work on the book about Z shell.

RCS was simple, but was not at all suitable for collaboration. I thought CVS and Subversion were, alas I never really understood how they worked. As in: why they worked, what was good practice, why you just couldn't seem to run it locally (without client-server architecture).

I happened to just read the release notes of the new subversion 1.7. And, reading the notes, I realize it wasn't me not being attentive or being plain stupid, it's just the interface as well as repository design that was totally messed up. For example:

Subversion 1.7 features a new subcommand called svn patch which can apply patch files in unidiff format (as produced by svn diff and other diff tools) to a working copy.

I mean – what?! This must be such a basic thing – receiving patches via a mailing list and trying to apply them, right? But apparently, there was no need for that since 2000...?

Or consider this great improvement of adding a --diff switch:

svn log can now print diffs

Oh, right. Diffs. In an SCM. Right. Might be of some use.

I'm glad I don't have to use plain SVN any more.

pictures in pictures

2011-10-06T22:07:00+02:00

While browsing I saw this one today:

If you look closely, someone calculates the check to be worth $536.49, which is of course wrong. (It's $0.002.) So I set out with the GIMP to draw a riposte:

By the way, these are called demotivator pictures.

mail server switchover

2011-10-03T21:43:00+02:00

Today I migrated the last big part of my old server: the mail system. Since I and other people depend on this server for their day-to-day mailing, I had to switch over without losing a single e-mail. This is how I did it:

Step one: Port the configuration, make the environment run on the new server. Copy also user metadata like passwords and make sure the overall structure is working (incoming SMTP works, POP3 access, etc.)

Step two: Copy over all mails from the old server. The mails will be synced later on again, so this can happen some minutes in advance. (I actually lost a few days doing this, because I discovered unused mailboxes with 995,000 mails, >99% of them being spam. I had to ask the owner first, though, whether I could delete them.)

Now comes the time-critical path. It took me an overall 70 seconds to do steps three to five.

Step three: Stop the daemons that receive mail or give the user access to it. For example: for s in postfix courier*; do /etc/init.d/$s stop; done – connecting to the host will now give a "connection refused" error message. MTAs trying to deliver mail will usually try again ten minutes later. (So no mail gets lost.)

Step four: Sync the emails again. There might have arrived new messages, or users have deleted some of their inbox. I used this command: rsync -avhP --delete vmail@eris.feh.name: .

Step five: Apply iptables rules to forward connections to the new server. This is due to the fact that DNS information is slow to spread. For a few days I don't care whether mail.feh.name resolves to 88.198.158.101 or to 176.9.247.89. Both will effectively talk to the new server.

iptables -t nat -A PREROUTING -p tcp -s ! 176.9.34.52 --dport 25 \
  -j DNAT --to-destination 176.9.34.52:25
iptables -t nat -A PREROUTING -p tcp -s ! 176.9.34.52 --dport 110 \
  -j DNAT --to-destination 176.9.34.52:110
iptables -t nat -A POSTROUTING -d 176.9.34.52 -j MASQUERADE
iptables -A FORWARD -p tcp -d 176.9.34.52 -j ACCEPT
iptables -A FORWARD -p tcp -s 176.9.34.52 -j ACCEPT

So this establishes forwarding for SMTP and POP3. The old server will simply act as a NATing gateway to the new server.

Step six: Adjust the DNS. As said above, you can take your time for this; but the information will eventually spread. To have an indicator of how many connections still arrive at the old host, try iptables -t nat -L -vn, it'll print packet and byte counts for each rule in the NAT table.

Done! And just one minute of outage. *like*

git and the unix philosophy

2011-10-03T18:56:00+02:00

Good, but slightly older article: Too Smart for Git.

Git follows Linux's philosophy of refusing to protect you from yourself. Much like Linux, Git will sit back and watch you fuck your shit right up, and then laugh at you as you try to get your world back to a state where up is up and down is down. As far as source control goes, not a lot of people are used to this kind of free love.

His conclusion holds true for many other programs also:

The problem isn't that Git is to hard, it's that smart developers are impatient and have exactly zero tolerance for unexpected behavior in their tools.

dead code easter egg

2011-10-02T19:57:00+02:00

I was just researching on how the file format of the xt_recent module works. That's where I found this nice easter egg: instead of writing down the size of an IPv6 address plus one, they simply used a dummy string +b335:1d35:1e55:dead:c0de:1715:5afe:c0de", reading "beesides less dead code it is safe code". Hehe.

migrating domains

2011-09-27T21:23:00+02:00

Today, I migrated all my domains to the new server. While at it, I set up a mirror of my blog repository at http://git.plenz.com/blog/. You could view the pure markdown files there, or see how I messed around with Jekyll stuff.

The sync is called from a post-receive hook with the command:

env GIT_SSH=`pwd`/hooks/blogpush git push -f git@git.plenz.com:blog master

... where blogpush is just a little wrapper to make SSH use the public key that has access to the repository:

#!/bin/sh
ssh -o ControlMaster=no -i /home/feh/blog.git/hooks/fehblog $@

gitolite and interpolated paths

2011-09-25T23:04:00+02:00

The Git daemon supports a feature called Interpolated Path. For example, my Git daemon is called like this:

git-daemon --user=git --group=git \
    --listen=0.0.0.0 --reuseaddr \
    --interpolated-path=/var/git/repositories/%IP%D \
    /var/git/repositories

What it does is it translates the request for a git repository before actually searching for it. When I do a git clone git://git.plenz.com/configs.git what actually happens is that the Git daemon will deliver the repository at /var/git/repository/176.9.247.89/configs.git. That is especially nice in environments where you share one git daemon for several people/projects and have sufficient IP addresses.

However, Gitolite doesn't seem to have a config option for this. Now what I did was patch Gitolite so that it'll prepend the repository name with the IP of the network interface the call came through.

This information is available from the SSH_CONNECTION environment variable.

(The patch to gl-compile-conf is needed so that Gitolite will create a projects.list per virtual host. That way, you can use different configuration files for each CGit instance, according to your VHost.)

diff --git a/src/gl-auth-command b/src/gl-auth-command
index 61b2f5a..a8d1976 100755
--- a/src/gl-auth-command
+++ b/src/gl-auth-command
@@ -124,6 +124,12 @@ unless ( $verb and ( $verb eq 'git-init' or $verb =~ $R_COMMANDS or $verb =~ $W_
     exit 0;
 }

+# prepend host's ip address
+# SSH_CONNECTION looks like this: 92.225.139.246 41714 176.9.34.52 22
+#                                 from-ip        port  to-ip       port
+my $connected_via = (split " " => $ENV{"SSH_CONNECTION"})[2] // "";
+$repo = $connected_via . "/" . $repo;
+
 # some final sanity checks
 die "$repo ends with a slash; I don't like that\n" if $repo =~ /\/$/;
 die "$repo has two consecutive periods; I don't like that\n" if $repo =~ /\.\./;
diff --git a/src/gl-compile-conf b/src/gl-compile-conf
index 2d4110f..6f8f0d7 100755
--- a/src/gl-compile-conf
+++ b/src/gl-compile-conf
@@ -577,6 +577,22 @@ unless ($GL_NO_DAEMON_NO_GITWEB) {
     }
     close $projlist_fh;
     rename "$PROJECTS_LIST.$$", $PROJECTS_LIST;
+
+    # vhost stuff
+    my %vhost = ();
+    for my $proj (sort keys %projlist) {
+        my ($ip, $repo) = split '/' => $proj => 2;
+        $vhost{$ip} //= [];
+        push @{$vhost{$ip}} => $repo;
+    }
+    for my $v (keys %vhost) {
+        my $v_file = "$REPO_BASE/$v/projects.list";
+        my $v_fh = wrap_open( ">", $v_file . ".$$");
+        print $v_fh $_ . "\n" for @{$vhost{$v}};
+        close $v_fh;
+        rename $v_file . ".$$" => $v_file;
+    }
 }

 # ----------------------------------------------------------------------------

With this patch applied, I can git push git.plenz.com:config.git and it will end up pushing to 176.9.247.89/configs.git, fully transparent to the user.

N.B.: This strictly is a "works for me" solution. It's not very clean – but I don't plan on properly implementing a config setting. As I said, it works for me. ;-)

cgit and setuid

2011-09-25T22:26:00+02:00

I'm still in the process of setting up my new server. Today, I migrated the Git repositories. I wanted a more secure setup, that is I don't want my web server to be able to read the repositories. (It spawns CGit, which has to read them somehow.)

So I created all repositories below /var/git/repositories, such that they are only readable by the system user "git".

However, that presents a problem to the CGit CGI: It cannot access the repositories any more. My first approach was: Just chown git.git /usr/lib/cgi-bin/cgit.cgi and set the setuid bit on it.

But, alas, that will only make the CGit binary run with effective user ID "git". What's needed to actually access the files is a real user ID "git".

The only way to set the real user ID is to call setuid() with effective root privileges. So I created a wrapper that

searches for git's user ID
setuid()s to that ID (real an effective)
executes the actual CGit CGI

That's what I came up with:

 #include <unistd.h>
 #include <sys/types.h>
 #include <pwd.h>
 #include <string.h>

int main(int argc, char *argv[])
{
    struct passwd *p;
    uid_t git_uid = 0;

    while((p = getpwent()) != NULL) {
        if(strcmp(p->pw_name, "git"))
            continue;
        /* got user "git" */
        git_uid = p->pw_uid;
    }

    endpwent();

    if(!git_uid)
        return 1;

    setuid(git_uid);
    execv("/usr/lib/cgi-bin/cgit.cgi", argv);

    return 0;
}

Provide it with a Makefile... (beware, use real tabs!)

default:
        gcc -Wall -o cgit-as-git.cgi cgit-as-git.c

install:
        install -o root -g root -m 4755 cgit-as-git.cgi /usr/lib/cgi-bin

... and change the Lighttpd configuration accordingly:

$HTTP["host"] == "git.plenz.com" {
    setenv.add-environment += ( "CGIT_CONFIG" => "/etc/cgit/plenz.com.conf" )
    alias.url = (
        "/cgit.css" => "/usr/share/cgit/cgit.css",
        "/cgit.png" => "/usr/share/cgit/cgit.png",
        "/cgit.cgi" => "/usr/lib/cgi-bin/cgit-as-git.cgi",
        "/"         => "/usr/lib/cgi-bin/cgit-as-git.cgi",
    )
    cgi.assign = ( ".cgi" => "" )
    url.rewrite-once = (
        "^/cgit\.(css|png)" => "$0",
        "^/.+" => "/cgit.cgi$0" 
    )
}

Done! Lighttpd can now call CGit as if it was a "usual" binary, but it will get wrapped and get the real and effective user ID of the system's user "git".

N.B.: If you want to use the wrapper as well, you might want to change the (hard coded) user name "git" and the binary (see execv call).

software design

2011-09-21T01:26:00+02:00

I'm in the process of setting up a new server. I struggled for some minutes now with the follwing strange behaviour: Lighttpd is very picky (and not at all verbose) about when you use alias.url.

As an actual example, look at these two blocks:

$HTTP["host"] == "HOST" {
  alias.url += ("/munin" => "/var/www/munin")
}

$SERVER["socket"] == ":443" {
  $HTTP["host"] == "HOST" {
    alias.url += ("/wiki" => "/usr/share/dokuwiki",)
  }
}

Now I, the innocent user, would expect the following to happen:

Munin is available via HTTP and HTTPS via /munin
The Wiki is available only via HTTPS at /wiki

But, alas, nope. It's quite different. The wiki is available alright if called before the munin part, and vice versa. The other one will be a 404, though. To make things work I actually have to double the alias part for munin – once outside the part that matches for SSL and once inside.

Oh, and don't confuse == and =~ – even if both match, the exact match wins. (Meaning you cannot put the line in a condition like =~ ":(443|80)".) So there's no real cascading or so. Bad design in my opinion, but it seems to be intentional. WTF?

Correct solution:

$HTTP["host"] == "HOST" {
  alias.url += ("/munin" => "/var/www/munin")
}

$SERVER["socket"] == ":443" {
  $HTTP["host"] == "HOST" {
    alias.url += ("/munin" => "/var/www/munin")
  }
}

$SERVER["socket"] == ":443" {
  $HTTP["host"] == "HOST" {
    alias.url += ("/wiki" => "/usr/share/dokuwiki",)
  }
}

Holy crap. Explain that to someone unfamiliar with how difficult it is to implement data structures in C. – They'd just call you a moron for not implementing a more intuitive aproach. – Wait, I do, too!

Git User's Survey

2011-09-05T17:34:00+02:00

The Git User's Survey is up from now until the beginning of october. Take part!

cryptome RSS feed suspended

2011-09-02T17:27:00+02:00

Great. :-(

25 August 2011: Due to unceasing abuse by clouds (now the worst offenders), bots, siphons, aggregators, search engines, and other data thieves the Cryptome RSS feed has been suspended.

debugging a zsh completion function

2011-09-01T23:27:00+02:00

There's a Perl module threads::shared. However, it won't show up as a completion match to the perldoc command. Why is that?, a colleague asked me. Now, that obviously is a quirk, so let's debug it.

We start at /usr/share/zsh/functions/Completion/Unix/_perldoc, where we find this:

_alternative \
    'modules:module: _perl_modules -tP' \
    'pods:base pod: _perl_basepods' \
    'files:module or .pod file:_files -g "*.(pod|pm)(-.)"' &&
    ret=0

So, go on to _perl_modules: It searches for paths where Perl would store modules and their documentation (line 85):

inc=( $( $perl -e 'print "@INC"' ) )

Then, they do some complicated globbing stuff on the directories (line 104), where a loop iterates over $inc and stores each element in $libdir:

new_pms=( $libdir/{[A-Z]*/***/,}*${~sufpat}~*blib* )

There's the catch: This globbing expression will only recurse into subdirectories that start with an uppercase letter. Lowercase modules on the highest level are okay, though. That's why threads appears (from threads.pm), but threads::shared doesn't (from threads/shared.pm).

The search for all modules that are missed this way will translate to the following Z-Shell command:

$ print -l ${^=$(perl -e 'print "@INC"')}/[a-z]*/***/*.pm(.N)
/usr/lib/perl/5.10/threads/shared.pm
/usr/share/perl/5.10/encoding/warnings.pm
/usr/share/perl/5.10/warnings/register.pm

tmux line continuation patch

2011-08-25T19:58:00+02:00

Yay! My patch to tmux was accepted yesterday. With it, tmux now supports \ as a line continuation character. Thus, you can put statements like this in your config:

bind P run-shell \
  "tmux set-b $(tmux sa - | curl -F 'sprunge=<-' http://sprunge.us) && \
  tmux display-message 'sprunge: upload sucessful!'"

dist-upgrade gnah

2011-08-24T09:07:00+02:00

I'm not sure why, but AFAIR I've never done a dist-upgrade that was entirely successful. That is true to my Debian system at home as well as to various other Ubuntu systems I've laid hands on.

My Debian system was a mix of stable and testing packages, and somehow aptitude came to a point where it'd just not be able to resolve some dependencies, meaning I could not install nor remove any new packages. So I opted for a dist-upgrade to Testing. Here's how it went, roughly:

Everything installs fine, until aptitude tries installing libc, where the installation process will go into a deadlock. Error message and solution is posted here (yes, you have to manually patch a Perl module). linux-base has the same problem.
System is left in a half-upgraded state, but aptitude won't do any further upgrading. apt-get dist-upgrade continues, though.
Reboot because of new kernel.
Sound is all gone. pcspkr module is appearently now called snd_pcsp and is thus loaded, strangely overriding all other sound drivers. I blacklist it manually, reboot.
Seems to work. Now install the new package I did all the upgrading for... right. There's still some 500 upgrades left to do! Ok, now...
Upgrade aborts with a problem: libcaca0 wants to overwrite a file which is also present in libcucul0. Purging libcucul0 will reinstall libcaca0, which in turn will fail. Turn to manual override and uninstall all packages libcaca* and libcucul* using dpkg.
Upgrade runs through fine. I go to sleep.
Today, I power up the computer. Alt+Enter to start a terminal – duh, nothing happens. Start an xterm. Inspect the situation: rxvt and rxvt-unicode are missing. Well, thanks, so here I go, aptitude install rxvt. But guess what: apt-get uninstalled aptitude! Yeah, great.
apt-get install rxvt-unicode, finally a decent terminal. Fire up mplayer. It's not installed any more. Like, what?!
Install mplayer, start it on a file. No sound. After various tries, a pattern ~~evolves~~dissolves: Sometimes it just won't play (like muted), sometime's there's an error message of various audio drivers. aplay works just fine.
mplayer -ao alsa works. Put it into the config. Happy now with music.

Now, what I ask myself: Manually patching a Perl module, manually resolving dependencies and invoking dpkg, being (momentarily) deprived of the few programs I use on a daily basis ... how will this packaging system ever be remotely feasible for someone who's not an expert of sorts and well-versed in debugging in a Unix wold?

Update: Some crap program said it depended on DECnet stuff. After a reboot, this caused the MAC addresses of all my interfaces (yes, both wireless and wired!) to be the same, i.e.: aa:00:04:00:0a:04. Solution is removing libdnet* and dnet-common. – You kidding me?!

URL shortener

2011-08-07T21:01:00+02:00

The bit.ly service requires some sort of registration or an API key for some time now. I used to use it in shell scripts to automate shortening of links.

Similar to the sprunge pastebin service, there's come up a new, easy-to-understand URL shortening service called gnzkrz (short for German "ganz kurz", "very short"). And with it's latest commit, it acquired a simple API. I use it as such:

#!/bin/sh
URL=`xclip -o`
SHORT=`wget -qO- "http://krzz.de/_api/save?url=$URL"`
(echo -n $SHORT | xclip -i -display :0 -loops 0 ) &
/home/feh/bin/notify-wrapper "krzz.de: switched URL in clipboard" "$SHORT"

Now I simply bind a hotkey to call this program, and it will exchange the URL from my X clipboard with a shortened version.

statically linking dwm against X11 and XCB

2011-08-05T01:55:00+02:00

Today, virtually all binaries used on linux systems are dynamically linked to several libraries. While it is commonly accepted statically linking applications is bad – most notably in terms of security concerns: fixing a library's bug means you won't have to recompile all applications that are using that special library, they'll simply load the version available at run-time – there are in fact good reasons to use static linking. (And for those who claim statically linked binaries occupy much disk space: yeah, sure. As if a few megs compared to a few hundred kilobytes make that much a difference today, plus you don't have the overhead of looking up and loading the libs in the first place.)

As I mentioned in my post about tmux already, there's a huge advantage to static linking: you can compile bleeding edge software with bleeding edge library functions and still use them on reasonably outdated systems (think: Debian stable).

One division of rapidly evolving software I could never successfully link statically was window managers like dwm or awesome. However, especially considering the XCB development and adoption over the past few years, to me it makes perfect sense. I'll just distribute a copy of the window manager I use to different systems and have a guarantee it'll work there, no matter the libxcb version (or if it's available at all).

Usually, however, it's not possible to just pass a -static or -Wl,-Bstatic flag to the compiler (in my case, gcc). It'll fail to find several symbols that are located in libraries that don't have to be explicitly linked in. Such an error message might look like this:

/usr/lib/libXinerama.a(Xinerama.o): In function `find_display':
(.text+0x89): undefined reference to `XextCreateExtension'
/usr/lib/libXinerama.a(Xinerama.o): In function `XineramaQueryScreens':
(.text+0x255): undefined reference to `XMissingExtension'

To find the appropriate library, you may try to use pkg-config. I use a different approach, however. I have a shell function defined called findsym (beware, Z-Shell specialties apply):

findsym () {
  [[ -z $1 ]] && return 1
  SYMBOL=$1
  LIBDIR=${2:-/usr/lib}
  for lib in $LIBDIR/*.a
  do
    nm $lib &> /dev/null | grep -q $SYMBOL && \
      print "symbol found in $lib\n -L$LIBDIR -l${${lib:t:r}#lib}"
  done
}

Thus, I can simply go looking for the missing XMissingExtension symbol like this:

$ findsym XMissingExtension
symbol found in /usr/lib/libXext.a
 -L/usr/lib -lXext
symbol found in /usr/lib/libXi.a
 -L/usr/lib -lXi
symbol found in /usr/lib/libXinerama.a
 -L/usr/lib -lXinerama
symbol found in /usr/lib/libXrandr.a
 -L/usr/lib -lXrandr

Now, I use the readme file, some common sense or symple try'n'error to find out which library I'd best link in, too. In this case, it's adding a simple -lXext to the LDFLAGS part.

Thus, I come up with the following diff to dwm's config.mk:

--- a/config.mk
+++ b/config.mk
@@ -16,7 +16,7 @@ XINERAMAFLAGS = -DXINERAMA

 # includes and libs
 INCS = -I. -I/usr/include -I${X11INC}
-LIBS = -L/usr/lib -lc -L${X11LIB} -lX11 ${XINERAMALIBS}
+LIBS = -L/usr/lib -L${X11LIB} -static -lX11 ${XINERAMALIBS} -lxcb -lXau -lXext -lXdmcp -lpthread -ldl

 # flags
 CPPFLAGS = -DVERSION=\"${VERSION}\" ${XINERAMAFLAGS}

There's one important point here: libX11 will (to me, it seems, inevitably) load another library, not sure why or which one. Thus, it is vitally important to statically link in libdl, the library that dynamically loads another library. Otherwise, the follwing error messages appear:

/usr/lib/libX11.a(CrGlCur.o): In function `open_library':
(.text+0x3b): undefined reference to `dlopen'
/usr/lib/libX11.a(CrGlCur.o): In function `fetch_symbol':
(.text+0x6b): undefined reference to `dlsym'
/usr/lib/libX11.a(CrGlCur.o): In function `fetch_symbol':
(.text+0x88): undefined reference to `dlsym'

With the above modification to config.mk, dwm will compile and link just fine:

$ file dwm
dwm: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
statically linked, for GNU/Linux 2.6.18, not stripped

You can reduce the binary's size by a few hundred kilobytes by manually calling strip(1).

The binary works very well for me. I'll try to use it on different systems over the next few weeks and see what happens. If that works out well, I'll also try to get lucky with awesome and zathura, as these (and the libraries needed) are not installed on many systems, either.

switching screen for tmux

2011-07-21T21:35:00+02:00

I had invited Nicholas Marriott to LinuxTag 2011 in Berlin to give a talk on tmux. I was conviced I should drop screen for tmux, but ... Yesterday would be the day I first installed the program, and by now I'm actively using it.

It was not without pressure, though. I had volunteered to give a talk on tmux at work, and today was the date. See the quick'n'dirty slides here (in German).

tmux needs a fairly recent libevent. So if you want to run a recent tmux version (1.5) on a Debian stable system, you should consider linking it statically like so:

$ ./configure --enable-static
$ sed -i '/^LIBS =/s/$/ -lresolv/' Makefile
$ make CC='gcc -static'
$ cp tmux ~/bin

The sed call is needed to make the static linking work, otherwise gcc will fail to find the symbol __b64_ntop, which is part of glibc's libresolv.

First action to be taken is to make Ctrl-A the prefix. I mean, seriously, why would you use Ctrl-B? It's for "previous page" or previous argument!

unbind C-b
set -g prefix C-a
bind a send-prefix
bind C-a last-window

Next is the status bar. The syntax is pretty self-explanatory.

set -g status-bg yellow
set -g status-fg black
set -g status-interval 5

set -g status-left "#[fg=red]#S %H:%M "
set -g status-right "#H: #(cut -d' ' -f1-3 /proc/loadavg)"

set -w -g window-status-current-bg red
set -w -g window-status-current-fg yellow

Also, cycling through the windows becomes easier. The -r switch to bind allows half a second time (set via repeat-time) to perform the next keystroke without pressing the prefix again. So <prefix><space><space><space> will move you three windows ahead:

# repeated space/backspace toggles windows forward/backwards
bind -r Space next-window
bind -r C-Space next-window
bind -r C-h previous-window
bind -r C-? previous-window

This feature is very handy for resizing or switching panes repeatedly. (Read more about sessions/windows/panes in the docs...)

bind -r h select-pane -L
bind -r j select-pane -D
bind -r k select-pane -U
bind -r l select-pane -R

Having multiple panes in one window is easy enough. Splitting panes, however, is sort of counter-intuitive: A horizontal split (split-window -h) will divide the current pane into two panes that are horizontally next to each other – i.e., a horizontal split will introduce a vertical split line between to panes. This resource has a nice idea to make splitting more idiomatic:

bind | split-window -h
bind - split-window -v

For a great feature, check out link-window. If you do name your sessions and windows in a predictable way so that you can match them via fnmatch(3), you'll be able to do cool things like this:

bind M link-window -s comm:mutt*
bind I link-window -s comm:irssi*
bind K unlink-window

So with Prefix-M you'll quickly bring up the mutt instance of your communication's session, and will detach that single window with Prefix-K again. That's especially handy for things like Mail and IRC client, music player, Todo Lists ...

Side note: Sadly, it's not possible without some tricks to link a window from another session to a pane (sub-window) in the current session. (Think a ten-line-high IRC client at the bottom of your current window.) The FAQ says "[fixing this] is a big todo item but quite invasive".

Sprunge, the fine pastebin service

2011-07-20T23:55:00+02:00

I have never understood why automatically pasting to a pastebin service should be so hard as in "I need a huge script to do the job!".

Luckily, once in a while there are smart people that build good and simple applications. Enter sprunge, the first truly sane pastebin service. Need to know how it works? You'll find the man page on their website (no, really). – No listing of pastes. No annotation, comments, amendments. No foobar. Paste away!

$ sprunge < ~/bin/sprunge 
 http://sprunge.us/OcPR

$ wget -qO- http://sprunge.us/OcPR
#!/bin/sh
exec curl -F 'sprunge=<-' http://sprunge.us

Represent directory structures in LaTeX

2011-07-18T12:13:00+02:00

When writing the git book we faced a problem: when dealing with an example repository, how do you describe a directory structure?

Simply "writing" about it is the cheap apporach, although pretty stupid: most readers (including me) would skip over the paragraph. Later, they'd be confused when we'd refer to a specific property of the repository layout.

The intermediate approach was to simply paste the output of the tree unix command. That was hard to "parse" though, and didn't look well.

It is clear that a graphic representation of the directory structure is optimal. But: how? The easiest way would be to install some graphical application that can represent tree-structures, such as Nautilus. There are no upsides to this, but two minor downsides: it looks crappy, and it requires you to install some hundred megabytes of gnome-wtf-libs.

The major downside is this: If you want to re-do the screenshot, you'll need the same directory structure, the same setup, GNOME styles, same selection to grab, ... there is no easy way to automate this.

Not wanting to settle for the cheap way, what were the goals our solution had to meet?

can be understood at one glance (important)
can be automated and tracked in git (important)
does not eat up much space on the page
doesn't depend on colors

An alternative that came to my mind was to use GraphViz's neato tool. I wrote a Perl script to convert a directory structure to a diagram. The code's too cruel to be released to the world. You can imagine what it looks like by invoking:

neato -Txlib <<EOF
graph G {
  n1 [shape=folder width=1 style=filled label="baz" pos="0.00,0.00!"];
  n2 [shape=box height=.2 width=1 labelloc=t label="file" pos="0.00,-0.60!"];
  n3 [shape=folder width=1 style=filled label="foo" pos="0.00,-1.20!"];
  n4 [shape=folder width=1 style=filled label="bar" pos="1.20,-1.20!"];
  n3 -- n4;
  n1 -- n2 -- n3;
}
EOF

However, this output did not meet above criteria #3&4. It looked even cheaper than simple tree output.

The solution we settled for was dirtree.sty. It works like this: You download the dirtree.sty and dirtree.tex file, place them within your project and add a \usepackage{dirtree}. In theory, you can then include a \dirtree{} declaration (see the docs for details).

For example, this code (note the percent sign on line #1)...

\dirtree{ %
.1 .git/.
.2 HEAD.
.2 config.
.2 hooks/.
.2 index.
.2 info/.
.2 logs/.
.3 HEAD.
.3 refs/.
.2 objects/.
.3 info/.
.3 pack/.
.2 refs/.
.3 heads/.
.3 remotes/.
.3 tags/.
}

will produce the following graphic:

However, since it's not part of the LaTeX standard distribution, you cannot be sure about it's stability. It may use "dangerous" LaTeX constructs, etc... certainly nothing you want to include "as-is" in a book that'll eventually be printed. Also, the book will go through a conversion process to be distributed as an Ebook. You wouldn't want such a little LaTeX hack to be a show-stopper – or, worse yet, discover document corruption just after you got some thousand fresh copies from the printing press.

EPS/PDF files are stable, however. So what we did was this:

Put the \dirtree{} statements into separate files, including some \begin{document} stuff so that it will compile to a PDF file containing just the diagram.
Remove any whitespace. We use pdfcrop for that.
Convert to EPS.

In terms of Makefile statements, Valentin came up with this:

files=objektmodell-programm-crop\
    svn-stdlayout-crop\
    svn-nonstdlayout-crop\
    svn-branches-crop\
    git-branches-crop\
    git-dir-crop

pdfs=$(addsuffix .pdf, $(files))
epss=$(addsuffix .eps, $(files))

all: $(pdfs) $(epss)

clean:
    -rm -v *.pdf *.eps *.aux *.log

%.pdf : %.tex
    pdflatex $<

%-crop.pdf : %.pdf
    pdfcrop $<

%-crop.eps : %-crop.pdf
    pdftops -eps $< $@

Now it's as simple as make -C dir-listings && git add dir-listings (with an appropriate .gitignore file) to record changes to diagrams and recompile the PDF and EPS files. If you don't plan to keep the compiled files in your repository, you can also add a dependency for the subdirectory to your main Makefile.

I failed the social web, again

2011-07-12T12:59:00+02:00

For some days now, G+ has opened its doors to beta testers. All the people want invitations, and Reddit realized for a long time already that Facebook will be the new MySpace.

Now, last tuesday or so I saw G+ at a friend's, who sent me an invitation. Friday, I pondered the question of whether or not to sign up for G+ – literally for hours. (I have to admit, though, that while lying on my bed and considering the arguments, I fell asleep for a short period.)

I had seen some of the freshly-made Google profiles of the avant-garde – that is, technophile persons whose blogs I enjoy from time to time – and was rather impressed of the quality content they posted. So I decided to give it a try.

Now this was a major step! G+ would be the first Google product I would use that requires registration. So I went through the process and was eager to try out G+. However, as I had heard already, getting an invite doesn't mean you can use it right away, it might take a day or two.

To make a long story short: While waiting for this timespan to elapse, I browsed through various circles. And what I saw struck me as a Facebook for grown ups: People I don't know talking about stuff I don't care about, in a way that might be appropriate in a bar atmosphere, but not to be archived publicly until ... forever or something.

A day later I got a second invite, and I could continue to the G+ thing. However, the site that came up told me my browser was not supported (it's FF 3.5). So, it seems I'm not avant-garde enough to use G+.

I just deleted my freshly created Google account. I mean, come on, it's sunny outside and there's a pile of unread books waiting!

Southern Sudan's Secession

2011-07-07T19:35:00+02:00

I am closely following the secession of Southern Sudan. It is a very intriguing situation – the birth of a new state.

Information is scarce about this topic. The Sudan Tribune provides some insights, but most quality coverage is done by Al Jazeera here. Some facts to consider (quoted from here):

The political system is dominated by one party [the SPLM]
Other opposition parties are extremely weak
Parliamentary elections in April 2010 were marred by widespread reports of intimidation and fraud
Juba [the Capitol-to-be] produces more than three-quarters of Sudan's 500,000 barrels per day of oil
98 per cent of government revenue comes from the petroleum sector
Southern Sudan has just 60 kilometres of paved roads, and electricity is provided only intermittently by generators
southern officials estimate that between 150,000 and 200,000 people – in a region of 20 million – are on the army's payroll

Eritrea, Libya and Iran have already stated they won't recognize Southern Sudan as a state. Most western countries probably will, as plans are persued to open embassies in Juba. (Juba has just been getting a new international airport – I wonder how much fuel they keep on stock to operate it just with generator power?)

Perl: use feature;

2011-07-02T14:38:00+02:00

With it's 5.10 version, Perl learned some pretty interesting new features. Some of them modify the syntactic interpretation of Perl code, so to stay backwards-compatible, you have to manually enable them. To do this, add a line like

use feature qw(:5.10);

to your other use declarations. (For one-liners, add the -E command line switch.) The documentation is available via perldoc feature or online.

There's one pretty trivial change that nonetheless has bothered me for a long time already. Why is there no function in Perl to print out some strings, and then finish with a newline? I mean, almost every sane programming language has this, and it helps writing clean code a lot (no fiddling with \n and so on). – Now, they backported the say function from Perl 6, which does just that: add an omplicit newline.

The new concept of state variables is nothing I need for now; however, the feature they strangely call switch (although the actual keywords are given/when) is pretty nice: It's pretty much like what you'd expect with a typical C construct like this:

switch(var) {
    case 1:
        /* do something */
        break;
    ...

    default:
        /* do default thing */
        break;
}

In Perl that's now:

given($var) {
    when(condition1) {
        # do something
    }

    do_other_thing when condition2; # as one-liner

    default {
        # fallthrough case
    }
}

Update: The statement when condition; one-liner doesn't work. Don't know where I got that from.

You can use simple strings (maps to $_ eq "string"), regular expressions (maps to $_ =~ /regex/) or function references like \&myfunc (maps to myfunc($_)).

The best improvement, however, are so-called named captures. (You don't have to enable those specifically.) It allows regexes to contain pairs of parentheses that you can assign a name to later extract the match. That means: no stupid and mind-boggling re-numbering of $2 to $3 etc., just because you added a set of brackets around something you want to extract from a string. Consider this example regex:

my $re = qr/
    (?<user>[^\@\s]+)
    \@
    (?<domain>[^\s">]+)
    /x;

You can now access the user-part of the e-mail address via $+{user}, not necessarily via $1. The immediate gain is obvious: if you add parentheses around the whole expression (to capture the whole e-mail address) the user part is still available via $+{user}. The numbered variable would be $2 now, though.

Another side-effect is that you can test several regexes in a row now, and later extract matching parts from all of these – of course only if you use distinct capture names. Hash entries in %+ will only be updated upon a match (and won't be cleared AFAIK, at least in the lexical scope).

ending the silence

2011-06-13T18:14:00+02:00

It has been a little more than three months since I last posted something here in my blog. Considering that the first post ever in this blog was from 1st of January this year, this pretty much looked like a "tried to blog, but gave it up again" thing.

I was really busy, however, and was simply not able to write a single post. What really ate up all my time was my latest pet project, writing a German book about Git. Valentin (my co-author) and I worked really hard throughout the past few weeks – only got up once in a while to get something to eat and stock up on Club Mate. The book is being published at Open Source Press and will be available from the end of June. Go buy it!

Now, with a lot of free time on my hands, I can finally get back to my studies (yes, really). Also, I will devote more time to this blog. :-)

git: find absolute path to tracked file

2011-03-08T03:22:00+01:00

If you need to name a blob by name, you can't trust Git to honor the current working directory. If you are in a subdir, git show HEAD^:file won't work unless you prepend the subdirectory name.

But there's a sort of basename for absolute paths within a Git repo:

git ls-tree --name-only --full-name HEAD file

This honors $PWD and thus returns a pathname that's unique across the repository. You can build a simple "diff to previous version" script with this now:

#!/bin/sh

test ! -z "$1" || exit 1
temp=${1%.*}.HEAD^1.$$.${1#*.}
fullpath=`git ls-tree --name-only --full-name HEAD^ $1`

test ! -z "$1" || exit 2

echo "extracting '$fullpath' from HEAD^..."
git show HEAD^:$fullpath > $temp
vim -fd $1 $temp
echo "cleaning up..."
rm $temp

This can act as a simplified mergetool when you just want to review changes made to some file.

git: forget recorded resolution

2011-03-01T05:24:00+01:00

Git's rerere is immensely useful. However, if you erroneously record a wrong conflict resolution, it's not entirely clear how to delete it again. The preimage file is stored under $GIT_DIR/rr-cache/some-sha1-sum/preimage, which is not a convenient place to look for that one wrong resolution if you have hundreds of them. ;-)

However, v1.6.6-9-gdea4562 introduced rerere's forget subcommand, which was not documented until recently (v1.7.1.1-2-g2c64034). So when rerere replays a wrong resolution onto your file, simply use git rerere forget path/to/file to forget the wrong resolution.

To (temporarily) replace the resolved version with the originial version containing conflict markers, use git checkout -m -- path/to/file. You can then issue a git rerere to replay the resolution again.

vi mode in GNU readline

2011-02-28T11:56:00+01:00

I am stuck in a course about computer algebra systems. We are using Singular. I searched for some minutes how to switch on the vi mode on the Singular command line, but apparently, there is none.

However, since Singular uses the GNU readline libs, you can simply switch to vi mode with Ctrl-Meta-j. – How convenient is that. I wonder if there's a possibility to configure readline to switch to vi mode by default.

criss-cross subtree merges

2011-02-19T19:55:00+01:00

Note to self: It is bad to rely on Git's magic of finding out whether you can fast-forward when doing a merge. The more clever way is to explicitly state what you want to do with aliases:

nfm = merge --no-ff     # no-ff-merge
ffm = merge --ff-only   #    ff-merge

I stumbled upon this today when I played around with subtree merges. Strangely, the results with the strategy subtree were not the same as with recursive's subtree=path option:

$ git merge -s recursive doc
$ git merge -Xsubtree=Documentation doc

Turns out that – in the special case I had created – Git could fast-forward the branch although it was a subtree merge, effectively ignoring any strategy option. Duh. I had already read, understood and tried all the tests related to merging before I had figured that out.

Zathura

2011-02-15T18:38:00+01:00

I like small, clean software, that focuses on keyboard usability. PWMT, "programs with movie titles", has a nice and fast(!) PDF viewer called Zathura. The user interface looks just like Vimperator (or Vimprobable, for that matter). I like!

The config file syntax is not documented. (Only on the web site, which is not available at the moment.) So you'll have to change the colors within the code itself. libpoppler-glib-dev is required to build Zathura.

git tracking branch management

2011-02-10T16:33:00+01:00

There are different approaches 'round the net on how to deal with git remote tracking branches in an effecient way. Some people have written Ruby gems (uggh) that introduce lots of not-so-intuitive git subcommands to manage all kinds of tracking branches.

In recent versions of Git however, I have no need for these scripts. I can live with the following aliases:

lu = log ..@{u}
ft = merge --ff-only @{u}
track = branch --set-upstream

To track a branch, I enter git track master origin/master, for example. (By the way, if you create a new branch and want to push it to a remote, use push -u – it will automatically set up your current branch to track the remote branch you just created.)

Then, my regular workflow looks like this:

$ git remote update     # fetch changes
$ git lu                # log ..upstream -- review changes
$ git ft                # ff tracking branch -- integrate

The key here is using @{u}, which is a short form of @{upstream}, a reference to the upstream branch (that is, the branch your HEAD is tracking). This special notation first appeared in Git 1.7.0.

If you're not sure about the current status of your branches and what they are tracking, simply use git branch -vv. You'll find the tracking information in brackets just before the commit description.

Egypt's net censorship and DNS tunneling

2011-02-03T18:19:00+01:00

I got a call yesterday – from Egypt. It was around noon local time, i.e. shortly after Egypt's government had restored regular internet access (see also, more graphs). At the time of the call, I didn't know that, though.

The guy asked me whether I could help him set up DNS tunneling. Today, he called again and inquired how things were going. He told me he wanted to have a backup internet connection if things would get messy again.

I think, however, that DNS tunneling wouldn't be able to bypass the internet block. On the day of the shutdown (Jan 27th, shortly before midnight UTC),

"Approximately 3,500 individual BGP routes were withdrawn, leaving no valid paths by which the rest of the world could continue to exchange Internet traffic with Egypt's service providers, ... Virtually all of Egypt's Internet addresses are now unreachable, worldwide."

For a more technical analysis, see this BGPmon blog post.

What this means is that Egypt was essentially split off the net. Thus, even a DNS query from inside Egypt (which would be relayed to a server in, say, Germany) could not be forwarded by the provider's DNS servers. Neither could a DNS request from Germany reach a name server located in Egypt. This blog post claims that the providers did shut down DNS access as well (which wouldn't matter much in that situation, anyway).

So, to evade this blockade (i.e., IP won't get routed) it takes more than just trying to sneak data out using covert IP channels.

There are other methods, of course. Protesters set up a voice to tweet converter:

"Over the weekend we came up with the idea of a speak-to-tweet service – the ability for anyone to tweet using just a voice connection," they said.

Voice mail messages left at +16504194196; +390662207294 or +97316199855 will instantly be converted into text messages, referred to as tweets, and posted at Twitter with an identifying "hashtag" of #egypt.

You could use other radio infrastructure as well to communicate with other countries, which would then feed your data stream into the net. That's not possible on a large scale and for the typical user, however.

Firefox: Open in Browser

2011-02-01T23:49:00+01:00

It's been bothering me for years. There are a lot of file types that are essentially plaintext, but are labeled differently to make the browser's/mailer's decision easier which program is best to handle this type of file. For example, text/x-diff.

Now, how hard could it be to include a little checkbox "display in browser" in the file download dialog? I mean, it's text after all. But, apparently, it is quite hard. Yes, that's no display failure, that thing has been suggested at the end of 1999 and the bug report pushed back and forth since then. Stupid morons.

However, there's a nice addon which does just this job.

On a related note, since the last upgrade, opening context or dropdown menus will take Firefox up to 30 seconds sometimes. No heavy CPU or disk usage, the application just freezes.

I hope I'll find some time during the next few weeks to migrate to a saner alternative.

Google: Fast search result retrieval

2011-01-31T02:23:00+01:00

This whole Google jQuery autosuggestion bothers the hell out of me. I mean, why would you use several keystrokes and possibly the mouse to find a page, continuously requesting suggestions in the background?

Often I use Google to find a page I can pretty safely identify with some simple keywords. For example, in an IRC conversation I want to give someone a link to Al Jazeera's page about current events in Egypt.

Stupid way: 1. fire up browser, 2. enter search terms in address bar (:tabopen in Vimperator), 3. wait for the kilobytes to trickle through the line, 4. find your way around the page, 5. select link, 6. copy it somehow, 7. switch back to IRC window, 8. paste URL. Estimated time amount: ranging from anywhere between 10 seconds to 30 seconds.

Clever way: write a shell script to extract "I am feeling lucky" result.

#!/bin/sh

test -z "$1" && exit 1

URL="`curl -s -i -e 'http://www.google.com' -A 'Firefox 23' \
    -G -d hl=en -d btnI=lucky --data-urlencode q=\"$1\" \
    http://www.google.com/search | grep '^Location:' | \
    head -n 1 | cut -d' ' -f2 | tr -d '\r\n'`"

echo -n "$URL" | (xclip -i -display :0 -loops 0 ) &
/home/feh/bin/notify-wrapper "Google 1st result" "$URL"

(Footnote: You need a more or less sensible User-Agent string (-A) as well as a request on www.google.com, otherwise you won't be served a result.)

Then, bind a key to open a prompt in which you'll enter your search term. For awesome, this is:

awful.key({ modkey }, "g",
  function ()
    awful.prompt.run({ prompt = "Google: " },
      mypromptbox[mouse.screen].widget,
      function (s)
        awful.util.spawn("/home/feh/bin/glucky '" .. s .. "'")
      end,
      nil, nil)
  end
),

The notify-wrapper script uses DBus's notify-send to make a little notification box appear as soon as the URL is retrieved (which is, like, instantaneous).

Clever way works like this now: 1. Press Mod1-g, 2. enter search term, 3. paste URL. Time you need to find a popular link: Time to type search string + 1 second. Huge time saver.

Upgrading to Gitolite and CGit

2011-01-31T00:42:00+01:00

I finally found some time to update the git version used on my server. It's running a 1.7.1 now, which is just slightly better than the previous 1.5 version. ;-)

Debian package maintainers bother me again, though. The simply erase the git user and put a gitdaemon user in it's place, without asking for confirmation. Will probably happen with the next upgrade, too...

Now, having a recent git version enables me to set up gitolite instead of the old gitosis. Setup is easy, straight-forward and described in many places.

I used this script to convert my existing gitosis.conf to match gitolite's syntax; I noticed a major hiccup though: setting a description automatically enables gitweb access. I had to read the source to find this out, because I couldn't remember it from the docs and it didn't once cross my mind that you could actually want the behaviour to be like that. – As in, you could just describe a project without wanting to make it public – no?

However, with gitolite, I can finally use CGit too (because only gitolie produces a repositories.list which is in the format that CGit understands, ie. without description appended to the lines).

So now the repository page is all new and shiny. See, for example, this side-by-side diff. Nice, and faster than Gitweb.

Integrating CGit with Lighttpd was another two-hour fiddle-try-fail-repeat session. I finally came up with this snippet:

$HTTP["host"] =~ "^git\.(plenz\.com|feh\.name)(:\d+)?$" {
  alias.url = (
    "/cgit.css" => "/usr/share/cgit/cgit.css",
    "/cgit.png" => "/usr/share/cgit/cgit.png",
    "/cgit.cgi" => "/usr/lib/cgi-bin/cgit.cgi",
    "/"         => "/usr/lib/cgi-bin/cgit.cgi",
  )
  cgi.assign = ( ".cgi" => "" )
  url.rewrite-once = (
    "^/cgit\.(css|png)" => "$0",
    "^/.+" => "/cgit.cgi$0"
  )
}

It's important here that you noop-rewrite the CSS and PNG file to make Lighty expand the alias. Also, rewriting to an absolute filepath won't work; that's why the cgit.cgi aliases have to appear twice.

add -p for new files

2011-01-30T22:19:00+01:00

If you have a file that's unknown to git (ie., it appears in the "untracked" section of git status), and you want to add only part of the file to the index, git won't allow it: No changes.

Instead, mark the file for adding, but without adding any actual line:

$ git add -N file
$ git add -p file

-N is short for --intent-to-add and does just that: Create an index entry with an empty content field.

dec2bin for the poor

2011-01-26T21:18:00+01:00

Convert 0x53 to binary, quick! – Now you start to think: In which language could you implement a dec2bin function within a few seconds? Or just do it with pen and paper, old school?

But wait, there's poor man's dec2bin!

$ echo -n "\x53" | xxd -b
0000000: 01010011

;-)

Strip whitespace on rebase

2011-01-25T17:12:00+01:00

Normally, I use the following line in Git's pre-commit hook:

git diff --cached --check || exit 1

This makes a git commit abort if there are any whitespace-related errors. However, if you have code that already has whitespace issues, you can simply pass an option to git rebase to automatically correct them:

git rebase --whitespace=fix reference

N.B.: Rebase just passes this argument to the git apply command.

Which files are accessed?

2011-01-24T01:40:00+01:00

Often, if you need to know which files are accessed during the startup of an application, you are stuck searching for them in the manpage. Also, probably not all of them are documented there.

I use strace to find these files:

strace -efile /usr/bin/program 2>&1 1>/dev/null | grep ^open | cut -d\" -f2

Note that unless you place the grep expression in between this will give you all syscalls that involve files, i.e. also simple checks whether a file exists or has a certain permission set.

ACPI interface in /proc deprecated

2011-01-16T21:53:00+01:00

With a new kernel, /proc/acpi/battery/* was suddenly missing. Turns out it the interface is is deprecated but can be turned on with a config option at compile time. The same information is available in a different form in the /sys filesystem.

I didn't see that coming; almost all programs I use to check the battery state fail. It had been announced, actually:

The correct way is for upstream to announce the change in feature-removal-schedule.txt, which they did 2 years ago.

That's roughly how the Vogon Hyperspace Planning Council operates. Stubborn dev replies:

If the tools are not able to use them, it is their bug. No bug in the kernel.

However, after updating vicious and adjusting some API changes, everything works fine again.

Dubstep – it's kinda like that

2011-01-16T21:11:00+01:00

Using OpenWRT (again)

2011-01-14T21:38:00+01:00

The old and cheap wifi router we used in our flat got buggy; it had some 20% packet loss at times, and the LAN clients were not able to make connections to each other (only WAN connections were possible).

So I dug up my old WRT54G, reset the factory settings, and used that one as access point. This worked for about one week; today, when I came home, the internet connection would not work any more.

Ping statistics from the router (delivered to the web interface) showed that the internet connection was established. However, the router would not forward internet traffic from the LAN.

An hour ago I installed OpenWRT. I had been using it some years ago and was astonished that "White Russian" is still the stable release. Anyhow, there's a web interface now which is pretty nice. (With a link "Review settings", where you'll get a shell script that will get executed once you hit "apply" – nice if you want to find out the commands!)

There was some manual work to do, I leave this for search engines in case someone else has these issues.

WPA2-PSK setup

WPA2 is not natively supported. Although the nvram keys exist, the wifi driver cannot use them properly. iwconfig will display Encryption key:off regardless of what you configure.

To make WPA2 work, install the nas package (documentation). The /etc/init.d/S41wpa script actually manages WPA2 settings (and retrieves the configuration keys from nvram):

# ipkg install nas
# /etc/init.d/S41wpa restart

The relevant nvram keys for my wireless are:

wl0_akm=psk psk2
wl0_crypto=tkip+aes
wl0_ssid=...
wl0_wpa_psk=...

PubKey Authentication with DropBear

Simple, once you know it:

$ scp ~/.ssh/id_dsa.pub root@192.168.1.1:/etc/dropbear/authorized_keys

NTP

Keep up to date!

# nvram set ntp_server=time.fu-berlin.de
# nvram commit
# ipkg install ntpclient

Interestingly, the signal strength and bandwidth throughput is better with OpenWRT than using the LinkSys firmware. :-)

Automatic screenshot processing

2011-01-11T21:31:00+01:00

Cut out window borders and application bars from screenshots automatically: note down some corner points (in Gimp), then use ImageMagick:

for f (*.png) { convert -crop 975x559+2+30 $f $f; }

Task done!

NB: This command replaces file contents. So when not sure about the numbers, append some extension to it and view the results first. Or, simply kepp the images in a git controlled directory. That way, a simple git reset -- . will restore your previous images if you added them in the first place.

Kamusi and jQuery

2011-01-10T23:10:00+01:00

So some years ago I was what what one would call a Web Designer, doing CSS and XHTML 1.0 and that stuff. I got bored, however, and nowadays I hate these websites that are so over-styled and have so much JavaScript in there. I have been turning away from any web-related stuff since the times writing valid XHTML 1.0 was still "in".

However... today I tried this whole "Use JavaScript instead of regular links and reloads" stuff out. Friends of mine suggested using jQuery, of which I grabbed the "mini" version. (There have been some really clever code golfers at work to shrink the whole library down to some 77K. In that format, it's barely human-readable, though.)

jQuery is nice, because it doesn't have many JavaScript parts in it. The "usual" stuff seems to work by simply combining elementary jQuery functions, for which there's a great API documentation, and identifying HTML objects in a syntax similar to CSS.

So without really understanding how and why it works, I created a Web2.0-ish version of my kamusi command line client. Kamusi is the Kiswahili word for dictionary, and the program I wrote is a text-based interface written in Perl for the extensive database of words from http://kamusi.org/. Their web search, however, is very slow and irritating to use.

My hack-away implementation of a cleaner, faster search using jQuery and a custom CGI to retrieve the results can be found at http://k.plenz.com. Some sample queries you might try: -nia, bongo or -pa. Source code is availabe in the "cgi" branch on Github.

This jQuery stuff is hard to debug, actually. (JavaScript in general is, IMO.) However, the syntax is pretty straight-forward. I do realize now why so many of these Web2.0 pages use this fading and scrolling-in stuff so much (and autosuggest, as well!). It's just plain easy and provides a feel of using a "dynamic application".

I can understand now why these Web Designers are so proud of themselves.

Wikileaks mass mirroring project

2011-01-09T22:26:00+01:00

Since a day after Wikieleaks called for mass mirroring support I've been hosting a Wikileaks mirror at http://wikileaks.feh.name.

If you're curious how to allow to upload files via rsync, but just to a certain directory with no possibility to access other system parts, take a look at rrsync, which should ship with rsync. On a Debian system, simply do:

# cp /usr/share/doc/rsync/scripts/rrsync.gz /usr/bin
# gunzip /usr/bin/rrsync.gz

Now, I have this line in ~wl/.ssh/authorized_keys:

command="rrsync www" ssh-rsa AAAAB3N[...]w== wwwsync@wikileaks.org

So any rsync mirroring command issued will have a $HOME/www/ prepended. Thus, the wikileaks upload script can only modify files below a certain subdirectory. If you touch the file rrsync.log, each login will be logged as well.

Now, there are some 1400 sites mirroring the content now. How to access a random mirror, then? The German taz had a call for action page with a strange bash script which would always link to the first mirror in the list.

However, there are more clever approaches. http://wikileaks.de, for example, uses round robin DNS to display the cablegate pages of a randomly chosen host. However, there seem to be only five hosts registered yet:

$ ( repeat 100 host wikileaks.de ) | cut -f3 | sort -n | uniq -c
    100 85.214.146.40
    100 87.230.26.252
    100 87.239.128.202
    100 88.198.21.150
    100 178.63.249.117

I just registered my IP to show up there. You can, too! (Be sure to click the "commit changes" radio button when you receive the confirmation link!)

Another possibility is to point your browser to http://r.whereiswikileaks.org/, which emits a 302 HTTP header ("found") and adds a Location header to redirect you to a randomly selected mirror:

$ curl http://r.whereiswikileaks.org/ -s --head | grep Location:
Location: http://wikileaks.my-niap.org

Some french guy made interesting pie charts about the distribution of the 1400 mirrors. German mirrors are most common (30% of all active), mirrors from the US, France, Germany and the Netherlands combined make up more than three quarter of all active mirrors. There are other vizualizations, too.

Finally, there's a site grouping wikileaks mirrors by netblock. Interestingly, more than 100 mirrors are within German ISP Hetzner's netblock (whose position on Wikileaks hosting had not been clear entirely during the first few days).

X41 keyboard problem

2011-01-06T21:23:00+01:00

Today I fixed that annoying keyboard problem I talked about earlier. It was really simple: remove keyboard, clean contacts, put it back in.

Tuning old hardware with slow hard drive

2011-01-05T23:54:00+01:00

My main work machine is a pretty old X41 with a 40GB hard disk and 512MB of RAM. It is more than five years old and is not without problems. (In recent months, I have to try several before switching it on successfully – in most cases, it just beeps twice and displays "Keyboard error, <F1> to configure" and the keyboard doesn't work.)

However, there's a thing which annoys me a lot: bad performance. I use a resource-friendly window manager with some urxvts running. Apart from the memory-hog Firefox, I very seldom use any graphical application (ie. any program using the GTK or Qt libraries).

For some weeks now I've been trying this cgroups hack, with mixed results. In some cases, the performance is better, sometimes it's not.

How bad could the overall performance be, then? – Unfortunately, very bad. Which has, in part, to do with my slow hard disk. It does uncached reading with 18MB/s in theory:

$ sudo hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads:   56 MB in  3.08 seconds =  18.21 MB/sec

In reality, it's rather some 16.5MB/s:

$ dd if=/dev/zero of=./zero bs=1048576 count=256
256+0 records in
256+0 records out
268435456 bytes (268 MB) copied, 10.0246 s, 26.8 MB/s

$ time cat zero > /dev/null
cat zero > /dev/null  0.01s user 0.25s system 1% cpu 16.246 total

Now, I have to live with this (SSD's are still very expensive!). The main problem here is that the Kernel swaps out data that wasn't accessed for a while (although, from a naïve perspective, there's no ultimate need to do so since there's still free memory left).

I actually notice that with two programs regularly:

When I hadn't taken a look at Firefox for a few minutes, it might take up to 10(!) seconds to redraw the window.
When I hadn't opened an urxvt for some time, opening a new one might take up to 5 seconds; initializing the shell another two.

Now I always thought this was the Linux Kernel being stupid. However I discovered a switch today. From the sysctl.vm documentation:

swappiness

This control is used to define how aggressive the kernel will swap
memory pages.  Higher values will increase agressiveness, lower values
decrease the amount of swap.

The default value is 60.

Debian (like all other distros) seem to keep this default value. After reading up on some articles I set vm.swappiness=0 in /etc/sysctl.conf. (You can do this interactively with sysctl -w vm.swappiness=0 also. Interestingly, Ubuntu recommends a value of 10 for desktop systems.)

For the past day or so, I have been monitoring the output of vmstat 1 every now and then (especially the swap in/out parameters si and so). But even after the first hour one thing is evident: the interactive system performance is much, much better. It feels like a machine upgrade.

Terminals open instantly (because the initialization parts of their binary doesn't get swapped out, for example). Switching to Firefox is instant. Switching tabs is fast. The system feels a lot more responsive.

Where's the drawback, then? If you could magically tune your system's performance, why wouldn't you want do that?

A case where this setup will give you a headache is when you actually do run out of memory. I easily accomplished that by opening Gimp on a huge (blank) file. Now, working with Gimp is easy now; switching to Firefox takes ages (heavy swapping). So there a not-so-agressive swapping policy would be better if you switch between several memory-hogging applications a lot.

(Side note: When there's a lot of free memory left – for example after closing Gimp – the kernel step by step swaps in certain blocks again, a few every second so as to not disturb system performance. I saw this going on for several minutes on a otherwise completely idle system.)

Conclusion: For the usage pattern I'm accustomed to, setting vm.swappiness=0 actually is a huge performance improvement. But your mileage may vary.

First post with posting tool

2011-01-04T01:29:00+01:00

This is my first post with a custom shell script. It automatically adds a YAML header on top of new posts, starts an editor on it and later converts the title to a nice filename.

Eventually, the script adds and commits the new post to the git repo and pushes the result. "Recovering" is supported – sort of. ;-)

Excision Shambhala 2009 Mix

2011-01-03T23:05:00+01:00

Excision Shambhala 2009 Mix by Excision

Blog software

2011-01-03T18:41:00+01:00

Which blog software to take? – Not a simple question to answer.

I had my mind on Ikiwiki for quite some time. I haven't got it working, though. After several tries, I gave up. (The problem had in part something to do with an outdated git version, which I cannot upgrade on the server without going through quite some hassle.)

I decided to use Jekyll now. As with other "static site generators", the documentation is pretty rudimentary (and largely not very helpful). After some hours of fiddling around, I finally found a way to integrate tags and a tag cloud. This blog post has been very helpful, although I had to patch the file. Strangely, also, Jekyll refused to load any plugins when calling it like

/var/lib/gems/1.8/bin/jekyll --no-auto <source> <destination>

To fix this, I set up git's post-receive hook to cd to that directory first, and then call Jekyll with . as source paramter.

Time for a quick listing of pros and cons:

pro

Write posts in vim with Markdown syntax
Store every modification in a git repo
Publish with a simple git push

contra

Written in ruby (I'd have liked Perl much better)
Lots of ruby/gem/rake/whatever stuff
No extensive Documentation or User's Guide available

The template mechanisms seem managable, though. As for security and speed, the pages are generated once (even for pagination stuff) and are copied as simple static HTML pages to the web server. This is nice.

In the weeks to come I might publish the git repo of this blog where my other git repos are at.

Starting a blog

2011-01-01T23:42:00+01:00

2011 is here. I will try to maintain a blog again. The last time I gave up because I had nothing to write any more (I thought). I discover, though, that a lot of people like to read what I write.

So I have a blog again. I'll post random stuff here. Might just be about UNIX tools; philosophy; math stuff; books I read; links I like. English or German, -- maybe other languages in the future.

Stay tuned. :-)