Brace yourself

The shell (in this case bash) is packed with features. So much so that you will probably never ever learn them all. Brace expansion is one of those things.

What is it? In the bash manual it says:

Brace Expansion

Brace expansion is a mechanism by which arbitrary strings may be generated. This mechanism is similar to pathname expansion, but the filenames generated need not exist

So what is it? You can make your shell generate strings, like so:

Read more →

EeePC Linux install

Just bought anothor EeePC, this time the 900HD variant. Installing Ubuntu as we speak and giving Windows the boot. The 4 minute encounter with Windows XP (which felt like 4 hours) gave me the impression of a fast machine. This was just a quick peek to see if the hardware was working.

Going with Ubuntu 9.04 - then removing Pulseadio and (maybe) upgrading the kernel to 2.6.30.4.

The trick is to disable the hardisk in the BIOS and to disable all quick boot stuff in there too. So that my SD card with the Ubuntu install will boot.

Read more →

Git filtering

Excellent idea Ton!

How to make a custom filter in git to expand the string $Hash$ to something more usefull, ala the $Id$ (which git already supports), but then with more info (committer, date, etc.).

Which also helped in this case was the Pro Git book which is, as of now, a must buy.

But as always is the case between Ton and me, I find his scripts too long :-) So I miekified his solution.

Read more →

String multiplication in zsh

In Perl you have this:

% perl -e 'print "a" x 5, "\n"'
aaaaa

With that you can easily create a separator string consisting out of 60 spaces.

I always missed this in my shell - until now.

In Zsh have the following expansion:

l:expr::string1::string2:

Pad the resulting words on the left. Each word will be truncated if
required and placed in a field expr characters wide. 

See zsh.dotsrc.org. There is also a r: variant which operates in the same way.

Read more →

Bash shell scripting and octal values

While rechecking my rdup test-suite one of the tests failed. On closer inspection it was due to the following line:

DAY_BEFORE=$(( $(date +%d) - 1 )) 

When $(date +%d) is 10 this yields:

$ DAY_BEFORE=$(( 10 - 1 ))
$ echo $DAY_BEFORE
9

Also with octal values (those starting with a leading zero), it also works:

$ DAY_BEFORE=$(( 06 - 1 )) 
$ echo $DAY_BEFORE
5

So when does this go wrong? When the day is 08 or 09 (as it is today):

Read more →

Performance plotting with atop and Gnuplot

A customer wanted some performance figures for a ESX cluster we built. We used our own atop and the trusted workhorse of plotting Gnuplot to make some performance plots.

Creating the data

As said, we used atop for this, it has a nice data gathering mode which prints out lines like these:

13:40:07  cpu   %usr  %nice   %sys  %irq %softirq   %steal  %wait  %idle
13:40:08  all      0      0      0     0        1        0      4     95
13:40:09  all      0      0      1     0        0        0      0	  99

We were interested in the values of %usr, %sys, %irq + %softirq, %wait and %idle.

Read more →

How to mess up git

So you think you know git… today I found out something “funny”. In a git repository:

$ git log
$ fatal: object 1fcc8de9361c56e538ff35d8cc4b07a9c95b7bf3 is corrupted

Okay, WTF? Lets look in the .git directory:

$ cd .git/objects/1f
$ ls -l
total 4
-r--r--r-- 1 root root 1057 2009-06-23 19:03 03db070bcb47bff3f8106f2ec7028b3496aaa8
-r--r--r-- 1 root root    0 2009-08-03 21:41 cc8de9361c56e538ff35d8cc4b07a9c95b7bf3

Ah, 0 bytes, that explains it (probably due to the weird reboot of last night and ext4).

Read more →

Die Pulseaudio, die die die!!!

Finally made the switch to jackd which works so much better than Pulseaudio. Ubuntu did not make this easy, but with some perseverance it works. One of the things I really hate about Pulseaudio is that when I click on a new song in audacious it would take about 1 second before the audio stabilized. Also with flash audio would stutter for the first few seconds.

First (if you don’t care about gnome-desktop):

Read more →

DNS classes

If you ever administered a DNS server you are probably familiar with the IN (internet) class, as in

localhost   IN	    A	    127.0.0.1

where you define an IP address for the name localhost. As you may, or may not, already know there are other classes defined (but hardly used) for the DNS. Two of those are the Chaos class (CH) and the Hesiod (HS) class. With these classes you can create some sort of parallel world where you can also define names. This feature of the DNS isn’t exactly in great use, but is is nice to know where they were used for.

Read more →

Case insensitive cd

Do I want this?

% ls -ld Joe_Cocker_-_The_Definitive_Collection 
drwxr-xr-x 2 miekg admin 4.0K Jul 20 22:20 Joe_Cocker_-_The_Definitive_Collection/
% cd *joe*
cd: no such file or directory: *joe*
% unsetop case_glob 
% cd *joe*
% pwd
/shared/vol/music/J/Joe_Cocker_-_The_Definitive_Collection
Read more →

SRM with KVM and DRBD

Currently we are building a fairly rock solid high availability cluster for a client. This has the “usual” ingredients: two locations, two NetApps, two clusters of three vmware ESX servers and a bunch of virtual machines running on top of the ESX servers. Also included in the mix is a VDI (now called View) virtual desktop infrastructure for running virtual windows XP clients.

This is all managed by SRM (site recovery manager) and it is almost working. But that is another story.

Read more →

A library at home

I’ve told a few times about how to build a library.

But there a some extra things that must be done before you can call yourself a complete librarian.

  1. ID-ing the books you have
  2. Storing the information
  3. Marking the books
  4. Sorting and shelving the books
  5. Remembering who borrowed what

ID-ing books

For this I bought a barcode reader that reads the ISBN number of the book’s backcover. With book_get (Perl code) I can retrieve the author, title, genre and year published from Google.

Read more →

Kernel logs

So, there I am browsing my logs.

BAM

[650521.310292] INFO: task mysqld:2730 blocked for more than 120 seconds.
[650521.310320] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[650521.310364] mysqld        D 00000000     0  2730   2674
[650521.310367]  f6183ce0 00000086 c06ef6cc 00000000 f61c8c70 7834922a 00000dc2 00020050
[650521.310373]  00000296 c0796fe0 c0796fe0 f61c8f04 c2c35fe0 00000003 9500b16e 00024f83
[650521.310379]  00000002 00000002 f66bb500 f6d02e40 c2bf2090 f6e2e680 f6183ce8 c050eb68
[650521.310384] Call Trace:
[650521.310388]  [<c050eb68>] schedule+0x8/0x20
[650521.310391]  [<c0266701>] do_get_write_access+0x2a1/0x3d0
[650521.310395]  [<c0127d93>] ? kmap_atomic_prot+0x43/0xe0
[650521.310398]  [<c0150030>] ? wake_bit_function+0x0/0x60
[650521.310402]  [<c02669a3>] jbd2_journal_get_write_access+0x23/0x40
[650521.310405]  [<c0252a36>] __ext4_journal_get_write_access+0x26/0x60
[650521.310409]  [<c023e9d4>] ext4_reserve_inode_write+0x34/0x70
[650521.310413]  [<c023ea45>] ext4_mark_inode_dirty+0x35/0x140
[650521.310416]  [<c02669ac>] ? jbd2_journal_get_write_access+0x2c/0x40
[650521.310420]  [<c02446cf>] add_dirent_to_buf+0x12f/0x340
[650521.310423]  [<c0245032>] ext4_add_entry+0xe2/0x1d0
[650521.310427]  [<c024540a>] ext4_add_nondir+0x1a/0x70
[650521.310430]  [<c0245814>] ext4_create+0xc4/0x100
[650521.310434]  [<c01d0ced>] vfs_create+0xcd/0x170
[650521.310437]  [<c01d2562>] __open_namei_create+0x42/0xa0
[650521.310441]  [<c01d2be0>] do_filp_open+0x620/0x660
[650521.310444]  [<c01ca888>] ? cp_new_stat64+0xe8/0x100
[650521.310448]  [<c02d3a67>] ? strncpy_from_user+0x37/0x60
[650521.310451]  [<c01c52a1>] do_sys_open+0x51/0xe0
[650521.310455]  [<c01c5399>] sys_open+0x29/0x40
[650521.310458]  [<c0102ebc>] sysenter_do_call+0x12/0x28

Okay, what the hell…Two questions.

Read more →

Playing with dhcpd

Yes! At work today we made dhcpd crash on a /etc/dhcpd.conf :) We were working towards the following (now working) setup:

class "igel" {
	match if binary-to-ascii(16, 8, "-", substring (hardware, 0, 5)) = "1-0-e0-c5-67";
}
pool {
	allow members of "igel"; 
	range 192.84.30.161 192.84.30.170; # igel[0-9]
}

But while figuring this out, dhcpd went boom :)

[root@vmaster cfengine]# dhcpd -d -f -cf hosts/vnet/etc/dhcpd.conf
Internet Systems Consortium DHCP Server V3.0.5-RedHat
Copyright 2004-2006 Internet Systems Consortium.
All rights reserved.
For info, please visit http://www.isc.org/sw/dhcp/
hosts/vnet/etc/dhcpd.conf line 20: can't override match.
    match if 
    ^
hosts/vnet/etc/dhcpd.conf line 22: pool declared outside of network pool
				^
Segmentation fault
Read more →

Using LaTeX with a label writer

I wanted to pretty print my labels on my (new) Dymo 400 label writer. All the howtos I found were detailing how to use LaTeX to print a sheet of labels. But I needed some LaTeX-foo to print only one. So /me to the rescue…

My labels are 36mm X 89 mm (Dymo part #99012) after some trial and error and four misprinted labels I came to the following LaTeX code which fits the text on the labels:

Read more →

Making a library

We (my wife and I) have a lot of books, currently some 637…

$ sqlite3 biblio.db 'select count(*) from books' 
637

But we seem to loose books occasionally because we lend them to people and forget about it. So Linux (and Perl) to the rescue. I already own a bar code reader to read the ISBN number of books and then I use Google to give the author, genre, etc. (fully automated of course).

Read more →

Full file system and I/O redirection

This is an English translation of a blog item I wrote for AT Computing

While giving a course a student showed me the following:

$ ps -ef > /tmp/file

Where /tmp is 100% filled yields no errors and seems to have worked!

Lets try to see what is going on here.

Firstly, lets fill up a file system. We are going to use an fs mounted under /media/disk:

$ cp /dev/zero /media/disk/HUGE                 
$ cp: writing `/media/disk/HUGE': No space left on device 

Further we need a little program to test a few things:

Read more →

2.6.30 and Ubuntu Jaunty

Well, thanks to Ubuntu I’m now running the new (new!) 2.6.30 kernel on my systems.

No ill effects as of yet…

Btw, I’ve written the following script to download the latest kernels from Ubuntu:

SYNOPSIS: latest 29.4, this will fetch 2.6.29.4 or latest 30 which will get 2.6.30.

#!/bin/bash
# download the latest ubuntu mainline kernels
ubuntu="http://kernel.ubuntu.com/~kernel-ppa/mainline/"
version="2.6.$1" # need 29.n as argument
arch="i386"	 # or amd64
major=${version%.[0-9]}
minor=${1%.[0-9]}
patch=${version#2.6.*.}

[ -z "$1" ] && { echo -e "Usage: $0 MINOR\n$0 29.3"; exit 1; }
[ -n "$2" ] && arch="$2"

if echo "$patch" | grep -q '\.'; then
printf -v versionstr "0206%d" $minor
else
printf -v versionstr "0206%d%02d" $minor $patch
fi

printf -v generic "%s-${versionstr}-generic_%s-${versionstr}" $major $major
printf -v all "%s-${versionstr}_%s-${versionstr}" $major $major

echo "${ubuntu}v${version}/linux-headers-${generic}_${arch}.deb"
wget --progress=bar "${ubuntu}v${version}/linux-headers-${generic}_${arch}.deb"
echo "${ubuntu}v${version}/linux-headers-${all}_all.deb"
wget --progress=bar "${ubuntu}v${version}/linux-headers-${all}_all.deb"
echo "${ubuntu}v${version}/linux-image-${generic}_${arch}.deb"
wget --progress=bar "${ubuntu}v${version}/linux-image-${generic}_${arch}.deb"
Read more →

Group policy in Linux

Suppose you want to limit access to some servers, only people member of a specific group (or multiple groups) may log in.

The following is one way to tackle this. In this example I will configure ssh access in such a way that only people from the admin group can login. The nice thing is that this will work regardless of any Kerberos or LDAP usage.

Preparation

In /etc/pam.d find the “service” which you want to add a group policy to. For instance sshd, edit that file (this is with Ubuntu):

Read more →

Kerberos

Mental note to self

Having a host listed in a listprincs output isn’t enough to have single-sign-on working.

You have explicitly add it with ktadd host/your.host.com

So in my case:

# kdadmin.local
....
kadmin.local:  addprinc -randkey host/charm.atoom.net
...
kadmin.local:  quit

And then you can do a (on charm.atoom.net):

% kinit
Password for miekg@ATOOM.NET: 
% slogin elektron.atoom.net

And have a password-less login to my server.

Read more →

Zsh prompt updates

I wanted some git information in the prompt I currently have. While thinking and googling about this, a quote on Bart’s log got me thinking. I don’t need to see my current host, nor do I need to see my current user name.

I always login under my own account (miekg) and if I do change accounts, I probably will be forced to use bash. So showing the current user name is quite useless.

Read more →

DNSSEC deployment

I’ve done some work in the development of the DNSSEC protocol, this culminated (for me) in RFC 4641. At the time I was a big proponent of DNSSEC. I still think the DNS should be improved and also believe DNSSEC is one of the best solutions. I’m however not as sure about this as I once was. This is because of several reasons.

The first one being the development of the NSECn record.

Read more →

Short URLs

Not wanting to miss anything on this short urls business, I’ve implemented something similar in nanoblogger.

How? With a shell script and symlinks, my dear Watson. As with all shell scripting this is probably something you can do drunk.

First I need to get a list of all my articles so that I can link to them. Next I take this permalink address, pipe it through sha1sum, take the last 10 characters and make a symlink from the permalink path and presto: short urls

Read more →