Well, today I was looking into using LVM snapshots to allow a client OCN use Linux as a Netapp replacement…. Boy was I in for a disappointment.

LVM an sich is working great, but the moment you turn on snapshots the (in this case) write performance goes to hell. Using LVM is easy enough. The system I was on has 32 GB ram and 2 disk arrays with hardware RAID.

Setting up:

See this?

bond0: Warning: failed to get speed and duplix from eth*

Do this:

insmod bonding miimon=100

why?

When using Linux network bonding the kernel may be to quick to enslave the interfaces. When an interface is too slow to report it capabilities it will be set to 100Mbit and Full Duplex. Which is a bit sad when you have 6 Gigabit network cards…

For a few weeks months now I’m learnings how to use VIM text-objects. There is an extended help wth help text-objects in VIM. I’m trying to condense the VIM help in a smaller blog entry here.

What are a text-objects in vim? Text-objects are things like a ‘paragraph’ or the text between braces or something like a word. Text-objects can be used with the normal vim commands y, d and c. To make you really understand it, I can recommend using control-V to actually visually select your text object.

As I’m on an upgrade roll I decided to upgrade my new server too. It is configured with kerberos and ldap… and this is were the trouble.

ldap

When a service is upgraded in Ubuntu/Debian it is first stopped and than later restarted when the new files are there. When all your user information is kept in ldap, the following sucks:

Preparing to replace slapd 2.4.11-0ubuntu6.1 (using
    .../slapd_2.4.15-1ubuntu3_i386.deb) ...
Stopping OpenLDAP: slapd.
  Dumping to /var/backups/slapd-2.4.11-0ubuntu6.1:
- directory dc=atoom,dc=net... done.
Unpacking replacement slapd ...

And there goes the neighborhood… everything slows to a crawl, because every getpwnam call needs to timeout before /etc/passwd and friends are consulted.

A short guide on how to build the new notification system, which is scheduled for Jaunty, for Intrepid.

First download the source packages from: http://packages.ubuntu.com/source/jaunty/notify-osd

You will need the .dsc and the .tar.gz files, then you need to follow the Debian way of building the package.

dpkg-source -x notify-osd_0.9ubuntu2.dsc
cd notify-osd-0.9ubuntu2/

One important tweak, otherwise it will not install in Intrepid and will complain about an human-icon-theme that is tool old

I’ve used the Debian xfce pkg stuff to build Ubuntu packages of xfce 4.6.

The … release … rocks :)

You can find the debs in files/xfce/.

Be my guest…

Today if removed two annoyances I had with VIM.

Command mistyping

I often mistype the following:

:w! becomes :W!

or

:q! becomes :Q!

Which is annoying because W and Q do not mean anything, and I don’t write or VIM does not quit. VIM has a nifty feature called commands which you can (re)define or add new commands.

Lets try some to fix this, in a running vi:

:com -bang W write!

or

The following might be helpful to others too. I was trying to setup a new raid1 device from two partitions /dev/sda4 and /dev/sdb4. I wanted to do this the “right way” and use UUID everywhere, i.e. in /etc/mdadm/mdadm.conf and in /etc/fstab.

I hit a few snags along the way.

create the array

# mdadm --create --verbose /dev/md6 --level=1 --raid-devices=2 \
/dev/sda4 /dev/sdb4

Get the uuid mdadm uses:

# mdadm --detail  /dev/md6 | grep UUID
  UUID : dc9aba5e:ed1a70d4:770765d8:b0f56d86 (local to host elektron2)

Check. Add that to /etc/mdadm/mdadm.conf:

In bash (and other shells) you can use the [[ construct

From the bash manpage:

[[ expression ]]
          Return a status of 0 or 1 depending on  the  evaluation  of  the
          conditional  expression expression.  Expressions are composed of
          the primaries described  below  under  CONDITIONAL  EXPRESSIONS.

          Word  splitting  and pathname expansion are not performed on the

          words between the [[ and  ]];  tilde  expansion,  parameter  and
          variable  expansion, arithmetic expansion, command substitution,
          process substitution, and quote removal are  performed.   Condi‐
          tional operators such as -f must be unquoted to be recognized as
          primaries.

So no word splitting and pathname expansion.

A colleague of mine had some nice improvements to gitvi.

Right now gitvi can be edited by itself, because the magic sequence $Hash$ is escaped like this

sed -i -e 's/\$[H]ash:.*\$/$H''ash$/' "$base"

(This was Ton’s idea)

Some other improvements like a -m MSG switch which allows you to enter a commit message for all files you are editing.

And of course it then also needs to have a -h switch.

Re-selection of a visual

When you use control-v or shift-v in vim to do a visual selection and then use (for instance) y to copy the selected text, you loose the selection.

More often than not you want to do another thing with the selection you just had. How?

gv

Copy to clipboard

Make a selection and use

"*yy

Now the selection is ‘under your mouse’. If think I’m going to bind this sequence to something more easy, like *Y.

Update (200911-08): I’ve stopped using this, because it made the archive generating fail in mysterious ways…

As you might have seen I have now these nice (opinions may vary) “more…” links in my blog. This is done with some JavaScript and shell foo and in this entry I will explain how I did it.

The setup also works with text browsers and people who disable JavaScript. In the latter case you will get the normal nb behavior where you see the entire article. When using a text browser you also see the entire article but there is also a (non functional) ‘more’ link.

Basicly these blog items serve as a personal reminder and documentation. Also the code needs to be refactored a bit and some html tags need to be removed from it.

Files needed

The following files are needed for this comment system:

• comment.php, this is the PHP file the implements everything

• nbadmin, a small shell script that implemented the comment moderation (just a mv of the comment to the correct directory)

I’ve implemented a simple comment system in which

• there are no user accounts
• has a preview function
• once a comment is submitted you cannot edit it anymore
• all comments are moderated
• a small set of BB tags are allowed
• implemented as one PHP file
• the PHP script has some html in it
• needs a writeable directory in your document root

You need this file: comment.php.

The first few lines allow for some customization:

I’ve been thinking about comments on my blog ever since I started using nanoblogger. While nb is great (with VIM and markdown) I’m starting to miss the comment stuff.

Comment systems for nb

I’ve been searching for a good comment system that I want to use for nb. It should meet the following criteria:

1. Simple moderation system. With all the spam nowadays every comment made should be moderated.
2. Should not require a database, the filesystem is enough of a database to handle these tasks. This does imply that there will be no user accounts, nor the possibility to edit your comments.
3. Simple interface with nanoblogger.

I’ve looked at blogkomm, haloscan and cgicomment.

You are using the ControlMaster feature of ssh and you are having problems remembering which shell is the master?

That’s why I’ve made zsh display this in the prompt. You get a m@ when a connection is a master connection and an @ if it is a slave.

setting up

The check if we are a master works by creating a file which has $SSH_CLIENT as the name, as the following snippet shows.

The script in using git and vi together was a little bit borked, as too many git repo’s were created. Hence a new and improved version. This one will look up the directory tree to spot an existing git repository - is nothing found a new one will be created in $PWD.

code

#!/bin/zsh
# a wrapper around git and vi
# expands $Hash$ to $Hash: file short_hash epoch committer $
# git checkout HEAD $file? when would I need this
# TODO: spaces in filename

[[ ! -x =git ]] && exit 1

who=$SUDO_USER
who=${who:-$LOGNAME}

function search_git_dir {
    gpath="$1"

    [[ -d "$gpath/.git" ]] && echo "$gpath" && return
    [[ -z "$gpath" ]] && echo "" && return

    # strip that last path component and try again
    search_git_dir "${gpath%/*}"
}

for file in "$@"; do
dir=$(dirname $file); cd $dir
base=$(basename $file)

if [[ -z $(search_git_dir $PWD) ]]; then
    # make a new one in $PWD
    git init || exit 1
else
    #echo FOUND ONE
fi
chmod +w $base 2> /dev/null

if vi $base; then
    [[ ! -e $base ]] && exit 0
    git add $base
    # collapse $Hash: id $ line
    sed -i -e 's/\$Hash:.*\$/$Hash$/' $base
    git commit $base
fi

id=$(git-show -s --pretty=format:$base\ %h\ %ct\ $who%n -- $base) 
[[ -z $id ]] && exit 1

# re-add $Hash: sha1hash$ line
sed -i -e 's/\$Hash\$'/\$Hash:\ $id\ \$/ $base

chmod a-w $base 2> /dev/null
done

manpage And ofcourse the manpage.

A long while back I used vi together with rcs to manage files in /etc. This worked, but I found rcs to be clunky. So welcome to the 21st century and my git + vi script.

It is a wrapper around git, which will create a new git repository, add the file and commit it when changed.

The script will also expand $Hash$ to

$Hash: basename-of-file short-commit-hash epoch committer`

a typical example is

Ok, so far I’ve created a cfengine setup that works. On my main host ’elektron’ I’ve created a /export/nfsroot/x86 directory. This holds the general files for the x86 Ubuntu lazy clients. With 2 scripts (see below) I can easily create a whole new x86 from scratch. Bringing up new lazy clients is even more easy, once the groundwork is there. I only need to do three things:

• fix the DNS (forward and reverse)
• fix the DHCP and add an ip address.
• add this ip address to some cfengine file (2 in total)
• optionally add some extra configuration in cfengine for this specific host.

Create a generic lazy client from scratch

There is one script that creates a new host in a chrooted environment, it starts a debootstrap and after that calls a follow-up script to finish the configuration for this host - after this step cfengine takes over. The second stage script takes care of setting up the sources.list for apt and a preliminary /etc/hosts file. Cfengine fixes everything else.

In my ongoing effort to make my lazy client work I’ve embarked on the next phase of the project: Making a client boot from a generic NFS mounted root partition and then giving it its own /etc on a ramdisk.

This way I’m saving disk space and administration overhead on my server. The virtual /etc is provisioned by cfengine. This allows for small changes between the lazy clients while they still share the bulk of the rest of the files.

while playing with cfengine I kept seeing these messages:

Cfengine input file had no explicit version string

Well, I greped in the source of cfengine and this has helped me find the solution, in the file install.c we have the following:

3359 void VersionAuditFile()
3360 
3361 { char *sp;
3362 
3363 if (sp = GetMacroValue(CONTEXTID,"cfinputs_version"))
3364    {
3365    AUDITPTR->version = strdup(sp);
3366    }
3367 else
3368    {
3369    Verbose("Cfengine input file had no explicit version string\n");
3370    }
3371 }

(Line numbers are from VIM).

I want to quiet, full blown PC, without fans nor a harddisk in my living room. See my other blog about this subject

Currently I’m thinking about the following setup:

• NFS4 root
• some sort of configuration management

I wanted to use puppet, but after seeing it use more than 30% of my main memory – This is on a AMD64 Ubuntu server – I was ready to ditch it.

I’ve also have experience with cfengine, so I’m going to use that. In conjunction with svn. I’m going to store all my configs + cfengine files in one repository. The modus operandi will become:

Or how to create new builtins for use in bash. Short answer: you can, but it is not enabled by default (at least on my distribution, Ubuntu)

This is also something that is done much better in zsh.

Why do want this? SPEED!

Shell scripting is a very easy way to program, but forking all these helper programs takes a lot of time. So one way to speed up your shell program is to load these programs into the shell and making them a builtin.

Some older presentations I have given can be found here.

The all deal with DNSSEC - note that most of them have been written a few years ago.

Google is always helpful of course, so I’ve found this blog entry on how to set the background in xfce.

It boils down to setting a jpg in XFCE’s backdrops.list and then reloading xfdesktop. Crude, but it works.

This can de done with the following command:

cat <<EOF > ~/.config/xfce4/desktop/backdrops.list
# xfce backdrop list
background.jpg
EOF

And then

xfdesktop --reload 

However when running from CRON it is more easy to do a killall -USR1 xfdesktop.

Working on backup programs for almost a decade now, I’ve had plenty of time to think about file system wrt backups. Basically I want the following properties in a file system

• online snapshots in a .snapshots directory (for instance).
• the possibility to say the all those .snapshot directories must be mounted on a separate mount point.
• to ability so say the .snapshot directories should be encrypted and send to remote server.

I think this is too much to ask…

After some switching I have now settled on XFCE. With Ubuntu XFCE sometimes does not start, this is a dbus issue in combination with gnome-screensaver (a race condition between the two). This sucks, but can be resolved easily with

apt-get remove gnome-screensaver

(There aren’t any interesting option anyway…)

So my XFCE is now up and running, time for some tweaking.

In the quest for precious vertical pixels (especially on the EeePC 701 I own) I decided I must remove those pesky title bars.

Grrrrr

cd
rm -rf Desktop
mv Documents docs

I don’t want desktop environments messing with MY home directory.

Thank you.

I’ve battled with amavis too often and I started to dislike it a little. So when I wanted to configured my postfix setup to scan for viruses and spam and needed a solution which would work without amavis.

This is what I came up with.

debian administration was kind enough to do the hard work, but this is only the setup for clamav and I also want to spam scan.