A customer wanted some performance figures for a ESX cluster we built. We used our own atop and the trusted workhorse of plotting Gnuplot to make some performance plots.

Creating the data

As said, we used atop for this, it has a nice data gathering mode which prints out lines like these:

13:40:07  cpu   %usr  %nice   %sys  %irq %softirq   %steal  %wait  %idle
13:40:08  all      0      0      0     0        1        0      4     95
13:40:09  all      0      0      1     0        0        0      0     99

We were interested in the values of %usr, %sys, %irq + %softirq, %wait and %idle.

Normalizing the data

To matter slightly easier we decided to normalize the data a bit;

1. convert the time stamp to Unix epoch
2. kill the first 5 lines of output as this is a header

In short, the following tiny Perl program was used for the conversion.

#!/usr/bin/perl -w
use Time::Local;
use strict;
<>;<>;<>;<>;<>;<>; # kill first 5 lines
my @l;
while(<>) {
chomp;
if (/^#/) { print; next }
@l = split;
my ($h, $min, $sec) = split /:/, $l[0];
shift @l;
$h =~ s/^0//; # don't make it look octal
$min =~ s/^0//;
$sec =~ s/^0//;
my $time = timelocal($sec, $min, $h, (localtime)[3,4,5]);
print "$time @l", "\n"; # show the normalized line
}

Generating the plot

This was actually the most difficult step as Gnuplot is such a complicated program - you can do a lot of plotting with it. So it was a matter of getting it to plot exactly as we needed it to plot.

We wanted to plot the data as a histogram were each bar in the plot would show the percentage of time spend in the different statuses (usr, irq, sys and idle). Gnuplot calls this a rowstacked histogram. With this knowledge we came to the following Gnuplot script (script.gnuplot):

set yrange [0:100]
set xrange [0:]
set ylabel "% of total"
set xlabel "seconds"
set style data histograms
set style histogram rowstacked
set style fill solid
set key outside
set terminal jpeg
# output to /tmp/a.jpg
set output "/tmp/a.jpg"
# read the data from /tmp/a
plot '/tmp/a' u 3 t '%usr' lt rgb 'red', \
'' u 5 t '%sys' lt rgb 'black', \
'' u 6 t '%irq' lt rgb 'gray', '' u 7 t '%softirq' lt rgb 'gray', \
'' u 9 t '%wait' lt rgb 'yellow' , '' u 10 t '%idle' lt rgb 'blue'

Gnuplot would read its data from /tmp/a and generate a jpg named /tmp/a.jpg.

The full Monty

After gathering the performance data, making the plots was as easy as:

./normalize < performance.data > /tmp/a && \
gnuplot gnuplot.script && \
rm /tmp/a

Then /tmp/a.jpg could be renamed to something more suitable.