A customer wanted some performance figures for a ESX cluster we built. We used our own atop and the trusted workhorse of plotting Gnuplot to make some performance plots.

Creating the data⌗

As said, we used atop for this, it has a nice data gathering mode which prints out lines like these:

13:40:07  cpu   %usr  %nice   %sys  %irq %softirq   %steal  %wait  %idle
13:40:08  all      0      0      0     0        1        0      4     95
13:40:09  all      0      0      1     0        0        0      0	  99

We were interested in the values of %usr, %sys, %irq + %softirq, %wait and %idle.

Normalizing the data⌗

To matter slightly easier we decided to normalize the data a bit;

  1. convert the time stamp to Unix epoch
  2. kill the first 5 lines of output as this is a header

In short, the following tiny Perl program was used for the conversion.

#!/usr/bin/perl -w
use Time::Local;
use strict;
<>;<>;<>;<>;<>;<>; # kill first 5 lines
my @l;
while(<>) {
chomp;
if (/^#/) { print; next }
@l = split;
my ($h, $min, $sec) = split /:/, $l[0];
shift @l;
$h =~ s/^0//; # don't make it look octal
$min =~ s/^0//;
$sec =~ s/^0//;
my $time = timelocal($sec, $min, $h, (localtime)[3,4,5]);
print "$time @l", "\n"; # show the normalized line
}

Generating the plot⌗

This was actually the most difficult step as Gnuplot is such a complicated program - you can do a lot of plotting with it. So it was a matter of getting it to plot exactly as we needed it to plot.

We wanted to plot the data as a histogram were each bar in the plot would show the percentage of time spend in the different statuses (usr, irq, sys and idle). Gnuplot calls this a rowstacked histogram. With this knowledge we came to the following Gnuplot script (script.gnuplot):

set yrange [0:100]
set xrange [0:]
set ylabel "% of total"
set xlabel "seconds"
set style data histograms
set style histogram rowstacked
set style fill solid
set key outside
set terminal jpeg
# output to /tmp/a.jpg
set output "/tmp/a.jpg"
# read the data from /tmp/a
plot '/tmp/a' u 3 t '%usr' lt rgb 'red', \
'' u 5 t '%sys' lt rgb 'black', \
'' u 6 t '%irq' lt rgb 'gray', '' u 7 t '%softirq' lt rgb 'gray', \
'' u 9 t '%wait' lt rgb 'yellow' , '' u 10 t '%idle' lt rgb 'blue'

Gnuplot would read its data from /tmp/a and generate a jpg named /tmp/a.jpg.

The full Monty⌗

After gathering the performance data, making the plots was as easy as:

./normalize < performance.data > /tmp/a && \
gnuplot gnuplot.script && \
rm /tmp/a

Then /tmp/a.jpg could be renamed to something more suitable.