A picture is worth a thousand words, they say. Well that's certainly the case when you're trying to view the impact of some code changes over a period of time.
One thing I needed to do when writing the new background garbage collection feature for the yaffs file system was to see how the background garbage collection heuristics were working. In particular I needed to trace how the erased space compares to the free space (the garbage collector changes dirty free space into erased free space).... and I wanted a graphic trace in real time.
This really helped to monitor and debug some interesting problems where the background garbage collector should have stopped working but did not.
There are quite a few ways to do this for static logs (gnuplot, spreadsheets, etc) and there are also quite a few logging database tools (rrdtool etc) but those just all seemed too much effort to set up or docs to read. Why use a chainsaw to cut butter?
I was able to crank out something simple using bash and gnuplot. The bash script has three parallel processes: data gathering and driving gnuplot and gnuplot itself.
#!/bin/sh
# Script that gathers data erased vs free data from /proc/yaffs_stats and simultaneously \
# plots it using gnuplot.
#Gather settings
log_file=data
gather_delay=1
# Plot settings
trunc_file=trunc_data
plot_samples=1000
plot_delay=2
# Gathering task
gather_data() {
i=0;
rm -f $log_file
while true; do
str=$(cat /proc/yaffs_debug)
echo "$i, $str"
echo "$i, $str" >> $log_file
let i=$i+1
sleep $gather_delay
done
}
# Plotting task
# Periodically creates a truncated version of the log file and
# outputs commands into gnuplot, thus driving gnuplot
drive_gnuplot(){
sleep 5
tail -$plot_samples $log_file > $trunc_file
plot_str=" plot '$trunc_file' using 1:3 with linespoints title 'free', '' using 1:4 with linespoints title 'erased'"
echo "set title 'yaffs free space and erased space'"
echo $plot_str
while true; do
sleep $plot_delay
tail -$plot_samples $log_file > $trunc_file
echo replot
done
}
echo "Start gathering task in background"
gather_data &
echo "Run plotting task"
drive_gnuplot | gnuplot
The data gatherer periodically reads from a Linux kernel procfs entry and dumps a new data record onto the end of a log file.
The gnu-plot driver process uses tail to truncate the last n lines of the log into a truncated log file, then sends commands to gnuplot to either start a plot (for the first time around) or replot (on subsequent iterations). The nifty bit here is the pipe. Since the process is piping into gnuplot, output from this process is treated as if a user typed the command in gnuplot.
The above script gets run in a terminal and we get a nice gnuplot picture that automatically updates every few seconds.
The same basic idea is readily modified for all sorts of applications. For example if your embedded system spits out stats periodically on a serial port that info can get logged and run through a similar plotting task.
