Pen Drive Smoker HOWTO

Overview

This is a brief how-to, with support scripts, for running CPAN::Reporter::Smoker from a machine without a hard drive (i.e. no risk of loosing data), booted from an out-of-the-box liveCD1, using a pen drive for persistent data (configuration, reports log). Great use for that spare laptop2!

Last Updated 2009-01-10

Preparation

A live CD or other means of booting a system with perl & compiler tools.

Something (usb pendrive, external harddrive, nfs mount, samba mount, etc) that's read-write mountable.

The smoke.tar.gz archive extracted onto whatever media. Then YOU MUST EDIT the following files in the resulting smoke/ directory:

You can browse the files here: smoke/

Execution

Boot & mount

Boot (preferably into runlevel 3 -- to save resources).

Mount your media of choice (cat /etc/fstab may be helpful in finding it), and change to the smoke/ directory.

cat /etc/fstab
mount /media/sda1    # e.g. a USB stick
cd /media/sda1/smoke

Setting Up

At this point, the only item of information needed for setup is a filename of a temporary filesystem to use, and its desired size (in MB). This will be a ext2 filesystem file -- this is done for flexibility, e.g. for using VFAT formatted USB pen drives or samba-mounted drives.

Size over 500MB is probably fine; 1GB is pretty safe. Can go lower (e.g. 250MB), but going to need occaissional restarts after the space fills up.

This file can be on the main system (e.g. the ramdisk if this is a liveCD), a local hardrive, the current (smoke/) directory, a second usb drive, a remote nfs/samba mount, or whereever. Some examples:

# Simplest -- using the F directory (i.e. same piece of media):
./smoke-setup smoke.fs 1000

# Using a second USB stick:
mount /media/sdb1
./smoke-setup /media/sdb1/smoke.fs 1000

# Using a LAN samba share:
mkdir /mnt/smb
mount -t smbfs -o rw,lfs,workgroup=WORKGROUP,username=Administrator,password= //192.168.1.101/_smb /mnt/smb
./smoke-setup /mnt/smb/smoke.fs 1000

Whereever it goes, it's best to reuse the same one later (but not required -- will just have slower startup times while it reinstalls the toolchain).

Running

Now, to get things going, simply do:

./smoke-start

And you're off!

Monitoring

These are all useful monitoring commands:

# Watch it all spew by:
tail -f /tmp/smoke.out

# Watch the reports being sent out:
tail -f .cpanreporter/reports-sent.db

# Dashboard of free disk, memory, smoker status, reports counts and listings.
watch -n30 ./smoke-status

CAVEATS

All the standard caveats about smoke testing apply. However, this is a very safe and easy approach if you're using a live CD, no hard disk, and a cheap usb pen drive w/no other data on it.

You will just want to periodically save the smoke/ directory to another system since, in theory, something being smoke-tested could destroy it. Copying the smoke.fs file is optional (and you probably don't want to bother with the size) -- it can always just be re-generated.

INTERNALS

The basic concept here is to keep just the CPAN & CPAN::Reporter config files on a writeable media (i.e. usb drive), and kick off CPAN::Reporter::Smoker and sit back and watch. There are two main needs, though, which is where the smoke.fs filesystem file comes in.

First, need to have the newest versions of CPAN, CPAN::Reporter, toolchain modules, and deps installed. For this, the CPAN config file sets the PREFIX to be on the smoke.fs mount, and the first things that smoke-start does is update/install those modules.

Second, need room for CPAN to put the build and sources directories -- this is what takes up the bulk of the smoke.fs mount, and again is specified in the CPAN config file.

smoke-setup takes care of creating and mounting the smoke.fs mount, and also creates symlinks to all the smoke/ from the main system, so that any updates (most importantly to .cpanreports/reports-sent.db are preserved, since it's the one persistent RW location.

smoke/smoke-setup

   1 #!/bin/bash
   2 
   3 # USAGE: smoke-setup filesystemFile sizeMB
   4 # MUST be run from smoke directory on USB drive.
   5 
   6 d=`pwd`
   7 fs=${1:?"must give filename for temp fs"}
   8 sizeMB=${2:?"must give size in MB for temp fs"}
   9 
  10 if ! test -f $fs ; then
  11   dd if=/dev/zero of=$fs bs=1024 count=${sizeMB}000
  12   mkfs.ext2 $fs
  13 fi
  14 d2=/mnt/smoke
  15 mkdir -p $d2
  16 umount $d2
  17 mount -o loop $fs $d2
  18 mkdir -p $d2/.cpan
  19 mkdir -p $d2/perl
  20 
  21 df -h
  22 
  23 function make_symlink {
  24   actual=$1
  25   link=$2
  26   mkdir -p `dirname $link`
  27   /bin/rm -rf $link
  28   ln -s $actual $link
  29 }
  30 
  31 cd $d
  32 make_symlink $d/Config.pm        /etc/perl/CPAN/Config.pm
  33 make_symlink $d/.cpanreporter    /root/.cpanreporter
  34 make_symlink $d/.cpanreporter    /.cpanreporter
  35 make_symlink $d/prefs            /root/cpanprefs
  36 make_symlink $d2/.cpan           /root/cpanbuildcache
  37 make_symlink $d2/perl            /root/perl
  38 
  39 

smoke-start updates/installs the prereqs, then kicks off CPAN::Reporter::Smoker

smoke/smoke-start

   1 #!/bin/bash
   2 
   3 p=/mnt/smoke/perl
   4 
   5 export PERLLIB="$p/lib/perl:$p/share/perl:$p/lib/perl/5.8:$p/share/perl/5.8:$PERLLIB"
   6 export PERL5LIB="$PERLLIB"
   7 export PATH="$p/bin:$PATH"
   8 export MANPATH="$p/man:$p/share/man:$MANPATH"
   9 
  10 # make sure toolchain is up to date
  11 cpan CPAN ExtUtils::MakeMaker Module::Build
  12 # make sure Reporter & Smoker is up to date
  13 cpan CPAN::Reporter CPAN::Reporter::Smoker
  14 # update additional toolchain modules
  15 cpan Digest::SHA Proc::ProcessTable YAML
  16 
  17 out=/tmp/smoke.out.`date +%Y%m%d.%H%M`
  18 date > $out
  19 wc -l .cpanreporter/reports-sent.db >> $out
  20 echo >> $out
  21 
  22 rm /tmp/smoke.out
  23 ln -s $out /tmp/smoke.out
  24 
  25 perl -MCPAN::Reporter::Smoker -e start >> $out 2>&1 &
  26 
  27 cat <<-EOF
  28 	==== To Monitor Progress: ====
  29 	++++ See CPAN output in real-time:
  30 	    tail -f /tmp/smoke.out
  31 	++++ See reports being sent:
  32 	    tail -f reports-sent.db
  33 	++++ See mem & disk status, and current package:
  34 	    ./smoke-watch
  35 EOF
  36 

watch ./smoke-status is useful to leave up (can switch terminals between it and tail -f /tmp/smoke.out), and is (hopefully) self-documenting.

smoke/smoke-status

   1 #!/bin/bash
   2 
   3 r=.cpanreporter/reports-sent.db
   4 statusfile=`ls -rt /tmp/smoker-status-*.txt|tail -1`
   5 
   6 free -m
   7 echo
   8 df -h /mnt/smoke
   9 echo
  10 head -2 /tmp/smoke.out
  11 echo
  12 wc -l $r
  13 echo `cat $statusfile`
  14 echo
  15 tail -20 $r
  16 

CONTACT

David Westbrook (CPAN: davidrw), <dwestbrook at gmail.com>

SEE ALSO

http://cpantesters.org et al

CPAN::Reporter::Smoker

CPAN::Reporter, CPAN::Reporter::Config

CPAN, CPAN::Distroprefs, CPAN::FirstTime


1. Others I tried we're suitable because they either did not have a full perl install, or would not boot on my laptop. The distro i'm using is perl-5.8.8 i486-linux-gnu-thread-multi 2.6.18.3.

2. My testing was done on a T43 IBM Thinkpad.

View Source (POD)