Tip 2: Makefiles and qsub

Tip #2 is about using Makefiles to create repeatable experiments without hand-running everything. Makefiles are often used for compiling programs, but they can be adapted for anything. One thing I often use them for is running a set of experiments. For example, let's say you have a set of benchmarks (ami33, ami49, apte, etc. ) in a certain directory (../benchmarks/hard).

You can create a Makefile.hard.area like this:

BENCHDIR = ../benchmarks/hard
BENCHMARKS = ami33 ami49 apte hp n10 n100 n200 n30 n300 n50 xerox

TAG=hard_area

OPTS=
SRC=../simanneal/simanneal
LOG=$(TAG)_results

all:    clean $(BENCHMARKS)

$(BENCHMARKS):
       mkdir -p ${LOG}
       $(SRC) $(OPTS) -i $(BENCHDIR)/${@}  2>&1${LOG}/${@}.log

clean:
       rm -f ${LOG}/*

If you run "make -f Makefile.hard.area" it will run every benchmark like this:

../simanneal/simanneal -i ../benchmarks/hard/ami33 2>&1  hard_area_results/ami33.log
../simanneal/simanneal -i ../benchmarks/hard/ami49 2>&1  hard_area_results/ami49.log
etc.

and the results will all be in the output log for later processing. The "-f Makefile.hard.area" specifies a makefile name. If you omit it, it will look for just "Makefile", but this isn't good if you have a bunch of experiments in different makefiles.

You can also run a single benchmark or remove all of the logs by doing:

make -f Makefile.hard.area ami33
make -f Makefile.hard.area clean

An enhancement to this Makefile is to use the mada queue system. You need to run this on one of the mada machines (mada0, mada1, etc.). It will submit the jobs individually to the queue and you will get an email when they are finished. You can check the status of you jobs with "qstat" and you can delete all of your jobs with "qdel -u <user>" or a specific job with "qdel <jobid>".

BENCHDIR = ../benchmarks/hard
BENCHMARKS = ami33 ami49 apte hp n10 n100 n200 n30 n300 n50 xerox

TAG=hard_area
USER=$(shell whoami)
EMAIL=$(USER)@soe.ucsc.edu
QSUB=qsub -S /bin/sh -m es -M $(EMAIL) -cwd -b y

OPTS=
SRC=../simanneal/simanneal
LOG=$(TAG)_results

all:    clean $(BENCHMARKS)

$(BENCHMARKS):
       mkdir -p ${LOG}
       $(QSUB) -N "$(TAG)_${@}" -e ${LOG}/${@}.err -o ${LOG}/$ 
{@}.log $(SRC) -- $(OPTS) -i $(BENCHDIR)/${@} -o ${LOG}/${@}
clean:
       rm -f ${LOG}/*

Note that Makefiles are space sensitive. You need to press <tab> whenever there is an indentation above.