Table of Contents

Name

scanmaster -- farm scanning jobs out to hosts running scanslave

Synopsis

scanmaster [-Hh] [-A authmode] [-a aggregator] [-c post-chunk] [-d dir] [-j jobs] [-p pre] [-w workers] -i input -n name -r remote [-- scanslave-passthrough-args]

Description

The scanmaster utility is part of the “scanmaster” suite of scripts. It is the primary invoking command and hence named the same. It represents the portion of the process that farms a scan job out to a number of scanning nodes, which in turn invoke scanslave(1) .

scanmaster supports the following command-line options:

-A authmode
Specify the authentication mode. Valid options are “all", “password” and “pubkey". This is passed on to the scanslave(1) command.

-H
Do not use the headless user for ssh connections.

-a aggregator
The path to a script to aggregate the results from all slaves. This script is run after all slaves have finished and it is passed as the first argument the name of the scan, as the second argument the path to the original input file and as the third argument the date (YYYYMMDD) that the scan was started. If you require more flexibility, you may wish to use a wrapper script that invokes scanmaster without an aggregator script and the runs whatever other tools you have to post-process the data.

-c post-chunk
The script each slave should execute after it’s finished. As this flag is passed on to the scanslave(1) invocations, the path needs to be one that all slave nodes can resolve. Using a script on an NFS share is a good solution to this problem.

-d dir
The directory under which scanmaster should create all output. Preferably this is on an NFS share amongst all scanning nodes, though this is not a technical requirement (it just makes postprocessing of the scan results an order of magnitude easier). Please note that if no NFS share is used, the location still needs to be the same for all slave nodes. If not specified, defaults to “/mnt/scanmaster".

-h
Print a short usage and exit.

-i input
Specify the file containing the input list of hosts to scan.

-j jobs
The number of jobs scanhosts(1) should execute. This flag is passed on to scanslave(1) ; if not specified, use that tool’s defaults.

-n name
Specify the name of this scan. This name is used in a number of places during output file creation.

-w workers
Specify the file containing the list of worker hosts to farm this job out to. If not specified, defaults to “/usr/local/share/scanmaster/slavenodes".

-p pre
Specify the script to run before doing anything else. This script is invoked with the scan name as the single argument.

-r remote
Specify the script to execute on all target hosts. As this flag is passed on to the scanslave(1) invocations, the path needs to be one that all slave nodes can resolve. Using a script on an NFS share is a good solution to this problem.

Any arguments given after a -- will be treated as “scanslave passthrough" arguments. That is, they will be passed on to scanslave(1) , with the -- intact, meaning scanslave(1) will further pass them through to other tools it may invoke.

Details

Upon invocation, scanmaster will first run the script given to the -p flag (if any), passing it the name of the scan as the single argument. It then removes any files found under the shared directory for a scan by this name, creates a new sub directory (by date), and tries to set up the environment for the currently running (if any) ssh-agent(1) . Next, it splits the input file into N chunks, where N is the number of hosts found in the worker hosts file. After that, it will ssh to each worker host, create a screenrc file that will invoke scanslave(1) with the appropriate options; it then invokes screen(1) with a key-binding of ^Ww. The user needs to detach from each host’s screen session in order to let scanmaster move on to the next host.

After all hosts have started a screen session, scanmaster will wait for the scanning nodes to complete their respective jobs. Once every minute, scanmaster will look for the existence of a file called “.done” which is created in the well-defined shared location by scanslave(1) . When it finds all “.done” files, it will execute the script given via the a flag (if any), passing it the name of the scan as the first argument and the location of the original input file as the second argument.

Environment

scanmaster uses the following environment variables:

SCAN_ENV
scanmaster invokes scanslave(1) by calling

env ${SCAN_ENV} scanslave other-args This allows the user to pass anything into the environment of the invoked processes on the scanslave hosts. Obviously, SCAN_ENV should then contain key=value pairs.

Examples

The following examples illustrate common usage of this tool.

SCAN=info1
PREFIX=/mnt/scanmaster/jschauma/${SCAN}

scanmaster
\
-a ${PREFIX}/aggregator.sh
\
-c ${PREFIX}/post.chunk.sh
\ -i ${PREFIX}/files/${SCAN}-input \
-p ${PREFIX}/pre.sh
\
-r ${PREFIX}/remote.sh
\ -n ${SCAN}

See Also

autopw(1) , checkhosts(1) , scanhosts(1) , scanslave(1) , tkill(1)

History

The scanmaster utility was originally written by Jan Schaumann <jschauma@yahoo-inc.com> in July 2007 as “cmd.chunker".

Bugs

Please report bugs and feature requests to the author.

Table of Contents