chunksetup(1)                                                    chunksetup(1)



NAME
       chunksetup - Set up command files to process a volume in chunks

SYNOPSIS
       chunksetup [options] <command file> <input file>  <output file>

DESCRIPTION
       chunksetup will produce a set of command files to process a 3D image
       file in chunks and reassemble the chunks into a single image file.
       This procedure can be used when the processing operation is memory
       intensive, when it is more efficiently done in smaller chunks, or when
       one wishes to run the operation on multiple processors.

       To use this procedure, first make a master command file (the first
       required argument) that performs the needed operation, in the format
       required by Vmstocsh, as ordinarily run with subm.  Instead of using
       the actual filenames, the input file for the operation in this command
       file should be entered as INPUTFILE, and the final output of the opera-
       tion as OUTPUTFILE.  (Note that OUTPUTFILE should not be used for the
       name of the final output file given on the chunksetup command line.)
       The command file may perform multiple operations and may even contain
       INPUTFILE more than once.  If the procedure needs to produce temporary
       intermediate files, they should be named ending in ".$$" (e.g., chunk-
       temp.$$).  At the end of the command file, include lines to remove the
       specific intermediate files, e.g.
       $if (-e chunktemp.$$) rm -f chunktemp.$$
       If you are going to run the operation on more than one machine, it is
       probably best to put the files in a machine-specific location such as
       /usr/tmp.  Alternatively, you can use a more elaborate method for nam-
       ing temporary files.  Define a variable at the top of your command
       file:
       $set tmpext 'hostname'.$$
       and end your temporary file names with ".$tmpext".  In this case, you
       should remove the files at the end of the command file with statements
       like:
       $if (-e chunktemp.$tmpext) rm -f chunktemp.$tmpext

       The input and output files need not be in the same directory as the
       command files; a path specification can precede each filename.  Also,
       the master command file need not be in the current directory; the chunk
       command files will be placed in the directory where it is located.  In
       this case, however, you must enter the path to the input and output
       files relative to the command file's directory, not relative to the
       current directory.

       When chunksetup is run, it first calls Tomopieces, which analyzes the
       dimensions of the input file to determine how to chop it up optimally.
       It then produces command files for extracting each chunk with Taper-
       outvol and processing them, numbered sequentially with 3-digit numbers.
       If the name of the master command file is "comfile.com", the first com-
       mand file is comfile-001.com, INPUTFILE is replaced by comfile-001.in,
       and OUTPUTFILE is replaced by comfile-001.out, and similarly for the
       following files.  Each file contains a command to remove comfile-
       nnn.in, so only the output chunks accumulate as the files are run.
       Chunksetup also produces a command file to reassemble the pieces, com-
       file-finish.com, which will also delete the output chunks and the com-
       mand and log files for each of the chunks.  Finally, it produces a com-
       mand file, comfile-all.com, to run all of the chunks in sequence and
       run the finishing command file at the end.

       You have the option of running the top-level command file using "subm",
       or running the individual command files and running the finishing
       command file at the end.  If you have multiple processors available
       with access to the current directory, you can use the Processchunks
       script to run the jobs in parallel on the various processors or
       machines, such as with:
       processchunks machine_list comfile
       where machine_list is a list of the machines to use.


   Options
       -p <value>
              Set the number of pixels of padding on each edge to the given
              value.  The default is whatever the default is for Tomopieces (8
              pixels).

       -o <value>
              Set the number of pixels of overlap between the subvolumes to
              the given value.  The default is 8.

       -m <size>
              Limit each subvolume to "size" megavoxels.  The default is what-
              ever the default is for Tomopieces (20 megaVoxels).

       -xm <value>
              Set the maximum number of chunks in the X direction to the given
              value.  The default is -1, which is effectively no limit.  See
              the man page for Tomopieces for more details on the choices for
              maximum number of pieces.

       -ym <value>
              Set the maximum number of chunks in the Y direction to the given
              value.  The default is -1, which is effectively no limit.

AUTHOR
       David Mastronarde  <mast at colorado dot edu>

SEE ALSO
       tomopieces, taperoutvol, assemblevol, processchunks



BL3DEMC                             3.11.2                       chunksetup(1)