chunksetup(1)               General Commands Manual              chunksetup(1)



NAME
       chunksetup - Set up command files to process a volume in chunks

SYNOPSIS
       chunksetup  [options]  command_file  input_file  output_file

DESCRIPTION
       chunksetup will produce a set of command files to process a 3D image
       file in chunks and reassemble the chunks into a single image file.
       This procedure can be used when the processing operation is memory
       intensive, when it is more efficiently done in smaller chunks, or when
       one wishes to run the operation on multiple processors.

       To use this procedure, first make a master command file (the first
       required argument) that performs the needed operation, in the format
       required by Vmstocsh, as ordinarily run with subm.  Instead of using
       the actual filenames, the input file for the operation in this command
       file should be entered as INPUTFILE, and the final output of the opera-
       tion as OUTPUTFILE.  (Note that OUTPUTFILE should not be used for the
       name of the final output file given on the chunksetup command line.)
       The command file may perform multiple operations and may even contain
       INPUTFILE more than once.  If the procedure needs to produce temporary
       intermediate files, they should be named ending in ".$$" (e.g., chunk-
       temp.$$).  At the end of the command file, include lines to remove the
       specific intermediate files, e.g.
       $if (-e chunktemp.$$) rm -f chunktemp.$$
       If you are going to run the operation on more than one machine, it is
       probably best to put the files in a machine-specific location such as
       /usr/tmp.  Alternatively, you can use a more elaborate method for nam-
       ing temporary files.  Define a variable at the top of your command
       file:
       $set tmpext `hostname`.$$
       and end your temporary file names with ".$tmpext".  (Note that "host-
       name" must be enclosed in back-quotes not regular quotes - this charac-
       ter may not appear properly here.)  In this case, you should remove the
       files at the end of the command file with statements like:
       $if (-e chunktemp.$tmpext) rm -f chunktemp.$tmpext

       The input and output files need not be in the same directory as the
       command files; a path specification can precede each filename.  Also,
       the master command file need not be in the current directory; the chunk
       command files will be placed in the directory where it is located.  In
       this case, however, you must enter the path to the input and output
       files relative to the command file's directory, not relative to the
       current directory.

       When Chunksetup is run, it first calls Tomopieces, which analyzes
       the dimensions of the input file to determine how to chop it up opti-
       mally.  It then produces command files for extracting each chunk with
       Taperoutvol and processing them, numbered sequentially with 3-digit
       numbers.  If the name of the master command file is "comfile.com", the
       first command file is comfile-001.com, INPUTFILE is replaced by com-
       file-001.in, and OUTPUTFILE is replaced by comfile-001.out, and simi-
       larly for the following files.  Each file contains a command to remove
       comfile-nnn.in, so only the output chunks accumulate as the files are
       run.  Chunksetup also produces a command file to reassemble the pieces,
       comfile-finish.com, which will also delete the output chunks and the
       command and log files for each of the chunks.  Finally, it produces a
       command file, comfile-all.com, to run all of the chunks in sequence and
       run the finishing command file at the end.

       You have the option of running the top-level command file using "subm",
       or running the individual command files and running the finishing com-
       mand file at the end.  If you have multiple processors available with
       access to the current directory, you can use the Processchunks
       script to run the jobs in parallel on the various processors or
       machines, such as with:
       processchunks machine_list comfile
       where machine_list is a list of the machines to use, or just the number
       of processor to use on the local machine.


OPTIONS
       Chunksetup uses the PIP package for input (see the manual page for
       pip).  Options can be specified either as command line arguments
       (with the -) or one per line in a command file (without the -).
       Options can be abbreviated to unique letters; the currently valid
       abbreviations for short names are shown in parentheses.

       -master (-ma) OR -MasterComFile     File name
              Name of master command file.  If this option is not entered, the
              first non-option argument will be taken as this name.

       -input (-i) OR -InputImageFile      File name
              Name of input image file.  If this option is not entered, the
              second non-option argument will be taken as this name.

       -output (-ou) OR -OutputImageFile   File name
              Name of output image file.  If this option is not entered, the
              third non-option argument will be taken as this name.

       -p OR -PaddingPixels      Integer
              Number of pixels of padding on each edge of each subvolume.  The
              default is 8.

       -o OR -OverlapPixels      Integer
              Minimum number of pixels of overlap between the subvolumes.  The
              default is 8.

       -m OR -MegavoxelMaximum   Integer
              Limit each subvolume to the given number of megavoxels.  The
              default is whatever the default is for Tomopieces (20
              megaVoxels).

       -xm (-x) OR -XMaximumPieces    Integer
              The maximum number of chunks in the X direction.  The default is
              -1, which is effectively no limit.  See the man page for Tomo-
              pieces for more details on the choices for maximum number of
              pieces.

       -ym (-y) OR -YMaximumPieces    Integer
              The maximum number of chunks in the Y direction.  The default is
              -1, which is effectively no limit.

       -no (-n) OR -NoFFTSizes
              Do not increase padding to make suitable sizes for an FFT.  Use
              this option if no FFTs will be taken of the volumes.  It causes
              the "-nofft" option to be sent to Tomopieces and to be added
              to the input to Taperoutvol that is placed in the command
              files.

       -param (-pa) OR -ParameterFile      Parameter file
              Read parameter entries as keyword-value pairs from a parameter
              file.

       -help (-h) OR -usage
              Print help output

       -StandardInput
              Read parameter entries from standard input


AUTHOR
       David Mastronarde  <mast at colorado dot edu>

SEE ALSO
       tomopieces, taperoutvol, assemblevol, processchunks



IMOD                                4.10.10                      chunksetup(1)