subtomosetup(1)             General Commands Manual            subtomosetup(1)



NAME
       subtomosetup - Sets up command files for reconstructing multiple sub-
       volumes

SYNOPSIS
       subtomosetup  options

DESCRIPTION
       Subtomosetup creates a set of command files for directly reconstructing
       multiple numbered subvolumes from an aligned tilt series.  Its main use
       is to allow subtomogram alignment and averaging to be done on unbinned
       data after selecting particle positions on a binned-down tomogram,
       without having to build a full unbinned tomogram.  It also allows one
       to apply CTF correction to the aligned stack for a series of Z depths
       in the tomogram, and use the appropriate corrected stack in recon-
       structing each subvolume.  Note that if the unbinned tomogram already
       exists, the program Boxstartend can be used to extract subvolumes
       from it.

   Rules of Operation
       The program allows considerable flexibility as long as several restric-
       tions are followed.
         1) The program must be run from the dataset reconstruction directory,
       and the command files provided to it should ideally be ones managed and
       output by Etomo.  Do not try to use a tilt.com constructed with what
       seem to be the minimum required entries.
         2) The raw tilt series stack must still be present.  Alternatively, a
       file with the same name as the stack containing just the header can be
       used, but in this case the aligned stack must be present at the desired
       size and binning.
         3) The tomogram on which points were selected for reconstruction must
       be available and entered with the -volume option, although it need not
       be in the current directory.  Alternatively, a file with just the
       header of this volume can be supplied.
         4) The tilt series alignment, tomogram positioning parameters, the X-
       axis tilt, and the tilt angle offset in the Tomogram Generation panel
       of Etomo must not be changed between the binned tomogram and subvolume
       reconstructions.  Whether to use a GPU may be changed, and the current
       choice will determine whether a GPU is used for the reconstructions.
         5) If you apply any processing to that volume outside of Etomo, the
       header entries for pixel spacing, origin, and tilt angles must be pre-
       served in the file entered with the -volume option.  If you need to
       trim some more, use Trimvol; if you need to bin, use Binvol.
         6) If you need to apply other software that does not preserve these
       header entries, follow these procedures: Make sure that the file ana-
       lyzed has the same dimensions as the IMOD-based file from which it was
       derived.  Enter the name of the IMOD-based file instead of the analyzed
       file with the -volume option.  Supply a point coordinate file instead
       of a model file for the -center option (use Model2Point(1) with the
       -float option to make such a file).  Other alternatives are to load a
       model onto the IMOD-based file with the -m option and save it again, or
       to run Imodtrans on a model with the -image option and the name of
       the IMOD-based files, and use the resulting model.

       To make a file with a copy of the header from the raw stack or modeled
       volume, you can use the script "copyheader" with two arguments: the
       name of the input image file and the file in which to write the header,
       e.g.:
          copyheader setname.st setname_header.st

       You are free to make an aligned stack from a centered subarea or an
       over-sized area and to change this area between the binned and subvol-
       ume reconstructions.  You can apply any trimming options in the Post-
       processing panel in Etomo.  Note that the subvolumes will be reoriented
       the same way that the binned tomogram was.

   3-D CTF Correction
       3-D CTF correction is activated with the -zlevels option, which speci-
       fies the number of levels in Z at which to compute CTF corrections, or
       with the -extent option, which sets the range in Z over which each CTF
       correction is used  The total Z range of the subvolume centers will be
       divided into levels based on one or the other entry, and each subvolume
       will be reconstructed from the aligned stack for its Z-level.  The com-
       mand files are set up to process the Z-levels in order, with the CTF-
       corrected stack from the previous Z-level deleted before going on to a
       new Z-level.  The command file for correction ("ctfcorrection.com" by
       default) must already include the necessary options before running this
       program, including the correct pixel size for the aligned stack.  The
       aligned stack must not already be CTF-corrected, but it may have gold
       fiducials erased already.  Alternatively, fiducials can be erased in
       each corrected stack with the -erase option.  If so, the command file
       for erasing ("golderaser.com" by default) must also have the correct
       options before running this program.

   Reconstructions from the Raw Stack
       Reconstructions can also be done directly from the raw stack when doing
       3-D CTF correction.  The aligned stack need not exist in this case, but
       desired parameters must be set in the command file for CTF correction,
       and the defocus file must exist.  If 2-D filtering is to be done, it
       will be done on the raw stack by the command files produced here, and
       parameters must be set already in the command file for filtering.  If
       gold beads are to be erased, that will happen after creating the CTF-
       corrected raw stack for each defocus level, the parameters must be set
       in the gold erasing command file, and the model for erasing must exist
       already.

       Reconstructions from raw images cannot take account of corrections for
       a distortion field or magnification gradients and will not be valid if
       either of these corrections were used when making the aligned stack.
       The program will examine the Newstack log and/or command files from
       making both the coarse and final aligned stacks and issue a warning if
       either of these contains distortion corrections.

   Using One or More GPUs
       A GPU can be used for both correction and reconstruction, for neither,
       or for correction but not reconstruction.  The -gpu option can be used
       to control when the GPU is used.  If that is not entered, the command
       files determine what will happen.  If the reconstruction command file
       specifies use of the GPU, then it will be used for CTF correction as
       well, regardless of the setting in the command file for correction.
       However, if the reconstruction command file does not specify using the
       GPU, the setting in the correction command file determines whether cor-
       rection is done on a GPU.  When reconstruction is run on CPUs and cor-
       rection on a GPU, correction will be run on only one GPU, and the first
       machine in the machine list given to Processchunks must have a GPU.
       It may be more efficient to do correction on one GPU even when many
       CPUs are available, because correction with CPUs will be split into
       many jobs to be run in parallel, which incurs some overhead and delays.
       When correction is done on CPUs, the operation will be split up by
       assuming there are 8 CPUs, unless a specific number is specified with
       -proc.  Also, if reconstruction is set to be done on a GPU, correction
       (on the GPU) will be split into jobs when -proc is entered with a value
       higher than 1.

   X-axis Tilt
       When reconstructing from an aligned stack, if an X-axis tilt is
       included in either the reconstruction or correction command files, then
       the program evaluates whether it is similar enough in the two opera-
       tions.  If an X-axis tilt is being applied at all during correction, it
       should closely match the value used for reconstruction, otherwise the
       program gives a warning.  If there is an X-axis tilt in reconstruction
       but not correction, the program will determine the maximum difference
       in defocus produced by the two different X-axis tilt values, which
       occurs at the extremes of the reconstruction in Y.  If this defocus is
       larger than a certain fraction of the Z-level extent, the program
       issues a warning.

       When reconstructing from the raw stack, the X-axis tilt in the recon-
       struction command file is unconditionally used for the correction.
       There is no additional time involved in using the X-axis tilt in this
       case because the CTF correction has to be done in diagonal strips of
       full-sized FFTs anyway.

   Chunks, Subvolumes, and Running the Jobs
       Subtomosetup generates command files named with the root name of the
       Tilt command file ("tilt" by default) plus "-sub-001.com", etc. By
       default, each command file will produce ten subtomograms named "root-
       name-N.mrc" where "rootname" is the dataset root name, and "N" is a
       number that increases sequentially for all subvolumes computed, but
       with enough leading zeros so that all files have the same number of
       digits and will list in order.  The numbers will be in the same order
       as the points in the model or point file, but with 3-D CTF correction,
       they will not be computed in order.  If a position is too close to the
       top or bottom of the Y-extent that can be reconstructed from the cur-
       rent aligned stack, the particle will be skipped, but output files will
       still be numbered sequentially unless the -skip option is entered.  For
       positions not quite so close to the limits in Y, a Tilt run will be
       included to generate a subvolume with up to 1/6 of the extent blank in
       that direction.

       Multiple Tilt runs are put in each command file in order to reduce
       the overhead in starting and monitoring commands, and to reduce the
       number of .com and .log files in the directory.  The division of runs
       into jobs can be controlled by the -proc and -runs options.  When doing
       3-D CTF correction, runs at each Z level are apportioned into chunks
       using the same strategy, but with the same target for the total number
       of reconstruction chunks.

       The command files can be run with Subm, Processchunks, or the
       generic parallel processing interface in Etomo.  For example
          subm tilt-sub*.com
       to run all files in sequence, or
          processchunks 8 tilt-sub
       to run in parallel on 8 processors, as can also be done in the Etomo
       interface.  A GPU will be used if specified in the original command
       file.  The Etomo interface can be used to run on multiple GPUs located
       on different machines, but to run on multiple GPUs in one machine, pro-
       cesschunks must be run directly, such as with
          processchunks -G localhost:1:2:3


OPTIONS
       Subtomosetup uses the PIP package for input (see the manual page for
       pip).  Options can be specified either as command line arguments
       (with the -) or one per line in a command file (without the -).
       Options can be abbreviated to unique letters; the currently valid
       abbreviations for short names are shown in parentheses.

       -root (-ro) OR -RootName       Text string
              Root name of dataset.  For one axis of a dual-axis data set,
              include the "a" or "b" in this name.  This option is required.

       -center (-ce) OR -CenterPositionFile     File name
              Name of an IMOD model file or a point coordinate file with the
              center positions of the desired subvolumes.  A point coordinate
              file should have one point per line, with its X, Y, and Z coor-
              dinates separated by spaces, not commas.  These coordinates
              should correspond to pixels in the full file specified with
              -volume, where the first pixel in any dimension spans coordi-
              nates from 0 to 1. This option is required.

       -volume (-v) OR -VolumeModeled      File name
              Name of the image volume in which points were selected.  If a
              model file is supplied for the -center option, this entry should
              be the image volume that was modeled.  If a point coordinate
              file is supplied with -center, the coordinates must correspond
              to pixel positions in this volume file.  This option is
              required.

       -raw (-ra) OR -RawStackFile    File name
              Name of raw image stack.  This option is needed only if this
              filename cannot be determined by analyzing command files in the
              directory.

       -axis (-a) OR -AxisAngle       Floating point
              Angle that the tilt axis is rotated from vertical (counterclock-
              wise positive) in the raw image stack, used to determine whether
              the X and Y sizes of a full aligned stack are transposed from
              those of the raw stack.  This entry is needed only if it cannot
              be found in an alignment log file.

       -objects (-o) OR -ObjectsToUse      List of integer ranges
              If only a subset of objects in the model contain points to be
              reconstructed, this option can be used to enter a list of those
              objects (comma-separated ranges are allowed).

       -size (-si) OR -SizeInXYZ      Three integers
              Final size of subvolumes in X, Y, and Z, in unbinned pixels.  If
              the modeled volume was reoriented by rotation or flipping, sub-
              volumes will be treated similarly, and this entry specifies the
              size after reorienting.  If the aligned stack is binned, the
              actual size will the specified size divided by its binning.
              This option is required.

       -dir (-d) OR -DirectoryForOutput    File name
              Name of directory to place subvolumes into.  By default, the
              subvolumes will be written into the dataset directory; this
              option can be used to place them in a subdirectory or, in fact,
              in a directory located anywhere.  If the directory does not
              exist yet, it will be created.

       -chunk (-ch) OR -DirectoryForChunkFiles       File name
              Name of directory in which to place the command files.  By
              default, these chunk files will be written to the dataset direc-
              tory; with this option they can be in a subdirectory or other
              location.  The log files will be created in the same directory.
              If the directory does not exist yet, it will be created.  The
              command files must still be run from the dataset directory.

       -skip (-sk) OR -SkipSubVolNumbers
              Skip subvolume numbers whenever points near the border of the
              volume are skipped, thus keeping the subvolume and point numbers
              in register even when points are skipped.  The default is to put
              out sequentially numbered subvolumes with no gaps, which is
              required when processing the subvolumes in PEET with template
              specifications for a series of volumes.

       -com (-co) OR -CommandFile     File name
              Starting command file for running Tilt to make the reconstruc-
              tions.  The default is to use "tilt.com", so this option is
              needed for a dual-axis data set or to use a copy of the file.

       -binali (-b) OR -NewAlignedBinning       Integer
              Make a new aligned stack with the given binning instead of using
              the existing aligned stack.  Any operations that were done on
              the previous aligned stack need to be specified with options
              here; they will not be redone automatically.  Also note that the
              program will run Newstack to create a temporary aligned stack
              with one image in order to analyze its header settings.  This
              option cannot be entered with -unaligned.

       -newstcom (-n) OR -NewstackComFile       File name
              Name of command file to use for making a new aligned stack,
              which can be entered with or without its extension of ".com".
              If this option is not entered, "newst" will be substituted for
              "tilt" in the name of the command file for reconstruction.

       -unaligned (-un) OR -UseUnalignedImages
              Use the raw, unaligned stack for CTF correction and reconstruc-
              tion.  This should be the only option that is needed to use the
              raw stack, other than the -reduce option if desired, when this
              program is run inside a directory with a full set of processing
              files from one data set.  This option can be used only with 3-D
              CTF correction.  The option cannot be entered with -binali.

       -reduce (-red) OR -FourierReduceByFactor      Integer
              Reduce the raw stack by the given factor by using Newstack to
              crop its Fourier transform and use this reduced stack for pro-
              cessing from unaligned images.

       -zlevels (-z) OR -NumberOfZLevels   Integer
              Number of different depths in Z at which to make CTF-corrected
              aligned stacks.  Either this option or -thickness must be
              entered, but not both.  An entry of 1 is allowed and will result
              in CTF correction being done at the defocus corresponding to the
              midpoint of the subtomogram Z depths.

       -extent (-ex) OR -ExtentOfZLevelsInNm    Integer
              The maximum depth in Z over which to use one CTF correction, in
              nanometers.  A large value can be used to do CTF correction only
              once, in which case the correction will be at the defocus corre-
              sponding to the midpoint of the subtomogram Z depths.

       -invert (-i) OR -InvertZLevelOffsets     Integer
              Enter 1 to invert the sign of the offset in Z applied when doing
              the CTF correction for each Z level.  This would be needed if
              the tomogram is upside-down from its orientation in the micro-
              scope, by rotation and/or inversion in Z.  Tests with the most
              common sources of this problem indicate that this occurs when-
              ever tilt angles need inversion in Ctfplotter and Ctfphase-
              flip(1); thus the default is to use the value of the -invert
              option in the command file for CTF correction, with the option
              here available to override that value.  See the section on .B
              Inverting Tilt angles in the Ctfplotter man page for more
              details.

       -ctfcom (-ct) OR -CorrectionComFile      File name
              Command file for CTF correction, which can be entered with or
              without its extension of ".com".  If this option is not entered,
              "ctfcorrection" will be substituted for "tilt" in the name of
              the command file for reconstruction.

       -erase (-er) OR -EraseFiducials
              Run a command file to erase gold, which will occur after after
              CTF correction of the aligned stack, if any.  Gold may be erased
              in the aligned stack prior to CTF correction when using an
              existing aligned stack, but this option can be used to erase
              after correction, which is the preferred order of operations in
              Etomo and Batchruntomo.  That order is better in principle
              but may make little difference in practice.

       -goldcom (-go) OR -GoldEraserComFile     File name
              Name of command file to use for erasing gold, which can be
              entered with or without its extension of ".com".  If this option
              is not entered, "golderaser" will be substituted for "tilt" in
              the name of the command file for reconstruction.

       -filter (-f) OR -FilterIn2D
              Filter images in 2-D.  If filtering is desired, this entry is
              needed when reconstructing from a raw stack, but when using an
              existing aligned stack, the filtering may be done either before-
              hand or with this option.

       -2dcom (-2) OR -2DFilterComFile     File name
              Name of command file to use for 2-D filtering.  If this option
              is not entered, "mtffilter" will be substituted for "tilt" in
              the name of the command file for reconstruction.

       -gpu (-gp) OR -WhenToUseGPU    Integer
              The entry overrides the default behavior described above based
              on the entries in the command files.  Enter 0 to use just CPUs,
              1 to use GPUs for both reconstruction and CTF correction, or 2
              to use a single GPU for CTF correction and CPUs for reconstruc-
              tion.  In the latter case, the first computer in the machine
              list must have a GPU.

       -pixel (-pi) OR -RawPixelSize       Floating point
              Pixel size of raw image stack in nanometers.  This entry is
              needed when using unaligned images only if the pixel size is 1.0
              in the image file header, and the pixel size is not found in a
              track.com file.

       -xform (-x) OR -AlignTransformFile       File name
              File with the linear transformations that were used to align the
              images.  This option is needed when using unaligned images only
              if this filename cannot be determined by analyzing command files
              in the directory.

       -reorient (-reo) OR -ReorientionType     Integer
              This option can be used to specify the type of reorientation
              that was applied to the original reconstruction to obtain the
              modeled volume, in the unlikely event that the program cannot
              detect which was used.  Reorientation by rotation around the X
              axis can be detected by the tilt angles in the volume header;
              swapping of Y and Z can be detected only if the "clip: flipyz"
              title is still present.  If necessary, the program will assume
              that swapping occurred if the Y dimension is more than twice the
              Z dimension, or that no reorientation occurred if the Z dimen-
              sion is more than twice the Y dimension.  If the program makes a
              wrong assumption or insists that this option be used, enter a 0
              for no reorientation, 1 for swapping of Y and Z, or -1 for rota-
              tion around X.

       -proc (-pr) OR -ProcessorNumber     Integer
              Number of processing units, either CPUs or GPUs, that the jobs
              will be run on.  The program will divide the jobs into 10 com-
              mand files (chunks) per processor if this results in fewer than
              1000 chunks, or with fewer chunks per processor, down to 5, in
              an attempt to keep the number of chunks under 1000.

       -runs (-ru) OR -RunsPerChunk   Integer
              Number of Tilt runs per command file (chunk).  The program will
              create chunks with this number of runs, or with more runs to
              keep the total number of chunks under 100,000.  This option can-
              not be entered with -proc.  The default is 10.

       -help (-h) OR -usage
              Print help output

       -StandardInput
              Read parameter entries from standard input


FILES
       All temporary files are removed by each command file as it runs, and
       command and log files except "-finish.log" file are removed by the
       "-finish.com" file.

AUTHOR
       David Mastronarde

SEE ALSO
       tilt, subm, processchunks, boxstartend, trimvol, bin-
       vol(1), model2point, imodtrans

BUGS
       Email bug reports to mast at colorado dot edu.



IMOD                                4.12.35                    subtomosetup(1)