batchruntomo(1)             General Commands Manual            batchruntomo(1)

       batchruntomo - Align tilt series and generate tomograms automatically

       batchruntomo  [options]  directive_files...

       Batchruntomo runs all the operations required to align a tilt series
       and build a tomogram, following a set of instructions called "direc-
       tives" provided in a text file.  These directives indicate either
       parameter settings or which pathway to follow at various points in the
       processing sequence.  The program will perform operations in a fixed
       order determined by these choices; the order of the directives does not
       affect this order.  The program can work on more than one data set and
       can also move the tilt series from their current location into a sepa-
       rate directory for each data set, which is created if necessary.  It
       accepts tilt series with extensions ".mrc", ".st", ".hdf", ".tif", and
       ".tiff".  Depending on the image file naming style specified, it may
       change the extension to ".st" so that other programs will work cor-

       Directives consist of key-value pairs separated by equals signs, where
       the key has 2 to 4 components separated by periods.  The first compo-
       nent is a prefix that indicates the type of directive or the stage at
       which it is applied.  The value may be blank if appropriate but the
       equals sign must be present.  There are currently three prefixes:
       "setupset", for directives involved when setting up a data set; "com-
       param", for directives used to set options in command files; and "run-
       time", for directives interpreted by Batchruntomo or Etomo.  A descrip-
       tion of all directives is contained in the file
        "IMOD/com/directives.csv", which is in comma-separated values format.
       It is accessible in a more readable form from the IMOD help index via
       the link "Directives for batch processing and Etomo templates" (avail-
       able from "imodhelp" or when accessing the html package of IMOD docu-
       mentation with a browser).

       "setupset" directives have two forms.  The majority have the form
          setupset.copyarg.<copytomocoms short option name>=value
       These directives will be processed mostly generically by Etomo and sent
       to Copytomocoms as command line arguments.  The short option name
       must be given in full.  Unlike for the other two kinds of directives,
       whenever there is a pair of directives corresponding to a pair of
       options for each axis of a dual axis set, a directive for the first or
       only axis does not apply to the second axis.  A separate "b" directive
       must be supplied.

       Other "setupset" directives are interpreted exclusively by Batchruntomo
       or Etomo and have the form
          setupset.<parameter or option>=value

       "comparam" directives are interpreted generically when a command file
       is created either by Copytomocoms or by Makecomfile.  They have
       the form
          comparam.<root name of file>.<program>.<Long option name>=value
       The root name of the command file may include an axis letter ("a" or
       "b").  For a single axis data set, directives without a letter and with
       an "a" are treated as equivalent and ones with "b" are ignored.  For a
       dual axis data set, a directive without a letter can apply to either
       axis, but will be overridden by a directive with a letter matching the
       specific axis, regardless of which one occurs later in the processing
       sequence.  The long option name must be given in full.  Options to
       Tilt must match the case shown in the Tilt man page, even though
       Tilt accepts case-insensitive options.

       Because "comparam" directives are treated generically, they are not
       restricted to the ones listed in the table of directives.  Any valid
       option can be used, although some would be meaningless in the context
       of automation and some would conflict with the management provided by
       Copytomocoms or be overridden by Makecomfile.

       "runtime" directives have the form
          runtime.<Processing step>.<axis>.<parameter or option>=value
       Here the processing step and parameter have names that have been made
       up just for the directive.  The convention is for the processing step
       to be capitalized and the parameter to start with lower case.  The axis
       must be either "a", "b", or "any".  For a single axis data set, direc-
       tives with "any" or "a" are treated as equivalent and ones with "b" are
       ignored.  For a dual axis data set, a directive with "any" can apply to
       either axis, but will be overridden by a directive with a letter match-
       ing the specific axis, regardless of which one occurs later in the pro-
       cessing sequence.

       Boolean directives must have a value of either 0 or 1: 1 to turn the
       option 1 or 0 to turn it off; no other values (including blank) are
       allowed.  If a boolean directive is not present then the option is off,
       unless it has been turned on by an entry in a template file (see
       below).  For any other kind of directive, the value can be blank, which
       causes the option in question to be removed from consideration.
       Specifically, for a "comparam" directive, a blank value will cause the
       option to be removed from the command file if present.  For other kinds
       of directives, a blank value removes the setting of the parameter by a
       previous directive in a template file.

       If a processing step is being skipped or bypassed by an alternative
       method of processing, then all the directives related to it have no
       effect on the processing that is done.  It is thus generally safe to
       leave unused directives in a file.  Such directives with prefix "com-
       param" will be used to set parameters in command files produced by
       Copytomocoms, but directives for parameters in other command files
       will have no effect because those files will not be produced.  Direc-
       tives with prefix "runtime" are simply ignored if they are not relevant
       to the chosen processing sequence.

   Template Files
       Template files can contain parameters that can be used meaningfully for
       multiple, similar data sets.  Directives exist to define a microscope
       template file, a system template file, and a user template file.  The
       idea is for the microscope file to have parameters specific to a micro-
       scope or microscope/camera combination, and to have the other template
       files contain other parameters that would not be dataset-specific.
       Each file consists of a subset of directives just like those described
       for defining options for the batch processing.  There is a heirarchy
       among these directives.  They are always processed in the order scope
       template, system template, user template, and batch directive file, and
       entries later in this sequence override ones that appeared earlier.
       Batchruntomo reads the template files and uses their contents along
       with those of the directive file to determine the values for parame-
       ters.  It also passes whatever template files exist, plus the batch
       file, to Etomo for it to use when setting arguments for calling Copyto-
       mocoms(1), and Etomo passes the files on to Copytomocoms to use for
       setting parameters.  Batchruntomo passes them to Makecomfile to use
       similarly when making optional command files, as well as to Setupcom-

       Another directive file, "batchDefaults.adoc" from the IMOD "com" direc-
       tory, provides some default values needed when running with batch pro-
       cessing.  As of IMOD 4.10.15, these are read in to Batchruntomo and
       processed prior to the templates.  Prior to that, Etomo appended these
       directives to the batch directive file unless there was an overriding
       value in a template.  Unfortunately, if such a file is used as a start-
       ing batch file in the Etomo interface, these directives are indistin-
       guishable from others and override corresponding template values, if
       any, instead of being overridden.  Batchruntomo now recognizes this
       situation and will issue a warning if any such directives do override a
       template value.

       The program checks all directives after it reads them, including the
       ones from a template file, against the master list in "directives.csv".
       All "setupset" and "runtime" directives must match ones in this file.
       It is possible to enter "comparam" directives not listed in this file,
       provided that the command file name and program name match those of
       other directives in the master file.  The program will consult the
       autodoc file for the program to determine whether the option is valid.
       It will issue a warning if an autodoc cannot be found; otherwise, it
       will be able to determine validity definitively.

   Processing Steps Available
       This section describes some of the capabilities and current limitations
       of various steps.

       X-ray removal is optional; if selected, automatic X-ray removal will be
       done, and "clip stats" will be run on the fixed stack.  There is also a
       directive to run Archiveorig to make a compressed difference between
       corrected and original stacks, then delete the original stack.  A man-
       ual replacement model can be specified.  It is suggested that object 1
       contain patches specified in the default way, by a point on each pixel;
       object 2 contain lines, if any; object 3 contain patches specified by
       boundary contours, if any; and that directives be added to specify
       these choices and that objects 1-3 are all-section objects.  These sug-
       gestions will be enforced when there is a GUI for setting up batch
       directives.  If the manual replacement model is located elsewhere, it
       will be copied to the dataset and renamed to the standard name for such
       models, so that the model can be opened and added to when running in

       Detection of blank images and images with large dark regions at the
       highest tilt angles can be done before further processing by including
       the preprocessing directive "endExcludeCriterion" with a fractional
       value such as 0.5.  (The directive is ignored for montaged tilt
       series.)  The program will look at the standard deviation (SD) of
       images at each end of the tilt series and find ones whose SD is less
       that the criterion times the mean SD of the five adjacent less-tilted
       images.  Up to three images can be excluded on each end of the series
       by this analysis.  The program will then do a histogram analysis with
       Clip to see whether the intensity distribution has two peaks and how
       much of the area falls in the lower peak.  A view is excluded if inten-
       sity at the lower peak is less than 0.17 times that at the upper peak
       and more than 0.33 of the pixels are in the lower peak (these defaults
       can be modified with "darkExcludeRatio" and "darkExcludeFraction"
       directives).  Views can be eliminated up to 6 tilts from either end in
       this way, but with either analysis, only consecutive views will be
       eliminated.  View skipping/exclusion lists will be added or adjusted in,,, and

       Coarse alignment is always performed.  The binning of a prealigned
       stack can be specified.

       Fiducialless alignment can be chosen, in which case tomogram thickness
       must be specified.  The tomogram will not be reoriented at all (no
       angle offset or X-axis tilt applied).

       Patch tracking can be chosen.  Again, tomogram thickness must be speci-
       fied, but in this case the tomogram will be reoriented based on angles
       found by Tiltalign.  Also, if you specify gold erasing after 3D bead
       finding, the thickness must be specified for the tomogram used to find
       the beads.

       Gold fiducial tracking can be done either with automatic seed-finding
       then bead tracking, or with RAPTOR followed by optional bead tracking.
       For a second axis, Transferfid can be run, followed by automatic
       seed-finding to fill in the resulting model.  RAPTOR can be run instead
       for the second axis (unlike in Etomo), although there will then be no
       list of corresponding fiducials.  Beadtrack can be run more than
       once, because this completes a model more reliably than selecting fur-
       ther rounds of tracking in a single run.

       Fine alignment with Tiltalign can include local alignments in all
       cases.  There is a directive to enable the stretching (distortion)
       solution.  When enabled, stretching will be solved for only if there
       are gold fiducials on two surfaces, and if there are enough on the
       minority surface overall and relative to the total number.  Stretching
       may be solved for in the global solution but not in the local one if
       there are too few fiducials on the minority surface per local area.  In
       addition to making these decisions, Batchruntomo also runs the program
       Restrictalign, which will use cross-validation to test whether the
       number of variables being solved for should be reduced.  This poten-
       tially lengthy process will be run once with global variables, then
       once again if either stretching or local alignments are added.
       Restrictalign will turn off robust fitting if it gives no benefit,
       but if it does so when evaluating global variables and local alignments
       are being done, Batchruntomo will turn robust fitting on again for the
       local tests.  The fine alignment produces an angle offset and an X-axis
       tilt that is applied when building the tomogram, unless there are too
       few gold.  In addition, if there is gold on two surfaces, the distance
       between the gold particles provides an estimate of thickness that can
       be used as one basis for the tomogram thickness.  Otherwise, if gold is
       to be erased after 3D bead finding, the thickness for the required
       tomogram must be specified.

       Tomogram positioning can be done for both plastic sections and cryosam-
       ples by setting the "Positioning.sampleType" directive to 1 or 2,
       respectively.  Plastic section positioning is done with a binned-down
       whole tomogram for now.  If there is no "Positioning.binByFactor"
       directive, the program will pick a binning up to 4 that brings the
       binned size under 512 in its largest dimension.  If there is no "Posi-
       tioning.thickness" directive, the program will pick a thickness based
       on the largest dimension: 250 if size under 512, 400 if size under
       1024, 500 if size under 2048, and 600 otherwise.  The program will run
       Findsection and try to make a model for Tomopitch with 5 pairs of
       sample lines. Cryosample positioning is done with the script Cryoposi-
       tion(1).  In this case, the "Positioning.thickness" directive is
       required.  Cryoposition picks an appropriate binning, but if you do
       want to set this binning, it can be done with a "cryoposition.Binning-
       ToApply" directive instead of the runtime "Positioning.binByFactor"
       directive.  By default, 25 unbinned pixels will be added on each sur-
       face by Tomopitch; this can be changed with a "tomopitch.ExtraThick-
       ness" directive.

       When there is no positioning, angles are set from the Tiltalign out-
       put.  The Z height of the specimen in the tomogram will be whatever
       location results from the coarse alignment; thus the specimen will be
       centered in Z if features contributing to the cross-correlations are
       evenly distributed in depth.  If the correlations are dominated by fea-
       tures near one surface, the specimen will not be centered in Z.  If
       there are fiducials on two surfaces, one can use a "Position.centerOn-
       Gold" directive to apply the Z shift computed by Tiltalign for cen-
       tering on the range of fiducials in Z.  Another side-effect of using
       this directive is that the estimated thickness from alignment will be
       larger so that the tomogram will then contain all of the gold included
       as fiducials.

       The aligned stack can have specified binning and size.  Optional modi-
       fications of the aligned stack are performed in the standard order, or
       with gold erasing first if the -erase option is entered.  CTF correc-
       tion can be done with a single nominal defocus value.  Alternatively,
       Ctfplotter can be used to find the defocus automatically if the
       "autofitRangeAndStep" directive is present.  When Ctfphaseflip is
       run, it will use a "setname.defocus" file if it is present; otherwise
       it will use a "setname_simple.defocus" file created with the nominal
       defocus from the "setupset.copyarg.defocus" directive.  (The "set-
       name.defocus" file could derive from manual or automatic running of
       Ctfplotter, or could be created by other means).  Gold erasing can
       be done with automatic finding of beads in a tomogram, or by transform-
       ing the fiducial model (or the model put out by Tiltalign that has
       gaps filled in, if available).  For finding beads in the tomogram, the
       "GoldErasing.thickness" directive must be supplied unless there are
       beads on two surfaces.  The binning is set as the unbinned bead size
       divided by 5 and rounded to an integer, as in Etomo; thus, the "Gold-
       Erasing.binning" directive is no longer required.  Both the automatic
       finding of beads in 3D, and the detection of defocus, are steps that
       one might want to check before proceeding.  To allow intervention at a
       single stopping place, CTF correction and gold erasing steps are inter-
       leaved: defocus is detected, beads are found, CTF is corrected, then
       beads are erased.  2D filtering and dose weighting can be done by
       adding the directive "AlignedStack.filterStack".  There are various
       directives to enter the Mtffilter parameters for dose weighting.  To
       use an .mdoc file or the information inside an HDF file from SerialEM,
       add the directive "mtffilter.TypeOfDoseFile" with value 4.  The entry
       for the dose information file in the command file is managed appropri-
       ately by Copytomocoms except in one case: using the old naming style
       with descriptive extensions when the input stack is an MRC file with an
       extension other than ".mrc".  In this case you need the "mtffil-
       ter.DoseWeightingFile" directive with either the name of the .mdoc file
       or a generic entry that works for multiple data sets: the extension
       before the ".mdoc" in the name, typically ".st".

       The tomogram can be computed with regular weighted back-projection,
       with SIRT, or with a back-projection using a SIRT-like filter.  The
       iterations to leave can be specified for SIRT.  The regular back-pro-
       jection can be computed in the same run as a SIRT or SIRT-like one by
       adding the directive "Reconstruction.doBackprojAlso".  The thickness
       can be chosen automatically if there are gold beads on both surfaces or
       if tomogram positioning was done, otherwise it must be specified with a
       directive.  The "Reconstruction.fallbackThickness" directive is the
       most useful one because it can be used to specify a thickness to be
       used when there is no thickness available from either fine alignment or
       positioning.  The fallback thickness will also be used if the thickness
       from either is less than 40% of the fallback thickness, which would be
       the case if there are gold on only one surface and no positioning was

       A 3D CTF-corrected tomogram can be computed by adding the directive
       "ctf3dsetup.SlabThicknessInNm"; "ctf3dsetup.RunSlabsInParallel" can be
       added to make slabs in parallel and "ctf3dsetup.UseUnalignedImages" can
       be added to reconstruct directly from the unaligned images (the raw
       stack).  The result will be given the name of the regular weighted
       back-projection tomogram.  You can make a SIRT or SIRT-like filtered
       tomogram in addition to the 3D CTF-corrected one.  Steps for modifying
       the aligned stack are skipped when making just the 3D CTF-corrected
       tomogram, and they are applied during the reconstruction step.

       Tomogram combination for a dual-axis dataset is possible with all kinds
       of alignments and will proceed automatically.  The program first uses
       Findsection to find the Z limits, and decides whether to get the
       initial registration between volumes with Solvematch or Dualvol-
       match(1).  The patch correlations and fitting are done with Autopatch-
       fit(1); by default, it will try medium to extra-large patches and add
       target residual values of 0.4,0.45 for Findwarp on the final trial.
       There are "runtime.Combine" directives to control this behavior and to
       set a few other parameters for tomogram combination; also, "comparam"
       directives can be used to modify parameters in, patch-, and If both SIRT or SIRT-like and regular
       back-projection reconstructions are available, the latter will be com-
       bined unless the combine "doSIRTifBoth" directive specifies otherwise.
       The decision of whether to match A to B instead of B to A is made based
       on the ratio of the B to A thicknesses determined from their Z limits.
       This ratio is compared to a criterion that can be set with the "matchA-
       toBThickRatio" directive and whose default is 0.9.  With this crite-
       rion, A will be matched to B if B is thinner than A by more than ~10%.
       Lowering the criterion would make it match A to B only when the thick-
       ness difference is more extreme.  Raising the criterion toward 1 would
       make it match A to B with less thickness difference.  It can be set to
       a very low number (0.2) to guarantee that B is always matched to A, or
       a very high number (5) to have A always matched to B.

       Postprocessing can be done with Trimvol, and should be selected with
       the directive "runtime.Postprocess.any.doTrimvol".  For compatibility
       with old behavior, Trimvol will still be run if any of the "run-
       time.Trimvol" directives "findSecAddThickness", "reorient", "thick-
       ness", "sizeInX", "sizeInY", "scaleFromX", "scaleFromY", or "scale-
       FromZ" are present.  Findsection will be used to find the section
       limits if the "findSecAddThickness" directive is present.  The output
       file for a final reconstruction will have the usual name, setname.rec.
       For a dual-axis data set, it is possible to trim one or both of the
       single axis reconstructions; here the output file will be named with
       "_trim.rec".  If both SIRT or SIRT-like and regular back-projection
       reconstructions were made, the postprocessing "doSIRTifBoth" directive
       controls whether to trim one or both reconstructions; in the latter
       case the file for the back-projection will be named "_BP.rec".  When-
       ever a SIRT reconstruction is trimmed, the result on the final itera-
       tion will be used.  By default, the trimmed volume will be reoriented
       by rotating around X.  The entire volume will be used unless the "find-
       SecAddThickness", "thickness", "sizeInX", or "sizeInY" directive is
       present.  If one of the "scaleFrom" directives or the "scaleToMeanSD"
       directive is present, the volume will be scaled to bytes.  If "scale-
       FromX" or "scaleFromY" is not present, the area used for scaling in X
       or Y will be the Trimvol default of 80% of the range; if "scaleFromZ"
       is not present, the central one-third of slices will be used.  All of
       these size and scaling directives may be specified either as a number
       of pixels, or as a fraction between 0.02 and 1.

       Nonlinear anisotropic diffusion (NAD) with Nad_eed_3d can be run if
       both "runtime.NAD" directives are present for specifying the number of
       iterations and the K value.

       Data set cleanup can be done with Tomocleanup by adding the "run-
       time.Cleanup.doCleanup" directive.  There are several "run-
       time.Cleanup.keep..." directives to retain particular intermediate data
       set files.  The default is for Batchruntomo to include the Tomo-
       cleanup(1) option -sirt to retain the output from running a SIRT or
       SIRT-like reconstruction as well as a back-projection, but this can be
       overridden with a directive.

   Stopping and Restarting
       The -start and -end options allow datasets to be started and stopped at
       chosen steps in the sequence.  The most useful step to stop at would be
       10, to check CTF plotting and/or gold detection, then restart at 11.
       To check fine alignment before going on, stop and restart at 6.

       If you are setting up a situation where parameters are being changed
       before restarting in an existing dataset, you should experiment to make
       sure the changes have the desired effect and are not overridden.  Here
       are some general guidelines on that point:
       1) Changes in "comparam" directives will generally have no effect once
       a command file is created, because they are processed generically dur-
       ing command file creation.  In most cases, creation occurs during tomo-
       gram setup, but some files are created with Makecomfile just when
       needed, so it would be possible to change "comparam" directives for a
       file still to be created.  Some exceptions to this are "align.tiltal-
       ign.SurfacesToAnalyze" and "tilt.tilt.THICKNESS", for which the current
       directive value will be used instead of the value in the command file.
       2) Changes in "setupset" directives will also generally have no effect
       because they are interpreted during setup; the use of the defocus value
       mentioned above is an exception.
       3) Changes in "runtime" directives will generally have the desired
       4) Changes in parameters in the command files will generally have the
       desired effect, unless that parameter is managed by Batchruntomo on the
       basis of "runtime" directives and other information.  The biggest exam-
       ple of this is the treatment of "" when Tiltalign is rerun;
       the -use option is provided specifically to avoid rerunning Tiltal-

   Resource Usage
       The CPU list provided with the -cpus option is passed directly to Pro-
       cesschunks(1) for running operations that are divided into multiple
       chunks.  However, other operations that run from a single command file
       will generally be run on the local machine, unless the -single option
       is used to direct them to the first machine in the CPU list.  The
       exceptions are two time-consuming steps in dual-axis combination,
       Matchvol and Autopatchfit, which benefit from using multiple
       threads and are sent to the machine with the highest number of CPUs.
       When running single command files, the program sets a limit on the num-
       ber of threads equal to the number of CPUs designated in the CPU list
       for the machine being used, if it is known (-limit can be entered if
       this is not otherwise known for the local machine).

       Batchruntomo can be run in parallel on multiple data sets, and in this
       case it can also be run on a cluster queue.  See the section OPTIONS
       FOR PROCESSING DATA SETS IN PARALLEL for the options and environment
       variables involved; see the Splitbatch(1) man page for descriptions of
       resource usage and how to use a cluster.

   Running External Commands
       It is possible to specify an arbitrary external command to run either
       instead of a step or after one has run, with "runtime.ReplaceS-
       tep.any.#" or "runtime.RunAfterStep.any.#", respectively, where "#" is
       the step number given below for the -end option.  So far this is possi-
       ble only for CTF determination (step 9), aligned stack filtering (step
       13), and post-processing (step 20), but more steps can be added easily
       if needed.  The directive value must consist of a single line to run
       one command-line operation without piping or redirection.  The program
       will place this command after a "$" in a command file named "replaceS-" or "", and a log will be created when it
       is run.  The command can contain %{setname}, which will be substituted
       with the root name of the data set plus the axis letter, if any.  A
       replacement command will be run even if there are no directives to
       activate the normal running of that step.  However, a command to be run
       after a step will be run only if the normal step is run.

       Batchruntomo uses the PIP package for input (see the manual page for
       pip).  Options can be specified either as command line arguments
       (with the -) or one per line in a command file (without the -).
       Options can be abbreviated to unique letters; the currently valid
       abbreviations for short names are shown in parentheses.

         These options give information about files, directories, and the data
       set name

       -directive (-di) OR -DirectiveFile       File name
              Input file with directives for automating one or more data sets.
              The filename must end in .adoc.  All non-option arguments will
              be taken as such input files.  (Successive entries accumulate)

       -root (-ro) OR -RootName       File name
              A data set name can be provided with this option to override the
              name contained in the directive file, thus allowing directive
              files to be reused without editing.  If this option is used, the
              number of directive files must be either one or the same as the
              number of root names.  The current location of the tilt series
              must be indicated with the -current option if only one directive
              file is entered with multiple root names.  (Successive entries

       -current (-cu) OR -CurrentLocation       File name
              Directory where tilt series is currently located, or where it
              was originally located if it has already been delivered.  This
              entry overrides the data set directory contained in the direc-
              tive file.  It can be used in two different ways.  If the
              -deliver or -make option is entered, this option specifies the
              directory where a tilt series will be moved from, and it must be
              entered either once (if all data sets are in the same place) or
              once for each data set. If neither of those options are entered,
              then this option specifies the directory where the data set will
              be processed and must be entered once for each data set or not
              at all.  If the program is being rerun with the -deliver option
              and the data set has already been delivered on a previous run,
              the entry here can be either the original location of the stack
              or its current location; the program will recognize that its
              current location is under the delivery directory.  However, if
              the -make option was used and the data set was already deliv-
              ered, this entry must be the original location of the stack
              rather than its new location; otherwise the program will move it
              into a subdirectory of its data set directory.  (Successive
              entries accumulate)

       -deliver (-de) OR -DeliverToDirectory    File name
              Make a subdirectory of the specified directory for each data
              set, named with the root name of the data set, and move the tilt
              series into the subdirectory.  This option can be entered either
              once, or once for each data set to deliver to more than one
              location.  Several other files with the same root name as the
              tilt series will be moved from the current location of the data
              set: an associated metadata file with extension ".mdoc", a log
              of acquisition with extension ".log", and a file of tilt angles
              with extension ".rawtlt".  A raw boundary model for patch track-
              ing or autoseeding will also be delivered.  The -current option
              must be entered either once, or once for each data set.  (Suc-
              cessive entries accumulate)

       -make (-mak) OR -MakeSubDirectory
              Make a subdirectory for each data set at its current location,
              named with the root name of the data set, and move the tilt
              series into the subdirectory.  Other files will be moved there
              as well, just as for the -deliver option.  With this option, you
              can sort data sets into multiple top-level directories, and the
              program will take care of putting each one into its own subdi-
              rectory.  This option cannot be entered with the -deliver
              option.  The -current option must be entered either once, or
              once for each data set, and must specify the original location
              of the stack if the program is being rerun and the stack has
              already been delivered.

       -check (-ch) OR -CheckFile     File name
              File to check for quit and finish signals.  The default is
              batchruntomo.###.input in the directory where this program is
              started, where ### is the process ID.  This file should not have
              the same name as the file used by Batchruntomo to control Pro-
              cesschunks when it runs jobs, which is "processchunks.cmds" in
              the data set directory.  If a Q is placed in this file, it will
              terminate any running jobs and stop all processing.  If an F is
              placed in the file, the program will stop after finishing the
              current data set.  Termination will usually be quicker if a Q is
              placed into "processchunks.cmds" as well, with no delay between
              that action and writing Q to this check file.  Thus, it should
              be done by a program or by entering a single command line with
              two commands on it.

       -frompath (-fr) OR -TranslatePathsFrom   Text string
              Prefix from which to translate directive, location, and delivery
              paths.  When this program is being run in parallel, the path for
              accessing the directories on the current machine may be at a
              different mount point than on the machine where the runs were
              set up. When there is RemoteDirectory entry, Splitbatch(1) will
              add this option and the next one to indicate how to translate
              the paths for directive files, current locations, and delivery
              locations to work on any machine.  These entries should be
              equivalent to a mountrule defined in cpu.adoc (see cpuadoc).
              Batchruntomo will use these translated paths in all runs (hence
              they must work on all machines, even if they are accessed
              through links on some machines).  However, it will translated
              paths in messages to the log back to having the original pre-
              fixes so that Etomo can track file changes.

       -topath (-t) OR -TranslatePathsTo   Text string
              Prefix to which directive, location, and delivery paths will be

       -remote (-re) OR -RemoteDirectory   File name
              Path on remote machines to directory from which this program is

         These options give information about the processing resources and
       their limits in the simplest case of not running data sets in parallel

       -cpus (-cp) OR -CPUMachineList      Text string
              Machines to use in parallel processing, or number of local
              cores.  Each machine name can be followed by a ":" (colon), fol-
              lowed by the number of CPUs to use in that machine; e.g.,
              druid:4,tubule:4.  Prior to IMOD 4.8.25, a "#" sign was used
              instead of ":".  Using the "#" sign will now result in a warning
              when entering options on the command line and an error when
              entering options through standard input, because in the latter
              case it cannot be distinguished from the start of a comment.  If
              the environment variable MULTI_PROC_CPU_LIST is set, it over-
              rides this entry to the program.

       -single (-si) OR -SingleOnFirstCPU
              Run single command files on the first machine in the CPU list,
              instead of on the local machine.  The matchvol and autopatchfit
              steps in combine are not affected by this entry; they are always
              run on the machine with the most CPUs.

       -gpus (-gp) OR -GPUMachineList      Text string
              Machines to use for parallel processing with a GPU, or 1 for
              local GPU.  As when running processchunks with the -G option,
              multiple or specific GPUs can be specified by colon-separated
              numbers after a machine name, e.g., druid:2,tubule:1:2.  If the
              environment variable MULTI_PROC_GPU_POOL is set, it overrides
              this entry to the program.

       -nice (-n) OR -NiceValue       Integer
              Priority level or niceness for running jobs.  Note that this
              applies to all computations, not just those run with parallel
              processing.  The default is 15.

       -limit (-l) OR -LimitLocalThreads   Integer
              Limit on the number of threads when running a single command
              file on the local machine.  By default, this limit is set to the
              number of CPUs on the local machine indicated by the entry to
              -cpus.  If that entry consists only of remote machines, then the
              program has no information about this limit, and single command
              files containing programs parallelized with OpenMP will use all
              available cores.  This option allows one to enter a limit in
              this case, as well as to set a different limit than the number
              of CPUs in the CPU list.

         These options control the processing of multiple data sets in paral-
       lel, including on a cluster

       -parallel (-pa) OR -ParallelBatchRootName     Text string
              Root name that will be sent to Gpuallocator(1) when this program
              is being run in parallel with other instances and needs to use
              one or more GPUs.  In this case, the entry for -gpus needs to
              have the full list of GPUs available for all of the instances.
              This entry needs to be included to indicate batch is being run
              in parallel even when there are no GPUs.

       -maxGPUs (-max) OR -MaxGPUsInParallelBatch    Integer
              Maximum number of GPUs to use when running this program in par-
              allel.  The default is 4.  See Splitbatch(1) and Gpualloca-

       -queue (-q) OR -QueueCommand   Text string
              Command for running processes on a cluster queue when running in
              parallel, typically a Queuechunk command.  This entry would
              be the same as the "command" entry for a queue in a cpu.adoc
              (see cpuadoc and "IMOD/autodoc/cpu.adoc" for examples).  If
              the environment variable MULTI_PROC_QUEUE_COMMAND is set, it
              overrides this entry to the program.  With this entry, the pro-
              gram runs all operations that can be parallelized in chunks on
              the queue.  The treatment of single command files for other
              operations depends on whether MULTI_PROC_QUEUE_COMMAND was set:
              if it is, then Batchruntomo assumes it is itself running on a
              queue and runs each command file normally; otherwise it runs
              each command file on the queue.  With this entry, -cpus and
              -gpus entries are not allowed.

       -jobs (-jo) OR -MaxJobsOnQueue      Integer
              Maximum number of jobs to submit to the cluster queue, which
              influences how many chunks parallel operations will be divided
              into.  If the environment variable MULTI_PROC_MAX_QUEUE_JOBS is
              set, it overrides this entry to the program.

       -jcores (-jc) OR -CoresPerClusterJob     Integer
              Number of cores available to each job when running on a cluster;
              these cores will be used directly instead of submitting command
              files to the cluster queue.  Thus, this entry and -queue cannot
              be entered together, nor can -cpus and -gpus be entered.  This
              entry also sets the thread limit for processes whose command
              files are not split into chunks.  If the environment variable
              MULTI_PROC_JOB_CORES is set, it overrides this entry to the pro-

       -jgpus (-jg) OR -GPUsPerClusterJob       Integer
              Number of GPUs available to each job when running on a cluster;
              these GPUs will be used directly instead of submitting command
              files to the cluster queue.  This option cannot be entered
              unless -jcores is.  If the environment variable
              MULTI_PROC_JOB_GPUS is set, it overrides this entry to the pro-

       -gqueue (-gq) OR -GPUQueueCommand   Text string
              Command for running GPU-using processes on a secondary cluster
              queue, typically another Queuechunk command.  For such a
              process, either a single command file or separate chunks will be
              submitted to this queue, depending on how many jobs are allowed
              for this queue (the -gjobs entry). Ideally, this queue should
              provide access to just one GPU, because there is no mechanism
              for the submitted jobs to make use of multiple GPUs.  If the
              environment variable MULTI_PROC_GPU_QUEUE is set, it overrides
              this entry to the program.  The option can entered only if
              -queue or -jcores is entered.

       -gjobs (-gj) OR -MaxGPUJobsOnQueue       Integer
              Maximum number of jobs to submit to the secondary cluster queue
              specified with -gqueue; this corresponds to the "number" entry
              in a cpu.adoc section for that queue.  The maximum jobs will
              also be limited by the entry for -maxGPUs.  If the environment
              variable MULTI_PROC_MAX_GPU_JOBS is set, it overrides this entry
              to the program.

         These options allow doing a subset of processing operations

       -one (-o) OR -ProcessOneAxis   Integer
              Enter 1 to do setup and process A axis, or 2 to skip setup and
              process only B axis.  This entry applies to all dual-axis data
              sets being run.

       -end (-en) OR -EndingStep      Floating point
              Step to end processing with in each set and axis.  Steps are
              numbered as follows:
                 0: Setup
                 1: Preprocessing
                 2: Cross-correlation alignment
                 3: Prealigned stack
                 4: Patch tracking, autoseeding, or RAPTOR
                 5: Bead tracking
                 6: Alignment
                 7: Positioning
                 8: Aligned stack generation
                 9: CTF plotting
                 10: 3D gold detection
                 11: CTF correction
                 12: Gold erasing after transforming fiducial model or
                     projecting 3D model
                 13: 2D filtering
                 14: Reconstruction
                 14.5: Postprocessing on a/b axis reconstruction
                 15: Combine setup
                 16: Solvematch
                 17: Initial matchvol
                 18: Autopatchfit
                 19: Volcombine
                 20: Postprocessing with Trimvol
                 21: NAD (Nonlinear anistropic diffusion)
                 22: Cleanup
                These numbers can be relied on not to change again since this
              is now a floating point entry; fractional values will be used if
              necessary to insert more steps without renumbering.

       -start (-sta) OR -StartingStep      Floating point
              Step to start processing with.  Steps are numbered as described
              for -end.  Each set will start at this step unless the -first
              option is entered, in which case only the first set will start
              with this step and others will start from the beginning.  In the
              latter case, add 100 to start with this step in the second axis
              of a dual-axis data set.

       -first (-fi) OR -StartForFirstSetOnly
              Skip to starting step only for the first set; the following sets
              will be run from the beginning.

       -use OR -UseExistingAlignment
              This option can be used to rerun Tiltalign instead of using the
              existing fine alignment when the -start option is used with a
              value over 6 and the program will be running any steps that
              involve the aligned stack or reconstruction.  With an entry of 0
              in this situation, the program reruns Tiltalign iteratively
              in the usual fashion before proceeding (this can only be done
              with input on sequential lines, not from command line).  The
              default is not to do this, to keep the program from overwriting
              an alignment based on manually adjusted parameters.  Rerunning
              alignment in this situation was a necessity in early versions of
              the program but should no longer be needed; this option is being
              kept just for backwards compatibility.

       -exit (-exi) OR -ExitOnError
              Exit on error instead of going on to next axis or data set

         Aside from the email options, these are miscellaneous options for
       testing or for use with Etomo.

       -email (-em) OR -EmailAddress       Text string
              Address for sending email messages either when a set is aborted
              or when the run completes.

       -SMTP (-SM) OR -SMTPserver     Text string
              An SMTP server for sending email messages.  The default is
              "localhost", but if the local host cannot send email, a server
              can be entered with this option.

       -validation (-v) OR -ValidationType      Integer
              By default, the program will check all directives in the direc-
              tive and template files against the master list of directives in
              IMOD_DIR/com/directives.csv.  Enter 1 to run the program just to
              check a directive file and exit, 2 to check a template file and
              exit, or -1 to skip this step.  When checking a directive file,
              the directive file is first processed to find certain required
              values, and an error in that step will prevent the checking of
              directives.  Template files referenced by the directive file
              will be checked as well.  With an entry of 2, the file will be
              checked right after reading.  A batch directive file is required
              to contain only directives with a Y in the "Batch" column of the
              master directives file, and a template file must contain only
              directives with a Y in the "Tmplt" column.

       -style (-sty) OR -NamingStyle       Integer
              0 for output image files to have descriptive extensions like
              ".preali", 1 for extension ".mrc", or 2 for extension ".hdf".
              In the latter two cases the usual decriptive text is put before
              the extension and generated image files will have the corre-
              sponding type.

       -pcm (-pc) OR -MakeComExtensionPcm       Integer
              0 or 1 to name output command files with extension ".com" or
              ".pcm".  The latter is not yet supported by Etomo.

       -ext OR -StackExtension   Text string
              Extension of an existing raw stack, used when just renaming
              stacks(s), omitting the period.  With this entry, the program
              will perform the same renaming operations on one or two raw
              stacks in the current directory as when doing a reconstruction.
              The -root option must be entered once with the root name of the
              file, and the -style option must also be entered.

       -axis (-a) OR -AxisOfExtension      Integer
              When just renaming stacks, enter 0 for a single-axis dataset, or
              1 or 2 for whether the extension provided with -ext applies to
              the a or b stack of a dual-axis set.  With a non-zero naming
              style, the stack that this extension applies to will keep its
              name, and the other axis of a dual axis set will have its exten-
              sion changed to match if necessary.

       -etomo (-et) OR -EtomoDebug    Integer
              Run Etomo with the given debug level

       -bypass (-b) OR -BypassEtomo
              Run Copytomocoms directly instead of Etomo for the setup step.
              The program will add options to extract tilt angles if direc-
              tives for using a raw tilt file are not present, and to specify
              axis rotation angles if those directives are not present.
              Setupset directives in the batch file only are passed as options
              to Copytomocoms, but it may still need other directives there.
              Excluded views are not removed since Etomo does that.  An "edf"
              file is made with just the items that would be added by

       -PID   Print process ID

       -help (-h) OR -usage
              Print help output

              Read parameter entries from standard input

       Batch and template files must have the extension ".adoc".  The tilt
       series file must have an extension ".st" or ".mrc".  When Etomo is run
       to set up the data set, it copies the batch directive file to the
       dataset directory with the name "batchDirective.adoc".  If there are
       any template files, they are copied to the dataset with the the names
       "scopeTemplate.adoc", "systemTemplate.adoc", and "userTemplate.adoc".
       However, if any feature is used that requires modification of the batch
       directive file, namely use of the -root or -current option or template
       files listed without paths, Batchruntomo writes the modified file to
       the dataset directory as "batchDirective.adoc" and runs Etomo with this

       David Mastronarde

       Email bug reports to mast at colorado dot edu


IMOD                                4.12.40                    batchruntomo(1)