ctf3dsetup(1) General Commands Manual ctf3dsetup(1)
NAME
ctf3dsetup
SYNOPSIS
ctf3dsetup options tilt_com_file
DESCRIPTION
Ctf3dsetup creates command files for a tomographic reconstruction with
CTF correction in 3-D. These command files will generate a series of
aligned stacks corrected for different Z depths in the tomogram, and
subsets of the tomogram in Z at the corresponding Z depths. These sub-
sets are referred to as slabs. The number of slabs can be entered
directly, or a maximum slab thickness in nanometers can be entered
instead. When processing with the standard command files managed by
Etomo or Batchruntomo, only the name of the Tilt command needs
to be entered, but an alternative command file for CTF correction can
also be entered.
The program can be run either starting with an aligned stack, or by
reconstructing directly from a corrected, unaligned stack. The
requirements are different in the two cases.
Reconstructions from Aligned Images
To use an aligned stack, an uncorrected aligned stack must be avail-
able, as specified in the command file used for CTF correction, desired
parameters must be set in that file, and the defocus file must exist.
If you wish to apply 2-D filtering (i.e., dose weighting), you can
apply this to the uncorrected stack in advance without affecting the
validity of the correction. Alternatively, you can use the -filter
option to have this step done for you, but the parameters for this fil-
tering must already be set in the command file for filtering. If you
wish to erase gold beads in the aligned stack, you can also apply this
in advance, which is not the standard processing order, or you can use
the -erase option to have this step done on each corrected stack. In
the latter case, the model for erasing the beads must already exist,
and parameters must be set already in the command file for gold eras-
ing.
Reconstructions from Raw Images
To use just the raw stack, an aligned stack need not exist and is
ignored. Desired parameters must be set in the command file for CTF
correction, and the defocus file must exist. If 2-D filtering is to be
done, it will be done on the raw stack by the command files produced
here, and parameters must be set already in the command file for fil-
tering. If gold beads are to be erased, that will happen after creat-
ing the CTF-corrected raw stack for each defocus level, the parameters
must be set in the gold erasing command file, and the model for erasing
must exist already. In this case you may want to make an aligned stack
for assessing the erasing, but neither the binning of this aligned
stack nor that of tomogram on which beads are founds needs to match the
reduction, if any, of the raw stack.
Reconstructions from raw images cannot take account of corrections for
a distortion field or magnification gradients and will not be valid if
either of these corrections were used when making the aligned stack.
The program will examine the Newstack log and/or command files from
making both the coarse and final aligned stacks and issue a warning if
either of these contains distortion corrections.
Computational Resources and Strategies
CTF correction and reconstruction can be run on either CPUs or GPUs,
depending on whether use of a GPU is specified in the command file for
reconstruction. A different choice in the command file for correction
will be ignored. The jobs will run either on multiple CPUs or multiple
GPUs, not a mixture of the two, so the number of "processing units"
referred to next means the number of GPUs when running on GPU.
This program provides two different computational strategies; the
choice between them depends on what CPU or GPU resources are available.
It can produce one command file per slab, each one doing all operations
for that slab (CTF correction of the full image, optional gold erasing,
and reconstruction of the full slab.) These command files are suitable
for running the slabs in parallel with each other. The other strategy
is to do the slabs sequentially, but use parallel processing (if there
are multiple processing units) in both the CTF correction and recon-
struction steps. See the options -parallel and -procs for more
details. Running the slabs in parallel is probably more efficient if
there are multiple processing units but will require more temporary
storage. If you are running on the GPU and computing the slabs sequen-
tially with parallel processing, enter the number of GPUs you expect to
run it on with the -procs option. If you have just one GPU, you can
omit the -procs option unless you want to try loading it with more than
one job, which may or may not be useful.
Correcting for X-axis Tilt
When reconstructing from an aligned stack, if an X-axis tilt is
included in either the reconstruction or correction command files, then
the program evaluates whether it is similar enough in the two opera-
tions. If an X-axis tilt is being applied at all during correction, it
should closely match the value used for reconstruction, otherwise the
program gives a warning. If there is an X-axis tilt in reconstruction
but not correction, the program will determine the maximum difference
in defocus produced by the two different X-axis tilt values, which
occurs at the extremes of the reconstruction in Y. If this defocus is
larger than a certain fraction of the slab thickness, the program
issues a warning.
When reconstructing from the raw stack, the X-axis tilt in the recon-
struction command file is unconditionally used for the correction.
There is no additional time involved in using the X-axis tilt in this
case because the CTF correction has to be done in diagonal strips of
full-sized FFTs anyway.
OPTIONS
Ctf3dsetup uses the PIP package for input (see the manual page for
pip). Options can be specified either as command line arguments
(with the -) or one per line in a command file (without the -).
Options can be abbreviated to unique letters; the currently valid
abbreviations for short names are shown in parentheses.
-reccom (-rec) OR -TiltCommandFile File name
Command file for reconstruction, which can be entered with or
without its extension of ".com". If this option is not entered,
a non-option argument is used for this command file. This entry
is required one way or the other.
-ctfcom (-c) OR -CorrectionComFile File name
Command file for CTF correction, which can be entered with or
without its extension of ".com". If this option is not entered,
"ctfcorrection" will be substituted for "tilt" in the name of
the command file for reconstruction.
-slabs (-s) OR -NumberOfSlabs Integer
Number of reconstructions to make at different depths in Z with
different corrections for CTF. Either this option or -thickness
must be entered, but not both. This entry must be at least 3.
-thickness (-th) OR -SlabThicknessInNm Integer
The maximum thickness of reconstructions at different depths, in
nanometers. This value, if entered, must be less than half of
the tomogram thickness so that at least 3 three slabs are com-
puted.
-invert (-i) OR -InvertSlabZOffsets Integer
1 to invert the sign of the offset in Z applied when doing the
CTF correction for each slab. This would be needed if the tomo-
gram is upside-down from its orientation in the microscope, by
rotation and/or inversion in Z. Tests with the most common
sources of this problem indicate that this occurs whenever tilt
angles need inversion in Ctfplotter and Ctfphaseflip; thus
the default is to use the value of the -invert option in the
command file for CTF correction, with the option here available
to override that value. See the section on Inverting Tilt
Angles in the Ctfplotter man page for more details.
-adjust (-ad) OR -AdjustForAlignZShift
Adjust for Z shift of tomogram away from the height that would
have been produced by the cross-correlation alignment alone.
Without this option, the program assumes that the tomogram has
been positioned in Z so that the material providing the average
CTF signal is centered on the middle of the tomogram, and it
applies zero Z offset in the CTF correction for the slab in the
middle of the tomogram. However, the original cross-correlation
(coarse) alignment would produce a tomogram in which the con-
trast-producing material is centered in Z, on average. If the
same material is responsible for the signals used to measure the
CTF, then a CTF correction with zero Z offset would be appropri-
ate for the middle of a tomogram based on that alignment, not
the one resulting from positioning. With this option, the pro-
gram finds the total shift imposed by the Z offset values in
align.com and tilt.com and adds that to the Z offset used in
Ctfphaseflip so that the portion of the tomogram shifted by
that amount has zero Z offset in the correction. This option is
thus appropriate if the material responsible for the CTF signal
is not centered in tomogram after Z positioning.
-parallel (-pa) OR -RunSlabsInParallel
Make each slab with one processor or GPU by creating a chunk
file containing both the Ctfphaseflip and Tilt commands.
This arrangement is the most efficient if there are more slabs
than GPUs or processors, but requires more storage, as there
will be as many created aligned stacks as processors. The
default is to make each slab sequentially, with parallel pro-
cessing (if any) used to create the aligned stack then recon-
struct the slab. In this case only one corrected aligned stack
exists at any time.
-procs (-pr) OR -NumberOfProcessors Integer
Use this option to specify the number of processors or GPUs that
you expect to run the command files on. It is appropriate only
when computing slabs sequentially and cannot be entered with the
-parallel option. It is passed directly to Splittilt unless
it is 1, in which case the processing is not divided into
chunks. The default is 8 unless the Tilt command file contains
the option to use the GPU, in which case it is 1.
-perproc (-pe) OR -ChunksPerProcessor Integer
Use this option to set the number of chunks per processor when
correction and reconstruction are divided into chunks; it will
be multiplied by the number of processors to give a value for
the -TargetChunks option to Splittilt and used to determine
the number of slices to be corrected per chunk in Splitcorrec-
tion(1). The default is 3.
-erase (-e) OR -EraseFiducials
Run a command file to erase gold after CTF correction of aligned
stack
-goldcom (-g) OR -GoldEraserComFile File name
Name of command file to use for erasing gold. If this option is
not entered, "golderaser" will be substituted for "tilt" in the
name of the command file for reconstruction.
-filter (-f) OR -FilterIn2D
Filter images in 2-D. If filtering is desired, this entry is
needed when reconstructing from a raw stack, but when using an
aligned stack, the filtering may be done either beforehand or
with this option.
-2dcom (-2) OR -2DFilterComFile File name
Name of command file to use for 2-D filtering. If this option
is not entered, "mtffilter" will be substituted for "tilt" in
the name of the command file for reconstruction.
-unaligned (-un) OR -UseUnalignedImages
Use the raw, unaligned stack for CTF correction and reconstruc-
tion. This should be the only option that is needed to use the
raw stack, other than the -reduce option if desired, when this
program is run inside a directory with a full set of processing
files from one data set.
-reduce (-red) OR -FourierReduceByFactor Integer
Reduce the raw stack by the given factor by using Newstack to
crop its Fourier transform and use this reduced stack for pro-
cessing from unaligned images.
-raw (-ra) OR -RawStackFile File name
Name of raw image stack. This option is needed when using
unaligned images only if this filename cannot be determined by
analyzing command files in the directory.
-pixel (-pi) OR -RawPixelSize Floating point
Pixel size of raw image stack in nanometers. This entry is
needed when using unaligned images only if the pixel size is 1.0
in the image file header, and the pixel size is not found in a
track.com file.
-xform (-x) OR -AlignTransformFile File name
File with the linear transformations that were used to align the
images. This option is needed when using unaligned images only
if this filename cannot be determined by analyzing command files
in the directory.
-axis (-ax) OR -AxisAngle Floating point
Angle that the tilt axis is rotated from vertical (counterclock-
wise positive), for correcting unaligned images in their origi-
nal orientation. This entry is needed only if it cannot be
found in an alignment log file.
-vertical (-v) OR -VerticalSlices
Do vertical slices with interpolation between slices instead of
old-style X-axis tilting, regardless of the time penalty.
-oldstyle (-o) OR -OldStyleXtilting
Do old-style X-axis tilting, with direct backprojection into
each output voxel, instead of vertical slices with interpolation
between slices.
-tempdir (-te) OR -TemporaryDirectory Text string
Directory to use for temporary files. The default is that the
temporary files will be placed in the current directory.
-leave (-l) OR -LeaveTempFiles
Leave temporary CTF-corrected stacks and reconstructed slabs
-boundary (-b) OR -BoundaryPixels Integer
Set the number of boundary pixels saved in separate files to the
given value when slabs are computed sequentially and each CTF
correction and reconstruction is parallelized. The boundary
pixels are rewritten to the output file (aligned stack or slab
reconstruction) after all chunks are done.
-help (-h) OR -usage
Print help output
-StandardInput
Read parameter entries from standard input
EXAMPLES
These examples illustrate how to run with various sets of processing
units. They assume that 25-nm slabs are wanted and that all command
files have standard names.
To run on one GPU, make command files with:
ctf3dsetup -thick 25 tilt
These command files will create the slabs sequentially with separate
files for CTF correction and reconstruction. They can be run with
just:
subm ctf3d*.com
To run on several GPUs, you almost certainly have space for as many
temporary stacks as GPUs, so you can make command files to run slabs in
parallel with:
ctf3dsetup -thick 25 -parallel tilt
All of the operations for a slab are within one command file. If the
GPUs are on separate machines, you can use the parallel processing
interface in Etomo or run them with Processchunks with a command
like:
processchunks machine1,machine2,machine3,machine4 ctf3d
If the GPUs are on the local machine, run them with a command like:
processchunks -G localhost:1:2:3:4 ctf3d
so that jobs are directed to different GPUs by Processchunks;
whereas for multiple GPUs per remote machine you can use a command
like:
processchunks -G machine1:1:2,machine2:1:2 ctf3d
You can also run slabs in parallel on multiple CPUs if there is no risk
of running out of space for temporary files, using the ctf3dsetup com-
mand above. You can run them in the Etomo parallel processing inter-
face, or run them with processchunks using a command like
processchunks 12 ctf3d
for the local machine, or
processchunks machine1:4,machine2:4,machine3:4 ctf3d
for 4 cores per specified machine.
If space for temporary files might be limited, generate the slabs
sequentially, doing parallel processing within each slab, with a com-
mand like:
ctf3dsetup -proc 16 -thick 25 tilt
and run this with the Etomo parallel processing interface or the kinds
of commands just given for running on CPUs.
FILES
The command files have the prefix "ctf3d-". The final reconstruction
is named with "_3dctf" added to the root of the output file name in the
reconstruction command file.
AUTHOR
David Mastronarde
SEE ALSO
ctfphaseflip, tilt, ccderaser
BUGS
Email bug reports to mast at colorado dot edu.
IMOD 5.2.0 ctf3dsetup(1)