ctfplotter(1) General Commands Manual ctfplotter(1)
NAME
ctfplotter - estimate defocus values of a tilt series
SYNOPSIS
ctfplotter options
DESCRIPTION
This GUI program will plot the logarithm of a rotationally averaged
power spectrum of an input tilt series after subtracting the noise
floor. The method is based on periodogram averaging; namely, averaging
of spectra from small, overlapping areas of the images, referred to as
tiles. The user can interactively choose which projection views are
included in the averaging. It is also possible to run the program non-
interactively for automatic fitting.
Noise Files
The noise floor is usually based on actual measurements of noise in an
image taken with no specimen in the beam. The program does have an
alternative method of subtracting a background level that is based on
fitting two lines to the spectrum, one to the rapidly falling region at
low frequencies and one to the more gradually falling high-frequency
end. This method may work well enough with some data, particularly
from an electron counting camera. However, if this method does not
work, you need to supply noise images.
To get noise images, collect a set of blank images at a series of beam
intensities. They can be placed in separate files or all in one stack.
These noise images are specific not only to the microscope but also to
the camera used, the operating voltage, and the binning. However,
these are not factors that vary over time (unless there is some radical
change in the camera), so you should only have to collect these images
once. The mean counts in these images should span the range of the
mean counts in the data being analyzed. The interval between intensi-
ties can be a factor of 1.5; interpolation is used to estimate the
noise for an image whose mean counts do not match one of the noise
images. For example, for a camera that has a gain of 10 counts per
electron, one might collect images at mean counts near 100, 150, 225,
etc., up to 1710. With a 1 nm pixel size, these noise images could be
used with images where the dose recorded at the camera ranged from 0.1
to 1.7 electrons per square Angstrom. If you need images for analyzing
both unbinned and binned data, you can take one set of unbinned noise
images and then bin them with a Newstack command like this:
newstack -bin 2 -mult 4,0 input_file output_file
The multiplication by 4 (the square of the binning factor) is necessary
to make the counts be the same as they would be for binned images from
the microscope with the same exposure.
If there are significant numbers of X-rays in the noise images, you
should clean them with Ccderaser, such as with:
ccderaser -find -peak 8 -diff 6 input_file output_file
The noise images must have positive means; i.e., their values should be
proportional to the recorded electrons. If they are acquired with
software that subtracts 32768 before storing as signed integers, then
you need to add this offset back to the data in order to use them as
noise images.
If the noise images are in one stack, its file name is entered directly
with the -config option. If the noise images are in separate files,
they are listed in a simple text file, the configuration file specified
by the -config option. To be found easily from within Etomo, the lat-
ter file, or a single stack, should be placed in a directory named
/usr/local/ImodCalib/CTFnoise or in $IMOD_CALIB_DIR/CTFnoise if the
environment variable IMOD_CALIB_DIR has been defined differently from
the default. Noise files can be placed in a subdirectory of CTFnoise,
for example, F20. In this case, the configuration file would contain a
list like this:
F20/file1.st
F20/file2.st
F20/file3.st
Noise Files for Direct Detector Cameras
For direct electron detector cameras, the actual power spectra may not
fit the noise spectra very well, particularly if motion correction is
applied to tilt series movie frames, or if electron counting is used.
The fitting of a polynomial to the baseline of the power spectrum,
described below, will generally give a flat enough power spectrum at
higher frequencies. However, when using motion correction, there are
two potential solutions that may give better noise files. Spectra from
ordinary noise images will not fit the actual spectra if the aligning
and summing of movie frames attenuates the high frequencies, which is
inevitable if interpolation is done in real space to align the images.
The best remedy is to shift frames into alignment with phase shifts in
Fourier space instead of with interpolation. This operation preserves
the high frequency power. Frames that have been aligned in SerialEM's
plugin to DigitalMicrograph or in Alignframes have been shifted in
Fourier space, preserving the power. If you determine the frame align-
ment some other way, you can transform the frames in Newstack with
the -phase option to shift them in Fourier space. The second remedy
would be to collect movie frames for the noise images and apply small
shifts to them by the same method used to shift actual frames before
summing them. Since the noise frames cannot be aligned, the shifts
need to be obtained some other way; either take a set of shifts from
real images, or use some random numbers between 0 and 1.
At least currently, the response of an electron counting camera depends
on the dose rate, which raises the question of whether to vary exposure
time or beam intensity when taking the set of noise images. In princi-
ple, varying beam intensity will replicate the differences between
light and dark areas in the same image; varying exposure time will give
noise images applicable to a range of images taken with the same dose
rate but different exposure times. There does not seem to be a good
solution here.
As of IMOD 4.10.18, the baseline fitting option is turned on by default
since its performance has been improved. It can be disabled, or the
polynomial order set, in a template file when processing through Etomo;
for example, the template directive
comparam.ctfplotter.ctfplotter.BaselineFittingOrder = 2
will restrict the fitting to a parabolic baseline.
Program operation
When you start the program, it will first load and analyze all of the
noise images. Then it will read in the images within the specified
initial angular range and compute power spectra from the central tiles.
This initial computation of tile power spectra is the most time-consum-
ing step, whereas recomputing a summed power spectrum from the spectra
of individual tiles is quick. The program thus stores the power spec-
trum for each tile in a cache when it is computed. You will have to
wait several seconds whenever you change the angular range to include
views that have not been analyzed before. However, once all of the
tiles on all views have been analyzed, the program will respond quickly
when you change angular ranges or defocus values.
You may find that the power spectrum curves up or down at high frequen-
cies, indicating that the noise power spectra do not adequately
describe the noise in the actual data and that the baseline fitting is
not working optimally to make the power spectrum be flat at high fre-
quencies. This fitting is controlled by an entry set in the Fitting
dialog, whose value can be initialized with the -baseline option. To
do this fitting, the program first identifies points that could define
a noise floor, primarily using points at local minima in the spectrum,
but sampling other points if needed at the highest frequencies. The
program then fits a concave or convex polynomial of a selected order to
those points and subtracts it from the power spectrum. The result is
to flatten the spectrum, except perhaps past 0.45/pixel. The order can
be set to 1 to fit a line, 2 to fit a parabola, or 3 or 4 to fit 3rd or
4th order equations. (To keep higher order polynomials well-behaved,
they are constrained to have a monotonically changing first derivative,
as a parabola does.) This fitting should be adjusted when the baseline
deviates from flat by enough to impair the ability to visualize the
zeroes or fit the spectrum.
To read a general description of the plots and tool buttons, click the
handbook icon to access the HTML help page.
The defocus values found by the program, as well as astigmatism and
phase shift if those are also found, can be saved in a text file that
can be used as input to the CTF correction program Ctfphaseflip.
OPTIONS
Ctfplotter uses the PIP package for input (see the manual page for
pip). Options can be specified either as command line arguments
(with the -) or one per line in a command file (without the -).
Options can be abbreviated to unique letters; the currently valid
abbreviations for short names are shown in parentheses.
-input (-inp) OR -InputStack File name
Input stack whose defocus will be estimated. In principle, this
should be a raw stack, not an aligned stack, because the inter-
polation used to make an aligned stack attenuates high frequen-
cies and the noise power spectra would no longer match. Also,
the usable CTF signal at high frequencies would be reduced.
-offset (-of) OR -OffsetToAdd Floating point
The program must analyze data values where 0 corresponds to no
electrons recorded. Use this option to specify a value to add
to your input stack that will make values positive and propor-
tional to recorded electrons. The value will not be added to
noise files; they are required to be positive. The program
automatically sets this value to 32768 if it recognizes a file
from FEI data acquisition software with a minimum below 0, or if
the minimum and mean of a non-FEI file are sufficiently nega-
tive.
-config (-con) OR -ConfigFile File name
If a text file is entered, this configure file specifies a set
of single-image noise files used to estimate the noise floor,
one file per line. The files can be specified with either abso-
lute paths or with paths relative to the location of the config-
ure file itself. Alternatively, all noise images can be in one
stack and the name of the stack provided here.
-angleFn (-an) OR -AngleFile File name
File containing tilt angles for the input stack. Each line of
this file is the tilt angle for a view of the input stack. The
angles are listed in order starting from view 1. If no file is
entered, the angles are assumed to be 0. In order to make the
interface usable, the views are assigned angles at 0.01 degree
increments, but all spectrum computation uses angles of zero.
-invert (-inv) OR -InvertTiltAngles
Invert the sign of the tilt angles to compensate for a left-
handed coordinate system in the microscope. When the sign of
the tilt angles and the value of the tilt axis rotation angle
are such that reconstructions are generated with inverted hand-
edness, then this option is needed to keep power spectra for
off-center tiles from being shifted in the wrong direction. One
way to assess this need is to examine the images in a stack
aligned for reconstruction, with the tilt axis vertical. At
positive tilt angles, the right side of the images should be
more underfocused than the left side; if not, then this option
is needed.
-aAngle (-aA) OR -AxisAngle Floating point
Specifies how much the tilt axis deviates from vertical (Y
axis). This angle is in degrees. It follows the right hand
rule and counter-clockwise is positive.
-defFn (-defF) OR -DefocusFile File name
File to store found defocuses. Each entry in the file will con-
sist of the starting and ending view number of the range of
views being fit to (numbered from 1), the starting and ending
angle of the tilt angle range being fit to, and the defocus
value in nanometers (underfocus positive). The full specifica-
tion for defocus files acceptable to Ctfphaseflip is given in
a section at the end of that man page. When Ctfplotter writes a
new defocus file, it puts a version number (currently 2) at the
end of the first line. If this file already exists when the
program starts, it will be read in and the results displayed in
a table in the Angle Range and Tile Selection dialog. The previ-
ous version will become a backup file (with ~ added to its name)
when new results are saved to this file.
If you supply an initial defocus file with more than line, be
sure to use the exact angles from the tilt angle file specified
with -angleFn; do not round to one decimal place. Alterna-
tively, add the number "2" as an extra value at the end of the
first line of the file; this will prevent the program from
thinking that the view numbers might be off by one. If you use
the -invert option, you must do one of two things: 1) either
invert all the tilt angles in this file, 2) or start the file
with a line to indicate that it is a version 3 file in which the
angles need to be inverted. That line starts with a sum of
flags for various options, where 16 is the flag that angles need
to be inverted; if that is the only flag, then the line would be
"16 0. 0. 0 3". See Ctfphaseflip for a description of all
flags.
-pixelSize (-pi) OR -PixelSize Floating point
Image pixel size in nanometers.
-crop (-cr) OR -CropToPixelSize Floating point
Crop power spectrum to the given pixel size in nanometers. This
option spreads out the low-frequency part of the power spectrum
and throws away the highest frequencies; the new Nyquist fre-
quency is 2 divided by the entered value. Cropping may improve
the fits to closely spaced zeros, but it will average less data
in each frequency bin and thus make the power spectrum more
noisy. The entered pixel size must be larger than the actual
size for this option to have any effect.
-fitZeros (-fit) OR -FitZerosForPhase
When finding phase, try to find and fit to zero positions in the
spectrum instead of just fitting to the power spectrum. Fitting
to zero positions can prevent the first few zeros from dominat-
ing the fit and may improve the fit to the higher zeros, but it
may also result in the first zero being noticeably off.
-volt (-vo) OR -Voltage Integer
Microscope voltage in kV.
-cs OR -SphericalAberration Floating point
Microscope spherical aberration in millimeters. A value of 0
can be entered; it will be made slightly larger to prevent divi-
sion by 0 in the CTF equations.
-ampContrast (-am) OR -AmplitudeContrast Floating point
The fraction of amplitude contrast. For cryo-EM, it should be
between 0.07 and 0.14. The default is 0.07.
-degPhase (-deg) OR -PhaseShiftInDegrees Floating point
The expected value for the phase shift imposed by a phase plate,
in degrees. Equations relating the CTF and defocus will include
this phase shift when phase is not solved for. It can be
entered with either positive or negative sign; the absolute
value of the entered value will be subtracted when computing
phase within the program.
-phase (-pha) OR -PhasePlateShift Floating point
The expected value for the phase shift imposed by a phase plate,
in radians. See the -degPhase option, which should be used
instead. The two may not both be entered.
-phRange (-phR) OR -PhaseRangeToSearch Floating point
Total range of phase shift to search around the expected shift,
in degrees. The default is 120.
-cuton (-cu) OR -CutOnFrequency Floating point
A fixed value for cut-on frequency that attenuates phase at low
frequency, in reciprocal nanometers. Whenever the cut-on fre-
quency is non-zero, the phase shift displayed on the screen and
stored in the defocus file is actually the phase at a fixed fre-
quency of 0.3/nm rather than the shift at infinity.
-maxCuton (-ma) OR -MaxCutOnToSearch Floating point
Maximum frequency to test when searching for cut-on frequency,
in reciprocal nanometers. The default is the frequency of the
first zero at the expected defocus and phase shift.
-skip (-sk) OR -ViewsToSkip List of integer ranges
List of views to leave out of the analysis; views are numbered
from 1 and ranges can be entered. The program will behave as if
those views do not exist unless the option is set for skipping
only for analysis of astigmatism or phase shift. If it is,
defocus can be found for every view, but astigmatism and phase
will be solved from adjacent views to ones in the skip list.
-bidir (-bi) OR -BidirectionalNumViews Integer
Number of views in the first half of a bidirectional tilt
series. Entering this option will turn on the checkbox "Break
groups at view" to prevent combining of views for analysis
across the two halves of the series. Do not use this option for
a tilt series taken with the Hagen scheme. If the option is set
for skipping and breaking only for analysis of astigmatism or
phase shift, the program will still combine views for defocus
fitting across the boundary.
-onlyAP (-on) OR -SkipOnlyForAstigPhase
Skip views in the list of ones to skip, or avoid combining views
for analysis, only for analysis of astigmatism and phase shift.
With this option set, the program will exclude views in the skip
list and avoid combining views across the bidirectional boundary
only when getting a combined spectrum for finding astigmatism
and phase shift.
-expDef (-exp) OR -ExpectedDefocus Floating point
Expected defocus at the tilt axis in nanometers, with a positive
value for underfocus. The frequency of the first zero of the
CTF curve is first computed based on this expected defocus. The
segments of the CTF curve of the input stack around that fre-
quency are selected to be fitted.
-fpOffset (-fp) OR -FocalPairDefocusOffset Floating point
Normally, ctfplotter processes a single stack whose name is
given by InputStack. It can optionally process a pair of stacks
with identical tilt angles and mutually aligned projections,
taken with a constant defocus offset between them. If, for exam-
ple, InputStack is myData.st, one could place a stack with 6 um
underfocus in myData_1.st and a stack with 2.175 um underfocus
in myData_2.st and specify an ExpectedDefocus of 6000 and a
FocalPairDefocusOffset of -3825. There is some possibility that
this processing still works after the major changes in 2018, but
using it will prevent some of the new features from being avail-
able.
-range (-ra) OR -AngleRange Two floats
When the -autoFit option is not entered, this entry sets the
starting and ending tilt angles for the initial analysis and is
a required entry. Views with a tilt angle within this range are
used to compute the CTF curve. When -autoFit is entered, this
entry sets the whole extent over which steps will be taken in
autofitting.
-autoFit (-au) OR -AutoFitRangeAndStep Two floats
Do initial autofitting over the whole tilt series with the given
range of angles in each fit and step size between ranges. A
value of zero for the step will make it fit to each single image
separately, regardless of the value for the range. The range
entry can also be zero to fit to a single image. This aut-
ofitting differs from that invoked through the Angles dialog in
several respects: 1) Fitting is done at each angle with the cur-
rent defocus from the previous angle, if any, and the "Use cur-
rent defocus" radio button is left selected at the end, unless
the -useDef option is entered. If phase is being found, it will
also use current phase. 2) Three fitting iterations will be
done. 3) The size of the range is determined by the parameter
entered here, not by the starting and ending angles entered with
the -range option. This autofitting will not be done if there
are already values in the defocus file and if the -range option
is entered, in order to prevent an annoying message when opening
Ctfplotter after processing with Batchruntomo. Otherwise, if
autofitting is specified and there are existing defocus values,
the program will ask you to confirm whether to replace them.
-useDef (-use) OR -UseExpectedDefForAuto
Use the expected defocus value at each angle instead of the cur-
rent defocus when doing initial autofitting. This is the behav-
ior of the program before IMOD 4.10.18. At each angle, the
expected defocus is used the first time and the defocus estimate
is used for the next two iterations.
-frequency (-fr) OR -FrequencyRangeToFit Two floats
Starting and ending frequencies of range to fit in power spec-
trum. The two values will be used to set the "X1 starts" and
"X2 ends" fields in the fitting dialog after the pixel size that
governs the meaning of these frequencies has been adjusted for
any cropping of the spectrum. In other words, if you are also
entering the -crop option, enter the fitting range that works
for the cropping specified by that option.
-extra (-ext) OR -ExtraZerosToFit Floating point
By default, the ending frequency of the fitting range is set to
the expected location of the second zero. With this entry, the
range will be extended by the given multiple of the interval
between first and seconds zeros. For example, entries of 1 or 2
will fit approximately to the third or fourth zeros, respec-
tively. An entry of more than 0.5 will trigger fitting to two
exponentials, which is important for fitting multiple peaks
between zeros.
-vary (-va) OR -VaryExponentInFit
Vary exponent of CTF function when fitting a CTF-like curve
-baseline (-ba) OR -BaselineFittingOrder Integer
Baseline fitting is used to make the power spectrum be flat at
high frequencies; this entry initializes the "Baseline fitting
order" setting in the Fitting dialog. For details, see above.
The default is 4 unless the expected position of the first zero
is 0.25 reciprocal pixels or higher, in which case it is set to
1.
-find (-fin) OR -FindAstigPhaseCuton Three integers
Enter 1 to find astigmatism or 0 not to, 1 to find phase shift
or 0 not to, and 1 to find cut-on frequency also, or 0 not to.
-sAstig (-sA) OR -SearchAstigmatism
Search for astigmatism when fitting. This option cannot be
entered with -find.
-sPhase (-sP) OR -SearchPhaseShift
Search for phase shift when fitting. This option cannot be
entered with -find.
-sCuton (-sC) OR -SearchCutonFrequency
Search for cut-on frequency when finding phase shift. This
option cannot be entered with -find.
-minViews (-mi) OR -MinViewsAstigAndPhase Two integers
Minimum number of views for finding astigmatism and phase shift;
the default is 3 for astigmatism and 1 for phase.
-save (-sa) OR -SaveAndExit
Save defocus values to file and exit after autofitting. The
program will not ask for confirmation before removing existing
entries in the defocus table.
-psRes (-ps) OR -PSResolution Integer
The number of points over which CTF will be computed. The fre-
quency range up to Nyquist is divided into equal intervals
delineated by these points. The default is 101.
-tileSize (-t) OR -TileSize Integer
The tile size for computing spectra. The size is in pixels and
the tiles are square. The whole image is divided into tiles
that overlap by 50% in each direction. For each view, these
tiles are assigned to strips that are considered to have con-
stant defocus. The default is 256.
-defTol (-defT) OR -DefocusTol Integer
Defocus tolerance in nanometers defining the strip width. The
center strips are taken from the central region of a view that
has defocus difference less than this tolerance. Similarly, the
off-center strips are defined to include tiles whose defocus
range is than this amount. The default is 200.
-leftTol (-l) OR -LeftDefTol Floating point
Defocus tolerance in nanometers for strips to the left of the
center strip. When non-center strips are included in the aver-
age, strips to the left of center are included if their defocus
difference is less than the given value. The default is 2000.
-rightTol (-ri) OR -RightDefTol Floating point
Defocus tolerance in nanometers for strips to the right of the
center strip. When non-center strips are included in the aver-
age, strips to the right of center are included if their defocus
difference is less than the given value. The default is 2000.
-cache (-ca) OR -MaxCacheSize Integer
To speed up computation, ctfplotter uses a cache to hold the
rotationally averaged power spectra of strips of tiles at 20
times the final power spectrum resolution (or whatever is
entered with the -hyper option). This option controls the cache
size. The default value is 1000 megabytes if the system physi-
cal memory cannot be determined; otherwise the size is the mini-
mum of 15 GB, 3/4 of physical memory, and physical memory minus
1 GB, for memory up to 30 GB, and half of physical memory above
30 GB; but in any case at least 400 MB. These large limits were
implemented before caching was changed to hold strip sums
instead of individual tiles; with that scheme, not nearly as
much memory is needed.
-hyper (-hy) OR -HyperResolutionFactor Integer
Ratio of points in cached spectra to ones in computed curves
(default 20)
-sectors (-se) OR -NumberOfSectors Integer
Number of sectors for astigmatism analysis. A power spectrum is
stored separately for each sector; spectra can then be computed
fairly quickly for wedges of any size that is a multiple of the
sector size. The default is 36, giving 5 degree sectors.
-wedge (-w) OR -WedgeRangeAndInterval Two floats
Initial value for the angular range and interval of wedges for
astigmatism analysis. The default is a range of 90 degrees and
an interval equal to one sector.
-astigMax (-as) OR -MaximumAstigmatism Floating point
Maximum astigmatism, in microns. During the fitting to wedge
spectra, the defocus is allowed to vary from the global value by
more than half of this amount. The default is 1.2.
-colors (-col) OR -ChangeColors
Switch to colors that are better for some forms of color-blind-
ness; this option will turn on the checkbox for this change on
the plotter window. The power spectrum changes from magenta to
white, a fitted curve changes from green to magenta, and the
background is set to black.
-debug (-deb) OR -DebugLevel Integer
Debug level, 0-3. 0: quiet. 1: user messages. 2: cache, tile
iteration messages. 3: additional fitting messages. The default
is 1.
-param (-pa) OR -ParameterFile Parameter file
Read parameter entries from this file.
-help (-he) OR -usage
Print help output
-StandardInput
Read parameter entries from standard input
AUTHORS
Quanren Xiong
David Mastronarde
John Heumann
SEE ALSO
ctfphaseflip, newstack
BUGS
Prior to IMOD 4.0.29, Ctfplotter had a bug in which the view numbers
written to the defocus file were numbered from 0, not 1. When Ctfplot-
ter reads in an existing defocus file, it will do its best to detect
this situation and adjust all the view numbers up by 1. If it does
detect an inconsistency between view numbers and angular ranges, it
will issue a warning.
Email bug reports to mast at colorado dot edu.
IMOD 4.11.0 ctfplotter(1)