ctfplotter(1)               General Commands Manual              ctfplotter(1)



NAME
       ctfplotter - estimate defocus values of a tilt series

SYNOPSIS
       ctfplotter  options

DESCRIPTION
       This GUI program will plot the logarithm of a rotationally averaged
       power spectrum of an input tilt series after subtracting the noise
       floor.  The method is based on periodogram averaging; namely, averaging
       of spectra from small, overlapping areas of the images, referred to as
       tiles.  The user can interactively choose which projection views are
       included in the averaging.  It is also possible to run the program non-
       interactively for automatic fitting.


   Noise Files
       The noise floor is usually based on actual measurements of noise in an
       image taken with no specimen in the beam.  The program does have an
       alternative method of subtracting a background level that is based on
       fitting two lines to the spectrum, one to the rapidly falling region at
       low frequencies and one to the more gradually falling high-frequency
       end.  This method may work well enough with some data, particularly
       from an electron counting camera.  However, if this method does not
       work, you need to supply noise images.

       To get noise images, collect a set of blank images at a series of beam
       intensities.  They can be placed in separate files or all in one stack.
       These noise images are specific not only to the microscope but also to
       the camera used, the operating voltage, and the binning.  However,
       these are not factors that vary over time (unless there is some radical
       change in the camera), so you should only have to collect these images
       once.  The mean counts in these images should span the range of the
       mean counts in the data being analyzed.  The interval between intensi-
       ties can be a factor of 1.5; interpolation is used to estimate the
       noise for an image whose mean counts do not match one of the noise
       images.  For example, for a camera that has a gain of 10 counts per
       electron, one might collect images at mean counts near 100, 150, 225,
       etc., up to 1710.  With a 1 nm pixel size, these noise images could be
       used with images where the dose recorded at the camera ranged from 0.1
       to 1.7 electrons per square Angstrom.  If you need images for analyzing
       both unbinned and binned data, you can take one set of unbinned noise
       images and then bin them with a Newstack command like this:
          newstack -bin 2 -mult 4,0  input_file output_file

       The multiplication by 4 (the square of the binning factor) is necessary
       to make the counts be the same as they would be for binned images from
       the microscope with the same exposure.

       If there are significant numbers of X-rays in the noise images, you
       should clean them with Ccderaser, such as with:
          ccderaser -find -peak 8 -diff 6  input_file output_file

       The noise images must have positive means; i.e., their values should be
       proportional to the recorded electrons.  If they are acquired with
       software that subtracts 32768 before storing as signed integers, then
       you need to add this offset back to the data in order to use them as
       noise images.

       If the noise images are in one stack, its file name is entered directly
       with the -config option. If the noise images are in separate files,
       they are listed in a simple text file, the configuration file specified
       by the -config option.  To be found easily from within Etomo, the lat-
       ter file, or a single stack, should be placed in a directory named
       /usr/local/ImodCalib/CTFnoise or in $IMOD_CALIB_DIR/CTFnoise if the
       environment variable IMOD_CALIB_DIR has been defined differently from
       the default.  Noise files can be placed in a subdirectory of CTFnoise,
       for example, F20.  In this case, the configuration file would contain a
       list like this:
       F20/file1.st
       F20/file2.st
       F20/file3.st


   Noise Files for Direct Detector Cameras
       For direct electron detector cameras, the actual power spectra may not
       fit the noise spectra very well, particularly if motion correction is
       applied to tilt series movie frames, or if electron counting is used.
       The fitting of a polynomial to the baseline of the power spectrum,
       described below, will generally give a flat enough power spectrum at
       higher frequencies.  However, when using motion correction, there are
       two potential solutions that may give better noise files.  Spectra from
       ordinary noise images will not fit the actual spectra if the aligning
       and summing of movie frames attenuates the high frequencies, which is
       inevitable if interpolation is done in real space to align the images.
       The best remedy is to shift frames into alignment with phase shifts in
       Fourier space instead of with interpolation.  This operation preserves
       the high frequency power.  Frames that have been aligned in SerialEM's
       plugin to DigitalMicrograph or in Alignframes have been shifted in
       Fourier space, preserving the power.  If you determine the frame align-
       ment some other way, you can transform the frames in Newstack with
       the -phase option to shift them in Fourier space.  The second remedy
       would be to collect movie frames for the noise images and apply small
       shifts to them by the same method used to shift actual frames before
       summing them.  Since the noise frames cannot be aligned, the shifts
       need to be obtained some other way; either take a set of shifts from
       real images, or use some random numbers between 0 and 1.

       At least currently, the response of an electron counting camera depends
       on the dose rate, which raises the question of whether to vary exposure
       time or beam intensity when taking the set of noise images.  In princi-
       ple, varying beam intensity will replicate the differences between
       light and dark areas in the same image; varying exposure time will give
       noise images applicable to a range of images taken with the same dose
       rate but different exposure times.  There does not seem to be a good
       solution here.

       As of IMOD 4.10.18, the baseline fitting option is turned on by default
       since its performance has been improved.  It can be disabled, or the
       polynomial order set, in a template file when processing through Etomo;
       for example, the template directive
         comparam.ctfplotter.ctfplotter.BaselineFittingOrder = 2
       will restrict the fitting to a parabolic baseline.


   Program operation
       When you start the program, it will first load and analyze all of the
       noise images.  Then it will read in the images within the specified
       initial angular range and compute power spectra from the central tiles.
       This initial computation of tile power spectra is the most time-consum-
       ing step, whereas recomputing a summed power spectrum from the spectra
       of individual tiles is quick.  The program thus stores the power spec-
       trum for each tile in a cache when it is computed.  You will have to
       wait several seconds whenever you change the angular range to include
       views that have not been analyzed before.  However, once all of the
       tiles on all views have been analyzed, the program will respond quickly
       when you change angular ranges or defocus values.

       You may find that the power spectrum curves up or down at high frequen-
       cies, indicating that the noise power spectra do not adequately
       describe the noise in the actual data and that the baseline fitting is
       not working optimally to make the power spectrum be flat at high fre-
       quencies.  This fitting is controlled by an entry set in the Fitting
       dialog, whose value can be initialized with the -baseline option.  To
       do this fitting, the program first identifies points that could define
       a noise floor, primarily using points at local minima in the spectrum,
       but sampling other points if needed at the highest frequencies.  The
       program then fits a concave or convex polynomial of a selected order to
       those points and subtracts it from the power spectrum.  The result is
       to flatten the spectrum, except perhaps past 0.45/pixel. The order can
       be set to 1 to fit a line, 2 to fit a parabola, or 3 or 4 to fit 3rd or
       4th order equations.  (To keep higher order polynomials well-behaved,
       they are constrained to have a monotonically changing first derivative,
       as a parabola does.)  This fitting should be adjusted when the baseline
       deviates from flat by enough to impair the ability to visualize the
       zeroes or fit the spectrum.

       To read a general description of the plots and tool buttons, click the
       handbook icon to access the HTML help page.

       The defocus values found by the program, as well as astigmatism and
       phase shift if those are also found, can be saved in a text file that
       can be used as input to the CTF correction program Ctfphaseflip.


OPTIONS
       Ctfplotter uses the PIP package for input (see the manual page for
       pip).  Options can be specified either as command line arguments
       (with the -) or one per line in a command file (without the -).
       Options can be abbreviated to unique letters; the currently valid
       abbreviations for short names are shown in parentheses.

       -input (-inp) OR -InputStack   File name
              Input stack whose defocus will be estimated.  In principle, this
              should be a raw stack, not an aligned stack, because the inter-
              polation used to make an aligned stack attenuates high frequen-
              cies and the noise power spectra would no longer match.  Also,
              the usable CTF signal at high frequencies would be reduced.

       -offset (-of) OR -OffsetToAdd       Floating point
              The program must analyze data values where 0 corresponds to no
              electrons recorded.  Use this option to specify a value to add
              to your input stack that will make values positive and propor-
              tional to recorded electrons.  The value will not be added to
              noise files; they are required to be positive.  The program
              automatically sets this value to 32768 if it recognizes a file
              from FEI data acquisition software with a minimum below 0, or if
              the minimum and mean of a non-FEI file are sufficiently nega-
              tive.

       -config (-con) OR -ConfigFile       File name
              If a text file is entered, this configure file specifies a set
              of single-image noise files used to estimate the noise floor,
              one file per line.  The files can be specified with either abso-
              lute paths or with paths relative to the location of the config-
              ure file itself.  Alternatively, all noise images can be in one
              stack and the name of the stack provided here.

       -angleFn (-an) OR -AngleFile   File name
              File containing tilt angles for the input stack.  Each line of
              this file is the tilt angle for a view of the input stack.  The
              angles are listed in order starting from view 1.  This entry is
              required.

       -invert (-inv) OR -InvertTiltAngles
              Invert the sign of the tilt angles to compensate for a left-
              handed coordinate system in the microscope.  When the sign of
              the tilt angles and the value of the tilt axis rotation angle
              are such that reconstructions are generated with inverted hand-
              edness, then this option is needed to keep power spectra for
              off-center tiles from being shifted in the wrong direction.  One
              way to assess this need is to examine the images in a stack
              aligned for reconstruction, with the tilt axis vertical.  At
              positive tilt angles, the right side of the images should be
              more underfocused than the left side; if not, then this option
              is needed.

       -aAngle (-aA) OR -AxisAngle    Floating point
              Specifies how much the tilt axis deviates from vertical (Y
              axis).  This angle is in degrees.  It follows the right hand
              rule and counter-clockwise is positive.

       -defFn (-defF) OR -DefocusFile      File name
              File to store found defocuses.  Each entry in the file will con-
              sist of the starting and ending view number of the range of
              views being fit to (numbered from 1), the starting and ending
              angle of the tilt angle range being fit to, and the defocus
              value in nanometers (underfocus positive).  The full specifica-
              tion for defocus files acceptable to Ctfphaseflip is given in
              a section at the end of that man page.  When Ctfplotter writes a
              new defocus file, it puts a version number (currently 2) at the
              end of the first line.  If this file already exists when the
              program starts, it will be read in and the results displayed in
              a table in the Angle Range and Tile Selection dialog. The previ-
              ous version will become a backup file (with ~ added to its name)
              when new results are saved to this file.  If you supply an ini-
              tial defocus file with more than line, be sure to use the exact
              angles from the tilt angle file specified with -angleFn; do not
              round to one decimal place.  Alternatively, add the number "2"
              as an extra value at the end of the first line of the file; this
              will prevent the program from thinking that the view numbers
              might be off by one.  If you use the -invert option, you must do
              one of two things: 1) either invert all the tilt angles in this
              file, 2) or start the file with the line "16 0 0. 0. 0 3" to
              indicate that it is a version 3 file in which the angles need to
              be inverted.

       -pixelSize (-pi) OR -PixelSize      Floating point
              Image pixel size in nanometers.

       -crop (-cr) OR -CropToPixelSize     Floating point
              Crop power spectrum to the given pixel size in nanometers.  This
              option spreads out the low-frequency part of the power spectrum
              and throws away the highest frequencies; the new Nyquist fre-
              quency is 2 divided by the entered value.  Cropping may improve
              the fits to closely spaced zeros, but it will average less data
              in each frequency bin and thus make the power spectrum more
              noisy.

       -fitZeros (-fit) OR -FitZerosForPhase
              When finding phase, try to find and fit to zero positions in the
              spectrum instead of just fitting to the power spectrum.  Fitting
              to zero positions can prevent the first few zeros from dominat-
              ing the fit and may improve the fit to the higher zeros, but it
              may also result in the first zero being noticeably off.

       -volt (-vo) OR -Voltage   Integer
              Microscope voltage in kV.

       -cs OR -SphericalAberration    Floating point
              Microscope spherical aberration in millimeters.  A value of 0
              can be entered; it will be made slightly larger to prevent divi-
              sion by 0 in the CTF equations.

       -ampContrast (-am) OR -AmplitudeContrast      Floating point
              The fraction of amplitude contrast. For cryo-EM, it should be
              between 0.07 and 0.14.  The default is 0.07.

       -phase (-pha) OR -PhasePlateShift   Floating point
              The expected value for the phase shift imposed by a phase plate,
              in radians.  Equations relating the CTF and defocus will include
              this phase shift when phase is not solved for.  It can be
              entered with either positive or negative sign; the absolute
              value of the entered value will be subtracted when computing
              phase within the program.

       -degPhase (-deg) OR -PhaseShiftInDegrees      Floating point
              The expected value for the phase shift imposed by a phase plate,
              in degrees. See the -phase option, which should have been in
              degrees.  The two may not both be entered.

       -phRange (-phR) OR -PhaseRangeToSearch   Floating point
              Total range of phase shift to search around the expected shift,
              in degrees.  The default is 120.

       -cuton (-cu) OR -CutOnFrequency     Floating point
              A fixed value for cut-on frequency that attenuates phase at low
              frequency, in reciprocal nanometers.  Whenever the cut-on fre-
              quency is non-zero, the phase shift displayed on the screen and
              stored in the defocus file is actually the phase at a fixed fre-
              quency of 0.3/nm rather than the shift at infinity.

       -maxCuton (-ma) OR -MaxCutOnToSearch     Floating point
              Maximum frequency to test when searching for cut-on frequency,
              in reciprocal nanometers.  The default is the frequency of the
              first zero at the expected defocus and phase shift.

       -skip (-sk) OR -ViewsToSkip    List of integer ranges
              List of views to leave out of the analysis; views are numbered
              from 1 and ranges can be entered.  The program will behave as if
              those views do not exist unless the option is set for skipping
              only for analysis of astigmatism or phase shift.  If it is,
              defocus can be found for every view, but astigmatism and phase
              will be solved from adjacent views to ones in the skip list.

       -bidir (-bi) OR -BidirectionalNumViews   List of integer ranges
              Number of views in the first half of a bidirectional tilt
              series.  Entering this option will turn on the checkbox "Break
              groups at view" to prevent combining of views for analysis
              across the two halves of the series.  Do not use this option for
              a tilt series taken with the Hagen scheme.  If the option is set
              for skipping and breaking only for analysis of astigmatism or
              phase shift, the program will still combine views for defocus
              fitting across the boundary.

       -onlyAP (-on) OR -SkipOnlyForAstigPhase
              Skip views in the list of ones to skip, or avoid combining views
              for analysis, only for analysis of astigmatism and phase shift.
              With this option set, the program will exclude views in the skip
              list and avoid combining views across the bidirectional boundary
              only when getting a combined spectrum for finding astigmatism
              and phase shift.

       -expDef (-exp) OR -ExpectedDefocus       Floating point
              Expected defocus at the tilt axis in nanometers, with a positive
              value for underfocus.  The frequency of the first zero of the
              CTF curve is first computed based on this expected defocus.  The
              segments of the CTF curve of the input stack around that fre-
              quency are selected to be fitted.

       -fpOffset (-fp) OR -FocalPairDefocusOffset    Floating point
              Normally, ctfplotter processes a single stack whose name is
              given by InputStack. It can optionally process a pair of stacks
              with identical til angles and mutually aligned projections,
              taken with a constant defocus offset between them. If, for exam-
              ple, InputStack is myData.st, one could place a stack with 6 um
              underfocus in myData_1.st and a stack with 2.175 um underfocus
              in myData_2.st and specify an ExpectedDefocus of 6000 and a
              FocalPairDefocusOffset of -3825.  There is some possibility that
              this processing still works after the major changes in 2018, but
              using it will prevent some of the new features from being avail-
              able.

       -range (-ra) OR -AngleRange    Two floats
              When the -autoFit option is not entered, this entry sets the
              starting and ending tilt angles for the initial analysis and is
              a required entry.  Views with a tilt angle within this range are
              used to compute the CTF curve.  When -autoFit is entered, this
              entry sets the extent over which steps will be taken in aut-
              ofitting.

       -autoFit (-au) OR -AutoFitRangeAndStep   Two floats
              Do initial autofitting over the whole tilt series with the given
              range of angles and step size between ranges.  A value of zero
              for the step will make it fit to each single image separately,
              regardless of the value for the range.  The range entry can also
              be zero to fit to a single image.  This autofitting differs from
              that invoked through the Angles dialog in several respects: 1)
              Fitting is done at each angle with the current defocus from the
              previous angle, if any, and the "Use current defocus" radio but-
              ton is left selected at the end, unless the -useDef option is
              entered.  If phase is being found, it will also use current
              phase.  2) Three fitting iterations will be done.  3) The size
              of the range is determined by the parameter entered here, not by
              the starting and ending angles entered with the -range option.
              This autofitting will not be done if there are already values in
              the defocus file and if the -range option is entered, in order
              to prevent an annoying message when opening Ctfplotter after
              processing with Batchruntomo.  Otherwise, if autofitting is
              specified and there are existing defocus values, the program
              will ask you to confirm whether to replace them.

       -useDef (-use) OR -UseExpectedDefForAuto
              Use the expected defocus value at each angle instead of the cur-
              rent defocus when doing initial autofitting.  This is the behav-
              ior of the program before IMOD 4.10.18.  At each angle, the
              expected defocus is used the first time and the defocus estimate
              is used for the next two iterations.

       -frequency (-fr) OR -FrequencyRangeToFit      Two floats
              Starting and ending frequencies of range to fit in power spec-
              trum.  The two values will be used to set the "X1 starts" and
              "X2 ends" fields in the fitting dialog after the pixel size that
              governs the meaning of these frequencies has been adjusted for
              any cropping of the spectrum.  In other words, if you are also
              entering the -crop option, enter the fitting range that works
              for the cropping specified by that option.

       -extra (-ext) OR -ExtraZerosToFit   Floating point
              By default, the ending frequency of the fitting range is set to
              the expected location of the second zero.  With this entry, the
              range will be extended by the given multiple of the interval
              between first and seconds zeros.  For example, entries of 1 or 2
              will fit approximately to the third or fourth zeros, respec-
              tively.  An entry of more than 0.5 will trigger fitting to two
              exponentials, which is important for fitting multiple peaks
              between zeros.

       -vary (-va) OR -VaryExponentInFit
              Vary exponent of CTF function when fitting a CTF-like curve

       -baseline (-ba) OR -BaselineFittingOrder      Integer
              Baseline fitting is used to make the power spectrum be flat at
              high frequencies; this entry initializes the "Baseline fitting
              order" setting in the Fitting dialog.  For details, see above.
              The default is 4 unless the expected position of the first zero
              is 0.25 reciprocal pixels or higher, in which case it is set to
              1.

       -find (-fin) OR -FindAstigPhaseCuton     Three integers
              Enter 1 to find astigmatism or 0 not to, 1 to find phase shift
              or 0 not to, and 1 to find cut-on frequency also, or 0 not to.

       -minViews (-mi) OR -MinViewsAstigAndPhase     Two integers
              Minimum number of views for finding astigmatism and phase shift;
              the default is 3 for astigmatism and 1 for phase.

       -save (-sa) OR -SaveAndExit
              Save defocus values to file and exit after autofitting.  The
              program will not ask for confirmation before removing existing
              entries in the defocus table.

       -psRes (-ps) OR -PSResolution       Integer
              The number of points over which CTF will be computed.  The fre-
              quency range up to Nyquist is divided into equal intervals
              delineated by these points.  The default is 101.

       -tileSize (-t) OR -TileSize    Integer
              The tile size for computing spectra.  The size is in pixels and
              the tiles are square.  The whole image is divided into tiles
              that overlap by 50% in each direction.  For each view, these
              tiles are assigned to strips that are considered to have con-
              stant defocus.  The default is 256.

       -defTol (-defT) OR -DefocusTol      Integer
              Defocus tolerance in nanometers defining the strip width.  The
              center strips are taken from the central region of a view that
              has defocus difference less than this tolerance. Similarly, the
              off-center strips are defined to include tiles whose defocus
              range is than this amount.  The default is 200.

       -leftTol (-l) OR -LeftDefTol   Floating point
              Defocus tolerance in nanometers for strips to the left of the
              center strip.  When non-center strips are included in the aver-
              age, strips to the left of center are included if their defocus
              difference is less than the given value.  The default is 2000.

       -rightTol (-ri) OR -RightDefTol     Floating point
              Defocus tolerance in nanometers for strips to the right of the
              center strip.  When non-center strips are included in the aver-
              age, strips to the right of center are included if their defocus
              difference is less than the given value.  The default is 2000.

       -cache (-ca) OR -MaxCacheSize       Integer
              To speed up computation, ctfplotter uses a cache to hold the
              rotationally averaged power spectra of strips of tiles at 20
              times the final power spectrum resolution (or whatever is
              entered with the -hyper option).  This option controls the cache
              size.  The default value is 1000 megabytes if the system physi-
              cal memory cannot be determined; otherwise the size is the mini-
              mum of 15 GB, 3/4 of physical memory, and physical memory minus
              1 GB, for memory up to 30 GB, and half of physical memory above
              30 GB; but in any case at least 400 MB.  These large limits were
              implemented before caching was changed to hold strip sums
              instead of individual tiles; with that scheme, not nearly as
              much memory is needed.

       -hyper (-hy) OR -HyperResolutionFactor   Integer
              Ratio of points in cached spectra to ones in computed curves
              (default 20)

       -sectors (-se) OR -NumberOfSectors       Integer
              Number of sectors for astigmatism analysis.  A power spectrum is
              stored separately for each sector; spectra can then be computed
              fairly quickly for wedges of any size that is a multiple of the
              sector size.  The default is 36, giving 5 degree sectors.

       -wedge (-w) OR -WedgeRangeAndInterval    Two floats
              Initial value for the angular range and interval of wedges for
              astigmatism analysis.  The default is a range of 90 degrees and
              an interval equal to one sector.

       -astigMax (-as) OR -MaximumAstigmatism   Floating point
              Maximum astigmatism, in microns.  During the fitting to wedge
              spectra, the defocus is allowed to vary from the global value by
              more than half of this amount.  The default is 1.2.

       -colors (-col) OR -ChangeColors
              Switch to colors that are better for some forms of color-blind-
              ness; this option will turn on the checkbox for this change on
              the plotter window.  The power spectrum changes from magenta to
              white, a fitted curve changes from green to magenta, and the
              background is set to black.

       -debug (-deb) OR -DebugLevel   Integer
              Debug level, 0-3. 0: quiet.  1: user messages.  2: cache, tile
              iteration messages.  3: additional fitting messages. The default
              is 1.

       -param (-pa) OR -ParameterFile      Parameter file
              Read parameter entries from this file.

       -help (-he) OR -usage
              Print help output

       -StandardInput
              Read parameter entries from standard input


AUTHORS
       Quanren Xiong
       David Mastronarde
       John Heumann

SEE ALSO
       ctfphaseflip, newstack

BUGS
       Prior to IMOD 4.0.29, Ctfplotter had a bug in which the view numbers
       written to the defocus file were numbered from 0, not 1.  When Ctfplot-
       ter reads in an existing defocus file, it will do its best to detect
       this situation and adjust all the view numbers up by 1.  If it does
       detect an inconsistency between view numbers and angular ranges, it
       will issue a warning.

       Email bug reports to mast at colorado dot edu.



IMOD                                4.10.22                      ctfplotter(1)