ctfplotter(1)                                                    ctfplotter(1)

       ctfplotter - estimate defocus values of a tilt series

       ctfplotter  options

       This GUI program will plot the logarithm of a rotationally averaged
       power spectrum of an input tilt series after subtracting the noise
       floor.  The method is based on periodogram averaging; namely, averaging
       of spectra from small, overlapping areas of the images, referred to as
       tiles.  The user can interactively choose which projection views are
       included in the averaging.  It is also possible to run the program non-
       interactively for automatic fitting.

   Noise Files
       The noise floor is based on actual measurements of noise in an image
       taken with no specimen in the beam.  Thus, in order to use this pro-
       gram, you need to collect a set of such blank images, at a series of
       beam intensities, and place each image in a separate file.  These noise
       files are specific not only to the microscope but also to the camera
       used, the operating voltage, and the binning.  However, these are not
       factors that vary over time (unless there is some radical change in the
       camera), so you should only have to collect these images once.  The
       mean counts in these files should span the range of the mean counts in
       the data being analyzed.  The interval between intensities can be a
       factor of 1.5; interpolation is used to estimate the noise for an image
       whose mean counts do not match one of the noise files.  For example,
       for a camera that has a gain of 10 counts per electron, one might col-
       lect images at mean counts near 100, 150, 225, etc., up to 1710.  With
       a 1 nm pixel size, these noise files could be used with images where
       the dose recorded at the camera ranged from 0.1 to 1.7 electrons per
       square Angstrom.  If you need files for analyzing both unbinned and
       binned data, you can take one set of unbinned noise images and then bin
       them with a Newstack command like this:
          newstack -bin 2 -mult 4,0  input_file output_file

       The multiplication by 4 (the square of the binning factor) is necessary
       to make the counts be the same as they would be for binned images from
       the microscope with the same exposure.

       If there are significant numbers of X-rays in the noise images, you
       should clean them with Ccderaser, such as with:
          ccderaser -find -peak 8 -diff 6  input_file output_file

       The noise files must have positive means; i.e., their values should be
       proportional to the recorded electrons.  If they are acquired with
       software that subtracts 32768 before storing as signed integers, then
       you need to add this offset back to the data in order to use them as
       noise files.

       The noise files are listed in a simple text file, the configuration
       file specified by the -config option.  To be found easily from within
       Etomo, the latter file should be placed in a directory named
       /usr/local/ImodCalib/CTFnoise or in $IMOD_CALIB_DIR/CTFnoise if the
       environment variable IMOD_CALIB_DIR has been defined differently from
       the default.  You can then place the noise files in a subdirectory of
       CTFnoise, for example, F20.  In this case, the configuration file would
       contain a list like this:

   Noise Files for Direct Detector Cameras
       For direct electron detector cameras, the actual power spectra may not
       fit the noise spectra very well, particularly if motion correction is
       applied to tilt series movie frames, or if electron counting is used.
       The ultimate recourse is to enable the option for fitting a polynomial
       to the baseline of the power spectrum, described below.  However, when
       using motion correction, there are two potential solutions that may
       give better noise files.  Spectra from ordinary noise images will not
       fit the actual spectra if the aligning and summing of movie frames
       attenuates the high frequencies, which is inevitable if interpolation
       is done in real space to align the images.  The best remedy is to shift
       frames into alignment with phase shifts in Fourier space instead of
       with interpolation.  This operation preserves the high frequency power.
       Frames that have been aligned in SerialEM's plugin to DigitalMicrograph
       or in Alignframes have been shifted in Fourier space, preserving the
       power.  If you determine the frame alignment some other way, you can
       transform the frames in Newstack with the -phase option to shift
       them in Fourier space.  The second remedy would be to collect movie
       frames for the noise images and apply small shifts to them by the same
       method used to shift actual frames before summing them.  Since the
       noise frames cannot be aligned, the shifts need to be obtained some
       other way; either take a set of shifts from real images, or use some
       random numbers between 0 and 1.

       At least currently, the response of an electron counting camera depends
       on the dose rate, which raises the question of whether to vary exposure
       time or beam intensity when taking the set of noise images.  In princi-
       ple, varying beam intensity will replicate the differences between
       light and dark areas in the same image; varying exposure time will give
       noise images applicable to a range of images taken with the same dose
       rate but different exposure times.  There does not seem to be a good
       solution here.

       Most likely you will need to use the baseline fitting option when prob-
       lems arise with direct detector images.  Note that the fitting can be
       enabled by default in a template file when processing through Etomo;
       for example, the template directive
         comparam.ctfplotter.ctfplotter.BaselineFittingOrder = 2
       will enable fitting to a parabolic baseline.

   Program operation
       When you start the program, it will first load and analyze all of the
       noise files.  Then it will read in the images within the specified ini-
       tial angular range and compute power spectra from the central tiles.
       This initial computation of tile power spectra is the most time-consum-
       ing step, whereas recomputing a summed power spectrum from the spectra
       of individual tiles is quick.  The program thus stores the power spec-
       trum for each tile in a cache when it is computed.  You will have to
       wait several seconds whenever you add non-central tiles from the cur-
       rent angular range, or when you change the angular range to include
       views that have not been analyzed before.  However, once all of the
       tiles on all views have been analyzed, the program will respond quickly
       when you change angular ranges or defocus values.

       You may find that the power spectrum curves up or down at high frequen-
       cies, indicating that the noise power spectra do not adequately
       describe the noise in the actual data.  The easiest solution is to
       activate a fitting option that can make the power spectrum be flat at
       high frequencies.  This fitting is controlled by an entry set in the
       Fitting dialog, whose value can be initialized with the -baseline
       option.  To do this fitting, the program first identifies points that
       could define a noise floor, primarily using points at local minima in
       the spectrum, but sampling other points if needed at the highest fre-
       quencies.  The program then fits a polynomial of a selected order to
       those points and subtracts it from the power spectrum.  The result is
       to flatten the spectrum, except perhaps past 0.45/pixel. The order can
       be set to 1 to fit a line, 2 to fit a parabola, or 3 or 4 to fit 3rd or
       4th order equations.  An order of 2 is usually adequate, but a higher
       order may be helpful.  (To keep higher order polynomials well-behaved,
       they are constrained to have a monotonically changing first derivative,
       as a parabola does.)  This fitting should be used when the baseline
       deviates from flat by enough to impair the ability to visualize the
       zeroes or fit the spectrum.  It is not activated by default because it
       can give a slightly poorer fit to the power spectrum in cases where the
       baseline is already nearly flat.

       To read a general description of the plots and tool buttons, click the
       handbook icon to access the HTML help page.

       Found defocus values can be saved in a text file that can be used as
       input to the CTF correction program ctfphaseflip.

       Ctfplotter uses the PIP package for input (see the manual page for
       pip).  Options can be specified either as command line arguments
       (with the -) or one per line in a command file (without the -).
       Options can be abbreviated to unique letters; the currently valid
       abbreviations for short names are shown in parentheses.

       -input (-inp) OR -InputStack   File name
              Input stack whose defocus will be estimated.  This should be a
              raw stack, not an aligned stack, because the interpolation used
              to make an aligned stack attenuates high frequencies and the
              noise power spectra would no longer match.

       -offset (-o) OR -OffsetToAdd   Floating point
              The program must analyze data values where 0 corresponds to no
              electrons recorded.  Use this option to specify a value to add
              to your input stack that will make values positive and propor-
              tional to recorded electrons.  The value will not be added to
              noise files; they are required to be positive.  The program
              automatically sets this value to 32768 if it recognizes a file
              from FEI data acquisition software with a minimum below 0, or if
              the minimum and mean of a non-FEI file are sufficiently nega-

       -config (-co) OR -ConfigFile   File name
              The configure file specifies the noise files used to estimate
              the noise floor, one file per line.  The files can be specified
              with either absolute paths or with paths relative to the loca-
              tion of the configure file itself.

       -angleFn (-an) OR -AngleFile   File name
              File containing tilt angles for the input stack.  Each line of
              this file is the tilt angle for a view of the input stack.  The
              angles are listed in order starting from view 1.

       -invert (-inv) OR -InvertTiltAngles
              Invert the sign of the tilt angles to compensate for a left-
              handed coordinate system in the microscope.  When the sign of
              the tilt angles and the value of the tilt axis rotation angle
              are such that reconstructions are generated with inverted hand-
              edness, then this option is needed to keep power spectra for
              off-center tiles from being shifted in the wrong direction.  One
              way to assess this need is to examine the images in a stack
              aligned for reconstruction, with the tilt axis vertical.  At
              positive tilt angles, the right side of the images should be
              more underfocused than the left side; if not, then this option
              is needed.

       -aAngle (-aA) OR -AxisAngle    Floating point
              Specifies how much the tilt axis deviates from vertical (Y
              axis).  This angle is in degrees.  It follows the right hand
              rule and counter-clockwise is positive.

       -defFn (-defF) OR -DefocusFile      File name
              File to store found defocuses.  Each entry in the file will con-
              sist of the starting and ending view number of the range of
              views being fit to (numbered from 1), the starting and ending
              angle of the tilt angle range being fit to, and the defocus
              value in nanometers (underfocus positive).  The full specifica-
              tion for defocus files acceptable to Ctfphaseflip is given in
              a section at the end of that man page.  When Ctfplotter writes a
              new defocus file, it puts a version number (currently 2) at the
              end of the first line.  If this file already exists when the
              program starts, it will be read in and the results displayed in
              a table in the Angle Range and Tile Selection dialog. The previ-
              ous version will become a backup file (with ~ added to its name)
              when new results are saved to this file.  If you supply an ini-
              tial defocus file with more than line, be sure to use the exact
              angles from the tilt angle file specified with -angleFn; do not
              round to one decimal place.  Alternatively, add the number "2"
              as an extra value at the end of the first line of the file; this
              will prevent the program from thinking that the view numbers
              might be off by one.  If you use the -invert option, you must do
              one of two things: 1) either invert all the tilt angles in this
              file, 2) or start the file with the line "16 0 0. 0. 0 3" to
              indicate that it is a version 3 file in which the angles need to
              be inverted.

       -pixelSize (-pi) OR -PixelSize      Floating point
              Image pixel size in nanometers.

       -volt (-vo) OR -Voltage   Integer
              Microscope voltage in kV.

       -cs OR -SphericalAberration    Floating point
              Microscope spherical aberration in millimeters.  A value of 0
              can be entered; it will be made slightly larger to prevent divi-
              sion by 0 in the CTF equations.

       -ampContrast (-am) OR -AmplitudeContrast      Floating point
              The fraction of amplitude contrast. For Cryo-EM, it should be
              between 0.07 and 0.14.  The default is 0.07.

       -phase (-ph) OR -PhasePlateShift    Floating point
              A fixed value for the phase shift imposed by a phase plate, in
              radians.  Equations relating the CTF and defocus will include
              this phase shift.  It can be entered with either positive or
              negative sign; the absolute value of the entered value will be
              subtracted when computing phase within the program.

       -expDef (-exp) OR -ExpectedDefocus       Floating point
              Expected defocus at the tilt axis in nanometers, with a positive
              value for underfocus.  The frequency of the first zero of the
              CTF curve is first computed based on this expected defocus.  The
              segments of the CTF curve of the input stack around that fre-
              quency are selected to be fitted.

       -fpOffset (-fp) OR -FocalPairDefocusOffset    Floating point
              Normally, ctfplotter processes a single stack whose name is
              given by InputStack. It can optionally process a pair of stacks
              with identical til angles and mutually aligned projections,
              taken with a constant defocus offset between them. If, for exam-
              ple, InputStack is myData.st, one could place a stack with 6 um
              underfocus in myData_1.st and a stack with 2.175 um underfocus
              in myData_2.st and specify an ExpectedDefocus of 6000 and a
              FocalPairDefocusOffset of -3825.

       -range (-ra) OR -AngleRange    Two floats
              When the -autoFit option is not entered, this entry sets the
              starting and ending tilt angles for the initial analysis and is
              a required entry.  Views with a tilt angle within this range are
              used to compute the CTF curve.  When -autoFit is entered, this
              entry sets the extent over which steps will be taken in aut-

       -autoFit (-au) OR -AutoFitRangeAndStep   Two floats
              Do initial autofitting over the whole tilt series with the given
              range of angles and step size between ranges.  A value of zero
              for the step will make it fit to each single image separately,
              regardless of the value for the range.  This autofitting differs
              from that invoked through the Angles dialog in several respects:
              1) All tiles will be used for fits; the "All tiles" radio button
              will be selected at the end. 2) Three fitting iterations will be
              done, with the expected defocus used the first time and the cur-
              rent defocus estimate used for the next two iterations.  3) The
              size of the range is determined by the parameter entered here,
              not by the starting and ending angles entered with the -range
              option.  This autofitting will not be done if there are already
              values in the defocus file and if the -range option is entered,
              in order to prevent an annoying message when opening Ctfplotter
              after processing with Batchruntomo.  Otherwise, if aut-
              ofitting is specified and there are existing defocus values, the
              program will ask you to confirm whether to replace them.

       -frequency (-fr) OR -FrequencyRangeToFit      Two floats
              Starting and ending frequencies of range to fit in power spec-
              trum.  The two values will be used to set the "X1 starts" and
              "X2 ends" fields in the fitting dialog.

       -extra (-ext) OR -ExtraZerosToFit   Floating point
              By default, the ending frequency of the fitting range is set to
              the expected location of the second zero.  With this entry, the
              range will be extended by the given multiple of the interval
              between first and seconds zeros.  For example, entries of 1 and
              2 will fit approximately to the third and fourth zeros, respec-
              tively.  An entry of more than 0.5 will trigger fitting to two
              exponentials, which is important for fitting multiple peaks
              between zeros.

       -vary (-va) OR -VaryExponentInFit
              Vary exponent of CTF function when fitting a CTF-like curve

       -baseline (-b) OR -BaselineFittingOrder       Integer
              This entry can be used to make the power spectrum be flat at
              high frequencies; it initializes the "Baseline fitting order"
              setting in the Fitting dialog.  For details, see above.

       -save (-s) OR -SaveAndExit
              Save defocus values to file and exit after autofitting.  The
              program will not ask for confirmation before removing existing
              entries in the defocus table.

       -psRes (-ps) OR -PSResolution       Integer
              The number of points over which CTF will be computed.  The
              Nyquist frequency is divided into equal intervals delineated by
              these points.  The default is 101.

       -tileSize (-t) OR -TileSize    Integer
              The tile size each strip will be tessellated into.  The size is
              in pixels and the tiles are square.  Each view is first divided
              into strips that are considered to have constant defocus.  The
              deafult is 256.

       -defTol (-defT) OR -DefocusTol      Integer
              Defocus tolerance in nanometers defining the center strips.  The
              center strips are taken from the central region of a view that
              has defocus difference less than this tolerance.  These kind of
              center strips from all views within AngleRange are considered to
              have a constant defocus and are used to compute the initial CTF
              after being further tessellated into tiles.  The default is 200.

       -leftTol (-l) OR -LeftDefTol   Floating point
              Defocus tolerance in nanometers for strips to the left of the
              center strip.  When non-center strips are included in the aver-
              age, strips to the left of center are included if their defocus
              difference is less than the given value.  The default is 2000.

       -rightTol (-ri) OR -RightDefTol     Floating point
              Defocus tolerance in nanometers for strips to the right of the
              center strip.  When non-center strips are included in the aver-
              age, strips to the right of center are included if their defocus
              difference is less than the given value.  The default is 2000.

       -cache (-ca) OR -MaxCacheSize       Integer
              To speed up computation, ctfplotter uses a cache to hold the
              rotationally averaged power spectra of individual tiles at 20
              times the final power spectrum resolution.  This option controls
              the cache size.  The default value is 1000 megabytes, which
              should be enough for all slices in a 4Kx4K tilt series with the
              default tile size.

       -debug (-deb) OR -DebugLevel   Integer
              Debug level, 0-3. 0: quiet.  1: user messages.  2: cache, tile
              iteration messages.  3: additional fitting messages. The default
              is 1.

       -param (-pa) OR -ParameterFile      Parameter file
              Read parameter entries from this file.

       -help (-h) OR -usage
              Print help output

              Read parameter entries from standard input

       Quanren Xiong
       David Mastronarde
       John Heumann

       ctfphaseflip, newstack

       Prior to IMOD 4.0.29, Ctfplotter had a bug in which the view numbers
       written to the defocus file were numbered from 0, not 1.  When Ctfplot-
       ter reads in an existing defocus file, it will do its best to detect
       this situation and adjust all the view numbers up by 1.  If it does
       detect an inconsistency between view numbers and angular ranges, it
       will issue a warning.

       Email bug reports to mast at colorado dot edu.

IMOD                                 4.9.3                       ctfplotter(1)