mtffilter(1)                General Commands Manual               mtffilter(1)



NAME
       mtffilter - filter by inverse of MTF and general Fourier filter

SYNOPSIS
       mtffilter [options] input_file [output_file]

DESCRIPTION
       Mtffilter originated as a program for restoring contrast in camera
       images by multiplying them by the inverse of the camera's modulation
       transfer function (MTF), but has evolved to do general-purpose filter-
       ing in Fourier space as well as to apply some other specialized fil-
       ters.  It can apply a low pass filter to reduce high frequency noise,
       as well as a high pass filter to eliminate low frequencies.  Any combi-
       nation of these filters may be applied.  In fact, the program provides
       all of the options that other IMOD programs do for specifying a general
       Fourier filter.  Because images are automatically padded to dimensions
       suitable for taking an FFT, there are no restrictions on image size,
       unlike with Enhance.  This program can filter either real-space
       images in 2D planes, real-space images in 3D or 3D Fourier transforms
       in 3D.  The filter functions produced by these options can be visual-
       ized with the program Filterplot; see that man page for a full
       description of their effects.

       A specialized filter can be applied to perform dose weight-filtering of
       cryoEM images, particularly ones from tilt series.  The filter is as
       described in Grant and Grigorieff, 2015 (DOI: 10.7554/eLife.06980) and
       the implementation follows that in their "unblur" program.  At any fre-
       quency, the filter follows an exponential decay with dose, where the
       exponential is of the dose divided by 2 times a "critical dose" for
       that frequency.  This critical dose was empirically found to be approx-
       imated by a * k^b + c, where k is frequency; the values of a, b, c in
       that paper are used by default but can be modified with the -critical
       option.  This filter function is applied for all frequencies (complete
       attenuation above the "optimal dose" is no longer considered appropri-
       ate).  A variety of options are described below for providing dose
       information to the program.  This filter cannot be combined with the
       other Fourier filters.  However, when dose weighting is activated, the
       MaximumInverse, InverseRolloffRadiusSigma, and LowPassRadiusSigma
       options will simply be skipped and have no effect, in order to leave
       the standard mtffilter command file runnable if dose weighting options
       are inserted into it with a template.

       The program can also apply a third kind of filter that has been found
       useful for reducing fringe effects in EM images taken with a phase
       plate.  This filter is specified by a cutoff radius, and exponential
       power, and an amplification factor, the amount by which it amplifies
       low frequencies relative to high ones.  It goes from 1 at zero fre-
       quency to a floor of 1/ amplificationFactor at high frequencies and
       falls 1/e of the way to the floor at the cutoff radius.  A higher power
       increases the sharpness of the falloff.  This filter is referred to
       here as a low-frequency amplifier filter.  It is selected by entering
       the -amplifier option plus either the cutoff radius (with -cutoff) or
       parameters of the phase plate imaging (with -phase).  It cannot be used
       together with the low-pass and high-pass filter options.

       The program can also apply a filter in one dimension, in the X-direc-
       tion only, and specifically can apply an R-weighted 1-D filter such as
       is used in back-projection.  This R-weighted filter cannot be used
       together with inverse MTF filtering.

       Simply multiplying by the inverse of an MTF would amplify noise too
       much, so the inverse MTF filter is shaped by three parameters.  The
       first and most important is the maximum inverse value, which limits how
       high the inverse can become.  The other two parameters are a cutoff
       frequency at which to start a Gaussian rolloff of the inverse back to
       1.0, and the sigma value for this Gaussian rolloff.  The default values
       for these parameters (listed below) are based on limited experimenta-
       tion and are fairly conservative.  All of these parameters together
       will keep the inverse filter from amplifying high frequency noise.  The
       low pass filter's role is to filter out those high frequencies.

       If both filters are used, there are potentially 4 different frequency
       ranges:
         1) From 0 to the frequency at which the inverse reaches its maximum,
       the filter is actually the inverse of the MTF,
         2) From there to the cutoff frequency for the inverse rolloff, the
       filter equals the maximum inverse,
         3) Beyond this cutoff frequency, the filter progressively decays back
       to 1.0,
         4) Beyond the cutoff radius for the low-pass filter, the filter is
       multiplied by another Gaussian and decays to 0.

       The MTF curve to be applied should be read in from a file containing
       values for spatial frequency (in reciprocal pixels) and for the MTF,
       one pair per line.  The program has one built-in curve in which the MTF
       crosses 0.5 at 0.117/pixel.  This curve can be adjusted by scaling its
       axis, which will make it approximately correct for other situations.

       To apply only low-pass and high-pass filters, omit the -mtf and -stock
       options; to apply only an inverse filter, omit the -lowpass and other
       options for general filtering.  Similarly, to apply only a low-fre-
       quency amplifier filter, omit the -mtf, -stock, and other general fil-
       ter options.

       If the input file is a real image, then without the "-3d" option the
       program will take the FFT of each section, apply the filter, take the
       inverse FFT, and write out the filtered section.  With the "-3d"
       option, it will load the whole file into memory, tapered and padded
       just as in Taperoutvol, take the 3D FFT, filter, inverse FFT, and
       write the volume.  If the input file is a Fourier transform, it must be
       a 3D FFT (obtained from "clip fft -3d" or "fftrans -3d").  In this case
       the program will apply the filter to the transforms in three dimensions
       and write out a filtered FFT.

       The program allocates memory dynamically, so it is capable of filtering
       a rather large volume in 3D.  However, it will require 4 bytes of mem-
       ory per voxel; e.g., 4 GB for a 1 gigavoxel volume.  If the program
       claims that the padded volume is above the memory limit (which will
       depend on the amount of memory in the computer), or if it fails to
       allocate the memory, you can chop a volume into pieces, filter them,
       and reassemble the result.  To filter a large image file in 3D this
       way, simply make a file filterbig.com with one line:
          $mtffilter -3d <filtering options> INPUTFILE OUTPUTFILE
       where you insert your filtering options, but INPUTFILE and OUTPUTFILE
       are exactly as shown, and not the names of your actual input and output
       files.  Then run:
          chunksetup -p 0 -m 250 filterbig.com  input_file output_file
       where "-p 0" eliminates padding because Mtffilter will take care of
       padding, "-m 250" specifies the number of megavoxels per chunk, and
       this time you do put your actual input and output file names.  See
       Chunksetup for details.  You can execute the resulting command files
       with parallel processing (via Processchunks or Etomo) or sequen-
       tially with:
          subm filterbig-all
       Chopping up a volume used to be suggested as a quicker way to filter a
       large volume, because the time per voxel increases as the log of the
       volume size, but with current FFT speeds this is much less of an issue.


OPTIONS
       Mtffilter uses the PIP package for input (see the manual page for
       pip).  The following options can be specified either as command line
       arguments (with the -) or one per line in a command file or parameter
       file (without the -).  Options can be abbreviated to unique letters;
       the currently valid abbreviations for short names are shown in paren-
       theses.

       -input (-inp) OR -InputFile    File name
              Input file with images to be filtered

       -output (-ou) OR -OutputFile   File name
              Output file for filtered images.  If this file is omitted, the
              program will write filtered images back to the input file.

       -zrange (-z) OR -StartingAndEndingZ      Two integers
              First and last Z values in the file to filter.  Values are num-
              bered from 1 and the default is to do all sections.  This entry
              is allowed when doing a 3D FFT on real-space input, but not for
              FFT input.

       -mode (-mo) OR -ModeToOutput   Integer
              The storage mode of the output file; 0 for byte, 1 for 16-bit
              signed integer, 6 for 16-bit unsigned integer, or 2 for 32-bit
              floating point.  The default is the mode of the input file.
              This entry is allowed only when writing to a new output file and
              when the input is not an FFT.

       -3dfilter (-3) OR -FilterIn3D
              Filter data in 3D instead of in 2D.  The entire volume will be
              filtered, so it must fit into memory and -zrange.  If the volume
              will not fit in memory, you can try to use "clip fft -3d" to get
              an FFT, run Mtffilter on the 3D FFT, then inverse transform with
              "clip fft -3d -m mode", where mode is the desired output mode,
              typically the same as the input.  Otherwise you will need to
              process it in chunks.

       -1dfilter (-1) OR -OneDimensionalFilter
              Filter data in 1D (in X direction) instead of in 2D

       -lowpass (-l) OR -LowPassRadiusSigma     Two floats
              Cutoff radius and sigma for a low pass filter that imposes a
              high-frequency Gaussian roll-off to 0.0.  The default is no
              high-frequency filtering.  These entries correspond to the
              Radius2 and Sigma2 entries to Enhance and other programs; see
              the Enhance or Filterplot man pages for a full explanation
              of the effects of changing the sign of the Sigma2 or the Sigma1
              and Radius1 parameters entered with the next two options.

       -highpass (-hi) OR -HighPassSigma   Floating point
              Sigma for a high pass filter based on an inverted Gaussian that
              starts at 0.0 at zero frequency and decays up to 1 with the
              given sigma.  The default is no high-frequency filtering.  This
              entry corresponds to the Sigma1 entry to Enhance and other
              programs.  A negative Sigma1 can be used to get a band-pass fil-
              ter based on the second derivative of a Gaussian.

       -radius1 (-ra) OR -FilterRadius1    Floating point
              Cutoff radius for a high-pass filter that is 1.0 at this radius
              and falls off as a Gaussian to the left of this point with sigma
              equal to the Sigma2 value entered with -lowpass.  This entry
              corresponds to the Radius1 entry to Enhance and other pro-
              grams.  A negative Radius1 will make the inverted Gaussian
              invoked by -highpass be zero out to |Radius1|.

       -mtf (-mt) OR -MtfFile    File name
              File with MTF curve.  The format of the file is a series of
              lines, with a spatial frequency in reciprocal pixels and an MTF
              value on each line.

       -stock (-s) OR -StockCurve     Integer
              The number of the stock (built-in) MTF curve to use.  Since
              there is only one curve, only an entry of 1 is allowed.

       -maxinv (-ma) OR -MaximumInverse    Floating point
              Maximum value for inverse of MTF.  The inverse should always be
              limited to reduce noise.

       -invrolloff (-inv) OR -InverseRolloffRadiusSigma   Two floats
              Radius and sigma for gaussian roll-off of inverse to 1.0
              (default 0.12 and 0.05)

       -xscale (-x) OR -XScaleFactor       Floating point
              Scaling factor for X-axis of MTF curve.  Scaling the X axis is
              probably an adequate way to adapt a curve from one camera or
              binning to another.

       -noise (-n) OR -NoisePadding
              Use tapered noise based on nearby pixels along the edge of an
              image to pad an image before filtering.  The default is to pad
              with a simple taper that makes streaks outside the edge of the
              image. For low-dose images, low-pass filtering can spread these
              streaks into the image area after cropping and produce artifacts
              along the edge.  This option will avoid this effect, but is
              probably not suitable for higher-contrast images.

       -denscale (-de) OR -DensityScaleFactor   Floating point
              Scaling factor for image intensities after filtering.

       -rweight (-rw) OR -RWeightedFilter
              Apply an R-weighted filter in the X-dimension, as in back-pro-
              jection.  This option implies -1dfilter.  It cannot be used
              along with an inverse MTF filter.  The filter will be scaled to
              be 1.0 at the cutoff radius specified with the -lowpass option,
              if any, or at a frequency of 0.5.  This will likely result in a
              smaller range for the output values, which could lose intensity
              resolution by making integer values occupy too few gray levels.
              To overcome this problem, use the -denscale option to scale the
              data up, or change the output mode to floating point with "-mode
              2".

       -fake (-f) OR -FakeSIRTiterations   List of integer ranges
              Apply a filter to a standard R-weighted back-projection that is
              equivalent to doing a given number of iterations of SIRT. See
              the Tilt man page section on SIRT for a description of the
              filter and the literature reference. If one number is entered,
              the output file will have the supplied name.  If a list of num-
              bers (which can include ranges) is entered, a file is produced
              for each entered number, named by appending the number to the
              entered output name.  This filter can be combined with others
              (sensibly or not).  In particular, it could be combined with
              -rweight and and applied in 1-D to tilt series.  It cannot be
              applied in 3-D, and when applied in 2-D, it is appropriate only
              when applied to a reconstruction in its original orientation as
              a stack of X/Z slices.  In the latter case, the output here will
              match the output of Tilt very closely when there is no X-axis
              tilt or local alignments, and it will diverge increasingly with
              increasing X-axis tilt.  Whether these differences are signifi-
              cant depends on your application.  In any case, applying multi-
              ple filters would be an efficient way to find the desired number
              of iterations.  Iteration numbers must be under 1000.

   DOSE WEIGHT FILTERING OPTIONS
         These options control "dose weighting", which filters out high fre-
       quencies as a function of the dose already applied to a cryo-specimen.

       -dtype (-dt) OR -TypeOfDoseFile     Integer
              This option both enables dose weighting and indicates what kind
              of file is being provided with the dose information.  Types 1 to
              3 are simple text files with a line for each image in the input
              file containing one or two numbers.  These types contain just
              the dose for each image (type 1), the prior accumulated dose
              followed by the image dose (type 2), or the prior dose followed
              by the cumulative dose at the end of that image (type 3).  Type
              4 indicates information in the autodoc format: either the
              ".mdoc" file produced by SerialEM, or an HDF file that has
              incorporated that information into its metadata.  In the latter
              case, no filename would be entered with the -dfile option.
              Whichever form is used, the information in the file must be in
              the same order as the images in the tilt series file.  An origi-
              nal .mdoc file from SerialEM will not work if the input image
              file has been reordered so that the part of the series acquired
              first is at the end of the stack.

       -dfile (-dfil) OR -DoseWeightingFile     File name
              Name of file with dose information for dose weighting, or a
              short entry indicating how to derive the file name from the
              image input file name.  This entry is required if -dtype is
              entered, unless a 4 is entered and the input file is an HDF file
              with the equivalent metadata as in an ".mdoc" file.  Two kinds
              of entries can be used to derive the ".mdoc" name from the input
              name.  1) An entry with just an extension and starting with a
              period (like ".mrc") indicates to replace the extension of the
              input file with that entry and append ".mdoc".  For example, if
              the input is "setname.ali", an entry of ".mrc" will make it use
              the filename "setname.mrc.mdoc".  2) An entry starting with "_"
              and consisting of both a suffix and an extension will make it
              strip out the suffix from the input name (provided it is just
              before the extension), substitute the extension in the entry for
              the one in the input name, and append ".mdoc".  For example, if
              the input is "setname_ali.mrc", an entry of "_ali.mrc" will make
              it seek "setname.mrc.mdoc" and an entry of "_ali.st" will make
              it look for "setname.st.mdoc".

       -dfixed (-dfix) OR -FixedImageDose       Floating point
              Fixed dose for each image of the input file, in electrons/square
              Angstrom.  This option cannot be entered with -dtype.

       -initial (-ini) OR -InitialDose     Floating point
              Dose applied before any of the images in the input file were
              taken; this value will be added to all the prior dose values,
              however they were obtained.

       -bidir (-b) OR -BidirectionalNumViews    Integer
              Number of views in the first part of a bidirectional tilt series
              file, where the order of images in the input file is inverted
              from their order of acquisition.  This entry is essential for a
              bidirectional series if a fixed dose is entered with -dfixed or
              if just a dose for each image (dose file type 1) is entered.
              The program assumes that the part of the tilt series acquired
              first occurs at the beginning of the stack.  If this is not the
              case, you must enter the -reversed option.  The -bidir option is
              ignored for dose file types 2 and 3.  It might be used for type
              4 if there are no PriorRecordDose entries, but should not be
              needed in that case because the DateTime entries allow the ini-
              tial order of images to be deduced.

       -reversed (-re) OR -ReversedBidirectional
              A bidirectional series is stacked with the part acquired first
              at the end, and the second part at the beginning.  This entry is
              needed to make the -bidir option perform properly in the cases
              described there where it is needed.

       -volt (-vo) OR -Voltage   Integer
              Microscope voltage in kV; this value must be either 200 or 300.
              The default is 300; if 200 is entered, the computed critical and
              optimal doses are multiplied by 0.8.

       -optimal (-op) OR -OptimalDoseScaling    Floating point
              Factor by which to scale the computed optimal and critical doses
              that determine how much to attenuate a spatial frequency for a
              particular dose.  Enter a factor greater than or less than 1. to
              indicate that the specimen is more or less resistant to damage
              than the equations indicate.  Another use for this entry would
              be to adjust the critical dose for a voltage other than 200 and
              300 kV.

       -critical (-cr) OR -CriticalDoseFactors       Three floats
              Replacement factors a, b, and c in the equation
                 critical_dose = a * k^b + c
               where k is frequency in reciprocal Angstroms.  The default fac-
              tors are directly from Grant and Grigorieff and the unblur pro-
              gram.  Enter 0 for any of the factors to use the default for
              that factor.

       -verbose (-ve) OR -VerboseOutput    Integer
              Enter a 1 to output the cumulative and image doses used and the
              filter functions applied for dose weighting.

   PHASE PLATE FILTER OPTIONS
         These options control an amplifier filter used to correct artifacts
       in images obtained with phase plates having a hole.

       -amplifier (-a) OR -AmplifierFactorAndPower   Two floats
              Amplification factor and exponent for low-frequency amplifier
              filter.  Either -cutoff or -phase must also be entered.

       -cutoff (-cu) OR -CutoffForAmplifier     Floating point
              Cutoff radius in reciprocal pixels for amplifier filter.  This
              option cannot be entered together with -phase.

       -phase (-ph) OR -PhasePlateParameters    Three floats
              Parameters of phase plate imaging, used to compute a nominal
              cutoff radius for the low-frequency amplifier filter.  Enter the
              phase plate diameter in nanometers, the voltage in kilovolts,
              and the objective lens focal length in millimeters.

       -pixel (-pi) OR -PixelSize     Floating point
              Pixel size in nanometers.  A pixel size is needed for dose
              weighting and to compute the cutoff radius for an amplifier fil-
              ter from the phase plate parameters.  This entry is needed only
              if the pixel size in the image file header is incorrect.

       -param (-pa) OR -ParameterFile      Parameter file
              Read parameter entries from file

       -help (-he) OR -usage
              Print help output

       -StandardInput
              Read parameter entries from standard input.


HISTORY
       Added to package, 3/30/04
       Added ability to operate on 3D FFT, 6/19/04
       Added ability to take filter real volume in 3D, 5/20/08

BUGS
       Email bug reports to mast at colorado dot edu.



IMOD                                4.11.0                        mtffilter(1)