Frame Alignment Parameters Dialog

This dialog allows you to set parameters to control alignment of frames taken from a camera that can acquire frames.

With a K2/K3 camera, the alignment can be done in three different ways: with the native alignment procedures in DigitalMicrograph, with the 'framealign' module in SerialEM's plugin to DM, or independently of SerialEM with the Alignframes program in IMOD, which incorporates the same 'framealign' module and thus uses the same parameters. In the latter case, all that the plugin to DM does is write a command file for running the alignment in IMOD after all of the frames have been saved. The Framewatcher program in IMOD can be used to process these command files as soon as they appear.

With a OneView or Rio camera, alignment can similarly be done in the plugin to DM and applied to the image returned to SerialEM, or a command file can be written to run Alignframes.

With a Falcon camera, the alignment can generally be done in two or three similar ways: with the Thermo/FEI software that aligns frames from Falcon 3 camera, with the 'framealign' module incorporated into SerialEM, or with the Alignframes program in IMOD. Again, in the latter case all that SerialEM does is write a command file. The availability of the different forms of alignment and how data are processed depends on the details of the system. Specifically:

For Falcon 2 running through the standard scripting interface, Thermo/FEI software writes the frames as raw files on the microscope computer. They are read by SerialEM's microscope plugin or server and sent to SerialEM to be aligned and/or stored in an MRC file. Both methods of alignment with 'framealign' will always be available.
For Falcon 2 running through the advanced scripting interface, Thermo/FEI software writes the frames as an MRC stack in a standard storage location on the microscope computer. Alignment by SerialEM will always be available because SerialEM's plugin or server can read them and send them to SerialEM. Writing a command file for alignment with IMOD is available only if SerialEM has direct access to the file system on the microscope computer.
For Falcon 3 or 4, Thermo/FEI software writes the frames on the storage server. This dialog, and alignment with 'framealign' by either method, is available only if SerialEM has direct access to the file system on the storage server. Falcon 3 writes frames as MRC, while Falcon 4 allows frames to be written either as a gain-normalized MRC stack or as an unnormalized TIFF file in the compressed EER format. In the latter case, SerialEM will use the latest Falcon EER gain reference when aligning frames itself, or copy this reference to the directory with the frame stack when aligning in IMOD.

With a DE camera running a server version higher than about 1200, frame alignment can be done within SerialEM or by writing a command file to run Alignframes in IMOD, provided that SerialEM is running on the DE server. Alignment is done in SerialEM by reading in the frames from disk. In addition, there may be a frame alignment available through the DE server.

With a Tietz camera where frame saving and aligning are enabled, frames are obtained directly from the Tietz software and aligned in SerialEM, or a command file can be written to align them with Alignframes in IMOD.

Aligning in DigitalMicrograph or Thermo/FEI or DE Software

For a K2, the alignment in DM has one parameter, the filter that is used in the alignment procedure. The combo box should show the same set of filters that are available in DM; if not, the SerialEMproperties file needs to be modified to list the correct filters. This alignment is slow, especially with super-resolution frames. With a K3, alignment is done with the MotionCor2 built into DM. There is no control of parameters from SerialEM. With a GPU, the alignment is somewhat slower than the one available in the SerialEMCCD plugin and CTF fitting indicates no better performance for images requiring only a shift.

For Falcon 3, alignment with the Thermo/FEI software has no parameters but does require a certain minimum exposure time per frame and a minimum number of frames.

For a DE camera with the frame alignment option, again there are no parameters.

Aligning in the SerialEMCCD Plugin or SerialEM

Alignment with the 'framealign' module has many more parameters. Virtually all of the ones shown in this dialog correspond exactly to an Alignframes option, so the man page for that program can be consulted for more details about any particular option, as well as for general instructions and advice. The reason that there are so many parameters is that different settings are needed to obtain optimal results in fundamentally different situations. Still, it should be possible to set up a small selection of parameter sets that will work for your own data. SerialEM can store multiple parameter sets, and this dialog allows you to chose among them. The selection is stored as part of the individual camera parameter set. If you have no previously defined parameter sets, the program will add two sets suitable for use with counting mode and super-resolution frames. These default parameters are a good starting point and may well be adequate for you.

The dialog gives a choice among several alignment strategies. Which one is appropriate depends on the kind of data being taken and its noise level. One basic strategy is to find shifts for individual frames by fitting to measurements of shifts between many pairs of frames (the approach of the 'motioncorr' program from UCSF). This method uses comparable amounts of information for all of the frames. The implementation here has two important features: 1) the fits can be done to pairwise shifts measured for subsets of frames, thus giving the benefits of the approach without requiring as many measurements; and 2) the fitting is done with 'robust regression', which weights measurements less the more they deviate from the fitted solution used, so that a few bad correlations can be effectively ignored. The different options in the dialog allow the fits to be done to a fixed number of frames, half the frames, or all the frames. The other basic strategy involves correlating each frame to a sum of previously aligned frames; this is not as reliable in general because it uses less information for finding the shift of the first few frames, and is more susceptible to fixed pattern noise. However, for higher dose images (especially from stained specimens) this strategy should work well and will be somewhat faster.

Both methods can be followed by a refinement step in which each frame is correlated with the aligned sum of all other frames (this is similar to what is done in the unblur program from Grigorieff's group). This step probably has no benefit if the errors from the initial fit are low. The Alignframes man page describes how to evaluate whether one particular parameter selection is better than another.

The general advice about which approach to use is:

Fitting to pairwise shifts of 7-9 frames at a time is good for typical tilt series data with 5-15 frames, because there are generally enough features in the field to provide a reasonable signal-to-noise ratio.
Fitting to pairwise shifts of half the frames at a time is generally good for single-particle images with long exposures, as it both saves some time and avoids correlating images that may differ due to specimen damage.
Fitting to pairwise shifts of all the frames may be needed for noisier data or to handle situations where there may be very little contrast in the field.
For even noisier data, aligning sums of successive frames, called grouping, may be needed in the initial alignment; but grouping may not work well for early frames if there are substantial shifts between frames, so refinement is a good idea.
Alignment to a cumulative reference should be used only if there is a relatively high signal-to-noise ratio, such as for a stained specimen.

Some other main points about the parameters:

Low-pass filtering of low-dose images is essential.
The high degree of binning provides for both a speedup in alignment and the first level of filtering.
The second level of filtering is provided by a gaussian filter that starts at a specified cutoff filter and fall off with a sigma proportional to the cutoff. This is not like the traditional "B-factor" Gaussian filter that starts falling off from 0.
Filter cutoffs are in reciprocal pixels in the unbinned image (0.5 cycles/pixel is the Nyquist, the highest frequency that can be represented, in the unbinned image). This means that their effect stays the same if you change binning.
A cutoff value of 0.06 is good for typical data; some super-resolution frames may require 0.05; most data can probably tolerate somewhat higher values, possibly up to 0.1.
The binning for alignment is independent of the binning for the returned image, which is what is specified in the Camera Setup dialog.
The term "binning" is a misnomer in both cases: the image is reduced for alignment with real-space anti-aliased reduction; the final image is reduced by Fourier cropping.

The plugin will do cross-correlations for each frame as soon as it available, and will do the final shift of a frame and its addition into the aligned sum as soon as allowed by the parameters. If the time taken for alignment is a consideration, then you should be aware of which options make it unable to do any final shifts until all frames are received: doing pairwise fits among ALL the frames; refining the initial alignment; doing spline smoothing of the estimated shifts; and testing more than one filter cutoff without selecting to use 'hybrid shifts', which are available before the end. However, you will probably find that using these options costs less than 0.1 second per frame.

Frames can be acquired dark-subtracted or unprocessed instead of gain-normalized if they are also going to be saved as compressed TIFF files. The data will be normalized by the framealign module before being aligned, and the aligned sum will thus be gain-normalized.

When an alignment is done, results may be printed to the log. In their full form, they look like this:

Frame alignment results:          distance raw = 2.7 smoothed = 1.3
Weighted resid mean = 0.16  max max = 0.32  Mean unweighted max = 0.57
FRC crossings 0.5: 0.0335  0.25: 0.0468  0.125: 0.0898  is 0.1025 at 0.25/pix

Distance raw: the sum of all the measured shifts between successive frames, in unbinned pixels; measurement errors will make this number exceed the actual distance moved by the specimen.
Smoothed: the sum of measured shifts between successive frames after whatever smoothing is applied to them. This is the best indicator of the total amount of drift. If the raw distance is much bigger, it can be a sign of inaccuracies in the shift measurements.
Weighted resid mean: the single best measure of the quality of the alignment. A residual is the difference between a measured shift and the shift implied by a linear fit. A weighted mean residual is obtained for each individual robust linear fit and it appropriately ignores shifts with large errors just as they are ignored in obtaining the solution. The value here is this mean residual averaged over all of the fits. Values in the range of 0.1-0.4 indicate good fits overall.
Max max: the maximum value of the weighted residual found in any of the fits, which indicates the error in the worst shift that is actually used.
Mean unweighted max: This maximum residual is an unweighted one, the actual error between a measured shift and the one given by the fits. The value is the average of the maxima obtained in each of the fits. A high value is a sign that bad shifts are being measured between some pairs of frames; but if the "Max max" for weighted residuals is still fairly low, it means that the robust fitting successfully suppressed all of those bad shifts.
FRC crossings: Odd and even frames can be summed separately and the Fourier ring correlation computed. The first three pairs of numbers are the frequencies (in unbinned reciprocal pixels) at which the FRC falls below particular levels. In the example, it falls below 0.5 at 0.034/pixel, below 0.25 at at 0.047/pixel, and below 0.125 at 0.090/pixel.
FRC value at 0.25/pix: The final value is the FRC at half-Nyquist in the unbinned image.

If there are not enough frames for robust fitting, the output will be somewhat simpler and reflect only actual residual errors, since no weighting is being done. The number of lines of output can be set by the Set Frame Align Lines command in the Camera menu. The default is to omit the FRC values since these are the most difficult to interpret and depend on the sample and dose.

Aligning in IMOD

The same parameter settings are presented when writing command files for aligning with Alignframes as for aligning in the plugin. The principal difference is that one needs to select a folder where the command files will be written, and that one can align and stack an entire tilt series after completion. The files will actually have extension '.pcm' instead of '.com'.

Time should not be an issue when doing alignments with this option, since the alignments are generally fast enough to be done well before the next set of frames needs aligning. Thus you can use refinement, smoothing, or multiple filters routinely without worrying about whether they are actually helping.

K2/K3 frames can be taken unnormalized in order to save efficiently in compressed TIFF files. The command file will contain options to apply gain-normalization and defect correction. Similarly, EER frames from Falcon 4 are saved unnormalized, and the command file will contain the name of the gain reference, which incorporates a defect list.

Global Settings

The upper part of the dialog, down to the box for Parameter set name, allows one to set a few 'global' settings that apply to all alignments. The list box on the right shows the available parameter sets, which can be selected by clicking on one.

Use the GPU

Select whether to use the GPU on an NVIDIA card for the frame alignment. This setting is kept track of separately for aligning immediately versus aligning in IMOD, because the latter could be done by any computer with access to the frame data. In fact, if there are two K2/K3's, or a K2/K3 and a Falcon, it is kept track of separately for each camera. This option should be available if NVIDIA drivers are installed, and the files FrameGPU.dll, cudart64_41_28.dll and cufft64_41_28.dll are in the correct location: specifically, for a K2/K3 camera, in C:\ProgramData\Gatan\Plugins along with the SEMCCD-GM2-64.dll plugin; and for a Falcon or DE camera, in the same folder as the SerialEM executable.

Align only whole tilt series

This option is relevant when aligning in IMOD and only for a Record parameter set; when it is checked, a command file is not produced for individual Record images, only at the end of a tilt series. That command file will align all of the frame files for the tilt series and construct a new stack with appropriate header entries, using a copy of the .mdoc file for the original tilt series stack.

Set Folder for Align Com Files

Press this button to select the folder where the command files will be written when aligning in IMOD. This folder must be on the same drive as the frame files, because the command file needs to be written with relative paths that would be meaningful if processing on another machine. With a K2/K3 camera, if SerialEM is running on a microscope computer, then the folder name must be typed into the dialog box that comes up. If it is running on the K2/K3 computer, then a folder chooser will come up just as it does for selecting the frame folder. A folder chooser comes up for a DE camera, and for a Falcon 2 camera when SerialEM is running on the microscope computer.

Use same folder as frames

Select this option to have command files written into the same folder as the frames, thus overriding any entry that was made with the Set Folder button.

New Parameter Set

Press this button to create a new set of parameters that start out as a copy of the currently selected one.

Delete

Press this button to delete the currently selected set of parameters. This is possible only when there is more than one set, because the program always needs at least one set. If any other camera parameter sets have selected this set, their selection will be changed to the adjacent set; and if any of those have frame alignment enabled, the program will warn you about this change.

Parameter set name

Here you can edit the name of the set that appears in the list box.

Restrictions on When to Use a Parameter Set

This group of check-boxes lets you control whether a parameter set is to be used only in under certain conditions. Namely, you can specify that a set should be used only when Aligning in plugin (or in SerialEM) or only when Aligning with IMOD, and independently whether it should be only for non super-resolution frames or only for super-resolution frames. The restrictions based on size may be superfluous if one specifies binning to a target size rather than by a fixed amount for alignment. The restrictions are taken into account at several places in the program, where the currently selected parameters are tested for whether they meet the conditions. If they do not, the program analyzes whether the conditions are met by no, one, or several other parameter sets, and acts in various ways:

When you close this dialog, you will be given a choice of whether to use the selected set anyway (if there are no suitable ones), use the one suitable set (if there is one), or use the first suitable set (if there is more than one), or whether to stay in the dialog to change some settings.
When you switch between aligning in the plugin and with IMOD, it will automatically switch to another set if it is the only suitable one, otherwise leave the selection as it is.
In the Camera Setup dialog, when you switch between Counting and Super-resolution modes, it will automatically switch to another set if it is the only suitable one. Otherwise, you will be given a choice of whether to switch the mode back, open the dialog, use the selected set anyway (if there are no suitable ones), or use the first suitable set (if there is more than one).
When an image is to be acquired, it will automatically use another set if it is the only suitable one or the first suitable set if there is more than one; otherwise it will use the current set anyway.

The use of restrictions will work most conveniently if you have only one parameter set for each kind of situation, each one marked with different restrictions. For a K2/K3 or DE, if you have just the "4K default set" and "8K default set", simply restrict each one to the respective size of frames, and the parameter set will switch automatically when you switch between super-resolution and counting mode. However, if all the parameters except binning are the same, it may be simpler just to bin to a target size.

Basic Alignment Parameters

The parameters in the upper section are the ones most likely to need setting. The sections below refer to parameters being expressed in "unbinned" pixels, which really means the pixels of the frames that are supplied to the framealign module, before the reduction that it does. For Gatan and DE cameras, these would be super-resolution pixels when images are taken with super-resolution. However, for Falcon 4, it means the physical pixels of the chip when EER data are read as super-resolution.

Truncate values above n electrons (K2/K3 or Falcon 4 saving EER data only)

Select this option and set the number of electron counts to have values above that number replaced by the local mean. This is the number of electrons per acquired pixel, which cannot be higher than 15 for super-resolution frames. If your gain-normalized images have many extreme values, removing them from the frames may make the alignment more reliable, as well as making it easier to see the data in 3dmod.

Alignment strategy

Here one can choose among the options described above. Here are a few points to note. If the number of frames in the first option is set higher than the actual number of frames, it will have the same effect as fitting to ALL pairwise shifts. The second option, to fit to pairwise shifts of HALF the frames, will actually fit to a minimum of 7 frames. If alignment to a cumulative reference is chosen, grouping of frames and testing of multiple filter cutoffs are not available.

Process EER data with super-resolution (Falcon 4 only)

These radio buttons determine the resolution at which data are read in from EER files for alignment in IMOD: with no super-resolution, with 2x super-resolution to give an 8K image, or with 4x super-resolution to give a 16K image. When aligning in SerialEM, each summed frame is 4K and the electron events are summed into this image with antialiasing before gain correction. The capability to read with antialiasing was added in IMOD 4.12.3; before that version, reading at 2x or no super-resolution always involved binning, which would introduce some noise due to aliasing. In principle, the antialiasing provided by Fourier reduction in Alignframes should be slightly better than the real-space antialiasing done when reading the image, so you may wish to process images in IMOD at 2x super-resolution with a 4.12 version. You will certainly want to use at least 2x with IMOD 4.11. The 4x super-resolution may require too much time or memory, depending on your processing capability. To keep the effect of other parameters invariant when this option alone is changed, the binning, filter cutoffs, maximum frame shift, and the criterion for stopping iterative refinement are all scaled by this super-resolution factor before being placed in a command file. The value for reduction of the aligned sum placed in a command file will be 1 for 2x super-resolution or 2 for 4x super-resolution. Reading in as 2x for aligning in SerialEM could be supported if there is demand for it.

Alignment binning

Binning, actually antialiased image reduction, is one of the two most important parameters because it affects processing time as well as the accuracy of the fits. Binning should be set to bring images to a size of 1000 to 1250 pixels for a camera with a ~4K chip, or 1250 to 1500 pixels for a K3 camera; thus 3 or 4 for ~4K images and 6 or 8 for ~8K images. You can choose to set either a specific binning or a target size to which images should be binned.

Filter cutoff

The filter cutoff is the most important parameter; it is expressed in cycles per pixel in the unbinned frame, so that a value has about the same effect with different binnings. Optimal values are generally in the range of 0.05 to 0.07.

Maximum shift between frames

This entry sets the maximum shift that will be detectable from one frame to the next, in unbinned pixels. Larger shifts are allowed between more separated frames, but the shifts have to be in a range predicted by the relative shifts of the intervening frames. This parameter should probably not be in the basic section. The default in Alignframes is 20.

Group frames by n

Select the checkbox to have successive frames summed together (without alignment) for the initial correlations. The spin box sets the number of frames to sum. When frames are very noisy, grouping by 2 or 3 can bring the correlation peak out of the noise. All possible sums of successive frames are formed, but correlations are done only between sums that do not overlap. This increases the number of frames required to use this approach, which is shown after the spin button.

Refine alignment with up to n iterations

Select the checkbox to add a refinement step after all the initial correlation have been done. Even multiple iterations of refinement are relatively quick because all of the FFTs that need correlating are already available. The spinner sets the maximum number of iterations that will be run, but refinements often stop after fewer iterations because the changes have fallen below a threshold. Five iterations is generally sufficient unless refining with group sums.

Smooth shifts if at least n frames

Select this checkbox to apply spline smoothing to the shifts if there are at least as many frames as selected in the spinner. This smoothing may have a very small benefit but should only be used if there are enough frames to give a good assessment of the smoothing that is needed. The default in Alignframes is to smooth shifts when there are at least 15 frames, but this is left as a choice here because smoothing is one of the options that prevents any frames from being summed until all have been aligned.

Align subset of frames

Select this checkbox to align only a subset of the frames being saved, then fill in the starting and ending frames to align, numbered from 1. The option is irrelevant when no frames are saved; originally this was because in that case a drift settling time could be set for K2 or K3 to discard initial frames after the exposure starts. When frames are being aligned in the plugin to DM or in SerialEM, the image shown in SerialEM will be a sum of only the aligned frames. When frames are aligned with IMOD, the command file will contain the -frames option to Alignframes with the given starting and ending frames.

More Advanced or Less Commonly Needed Parameters

Refine in groups

When frames are exceedingly noisy, correlations between a single frame and an aligned sum may not be reliable enough, so this option is available to do the refinement with groups of aligned frames instead of single frames. The iterations for refinement should be set to 10 if this option is used.

Filter cutoff for refining

In this box, you can enter an alternative cutoff radius in cycles/pixel to use for the refinement step; otherwise the same cutoff will be used as for the initial alignment. Large increases will impair the alignment, but an increase of ~25% might be beneficial.

Stop refining when changes are below n unbinned pixels

This entry sets the threshold change in shifts for continuing to iterate the refinement of the initial alignment. The default in Alignframes is 0.1 unbinned pixel.

Other filter cutoffs

These two text boxes allow additional filter cutoffs to be tested at the same time as the main cutoff.

Use hybrid shifts

When multiple filters are being tested, the default is to accumulate all the alignment results, then pick one filter as best and apply the shifts from it to form the aligned sum. If this option is checked, then a best filter will be picked after each individual fit to a set of pairwise shifts, and the shift implied by the fit with that filter will be used for the one image whose shift is being set after that fit.

Ratio of falloff sigma to filter cutoffs

The ratio in this text box is used to scale the sigma of the Gaussian falloff of the high-frequency filter proportionally to the cutoff. Such scaling is done when using the '-vary' option in Alignframes, and the default ratio there (1/7) is based on the ratio of default values for cutoff and falloff.

Always process gain-normalized frames with full precision (K2 only)

This option should not be used; just make sure that you have not selected 'Divide 16-bit by 2' and the precision will be as good as with a K3. This option is available only when aligning in the plugin and for K2 frames only. It is relevant there when acquiring a gain-normalized image with an 'early return', and when using the GPU. In all other cases, the plugin passes a gain-normalized image to the framealign module in its original floating point form, before possibly scaling it to integers. With an early return, the plugin has to fetch frames from DM as fast as possible and store them in a local stack. By default the stack is in a scaled form (byte for super-resolution, 2-byte integer for counting mode). With this option checked, counting mode frames will be saved in their floating point form, and super-resolution frames will be multiplied by 4096 and saved as 2-byte integers, so the precision will be fully preserved, but the stack will occupy twice the memory. Choosing this option may delay the early return unless there is enough memory. When using a GPU, the plugin will do the usual scaling of frames to bytes or 2-byte integers before copying to the GPU, in order to reduce the copying time and allow more frames to be stacked on the GPU between the alignment and summing steps. With this option, floating point numbers will be copied, which will take longer and limit the number of frames that can be stack. However, this option is overridden by the option to scale saved frames by an additional factor of 100; if there is an early return or GPU use and frames are being saved with such scaling, the scaled frames will be passed to the framealign module.

Output aligned sums as floating point

This option is available only when aligning in IMOD. Select the checkbox to have aligned sums output as floating point instead of 2-byte integer, which will preserve a very small amount of precision in the sums. The regular scaling will still be applied to them in this case.