Guide to Particle Estimation for Electron Tomography (PEET)

University of Colorado, Boulder

Table of Contents

1. Introduction to PEET

PEET is a set of programs, separate from but typically used in conjunction with IMOD, to align and average particles / subvolumes extracted from 3D volumes.  It iteratively refines particle alignments against a reference volume over several iterations.  A new reference is typically generated from a subset of the aligned particles at the end of each iteration for use in the following iteration. IMOD's Etomo graphical user interface permits users to set up options (search parameters, etc.), and to control the execution of the process.  This guide will explain the use of that interface.  For details on individual PEET programs, please consult the program descriptions and manual pages accessible from the Etomo's Help Menu, by running "PEETHelp", or online at bio3d.colorado.edu.  Manual pages are also available at a shell prompt using the "man" command.  (Program descriptions and manual pages will only be available locally on systems with PEET already installed).

Each PEET project must reside in its own directory because of the potentially large number of intermediate and output files created.  Input files (IMOD models and volumes, initial motive lists, etc.) can also reside in this directory, but are not required to.

To access the interface and create a new PEET project, run Etomo and either press the New PEET button, or select File-->New-->PEET from the menu.  This will bring up a dialog allowing you to specify the directory where the new project is to be located, as well as a base or root name to be used in generating output and intermediate file names.  This dialog will also allow you to copy all the settings from an existing project as a starting point for the new one. 

Alternatively, to open an existing PEET project simply cd to the desired directory and run Etomo with the name of the desired project file, e.g. "etomo myProject.epe", or run Etomo with no arguments and select File-->Open-->PEET from the menu.  You can then browse to and select the desired project file.  (PEET project files have a .epe suffix, as described below).

The Etomo PEET gui has 2 tabs, labeled Setup and Run.  The Setup tab has options for defining the data to be worked on and setting up initial conditions and other features of the search.  The Run tab has a table for entering parameters to control each iteration of the search and options for the final averaging operation.

2. The Setup Tab

2.1. Root Name for Output

(Read only). Displays the base name that PEET will use to create output and intermediate files.  In the following examples, we will assume a base name of "myRun".

2.2. Directory

(Read only). Displays the directory containing the PEET project.

Assuming a base name of "myRun", some of the important text files in this directory are listed in the table below,  (Here, #k and #m stand for integers, and "*" is a wildcard which can match any series of characters.)

Files with .mrc extension in this folder are volumes stored in MRC format.  References and final averages created by PEET are important examples.  For example, myRun_Ref#m.mrc is the reference volume created for alignment during iteration #m.  Similarly, the average volume resulting from combining #n particles at iteration #m is stored in myRun_AvgVol_#mP#n.mrc.

2.3. Volume Table

The Volume Table can have multiple rows, each referring to a volume and corresponding model to be included in the alignment.  A row can be selected by clicking => near the left end of the row.  Each row can contain 5 entries, described in the following table. 

2.4. Action Buttons

Up, Down, Delete, and Dup buttons to the right of the volume table allow modifying the volume table by moving, deleting, or duplicating a selected existing row.  To select a row press => near the left of the row. The Insert button adds a new, blank row a the bottom of the volume table. Volume, Model, and Initial MOTL fields for any row in the volue table can be entered or modified either by selecting the file browser (folder) icon to the right of the field, or by typing directly in the text box.  Tilt ranges can be entered or modified by typing in the appropriate text box, or by pressing Read tilt file if you have the an appropriate file.  Selecting a row of the volume table and pressing Open in 3dmod will open and display the corresponding volume, assuming it is the name of a single volume.  If File names are templates is selected, any or all of Volume, Model, Initial MOTL, and Missing Wedge Mask can be templates mapping to many files, rather than the name of a single file. For example v1-60.mrc would be expanded to 60 volumes: v1.mrc, v2.mrc, ... v60.mrc. 

2.5. Reference Section

The Reference section allows specification of the initial reference to be used for the first alignment iteration. If the Particle radio button is pressed, a single, specified particle from one of the input volumes will be used as the initial reference.  Tomograms and particles are both numbered sequentially from 1, and particles are numbered as if all the contours were combined into one; e.g. if the first contour has 100 particles, then the 20th particle of the second contour will be particle number 120.  Note that PEET will align and average only particles and contours contained in the object 1.  Additional objects will be ignored during alignment and averaging, although they are used by some specific PEET programs (e.g. see the man pages for modTwist2EM, spikeInit, and stalkInit).

If User supplied file is selected, the specified MRC file will be used as the initial reference. In this case, the user is responsible for ensuring that the supplied reference is appropriately sized.  To support full searches, the reference should ideally be a cube large enough to contain a particle in any orientation, padded by twice the maximum search distance in any orientation.  (PEET will choose an adequate reference size automatically when the reference is not user-supplied).

Finally, if Multiparticle reference is selected the specified number (a power of 2 between 4 and 1024 chosen from the drop-down box) of randomly chosen particles will be aligned and combined using a binary tree structure, and the resulting average used as the initial reference.

2.6. Volume Size

Specifies the size in voxels of the subvolumes to be aligned and averaged.  Cubical volumes are prefered when possible, although they are not required.  NOTE: since PEET version 1.9.0 it is no longer necessary nor desirable that volume size be padded by twice the maximum Search Distance. Volume size should now be set only slightly larger than the volume of interest.

2.7. Missing Wedge Compensation Section

Check Enabled and specify Tilt Range in the Volume Table to use PEET's missing wedge compensation during alignment and averaging.  When selected, object space averaging of the aligned particles will be replaced by a weighted averaging in Fourier space, with each particle contributing only to regions in which its projections are informative, i.e. those lying outside its tomographic missing wedge.  Additionally, during alignment of particles against a reference with known missing wedge, correlations coefficients will be normalized by the fractional overlap between the informative regions of the particle and the reference. 

Edge shift specifies the number of pixels in from the edge of the missing wedge to include when averaging with missing wedge compensation.  Some useful information seems to spread into the missing wedge during backprojection.  A value of 1 voxel is typical appropriate.

An additional type of missing wedge compensation is available when using cross-correlation to select particles for averaging at the end of each iteration (i.e. when creating a new reference or final averages).  If the number of Weight groups is set to a number larger than 1, particles are divided into groups based on their missing wedge orientation.  Correlation scores are then scaled so that each group has approximately the same median score.  The result is that nearly equal numbers of particles from each of the orientation groups will included in the reference (or the final average), thus minimizing a missing wedge bias. The appropriate setting must be chosen heuristically, but depends on the number of available particles as well as the number of distinct orientations present in the data.  This option is most effective if particles assume a variety of original orientations.  If particles all have about the same orientation, (e.g. particles from a non-twisting microtubule in the X-Y plane), this type of compensation may not be necessary or helpful unless Random axial rotations (described below under 2.9. Particle Y Axis Section) are used, or a program like modTwist2EM is used to provide varying starting orientations. Randomized particle selection, described below under 3.6. Optional / Advanced Features, disables this type of missing wedge compensation.

Number of Tilt Axes determines how missing tomographic regions are specified. If 1 is selected, PEET will automatically generate single-tilt-axis missing wedges corresponding to the tilt ranges in the volume table. If 2 or more is selected, PEET will read binary missing wedge masks from the files whose names are specified under Missing Wedge Mask. For dual- or multi-tilt axis tomograms, program multiTiltMask can be used to generate the necessary mask file(s).

2.8. Masking Section

Users can specify a mask to limit the portion of the reference volume to be included in cross-correlation calculations.  The default, None,is to use the entire volume.  Spherical, cylindrical, or user-defined masks can be selected by checking the appropriate radio button.  For both spherical and cylindrical masks, both inner and outer radii can be specified, allowing definition of either solid regions or hollow shells. Voxels between the inner and outer radii will be included. Truncated cylindrical masks can also be generated using the optional Cylinder Height field.

To use an MRC volume containing an arbitrary mask select the User supplied binary file option, and enter the name of a file containing the mask volume.  A voxel of the reference volume will be masked out (i.e. excluded) if the value of its corresponding voxel in the mask volume is zero and included otherwise.  The mask volume size need not match that of the reference.  If they differ, the mask volume will be centered on the reference, and mask values outside the reference volume ignored.  Mask values outside the mask volume, but within the reference, will be determined by the value of the majority of the 8 corner voxels of the mask volume.  (I.e., if 5 or more corners are zero, the mask will be zero outside the volume;  conversely, if 4 or fewer corners are zero, the mask will be one outside the mask volume.) 

Masks will be centered on the initial reference, and, by default, a cylindrical mask will be oriented with its long axis along the reference's Y axis.  You can override the orientation of a cylindrical mask by selecting Manual Cylinder Orientation and specifying the desired Z and Y rotations in degres.

By default, masks are treated as binary, leading to an abrupt edge. To generate a soft-edged mask, enter a positive value for Blur mask by, which specifies the standard deviation of a 3D Gaussian to be convolved with the original, binary mask before use. The resulting soft edge will extend approximately 2.5 standard deviations on either side of the original abrupt transition.

2.9. Particle Y Axis Section

The particle Y axis (or twist axis) is the axis PEET uses for the first angular search, referred to as Phi in the iteration table.  (Particle X and Z axes are determined from Y to form a right handed coordinate system, with some arbitrary conventions to resolve ambiguities).  if Tomogram Y axis is selected, the particle Y axis will coincide with the tomogram Y axis.  In this case, Phi will be around the tomogram Y axis, Theta around the tomogram Z axis, and Psi around the tomogram X axis.

If Particle model points is selected, the first rotation axis will vary from particle to particle and will be the vector connecting 2 consecutive model points in the IMOD model contour.  With this option, Phi represents twisting around the contour axis; Theta represents bending or turning in the X-Y plane, and Psi represents dipping in Z.  This option is most useful when particles lie along continuous contours, e.g. along a filament such as a microtubule.  When using this option, be sure to model each filament in a separate contour.

End points of contour is similar to Particle model points, except that the vector between the first and the last points in the contour will be taken as the Y axis for each point in that contour.  This can be useful when the contours are nearly straight, as it provides a less noisy estimate of the Y axis.

2.10. Initial Motive List Section

Controls in this section provide several options for setting up an initial motive list with starting orientations for all particles.  This can allow limiting the angular search range, improving both throughput and accuracy, even when the particles have significantly different starting orientations. 

2.10.1. Set all angles to zero

All particles will have their initial orientation left unaltered. This is appropriate if the particles are scattered, independent particles, no prior alignment information is available, and the orientations are either not readily apparent from the images or if you choose not to extract such information, preferring instead to rely on a more extensive alignment search.

2.10.2. Align particle Y axes

Automatically generate initial motive list rotations to align each particle's Y axis with that of the reference. This is particularly useful when modeling filaments and Particle Y axis is set to Particle model points or End points of contour.

2.10.3. User supplied csv files

Use this option if you have a motive list file in .csv format specifying the approximate rotation (and, optionally, translation) required to orient each particle.  Such motive list (MOTL) files can be obtained from a prior PEET run or from an auxiliary program that computes initial orientations from a model for a particular situation.  PEET programs modTwist2EM, spikeInit, and slicer2MOTL are examples of such auxilliary programs.  Additionally, program modifyMotiveList can be used to incorporate additional rotations and translations into an existing motive list, e.g. when imposing symmetry.

2.10.4. Uniform random rotations

A random rotation drawn from a uniform distribution will be generated for each particle.  In conjunction with a limited range angular search, this can be useful in avoiding missing wedge bias for isolated particles.  With suspected icosahedral symmetry, for example, a useful approach is to use this option along with angular search ranges of roughly +/- 32 degrees.  Note that random rotations will only help avoid missing wedge bias if other knowledge allows using less than a full spherical search.

2.10.5. Random axial (Y) rotations

An initial motive list will be automatically generated which aligns the particles Y axis to that of the reference, followed by a random rotations around that axis.  This can be helpful in minimizing missing wedge bias, e.g. when averaging sections of a cylindrcal filament with similar missing wedge orientations and known axial symmetry (e.g. a 15 protofilament microtubule).  Random axial rotations will help eliminate missing wedge bias only if other knowledge allows restricting the axial (Phi) search range.

3. The Run Tab

3.1. Parallel Processing

PEET can distribute the computational load of aligning and averaging particles over multiple processors on one or more systems, as specified by the user.  This section allows you to select which systems and how many processors on each system are to be used.  See the Using Parallel Processing in the Etomo User's Guide for details on these controls and on configuring parallel processing. For information on using a cluster queue, see the IMOD man pages for processchunks and the cpu.adoc file.

3.2. Iteration Table

Each row of this table controls a single iteration and has five types of entries.

The Angular Search Range entries set search ranges and increments (step sizes) in degrees for the 3 rotations (Phi, Theta, and Psi, referring to rotations around Y, Z, and X axes).  For example, if Max and Step fields of Phi are set to 6 and 2 respectively, Phi will have a range of -6 to +6 and will assume 7 values (-6, -4, -2, 0, 2, 4, and 6).  A general guideline is that except for special cases (e.g. a no-search iteration to generate a new reference) earlier iterations should have larger ranges and step sizes than later ones.  Additionally, multiple iterations with larger step sizes are typically faster than a single search with a large range and small step size.  A reasonable compromise is to use a Max of 3 times Step, as in the example above.  Both Max and Step can then be reduced by a factor of 2 at each subsequent iteration, until the desired precision is achieved.  Searches with even fewer steps can be done by using a Max of twice the increment, provided that Max and Step are reduced by a factor of only 1.5-1.6 between iterations.  In many cases, it may be necessary to use more steps than this during the first iteration to obtain a reasonable initial alignment.  To skip searching around a particular axis entirely, set the Max to 0 and Step to 1.

Search Distance can be either a single number or three numbers, and specifies the amount of translation in pixels allowed in the X, Y, and Z directions during alignment searches.  For example, setting it to 2 will limit the translation to between -2 and +2 in each of the 3 directions.  These directions are in tomogram rather than particle coordinates, so using different limits would be useful only if particles all had similar orientations with respect to at least one tomogram axis.  This entry does not typically affect execution time, but small values can make the alignment more reliable by preventing spurious correlation peaks at higher translations from being chosen.  A search distance of 0 can be specified to disable searching, allowing the quality of the initial alignment to be determined.

High Freq. Filter specifies parameters for filtering out high frequencies (i.e. for a low-pass filter).  As in IMOD, frequencies are in reciprocal voxels, with 0.5 being the Nyquist frequency, and 0.86 corresponding to the highest frequency in the corners of 3D FFTs.  The Cutoff field sets the cutoff frequency, below which there is no attenuation, and above which there is progressively more attenuation. Above the cutoff, the response falls like the right half of a Gaussian with standard deviation given by the Sigma field.  Both the reference and the particle are filtered separately, so the correlation is effectively filtered twice.  One way to see the effect of a given filter on a particular volume or particle is to apply that filter using IMOD program mtffilter, e.g., with the command "mtffilter -3d -low cutoff, sigma input_file output_file".  Alternatively, you can explore the effect on a selected image plane interactively by opening the volume in 3dmod and then choosing Edit-->Image-->Process-->Fourier filter from the menu.

Reference Threshold controls how many particles are used to form the reference for the next iteration.  If a value larger than one is specified, it represents the number of particles to use, with higher correlation coefficients chosen preferentially (unless Randomized particle selection is specified under 3.6. Optional / Advanced Features).  A rule of thumb is to use two thirds of the total number of particles.  In some cases, however, you might want to include all particles, especially if limited data are available or at the first iteration to minimize any missing wedge bias in the new reference.  If a value less than one is specified, it represents a correlation coefficient threshold, above which particles will be included in computation of the new reference.  We have not found specifying the threshold in this latter manner particularly useful.

Duplicate Tolerance provides control over PEET's duplicate rejection logic.  When the search range is comparable to the spacing between particles, and especially when particles lie along filaments, an alignment search can sometimes result in mulitple particles pointing to essentially the same position and orientation.  If not corrected, these errors can bias subsequent references and averages, and can lead to overestimation of resolution using Fourier Shell Correlation.  If Remove duplicate particles after each iteration is checked (below the Iteration Table), PEET will attempt to identify cases of duplication, and will remove offending particles from further averaging or consideration during the current iteration.  The Shift and Angular tolerances specify the maximum separation in integer pixels and degrees, respectively, at at which particles can be considered duplicates.  Two particles are considered duplicates only if their separation exceeds neither of these tolerances. The shift tolerance is applied separately to each of the tomogram X, Y, and Z coordinates. Duplication is determined independently near the end of each iteration, so a particle ignored as a duplicate in one iteration is not necessarily excluded from later iterations.  A value of 0 for either or both of the angular and shift tolerances can be used to disable duplicate removal for the associated iteration.

As for the Volume Table, action buttons to the right of the Iteration Table allow adding, deleting, or rearranging lines in the table.

3.3.Spherical Sampling for Theta and Psi Section

This option is designed to make full angular searching at the first iteration more efficient by sampling orientation space at relatively uniform intervals.  This is referred to as spherical sampling because the second and third search angles (Theta and Psi) are chosen to given approximately uniform spacing when represented on the surface of a sphere.  (Phi corresponds to rotation around the particle's twist or Y axis).  Simply varying both Theta and Psi with regular increments results in oversampling near the poles.  With spherical sampling enabled, PEET avoids this oversampling.  If Full sphere is checked, PEET will sample the whole sphere.  If Half sphere is checked, PEET will only sample one hemisphere.  The Sample interval field specifies the Theta search interval in degrees, as well as the Psi search interval at the equator.  For a unit sample sphere, sample points are placed on latitude lines separated by the sample interval, with approximately 360 / Sample interval points along along each latitude line at the equator, decreasing with latitude to no more than a single point at each pole.  For either a full or half sphere angular search, enter a Max of 180, with an appropriate Step for Phi in the Iteration Table, as well as the Sample interval for spherical samping.  It is typically reasonable to choose the Sample interval equal to the Phi Step size.

3.4. Number of Particles to Average

Specify how many particles to use to form averages.  For example, if Start, Step, End, and Additional numbers are set to to 50, 50, 200, and 205, respectively, there will be 5 final averages, containing 50, 100, 150, 200 and 205 particles.  Values listed under Additional numbers must be comma separated, monotonically increasing, and greater than the End value of the arithmetic sequence.  The largest average requested should be no greater than the total number of particles available.

3.5. Use Equal Numbers of Particles from All Tomograms

These checkboxes provide another tool for reducing bias when combining data from multiple tomograms.  If checked, PEET will attempt to use equal numbers of particles from each tomogram when computing averages or new references.  (Unequal numbers may still be used if required to achieve the requested number of particles).  If left unchecked, it will choose particles without regard to which tomogram(s) they are in.

3.6. Optional / Advanced Features

This section contains a number of useful, but less often used controls.

If Align averages to have their Y axes vertical is selected, average volumes will be reoriented to have their Y axis approximately vertical.  This is particularly useful for particles along a straight or slightly curved filament.  Note that the reorientation is applied only to the particles as they are averaged and will not be reflected in the final motive list. 

if Use absolute value of cross-correlation is checked, PEET will maximize the absolute value of the cross-correlation, rather than its signed value, during alignment searches.  This can help prevent noise from reinforcing to match features in the reference.  This works well for many biological structures which are globular or irregularly shaped.  Use it with caution, however, with repeating or highly symmetric structures where there is a chance of aligning "out of phase" (i.e. dark on light).

If Save individual aligned particles is checked, each individual subvolume contributing to one of the final averages will be saved to an MRC file named aligned*.mrc.  Use this option judiciously, as it can consume large amounts of disk space.

Normally PEET treats the Angular Search Ranges and Search Distance from each iteration as completely separate from those at any other iteration. So, for example, if Search Distances of 5, 4, and 3 voxels were specified at each of 3 iterations, it is possible that a particle could be translated by up to 5+4+3 = 12 voxels in each of the X, Y, and Z directions.  If Strict search limit checking is checked, the maximum shift from the starting position in each direction is limited to the largest distance specified at any iteration... in this case 5 voxels.  (Maximum Angular Search Ranges are treated similarly). This can be useful when refining an initial alignment, since the optimum at each iteration is expected to be within the search range of the previous ones.

No reference refinement allows for alignment against a fixed reference (i.e. template matching).  Normally, PEET generates a new reference at the end of each iteration based on the current alignment.  If no reference refinement is checked, the initial reference (whether user supplied our automatically generated), will be used for all alignment iterations without modification.

Randomized particle selection indicates that particles should be selected for inclusion in averages or new references at random, rather than based on their cross-correlation scores. In some cases, this can minimize reinforcement of noise to match a biased initial reference selection. Randomized particle selection is incompatible with Missing Wedge Compensation during alignment, however, and will effectively set the number of Weight Groups to 1.

Particles per CPU controls the splitting of alignment searches into "chunks" for parallel execution on multiple CPUS.  No more than the specified number of particles will be sent simultaneously to a single CPU for alignment.  The smaller this value, the more command and log files will be created, and, within limits, the greater the overall throughput.  Too high a value, can lead to long delays should you wish to pause and then resume processing.  The default of 20 is a good choice for most purposes (runs with at least several hundred subvolumes); if Etomo reports more than a few thousand chunks to be processed, you may wish to raise this setting. Conversely, if you have many cpus and few particles, you can lower Particles per cpu for faster execution.  To make effective use of available processors, the total number of chunks reported by Etomo should be no less than the number of cpus times the number of alignment iterations to be performed.

Debug level (0, 1, 2, or 3) controls the amount of information included in the log files, with higher numbers leading to more verbose logs.

PEET can suppress low frequencies from both particles and the reference (i.e. apply a high-pass filter).  Low frequency cutoff is the frequency in inverse pixels below which the signal will be attenuated.  As in the iteration table, Sigma is the standard deviation of the Gaussian whose left half defines the shape of the resulting high-pass filter.  We have rarely found low frequency filtering to be useful, and do not recommend its routine use.

Average only members of classes can be used to create averages containing only selected subsets of the particles.  This is most often useful after classification with programs pca and clusterPca, but can be used following any other operation which places the desired numeric class labels in column 20 of the motive list.  Multiple classes can be specified separated by commas, with ranges indicated with ":".

3.7. Action Buttons

After reviewing and selecting Systems and / or processors in the Processing Table, press Run to start processing.  At this point, Etomo will write the parameter file and run the PEET program prmParser, producing a series of command (.com) files which will be executed by processchunks to carry out the requested computations.  During processing, status will be indicated by progress bar at the top of the page. Additional status information may be obtained from the messages in the optionally visible log window, as well as from the Parallel processing section.  Note that once a run has been started it is permissible to close the Etomo gui if desired;  if the same project is later opened in a new Etomo window on the same system from which it was originally run, it will reconnect to any processes still running and update the status accordingly. 

When processing is complete, press Open Averaged Volumes in 3dmod to view the computed averages in 3dmod.  The Zap and Isosurface windows will be opened automatically.  Note that the isosurface threshold can be adjusted independently for each average.  Similarly, pressing Open Reference Files in 3dmod will allow you to view the references generated at each iteration.  You can also select Remake Averaged Volumes to recompute averages using the existing alignment, e.g. after changing the number of particles to average or other settings.

3.8 Resampling Existing Volumes / Models

In some cases it may be desirable to change the origin, image size, or voxel size of an existing IMOD volume and its corresponding model for use with PEET.  For example, one may wish to create a binned version to reduce memory requirements or to speed preliminary characterization.  This is easy using IMOD, but must be done in a specific way to preserve compatability with both PEET and 3dmod.

First, it is necessary to ensure the starting model contains information about the associated volume.  To do this, simply open the volume and the model together in 3dmod, and then re-save the model;  equivalently, use "imodtrans -i" to update the existing model, as illustrated below.  Second, create the modified volume using one or more IMOD tools such as "binvol" or "clip -resize".  Finally, generate the revised model either by a) opening the modified volume and the original model together in 3dmod, or b) by using "imodtrans -i".

For example, to perform 2-fold binning on a dataset in which the original volume and model were named "old.rec" and "old.mod", respectively, one might use

imodtrans -i old.rec old.mod old.mod
binvol -b 2 old.rec new.rec
imodtrans -i new.rec old.mod new.mod

If the voxel size has been changed, parameter file settings (e.g. mask radii) must be modified accordingly.  Similarly, offsets which are stored in columns 11-13 of the motive list will need to be scaled manually; motive lists are stored in comma-separated-value (csv) format, so a spreadsheet program can be used to make any changes required.