JOURNAL OF STRUCTURAL BIOLOGY 116, 71-76 (1996)
ARTICLE NO. 0013

Copyright 1996 by Academic Press, Inc.
All rights of reproduction in any form reserved.

We have developed a computer software package, IMOD, as a tool for analyzing and viewing three-dimensional biological image data. IMOD is useful for studying and modeling data from tomographic, serial section, and optical section reconstructions. The software allows image data to be visualized by several different methods. Models of the image data can be visualized by volume or contour surface rendering and can yield quantitative information.

INTRODUCTION

IMOD was written to facilitate viewing of complex and convoluted 3-D image data of the type found in cellular and developmental biology; it affords a user several ways to view, analyze and model image data in order to obtain a better understanding of the structure under study. Most of the controls in IMOD were designed to have simple, yet complete, user interaction. The type of data most suitable for analysis by IMOD is 3-D image data from a volume of material, where each voxel has a single intensity value. Examples of these data are tomographic reconstructions, serial section images from an electron microscope, and optical sections from a light microscope.

MATERIALS AND METHODS

Preparation of the 3-D image data for use in IMOD. Tomographic data sets are calculated from a series of tilted views using the R-weighted back projection algorithm (Gilbert, 1972). The tilt series must first be aligned by creating a list of fiducial points, typically colloidal gold particles on the surface of the specimen. The program TILTALIGN uses the minimization approach described by Luther et al. (1988) to solve for the tilt angles and for the shifts, rotations, and size changes needed to align the views. TILTALIGN has two features that help to achieve accurate alignment. First, if necessary, individual fiducials can be labeled through only a subset of views, i.e., those in which a particular fiducial is seen clearly. (The program by Jing and Sachs (1991) also has this feature but cannot solve for tilt angles or section distortion.) Second, some parameters, such as ones that adjust for section distortion, can be expected to change slowly through the series. Several sequential views can be constrained to have the same, unknown value of such a parameter, substantially reducing the number of unknowns that must be solved for.

Serial section images must frequently be aligned using linear transformations that allow for overall change in size and stretch along an arbitrary axis as well as image translation and rotation (McDonald et al., 1991). Three methods (described more fully in Mastronarde et al., 1992) are available in separate programs for obtaining the needed transformations. The first program allows one to visualize the alignment between a pair of adjacent sections while manually adjusting the transformation applied to one of them. This program can also transform contours or points within a model, so that if a misalignment is noticed after modeling has begun, a correction can be made without having to remodel the newly aligned 3-D image. Second, one can make a list of fiducial points that correspond between adjacent sections and use these to solve for the linear transformations. Third, if enough image features are similar from one section to the next, the alignment can be done automatically, using a simplex algorithm to search for the transformation that minimizes a measure of image difference.

IMOD uses the MRC image file format developed at the Medical Research Council, Cambridge, England. We chose this format for IMOD because we had already developed numerous programs that used it to run on a VaxStation 3200 (McDonald et al., 1991). This format has the advantage of storing the 3-D image or stack of 2-D images in one file. Ancillary programs in the IMOD package can convert TIFF image files in the byte format, rgb files from Silicon Graphics Inc. (SGI), and raw image data to the MRC format. Once the 3-D image volume has been converted to MRC format and aligned, it is ready for viewing and modeling.

Viewing the 3-D image data with IMOD. IMOD allows 3-D image data to be visualized by several different methods. Each view is shown in its own window and is linked with others that show the same location in 3-D space from a different point of view. In this way, IMOD allows one to visualize 3-D image data interactively on a 2-D computer display. The windows representing these views are called the Zap (Zoom and pan) window (Fig. 1B), Model View window (Fig. 1C ), Tumbler window (Fig. 2A), Image Graph window (Fig. 2B), Slicer window (Fig. 2C), XYZ window (Fig. 2D), and Tilt window. All windows share the same current point within the 3-D data set and share the same brightness and contrast settings. IMOD's main program window (Fig. 1A) shows the status of the model being built, the current location of the displays in the 3-D image, and the current contrast settings. This window can also display text information, such as the areas within contours and error messages.

The Zap window allows the user to zoom and pan an image in real time. The view represented by the Zap window is a slice perpendicular to the Z-axis in the 3-D image volume (Fig. 1C). The Zap window overlays model contours and points on the image, and the model may be edited using a mouse. Panning from one area to another is quickly done by using the mouse to drag the image to the desired view.

The Model View window shows a Z-buffered wire frame rendering of model contour, line and point data with adjustable perspective (Fig. 1C). The model data can be visualized in cross-eyed or wall-eyed stereo, and adjustments for scaling, rotation, and translation can be made manually. The view of the model data can be rotating while one is editing it.

The XYZ window contains three views arranged like a mechanical drawing of a 3-D object (Fig. 2D). The lower left image is the same view shown in the Zap window; the top and right images are orthogonal planes cut through the volume at the current viewing point along the X- and Y-axes, respectively. The layout of this view allows one to know exactly what planes of the 3-D image data are being displayed.

The Slicer window displays a slice cut through the 3-D image data in any chosen orientation (Fig. 2C). The slice can be rotated about the Z-, Y-, and X-axes and can be zoomed up. The slice can be displayed quickly with nearest-neighbor interpolation or with higher quality, but slower, quadratic interpolation. The orientation of the slice in the 3-D volume is shown by lines that indicate the intersection of the slice's plane with the X, Y and Z normal planes displayed in the XYZ window (Fig. 2D)

The Tilt and Tumbler windows each show an orthographic summed voxel projection of the 3-D data; the former is based on a precalculated series of images, whereas the latter is calculated in real time. The Tilt window can display the original tilted views from which a tomogram was calculated, or a series of projections about the Y-axis of a 3-D volume. One can also use this window to view the current model contour within the 3-D data set in cross-eyed stereo. The Tumbler window shows a projection of a small volume around a selected point (Fig. 2A). The user can easily rotate the volume to any orientation.

Modeling areas of interest. The model structure in IMOD was designed to be flexible and fast by employing a data hierarchy. Models can contain several types of data including a list of scattered points, a list of contours describing a surface, a 3-D polygon mesh, a list of connected line segments, a list of points describing locations of similarly shaped items such as nuclear pores, or a list of scanline segments used for volume rendering. Data of one type can be grouped together into a single object that has attributes such as color, shininess, transparency, line width, point size, and name. All data types except the 3-D polygon mesh and the scanline list can be edited manually. Contours can be created by a semi-automated procedure that creates a contour around a selected area at a given threshold. Moreover, an entire contour-based model can be created by a fully automated threshold algorithm. Models generated automatically can be edited manually to improve the accuracy of the model.

Visualization of the modeled volume. Once a 3-D volume has been modeled, the model can be studied further. A projection of the 3-D model can be displayed either by surface rendering of the contour data or by volume rendering of the image data contained within the contours. Quantitative results, such as volumes, surface areas, nearest neighbor distances, and centroids can be obtained from the model data.

A smooth skin can be wrapped around the volume represented by the contours of a given object. The skin is represented by a surface mesh of connected triangles whose points are the same data points contained within the contours, along with additional points used to connect branches as needed. Adjacent contours are connected using an algorithm that minimizes the length of connecting lines between the two contours. Branches are handled by a method similar to that of Shantz (1981). Surface normals are also calculated, allowing a simple lighting algorithm to be used to display each object. The display program for this type of rendering, IMODV, is completely user-interactive. Volume rendering is done by a separate command-line-driven program that generates movies of the 3-D image data rotating about a fixed axis.

Other image processing tools. Several other command-line-driven programs are available to work with IMOD. Available operations include rotation, translation, stretching, cross-correlation, edge detection, calculating Fourier transforms, image averaging, filtering, and adjusting contrast and brightness. Most of these processes can be done in 2-D or 3-D. Montaged images can also be blended together to create a single large image.

RESULTS

High Voltage Electron Microscope Tomography

The tomograms shown in Fig. 2 were generated from two ±60° tilt series of a 0.25µm section of relaxed insect flight muscle. Micrographs were taken at 1° intervals around orthogonal axes and combined to create a single tomogram. The myofilaments were not perfectly aligned with the plane of section (Fig. 2D), so the Slicer window was used to bring a plane containing alternating thick and thin filaments into view (Fig. 2C).

Serial Section Reconstruction from Electron Micrographs

IMOD has also been used in several studies of 3-D reconstructions from serial sections imaged in an electron microscope. An example of one such reconstruction, of a mitotic nucleus from a wild-type yeast cell, can be seen in Fig. 3C. The small bright spheres represent the nuclear pores, which were modeled by manually selecting the center of each pore. One section was missing from the input data, but an option for the skinning program, IMODMESH, was used to fill in the missing data. We are now collecting statistics on the number and distribution of the nuclear pores from several models from differing strains of yeast.

Other forms of data

IMOD can be useful for studying any kind of isotropic 3-D images, such as ones from confocal microscopy or MRI scans. For example, image data of a 12-cell embryo of the nematode, C. elegans (Wood, 1988), were acquired from a light microscope using Nomarski differential interference contrast optics. The cell membranes and nuclei were modeled and then rendered as shown in Fig. 3D to illustrate the value of being able to make individual objects transparent. IMOD has thus proven useful for determining the connectivity between structures in light microscopic images as well as in electron tomographic data.

DISCUSSION

Advantages of using IMOD for viewing 3-D image data.

Most scientists who have worked with 3-D images have confronted the problem of studying and displaying their data in a reliable, comprehensible manner. IMOD was conceived and implemented in a laboratory that confronts these problems on a daily basis. The flexibility and power of this program reflects our strong belief that multiple approaches to display and analysis contribute to a valid understanding of 3-D data.

Each of the different views available within the IMOD program provides a different form of data visualization; our experience suggests that each form of display offers different insights into the 3-D structure under study. For example, slices viewed seriatim in the third dimension are extremely useful for developing a clear understanding of 2-D arrangements, but with such a display it is sometimes difficult to perceive the links between complex 3-D objects. Graphic contours drawn over 2-D data and assembled in 3-D can help one to perceive such linkages, but some form of rendering into a 3-D surface is often necessary to display the continuity clearly. On the other hand, it is not uncommon to introduce errors of interpretation while making such a model. Since identifying and correcting such errors in a rendered model is impossible, it is important to have effective ways of perceiving continuity in the original image data. IMOD's Slicer window, which allows one to cut a 3-D data set in a plane with easily chosen orientation and position, has been helpful in developing accurate models of cytoplasmic membranes. It has also helped in obtaining informative views of highly ordered structures, such as a myofibril. Finally, we have found that models are most accurate when they go through several generations of refinement. Our effort to develop semi-automated modeling routines, which provide draft structures that can subsequently be manually edited within IMOD, reflects our experience that a trained scientific observer is generally the best arbiter of difficult questions about the relationships between the model and the original data.

Display flexibility is also valuable in viewing models. One wants varying levels of model complexity, ranging from simple contours to wire frames to rendered surfaces. One also wants different levels of transparency to enhance viewing of certain model features.

An obvious value of building computer models of complex structures is the ease with which structural data may subsequently be quantified for objective analysis of hypothetical structural features. This ability has been particularly useful for us in our analysis of mitotic spindles (e.g., Mastronarde et al., 1993). IMOD's abilities to measure areas, volumes, and contour lengths, and to count objects of any model class constitute additional advantages of the software now available.

Requirements for using IMOD.

IMOD was developed on a Silicon Graphics Inc. (SGI) computer system and currently requires an SGI with 24-bit graphics in order to run. We plan to make IMOD available for more computer systems by using X-Windows/Motif and SGI's OpenGL for rendering of the graphic data. Using these programming interfaces would allow all 8-bit X-Windows-capable computers to run IMOD. The executable version of IMOD and all ancillary programs are freely available from the Boulder Laboratory for 3-D Fine Structure.

Future improvements to IMOD.

IMOD is a fairly new program, and we are working both on improvements to IMOD and on additional programs. We are currently seeking to enhance automated modeling by adding edge detection along with threshold detection (Yanowitz and Bruckstein, 1989) and by adding the ability to detect specific types of objects, such as the nuclear pores shown in Fig 3C. Users will be able to start outlining contours with a few points and have an interactive edge tracer complete the contour.

The Slicer window (Fig. 2C) will add the ability to display slices of different thicknesses and to view these thick slices in stereo. Volume rendering is now done statically, but it will be valuable to have this rendering occur in real time after the user selects the orientation of the model. Interactive or batch file control will be used to render 3-D movies automatically.

FIGURES

Note: If the figures appear too dark, please adjust the gamma level to 1.7.

FIG. 1.

FIG. 2.

FIG. 3.