Boulder Laboratory for 3D Electron Microscopy of Cells
University of Colorado
Using Parallel Processing
Other Interfaces in eTomo
eTomo provides a graphical user interface (GUI) to the IMOD Tomography package. The general user interface design idea on which eTomo is built is to present a separate panel for each distinct step in computing a tomogram with the IMOD package. This allows the user to focus on the current step in the processing sequence, presenting only the parameters and controls required for the current step.
eTomo can be started from the command line with the command
It may take a little while to come up; Java applications have a slow startup time. You can also open up a previous dataset by specifying the relative path to the .edf file on the command line, such as
When eTomo is first started, it shows its Front Page, which contains a set of buttons for starting different kinds of interfaces (see below). At this point you can either open a dataset that was previously being worked on by selecting it from the recent file list in the File menu, or you can use one of the buttons to open an interface for working on a new dataset.
Selecting New Tomogram will bring up the tomogram Setup Panel. To start working on a new dataset you need to fill out the fields in the Setup Panel.
The Dataset Name is the name of the file containing the projection sequence (or the name of one of these files in the case of a dual axis tomogram). You can enter the Dataset Name by opening the file selection dialog window using the folder labeled button associated with the Dataset Name field. Alternatively, you may enter the relative path to the projection sequence file directly into the edit box.
The Backup Directory is optionally available to save small working files every time you run a procedure. You can leave this field empty if you do not wish to use a backup directory.
Specify the pixel size, fiducial size and image
rotation in the appropriate fields. If the header of your image
file includes pixel size and/or image rotation information, press the Scan Header button to extract these
Specify the source for the tilt angles for either one or both axes, as appropriate. You can also optionally specify individual projections to exclude from the processing. The syntax for this exclude list is a comma separated list of ranges (i.e. 1,4-5,60-70)
Press the Create com scripts button to move on to generating the tomogram. You can also use an existing set of com scripts by pressing the Use existing coms button; just be sure that the setup parameters you have entered exactly match those that were entered using copytomocoms.
To return to a previous tomogram, select the .edf (eTomo Data File) file from the most recently used list on the bottom of the File menu, or the Open... item on the File menu if the .edf is not listed in the most recently used list. The file dialog will let you navigate around to select the .edf file associated with the tomogram you wish to work on.
The main window consists of several areas: on the left is a column of buttons (Process Control Buttons) that allow you to select the particular step of the tomogram computation to work on, the top center is the Process Monitor that will inform you of the status of the current process or the last process completed and a button to kill the current process, the bottom status bar presents some basic information about the dataset you are working on. If you are working on a dual axis tomogram this layout will be repeated for both the A and B axes.
The Process Control Buttons are arranged in
the suggested order of processing from top to bottom and are color
coded to signify the state of the process. Red indicates that the
process has not been started, magenta indicates that the process is
in progress, and green indicates that the process has been completed.
The state of the process is also indicated in text on the button.
When one of the buttons is selected the right side of the window will
fill in with information and fields associated with that process;
these forms are called Process Panels. They allow you to
modify the necessary parameters and execute the programs required by
that processing step. The parameters and buttons on each Process
Panel are typically laid out from top to bottom in the order they
should be executed. When you execute a process (by pressing a button
on one of the Process Panels) the Process Monitor will
indicate what the process is doing and when it is complete. In
many cases, the button that you pressed will turn dark to indicate that
the operation has been completed.
The text output of a particular process is written to a log file named for the particular process; this log file can be viewed by right clicking the mouse pointer over the window region associated with the process. This will open a menu that is split into three sections: the entries in the first section will open up the log files associated with the current process; the second section will allow you to open up man pages associated with the current process; the final menu section opens the general help documents. There is more information regarding online help resources in the section Accessing Help.
When you are done with all of the processes for a given step press the Done button; this will inform eTomo that the step is complete and will also move the GUI forward to the next step in the tomogram processing. Pressing the Postpone button will save all of the current parameters for that step and pressing Cancel will abort all of the changes made since the last process was run for that step. If you move between process panels by pressing the buttons in the left column, this will save the current parameters just as a Postpone does. The final button of interest at the bottom of most processing panels is the Advanced button. This will present a much more complete list of parameters for each step, which are mostly needed for unique or extreme cases.
Since the number of parameters that a particular
step requires can vary significantly, the amount
of screen space the
GUI needs to present that information also changes. The current version
of the program will adjust the window size automatically to fit tightly
around the Process Panel whenever you change
screens or change the amount of information displayed (such as by
changing to advanced mode). You make this fitting occur
automatically by selecting the Options/Fit Window
menu item or using Ctrl-F hot key. Whenever the window is
too small to display the information, scroll bars will appear so that
you can pan across the Process Panel. To reduce the need
for scrolling, several of the large panels are divided into sections
that can be opened or closed independently with a button in the
upper-left corner. This button is initially "-" when the section is open, and
becomes "+" when the section is
closed. In addition, a
button may appear in the upper right corner to allow a section to be
switched to advanced or basic mode independently of the others; this
button will be labeled A in
basic mode and B in
If you are working on a dual axis tomogram, each
tomogram is processed in a separate window. The upper left corner
of the Axis A window (or the Axis B window, if it is the only one
open) will have buttons to allow you to switch between displaying one
or the other axis or both axes. There are also hot keys, Ctrl-A,
Ctrl-B, and Ctrl-2, to display axis A, axis B, or both.
Every process panel has one or more buttons for
starting 3dmod to display an image file. To make it easier to
deal with very large datasets and problems that require more control
over the way that 3dmod is run, there are two ways to make 3dmod load
in images binned by 2 or to make it open with its startup window, so
that options can be selected. One way is to right-click on one of
these buttons, then select Open with
startup window or Open binned
by 2. The other way is to select the corresponding items
in the Options menu, which
will make 3dmod open that way every time.
For many processing steps, it is also possible to
run the operation and have the resulting images loaded into 3dmod
automatically. Right click on the button that starts the action
(e.g., Generate Tomogram) and menu will pop up with the choices And open 3dmod, And open 3dmod with startup window, And open 3dmod binned by 2. Select the desired
option to start the operation.
It is possible to load more than one dataset into
eTomo. Under the File
menu there are entries to open an existing dataset, open a new Setup
window for tomogram processing, or open a new window to setup joining
serial sections. If you start working on more than one dataset
by any of these routes, eTomo will place each dataset in a separate
tab so that you can go between them easily.
dialog presents several configuration options, such as one to change
the font size being used, and one to switch to a compact display in
which the buttons on the left are very small. It may be necessary
to close and reopen the program to see options like these take full
When you start processing a tomogram, a separate window will be opened called a project log. This window will receive selected output from various operations, and error messages that occur. You can also type your own notes into the window. There is an entry under the View menu, and a hot key Ctrl-L, for hiding or making it visible again. Even when it is hidden, output will be saved to it. The contents are automatically saved to a file named with the dataset name and "_project.log".
One of the significant features of the eTomo GUI is the immediate access to context sensitive online help. By context sensitive we mean that the help being presented is specific to the current processing problem at hand. Access to the online help is available through two methods. The first, commonly know as Tooltips, is accessed by leaving the mouse stationary over the parameter or other GUI object of interest. This will pop up a short text description of the parameter or function of the object. The object being described can be changed by moving the mouse around or the description can be removed by moving the mouse over a blank window region. The description will also be dismissed (disappear) after a short time if the mouse is not moved. Both the amount of time to delay displaying the Tooltip and the amount of time to keep the Tooltip visible can be set in the Options/Settings dialog.
The second method to access context sensitive help
is by right clicking the mouse over the processing region of
interest. This will open up a pop-up menu containing three sections.
The top section presents menu items that will open up a new window
containing a log file appropriate for the current processing step.
The second section presents menu items that will open up the
appropriate man pages for the current processing step. The final
section provides access to the Tomography Guide, Serial Section Joining
Guide, IMOD Guide and the 3dmod Introduction. If
the Tomography Guide is selected the Guide will be opened up to the
section that is appropriate for the current processing step. All
documentation is opened in the Qt Assistant, so that you can follow
hyperlinks and return from them, as well as have access to the full
IMOD documentation using the sidebar on the left.
The current state of the
processing and the value of any parameters you entered or changed (that
are not saved in the underlying com scripts) are saved automatically to
a file with the dataset name and the extension .edf. If you are
working on joining serial sections, the state is saved instead to a
file with extension .ejf. The state file is updated whenever you
run a process, change process panels, or exit the program. If you
wish to save this file with a different name, use the File/Save as
menu item. Be aware that if you reopen an earlier version of the
.edf file you may get unpredictable results.
Several steps in
the processing can now be split into chunks and run in parallel on
multiple machines or processors, including CTF correction, tomogram
generation, and the final
stage of combining volumes in a dual-axis dataset. This
capability is enabled if the machine on which you are running eTomo has
a configuration file called cpu.adoc located in the IMOD calibration
directory, which is /usr/local/ImodCalib by default. Run "man
cpuadoc" for a full description of how to set up such a file. A
two-line file is sufficient to use a single multiprocessor
system. There are two other alternatives when using a single
system. First, you can open the Options/Settings
dialog, turn on Enable Parallel processing, and then
fill in the number of CPUs. Second, you can enable parallel
setting an environment variable IMOD_PROCESSORS to the number of
processors in the computer. For example, if you have 4 processors
and use tcsh for your shell, place
setenv IMOD_PROCESSORS 4
in your .cshrc file, or if you use the bash shell, place
in your .bashrc file. If you are using multiple machines, you will also need to be able to log in via ssh without entering a password (see below) and run IMOD on all of the other computers that you want to use.
processing is available, you will be able to select
whether to use it with checkboxes on panels where it can be used.
With it selected, a parallel
processing table and controls will appear in the Process Monitor area
of the window. The table will show computers, their properties,
and their current loads. When you are ready to run the given
operation, select the computers and number of CPUs that you want to
use, then press the appropriate action button (e.g., Generate Tomogram). The table
will show how many chunks are completed on each machine. The
parallel processing area includes several controls:
The Kill button, as usual, will kill all running processes
The Pause button will stop the processing more gracefully by allowing all currently running chunks to finish.
The Resume button can be used after either a Kill or a Pause to resume processing without having to redo any chunks that were already completed.
The Save as Defaults button will save your current selection of computers as the default for future sessions.
The Nice control lets you select the "niceness" of your jobs; the higher the value, the lower the priority. The highest value is 19, which will compete with screen savers, while a value of 0 will compete equally with normal interactive usage of the machine.
The +/- button in the upper left can be used to minimize the table.
The </> button in the upper right
can be used to make the table smaller, so that it shows only the
selected computers and information needed while running.
To be able
to log in via ssh without a password, you need to generate ssh keys
that are present on each machine. First run
ssh-keygen -t dsa
and type Enter for all of the queries. Then do
cp $HOME/.ssh/id_dsa.pub $HOME/.ssh/authorized_keys2
If necessary, distribute this authorized_keys2 file to the .ssh directories on other machines that do not have the same home directory. The three different kinds of platforms (Linux/Unix, Mac OSX, Windows) may require separately generated keys. If this seems to be the case, run ssh-keygen once on each type of machine then combine the .ssh/id_dsa.pub files from each into one authorized_keys2 file. Distribute this combined authorized_keys2 file to the .ssh directories.
In addition to being done on multiple processors, tomogram generation can now be done on the graphics processing unit (GPU) of an NVIDIA graphics card. See the IMOD User's Guide for instructions on setting up a machine to use the GPU. Reconstructions can also be done in parallel using GPUs on several computers, although it is not yet possible to use more than one GPU on a single computer.
provides a good framework for developing new interfaces for managing
parameter files and running processes, so we continue to add interfaces
for operations other than generating a tomogram.
Interfaces for different kinds of projects are available on the Front Page
and through the File menu via the various selections
starting with New. They include interfaces
together tomograms from serial sections, opened by selecting File/New Join. See
the Guide to Joining Tomograms
for instructions. The processing state is saved to a file with
extension .ejf, which you can
reopen to resume your work.
Running the PEET volume alignment and averaging software, opened by selecting File/New PEET. The processing state is saved to a file with extension .epe, which you can reopen to resume working on a dataset.
parallel processing on an appropriate set of command files, such as the
ones created by the programs chunksetup,
sirtsetup, and splitcorrection. This
interface is opened by selecting File/New
Parallel Process then pressing the Generic Parallel Process
button. The program state is saved in a file with extension .epp, which you can reopen to
process files with the same name as previously.
by nonlinear anisotropic diffusion. This procedure often requires
finding the right parameter settings with a small test volume before
running on the full volume, so the interface provides tools for doing
this conveniently. All of the temporary and working files are
kept in a subdirectory that you can remove at the end. The
interface is opened by selecting File/New Parallel Process then
pressing the Nonlinear Anisotropic
Diffusion button. The program state is
saved in a file with extension .epp.
See the nad_eed_3d man page for
In addition to these kinds of projects, which have a saved state and can be resumed, interfaces for simpler operations can be started from the Tools menu as well as from the Front Page of eTomo. Currently there is just one tool, for flattening a tomogram without opening up its original dataset.
We very much appreciate feedback from our users; this includes bug reports, suggestions for enhancements and general comments on what ideas or use models do or do not work. It is often not initially obvious to either you or us whether a problem is occurring because of bug in eTomo, a bug in the rest of IMOD, or other difficulties with a particular dataset. To help diagnose problems, we have developed a script, tomosnapshot, that will collect system information and package up many of the small files that reflect the state of the processing. This script is now able to collect files for all of the kinds of projects that have a saved state (i.e, joining, PEET, NAD as well as tomogram generation). The best way to run this is to cd to the directory where the data are, start eTomo with the project file for the given data set, repeat the series of actions that give the error (if this is reasonably fast), then select the menu item File/Run Tomosnapshot. You can then send the resulting tar file (setname-snapshot), along with a description of what you did and the undesired result, to David Mastronarde or Sue Held (see web site for addresses). You can also report bugs and make suggestions or comments by posting a message on the IMOD mailing list at the University of Colorado. This allows others to observe the discussion and see if they are encountering similar problems; but please don't post your snapshot file to the list. To subscribe to the IMOD mailing list send an email to listproc@lists.Colorado.edu with the contents of the message being
subscribe IMOD your-full-name
replacing your-full-name with your name.