Using eTomo

Rick Gaudette

Boulder Laboratory for 3D Electron Microscopy of Cells

University of Colorado



Contents

Usage
Using Templates
Using Parallel Processing
Other Interfaces in eTomo
Reporting bugs



Usage

eTomo provides a graphical user interface (GUI) to the IMOD Tomography package. The general user interface design idea on which eTomo is built is to present a separate panel for each distinct step in computing a tomogram with the IMOD package. This allows the user to focus on the current step in the processing sequence, presenting only the parameters and controls required for the current step.

Starting eTomo

eTomo can be started from the command line with the command

etomo


It may take a little while to come up; Java applications have a slow startup time. You can also open up a previous dataset by specifying the relative path to the .edf file on the command line, such as

etomo path/to/dataset.edf



The Front Page and the Setup Panel

When eTomo is first started, it shows its Front Page, which contains a set of buttons for starting different kinds of interfaces (see below). At this point you can either open a dataset that was previously being worked on by selecting it from the recent file list in the File menu, or you can use one of the buttons to open an interface for working on a new dataset.

Selecting New Tomogram will bring up the tomogram Setup Panel.  To start working on a new dataset you need to fill out the fields in the Setup Panel.

Press the Create com scripts button to move on to generating the tomogram. You can also use an existing set of com scripts by pressing the Use existing coms button; just be sure that the setup parameters you have entered exactly match those that were entered using copytomocoms.

To return to a previous tomogram, select the .edf (eTomo Data File) file from the most recently used list on the bottom of the File menu, or the Open... item on the File menu if the .edf is not listed in the most recently used list. The file dialog will let you navigate around to select the .edf file associated with the tomogram you wish to work on.

The Main Window

The main window consists of several areas: on the left is a column of buttons (Process Control Buttons) that allow you to select the particular step of the tomogram computation to work on, the top center is the Process Monitor that will inform you of the status of the current process or the last process completed and a button to kill the current process, the bottom status bar presents some basic information about the dataset you are working on. If you are working on a dual axis tomogram this layout will be repeated for both the A and B axes.

The Process Control Buttons are arranged in the suggested order of processing from top to bottom and are color coded to signify the state of the process. Red indicates that the process has not been started, magenta indicates that the process is in progress, and green indicates that the process has been completed. The state of the process is also indicated in text on the button. When one of the buttons is selected the right side of the window will fill in with information and fields associated with that process; these forms are called Process Panels. They allow you to modify the necessary parameters and execute the programs required by that processing step. The parameters and buttons on each Process Panel are typically laid out from top to bottom in the order they should be executed. When you execute a process (by pressing a button on one of the Process Panels) the Process Monitor will indicate what the process is doing and when it is complete.  In many cases, the button that you pressed will turn dark to indicate that the operation has been completed.

The text output of a particular process is written to a log file named for the particular process; this log file can be viewed by right clicking the mouse pointer over the window region associated with the process. This will open a menu that is split into three or four sections: the entries in the first section will open up the log files associated with the current process; some menus will have a second section with choices for plotting various kinds of information; the next section will allow you to open up man pages associated with the current process; the final menu section opens the general help documents. There is more information regarding online help resources in the section Accessing Help.

When you are done with all of the processes for a given step press the Done button; this will inform eTomo that the step is complete and will also move the GUI forward to the next step in the tomogram processing. Pressing the Postpone button will save all of the current parameters for that step and pressing Cancel will abort all of the changes made since the last process was run for that step. If you move between process panels by pressing the buttons in the left column, this will save the current parameters just as a Postpone does.  The final button of interest at the bottom of most processing panels is the Advanced button. This will present a much more complete list of parameters for each step, which are mostly needed for unique or extreme cases.

Since the number of parameters that a particular step requires can vary significantly, the amount of screen space the GUI needs to present that information also changes. The current version of the program will adjust the window size automatically to fit tightly around the Process Panel whenever you change screens or change the amount of information displayed (such as by changing to advanced mode).  You make this fitting occur automatically by selecting the Options/Fit Window menu item or using Ctrl-F hot key.  Whenever the window is too small to display the information, scroll bars will appear so that you can pan across the Process Panel.  To reduce the need for scrolling, several of the large panels are divided into sections that can be opened or closed independently with a button in the upper-left corner.  This button is initially "-" when the section is open, and becomes "+" when the section is closed.  In addition, a button may appear in the upper right corner to allow a section to be switched to advanced or basic mode independently of the others; this button will be labeled A in basic mode and B in advanced. 

If you are working on a dual axis tomogram, each tomogram is processed in a separate window.  The upper left corner of the Axis A window (or the Axis B window, if it is the only one open) will have buttons to allow you to switch between displaying one or the other axis or both axes.  There are also hot keys, Ctrl-A, Ctrl-B, and Ctrl-2, to display axis A, axis B, or both.

Every process panel has one or more buttons for starting 3dmod to display an image file.  To make it easier to deal with very large datasets and problems that require more control over the way that 3dmod is run, there are two ways to make 3dmod load in images binned by 2 or to make it open with its startup window, so that options can be selected.  One way is to right-click on one of these buttons, then select Open with startup window or Open binned by 2.  The other way is to select the corresponding items in the Options menu, which will make 3dmod open that way every time.

For many processing steps, it is also possible to run the operation and have the resulting images loaded into 3dmod automatically.  Right click on the button that starts the action (e.g., Generate Tomogram) and menu will pop up with the choices And open 3dmod, And open 3dmod with startup window, And open 3dmod binned by 2.  Select the desired option to start the operation.

It is possible to load more than one dataset into eTomo.  Under the File menu there are entries to open an existing dataset, open a new Setup window for tomogram processing, or open a new window to setup joining serial sections.  If you start working on more than one dataset by any of these routes, eTomo will place each dataset in a separate tab so that you can go between them easily.

The Options/Settings dialog presents several configuration options, such as one to change the font size being used, and one to switch to a compact display in which the buttons on the left are very small.  It may be necessary to close and reopen the program to see options like these take full effect.

The Project Log

When you start processing a tomogram, a separate window will be opened called a project log.  This window will receive selected output from various operations, and error messages that occur. You can also type your own notes into the window.  There is an entry under the View menu, and a hot key Ctrl-L, for hiding or making it visible again.  Even when it is hidden, output will be saved to it.  The contents are automatically saved to a file named with the dataset name and "_project.log".

Accessing Help

One of the significant features of the eTomo GUI is the immediate access to context sensitive online help. By context sensitive we mean that the help being presented is specific to the current processing problem at hand. Access to the online help is available through two methods. The first, commonly know as Tooltips, is accessed by leaving the mouse stationary over the parameter or other GUI object of interest. This will pop up a short text description of the parameter or function of the object. The object being described can be changed by moving the mouse around or the description can be removed by moving the mouse over a blank window region. The description will also be dismissed (disappear) after a short time if the mouse is not moved. Both the amount of time to delay displaying the Tooltip and the amount of time to keep the Tooltip visible can be set in the Options/Settings dialog.

The second method to access context sensitive help is by right clicking the mouse over the processing region of interest. This will open up a pop-up menu containing three sections. The top section presents menu items that will open up a new window containing a log file appropriate for the current processing step. The second section presents menu items that will open up the appropriate man pages for the current processing step. The final section provides access to the Tomography Guide, Serial Section Joining Guide, IMOD Guide and the 3dmod Introduction. If the Tomography Guide is selected the Guide will be opened up to the section that is appropriate for the current processing step.  All documentation is opened in the Qt Assistant, so that you can follow hyperlinks and return from them, as well as have access to the full IMOD documentation using the sidebar on the left.

How Your Work Is Saved

The current state of the processing and the value of any parameters you entered or changed (that are not saved in the underlying com scripts) are saved automatically to a file with the dataset name and the extension .edf.  If you are working on joining serial sections, the state is saved instead to a file with extension .ejf.  The state file is updated whenever you run a process, change process panels, or exit the program.  If you wish to save this file with a different name, use the File/Save as menu item.  Be aware that if you reopen an earlier version of the .edf file you may get unpredictable results.

Using Templates

Default settings for many tilt series processing parameters can be customized in files called template files.  To provide flexibility, eTomo is set up to work with three different levels of template files, although there is no need to use all three.  The three levels are:

All template files must have the extension .adoc.  Two system templates are distributed with IMOD.  One is plasticSection.adoc, which just turns on the use of a Sobel filter to achieve better centering when tracking gold markers in the bead tracker.  The other is cryoSample.adoc, which also turns on the Sobel filter centering but with an appropriate smoothing parameter, and has some other settings that we have found appropriate for cryo-ET.  One way to see the settings in a template is to start a data set, select the template, and create the com scripts.  At this point you can select Templates/Save System Template in the File menu (to see a system template).  The items in the existing template are all checked for inclusion in a new template.  You may have to check Show only included to see just the items in the template.

The template files form a hierarchy and are considered in the order Scope - System - User, with later ones overriding earlier ones.  In other words, if the same parameter is found in more than one template, the setting in the last template is the one that is used.

The template editor can be used to save template files containing any of the parameters supported in the editor, but currently templates can be saved only from an actual data set.  The parameters are organized into sections corresponding to different processing steps; there is a set of checkboxes that control whether eahc section is open or closed.  Whether a parameter is saved is determined by the state of the Include checkbox to the left of each entry.  You can set these however you want, but the state of the Includes when you first open it depends on which kind of template you select to save.  If you select to save a Scope or System template, items will be checked for inclusion only if the data set was started with a Scope or System template, respectively, and the items checked will be the ones in that template.  Other items shown grayed out will be ones that eTomo thinks have changed since the data set was started.  The same applies if you go to save a User template and started the data set with a User template.  However, if no User template was used, the program will shows items it thinks have changed and select some items for inclusion based on how they are coded in a master file of parameters, excluding ones in Scope or System templates.  Unfortunately, there are still numerous problems with the program's ability to tell whether an item has changed.  In particular, it will think almost everything has changed for a data set started with a version of IMOD before 4.6.20, so it is best to open this editor only with a recently started data set.

The template editor is still under construction; we plan changes that will make it clear whether a parameter is contained in a lower-level template than the one being viewed, and we may change the logic by which items are selected for inclusion.  For now, it is important to look through all the sections of the editor and be sure you intend to save what is being saved.

Using Parallel Processing

Several steps in the processing can now be split into chunks and run in parallel on multiple machines or processors, including CTF correction, tomogram generation, and the final stage of combining volumes in a dual-axis dataset.  This capability is enabled if the machine on which you are running eTomo has a configuration file called cpu.adoc located in the IMOD calibration directory, which is /usr/local/ImodCalib by default.  Run "man cpuadoc" for a full description of how to set up such a file.  A two-line file is sufficient to use a single multiprocessor system.  There are two other alternatives when using a single system.  First, you can open the Options/Settings dialog, turn on Enable Parallel processing, and then fill in the number of CPUs.  Second, you can enable parallel processing by setting an environment variable IMOD_PROCESSORS to the number of processors in the computer.  For example, if you have 4 processors and use tcsh for your shell, place
  setenv IMOD_PROCESSORS 4
in your .cshrc file, or if you use the bash shell, place
  export IMOD_PROCESSORS=4
in your .bashrc file.  If you are using multiple machines, you will also need to be able to log in via ssh without entering a password (see below) and run IMOD on all of the other computers that you want to use. 

When parallel processing is available, you will be able to select whether to use it with checkboxes on panels where it can be used.  With it selected, a parallel processing table and controls will appear in the Process Monitor area of the window.  The table will show computers, their properties, and their current loads.  When you are ready to run the given operation, select the computers and number of CPUs that you want to use, then press the appropriate action button (e.g., Generate Tomogram).  The table will show how many chunks are completed on each machine.  The parallel processing area includes several controls:

To be able to log in via ssh without a password, you need to generate ssh keys that are present on each machine.  First run
   ssh-keygen -t dsa
and type Enter for all of the queries.  Then do
    cp $HOME/.ssh/id_dsa.pub $HOME/.ssh/authorized_keys2
If necessary, distribute this authorized_keys2 file to the .ssh directories on other machines that do not have the same home directory.  The three different kinds of platforms (Linux/Unix, Mac OSX, Windows) may require separately generated keys.  If this seems to be the case, run ssh-keygen once on each type of machine then combine the .ssh/id_dsa.pub files from each into one authorized_keys2 file.  Distribute this combined authorized_keys2 file to the .ssh directories.
 
In addition to being done on multiple processors, tomogram generation can now be done on the graphics processing unit (GPU) of an NVIDIA graphics card. See the IMOD User's Guide for instructions on setting up a machine to use the GPU. Reconstructions can also be done in parallel using GPUs on several computers, although it is not yet possible to use more than one GPU on a single computer.

Other Interfaces in Etomo

eTomo provides a good framework for developing new interfaces for managing parameter files and running processes, so we continue to add interfaces for operations other than generating a tomogram.  Interfaces for different kinds of projects are available on the Front Page and through the File menu via the various selections starting with New.  They include interfaces for:

In addition to these kinds of projects, which have a saved state and can be resumed, interfaces for simpler operations can be started from the Tools menu as well as from the Front Page of eTomo.  Currently there are two tools, for flattening a tomogram without opening up its original dataset and for testing the GPU.

Reporting bugs

We very much appreciate feedback from our users; this includes bug reports, suggestions for enhancements and general comments on what ideas or use models do or do not work.  It is often not initially obvious to either you or us whether a problem is occurring because of bug in eTomo, a bug in the rest of IMOD, or other difficulties with a particular dataset.  To help diagnose problems, we have developed a script, tomosnapshot, that will collect system information and package up many of the small files that reflect the state of the processing.  This script is now able to collect files for all of the kinds of projects that have a saved state (i.e, joining, PEET, NAD as well as tomogram generation). The best way to run this is to cd to the directory where the data are, start eTomo with the project file for the given data set, repeat the series of actions that give the error (if this is reasonably fast), then select the menu item File/Run Tomosnapshot.  You can then send the resulting tar file (setname-snapshot), along with a description of what you did and the undesired result, to  David Mastronarde or Sue Held (see web site for addresses).  You can also report bugs and make suggestions or comments by posting a message on the IMOD mailing list at the University of Colorado. This allows others to observe the discussion and see if they are encountering similar problems; but please don't post your snapshot file to the list. To subscribe to the IMOD mailing list send an email to listproc@lists.Colorado.edu with the contents of the message being

subscribe IMOD your-full-name

replacing your-full-name with your name.