File Properties Dialog

This dialog box provides options for file storage. The ones at the top have to do with how to treat unsigned 16-bit integer data, produced by 16-bit cameras. The MRC standard used to be that MRC files in integer mode stored data as signed integers, ranging from -32768 to 32767. However, 16-bit cameras have the potential to produce numbers higher than 32767. SerialEM provides five (!) options for dealing with this possibility.

  1. Data can be saved in the now-standard unsigned integer mode (mode 6). This mode has been supported in IMOD for many years, but there may still be some MRC file reading software that does not read it. Unless you know that your data are destined for such software, this is the best option to use.
  2. Data can be saved in a signed integer file with truncation at 32767. If the counts in the images to be saved are not expected to be higher than 32767, this should not lose any data.
  3. Data can be divided by 2 when saved to file. This will keep the data positive (and thus proportional to transmitted electrons through the specimen) but will lose one bit of precision. Since the Gatan 16-bit cameras simply amplify the signal from the CCD chip 4 times as much as the 14-bit cameras, dividing by 2 probably just removes one bit of noise, which makes this an acceptable option for current US1000/US4000 cameras.  This option should not be used for electron counting data from a direct detector camera.
  4. Subtracting 32768 and storing with the signed file mode will preserve the camera values completely, at the cost of producing negative values in the output file.
  5. Data can be divided by 2 as soon as they are acquired, by selecting Divide 16-bit By 2 in the Camera menu. They will thus never appear to be unsigned integers, making the options for treatment of 16-bit data irrelevant. If the camera noise level is high enough to make this an acceptable procedure, then it has the additional advantage that it will preserve negative values in gain-normalized images. Such values can occur and be meaningful in images with very low counts if the bias of the camera drifts downwards after taking a dark reference.

Independent of these choices of how to treat the numeric values, this box provides five options for the format of the file in which data can be saved.

  1. MRC files, which can be processed in IMOD and many other software packages. These files can hold multiple images in a stack, and have both a header for standard information about the file and an extended header in which SerialEM stores a small amount of additional information associated with each image, referred to as metadata.  This header is updated after every section saved in case the program crashes; this feature also allows a file to be opened in 3dmod as the file is still being saved to.
  2. HDF files, which can be processed in IMOD and EMAN2.  HDF stands for "high performance data format", which is a misnomer in our context. These files can also hold a stack of images and have several advantages over MRC files: they contain all of the metadata for each image that has to be saved in a separate '.mdoc' file for MRC files (see below); images can be saved with ZIP compression; and the output file can be rapidly re-ordered in place for bidirectional and dose-symmetric series instead of having to copy to a new stack.  A disadvantage of these files, aside from their limited portability, is that the time to write the header with all these metadata is significant and increases with the number of sections.  SerialEM limits the frequency of header updates so that the time consumed per section written is limited by the property HdfUpdateTimePerSection.  The consequence is that not all the saved sections may be available when opening the file with 3dmod during acquisition or after some kinds of program crashes that do not close files properly.
  3. TIFF files for single images. This option is available only when saving an image (either when saving if there is no open file, or when using Save Single/Save to other in the File menu). The file is closed after writing one image.
  4. Series of single-image TIFF files listed in a text file. The text file uses the 'autodoc' format from IMOD and is assigned the standard extension '.idoc', which stands for 'image autodoc'. This file will include all of the information about each image that would ordinarily be stored in the extended header of an MRC file plus additional metadata. This autodoc file allows a series of TIFF files to be used interchangeably with an MRC stack: images can be read back by section number or overwritten, the file can be reopened, and montaging can also be done with this file format. These files can also be read as image stacks by virtually all programs in IMOD, as of version 4.10.43.  If you are just looking for an easy way to get a series of numbered TIFF files, you can ignore all these features and, when you get to the file chooser, think of it as asking for the root name of the series.
  5. JPEG files for single images. This option is available only when saving a non-floating point image (either when saving if there is no open file, or when using Save Single/Save to other in the File menu). Data are converted to bytes with truncation of extreme values.  The file is closed after writing one image.

When saving to an MRC file, the dialog provides another option for saving metadata, into a file in the autodoc format. This file will be given the full name of the image stack plus the extension '.mdoc', which stands for 'metadata autodoc'. When you open or read from an existing MRC file, the program will automatically search for an associated '.mdoc' file and read metadata from the file whenever reading an image, thus restoring more information about the image (e.g., defocus) than would otherwise be available.

Both kinds of autodoc files, image and metadata, contain identical information about each image, described in File Formats and Other Documentation.  The files also have information at the top that replicates some of the information that would appear in an MRC header.  This includes the 'Pixel spacing', which is the pixel size in Angstroms.  The IMOD programs Extracttilts and Extractpieces can extract the same information from either type of file as they can from the extended header of the MRC file.  In fact, they will automatically look for information in an associated autodoc file if the values are not found in the extended header. In addition, Newstack and Edmont can transfer information from the autodoc file associated with an input file to a new autodoc for an output file. 

The same metadata are stored in an HDF file in a way that makes transfers between an HDF file and an autodoc file easy.  Newstack and Edmont can transfer metadata between HDF files, into an HDF file from an autodoc file, or into an autodoc file from an HDF file.  Extracttilts can get tilt angles and other information from an HDF file, Extractpieces can get the piece list out of an HDF file, and Extracttilts can be used to create an autodoc file with a command like:

     extracttilts -attr filename.hdf  filename.hdf.mdoc  

Save images to:

Select whether to save data in an MRC file, an HDF file, a TIFF or JPEG  file holding a single image (not always available), or a series of TIFF files listed in an autodoc file, as explained above. When you select the latter option, the file chooser for specifying the file will attach the extension '.idoc' if you type in a root name without an extension. Thus, the autodoc will be named rootname.idoc, and the TIFF files will be named rootnameNNNN.tif, where 'NNNN' is a 4-digit number starting with 0000.  Your selection here will be saved in the settings file when saving a single image with the Save Single/Save to Other command in the File menu.

Type of compression in TIFF or HDF file:

Select whether TIFF files will be saved with no compression, the type of compression used in Zip and PNG files, LZW (Lempel-Ziv-Welch) compression, or JPEG compression, or select whether HDF files are saved with ZIP compression. ZIP and LZW are both loss-less compression methods that may not save much space.  JPEG compression does lose a small amount of information but can save much more space. Your selection here will be saved in your settings file, and separate values are maintained for saving a single image with the Save Single/Save to Other command versus other situations.  In contrast, the default values for most other options in this dialog are controlled by entries in the SerialEMproperties.txt file. 

Save non-float data as:

Allows integer data to be saved as bytes or 16-bit integers. If there is a 16-bit camera and values are not being divided by 2, or if there are any buffers with unsigned data in them, an option to save in unsigned integer mode is also available.  These options are apply only if you are saving a non-floating-point image, because there is no provision yet for conversion of floats to integers or bytes.

When saving 16-bit data:

Controls the treatment of unsigned 16-bit integer data being saved to a signed integer file. The three options are to truncate at 32767, divide by 2, or subtract 32768, as described above.

Percentage of pixels to truncate converting to bytes:

Controls the fraction of pixels that will be truncated as black (0) or white (255) when images are scaled to bytes.

Images are captured as 16-bit integers, and the default is to save them to file in this mode. Images can be saved as bytes (range 0 to 255), but this requires scaling of the intensity values into this limited range. To keep the scaling from being dominated by a few extreme values (e.g., from X-rays), which would compress the dynamic range for the rest of the image data, the program will find a scaling which truncates a certain percentage of extremely dark and light pixels at 0 and 255, respectively. Every image is scaled independently.

If you select byte storage, saving to a TIFF file with JPEG compression, or saving to a JPEG file, you can control the fraction of pixels that will be truncated with these text boxes.  Your entries here will be saved in your settings file, and separate values are maintained for saving a single image with the Save Single/Save to Other command versus other situations.  However, if the data are very noisy and the image is large, you may find that the contrast is poor when viewing a zoomed-down image that was saved as bytes, and that increasing these truncation values does not help much.  The solution is to reduce the image before saving, using the Reduce Image command in the Process menu or the ReduceImage script command.

Save in extended header:

Select whether to save tilt angle, C2 intensity value, stage position, magnification, or dose in the MRC file header. The dose is referred to as 'exposure dose' because it is the dose incurred during the time when the camera is exposed to the beam, excluding any pre-exposure of the specimen. All of these values can be extracted from an MRC file with the IMOD program Extracttilts (use 'extracttilts -h' to see options). Regardless of which items are selected, all of these values and more can be saved in a metadata autodoc file.

Maximum number of sections:

Indicate maximum number of sections that space is needed for.

The program can store various information for each picture in the 'extended header' of the MRC image file. The check boxes allow you to choose which information is stored; namely tilt angles, the stage position, and magnification. If you are montaging, piece coordinates are automatically stored in the header as well. Because these data are placed in the header, the program needs to know when first opening the file how much space to allocate for it. This is the reason for the text box where you specify 'Maximum number of sections'. The admonition is to be generous because if you run out of header space your only choice is to start a new file.

Save extra information in a '.mdoc' metadata file:

Select this option to have metadata saved in a text file with the name of the image file and extension '.mdoc', as described above. When a montage file is opened and the range of coordinates is too large to store in the extended header, a '.mdoc' file is created automatically.

Skip this dialog (turn skipping off in the File menu):

Select this option to avoid seeing this dialog every time you open a new file.  Instead, existing values will be used.  With this option selected, the dialog will open once per session of the program.  When you want to see this dialog again, you can turn off Skip File Properties Dlg in the File menu.

Cancel:

Cancel file setup (and whatever else is being set up, when this dialog is one in a sequence, such as when montaging).