File Formats and Other Documentation

Image Metadata Files
Navigator Files
Importing Externally Defined Navigator Items
Settings and Calibration File Entries
Other Notes

Image Metadata Files

SerialEM stores metadata about images in text files in the IMOD "autodoc" format.  This format consists of keyword-value pairs organized into blocks called sections.  These sections can be of different types with different kinds of data in them, although SerialEM's files use essentially only one kind of section.  A section begins with a bracketed key-value pair:
      [sectionType = name]
where the section "name" or value will typically be unique but is not required to be so by the IMOD autodoc reader.  Lines below a section header of the form
      key = value
provide data associated with that section.  In addition, key-value pairs can occur at the beginning of the file, before any section header, and these are referred to as global values.

Although all values are stored as text strings in the autodoc file, software that expects information of a particular kind will often convert these strings to or from one or more floating point or integer values.  The autodoc module in the IMOD library libcfshr contains functions for storing or retrieving one, two, three, or many floating point or integer values in specified sections of the file.  The autodoc format was chosen for SerialEM because of the ready availability of this library, the convenience of using such functions, and compatibility with the autodoc-reading capability in IMOD.  IMOD also has a simple Python module, autodoc.py, with a function to take a set of autodoc lines and produce a top-level dictionary of the different types of sections, keyed by 'sectionType', with each entry containing an array of dictionaries, one for each section.  Other functions will return a list of integer or floating point values from the value in a key-value pair.

There are two kinds of image-associated autodoc files produced by SerialEM:

1) The ".mdoc" file provides data about an MRC file and has the same name as the image file, with the additional extension ".mdoc".  In these files, the section type is "ZValue" and the name for each section is the Z value of the image in the file, numbered from 0.

2) The ".idoc" file provides information that allows a series of single-image TIFF files to be treated interchangeably with MRC files.  The section type here is "Image" and the name for each section is the name of the TIFF file.

Global data replicate some of the information that is in an MRC file header.  In addition, the titles for an MRC file are stored in section headers of the type "T".

It is important to realize that no steps in IMOD tilt series processing rely on information directly from an .mdoc file or require the presence of one.  An .mdoc file is used for dose weighting, but only if the user specifies that as the source of dose information.  It can be used with Alignframes to create a tilt series from frame files.  Extracttilts and Extractpieces will fall back to looking for an .mdoc file if tilt angle or piece list values, respectively, are not in the image file header.

 The IMOD library used by SerialEM uses the same internal autodoc structure to access the metadata in HDF files.  It is thus simple to extract all of these metadata information into a text file in the autodoc format, with a command like:

    extracttilts -attr filename.hdf  filename.hdf.mdoc 

Global Data
DataMode MRC file mode
ImageSize X and Y size of images
Montage 1 if the file is a montage
ImageSeries 1 for an .idoc file describing a series of single-image files
ImageFile In a .mdoc file, contains the name of the image file
PixelSpacing Pixel spacing in Angstroms that would be in MRC header (or in 1/Angstroms for diffraction images)
Section Data
TiltAngle Tilt angle in degrees
PieceCoordinates Pixel coordinates in X and Y and section Z value for montage piece
StagePosition X and Y stage position in microns
StageZ Z stage position in microns
Magnification "Film" magnification value
CameraLength Camera length in mm for image in diffraction mode
MagIndex Magnification index
Intensity Raw intensity value between 0 and 1
SuperMontCoords X and Y pixel coordinates of frame in a supermontage
PixelSpacing Pixel spacing in Angstroms for individual image (or in 1/Angstroms for diffraction image)
ExposureDose Dose on specimen during camera exposure time in electrons/sq. A, excluding pre-exposure time
DoseRate Dose rate to the camera, in electrons per physical pixel per second
SpotSize Microscope spot size
ProbeMode For Thermo/FEI scopes, 1 for microprobe or 0 for nanoprobe
Defocus Relative defocus readout from microscope (microns)
TargetDefocus Current target for autofocus (microns)
ImageShift X and Y image shift in basic units (close to microns for Thermo/FEI, much larger than microns for JEOL)
RotationAngle Rotation of image from having the tilt axis along its X axis (CCW +).  This corresponds to the definition of the approximate rotation angles reported by Thermo/FEI scripting.  It is different from the 'Tilt axis angle' in the second title of the file header and .mdoc, which follows the IMOD convention of tilt axis rotation relative to the Y axis of the image. Thus, the latter will be 90 degrees less than the RotationAngle in a system with no axis inversions.
ExposureTime Image exposure time
Binning Image binning on the camera
UsingCDS 1 if image was taken with CDS mode on
CameraIndex Index of the CameraProperties section for the camera used
DividedBy2 1 if image was divided by 2
RotationAndFlip RotationAndFlip property of K2 or K3 camera
LowDoseConSet Control set index plus 1 if image taken in Low Dose, or negative of index if not in Low Dose (index 0-6 for View, Focus, Trial, Record, Preview, Search, Mont-map; 7 for tracking, 8 for montage)
MinMaxMean Minimum, maximum, and mean value for this image
PriorRecordDose For an image in a tilt series taken in Low Dose mode, the cumulative dose in the Record area prior to this image, including Record and Preview images and View images taken in tasks
XedgeDxy Edge displacement in X and Y for montage piece to the right of this piece
YedgeDxy Edge displacement in X and Y for piece above this piece
XedgeDxyVS Edge displacement for piece to right, computed with very sloppy option
YedgeDxyVS Edge displacement for piece above, computed with very sloppy option
XedgeMaxSD 95th percentile value of local SD in area correlated to find shift of piece to the right.
YedgeMaxSD 95th percentile value of local SD in area correlated to find shift of piece above.
XedgeMaxSDVS 95th percentile value of local SD in area correlated with very sloppy parameters to find shift of piece to the right.
YedgeMaxSDVS &95th percentile value of local SD in area correlated with very sloppy parameters to find shift of piece above.
StageOffsets When aligning montage pieces with image shift, this is the effective stage position (actual position plus the stage equivalent to the image shift adjustment) minus the nominal stage position of the piece.
AlignedPieceCoords Piece coordinates adjusted by the solved shifts for each piece when pieces are aligned in overview with 'Very sloppy' option off.
AlignedPieceCoordsVS Piece coordinates adjusted by the solved shifts for each piece when pieces are aligned in overview with 'Very sloppy' option on.
SubFramePath Directory or file in which subframes of exposure were stored
NumSubFrames Number of subframes stored
FrameDosesAndNumbers Dose per frame in electrons per square Angstrom followed by number of frames at that dose; variable-sized frame sums will have multiple pairs of such values
DateTime Time and date of image acquisition; the format of the date is dd-Mon-yy regardless of locale.  The year was changed to yyyy in SerialEM 4.1, 7/15/22.
TimeStamp Time of image acquisition in integer seconds since Jan. 1, 2020.
NavigatorLabel Label of Navigator item, added if Acquire at Items is being run and a tilt series is not
FilterSlitAndLoss Energy filter slit width and energy loss if slit is in, 0 0 otherwise
ChannelName Detector name for a STEM image
MultishotHoleAndPosition Hole identifier and position within hole when taking multiple record images; peripheral positions are numbered from 1 and the center is numbered 0.  For a regular pattern, hole identifiers are a pair of numbers relative to the center of the pattern; for a custom pattern, they are simply numbers starting at 1.
CameraPixelSize For diffraction image, the size of the image pixel on the camera in microns, namely the physical chip pixel size times the binning.
Voltage High voltage in kV for diffraction image.
FlashCounter Cumulative counter of number of FEG flashes done by SerialEM.
FEGCurrent The FEG beam current in nanoamps, which is read from the scope every 5 minutes.
Direct Electron Specific Section Data
DE12-ServerSoftwareVersion Server software version
DE12-PreexposureTime(s) Pre-exposure time
DE12-TotalNumberOfFrames Total number of frames in acquisition
DE12-FramesPerSecond Frame rate in frames per second
DE12-CameraPosition Whether camera was retracted or inserted
DE12-ProtectionCoverMode Whether protection cover is kept open or open and closed for this shot
DE12-ProtectionCoverOpenDelay(ms) Delay after opening prtection cover
DE12-TemperatureDetector(C) Detector temperature
DE12-FaradayPlatePeakReading(pA/cm2) Current reading from Faraday plate during exposure
DE12-SensorModuleSerialNumber Serial number of sensor
DE12-SensorReadoutDelay(ms) Delay before starting to read out sensor during exposure
DE12-IgnoredFramesInSummedImage Number of frames ignored in a summed image
Additional Data in MontSection
FullMontSize Full size of montage in X and Y if all pieces are present
BufISXY Image shift associated with overview buffer
ProbeMode 1 for microporbe, 0 for nanoprobe
MoveStage 1 if for a stage montage
ConSetUsed Control set used for the montage (0 = View, 3 = Record, 5 = Search, 6 = MontMap
MontBacklash The nominal backlash for this montage, or 0 0 if using an anchor at the center
ValidBacklash The nominal backlash for this montage, or 0 0 if using an anchor at the center or doing a zigzag pattern
DriftSettling Drift settling in control set used
CameraModes Shutter mode and read mode
FocusOffset Focus offset for low dose View or Search, assigned to map item DefocusOffset if a map is made
NetViewShifts Image shift offsets for low dose View or Search
ViewBeamShifts Beam shift offsets for low dose View or Search
ViewBeamTilts Incremental beam tilts for low dose View or Search
ViewDefocus Focus offset for low dose View or Search, assigned to image buffer when reading in
Alpha Alpha value on JEOL
FilterState 0/1 if slit is out/in, and slit width
AdjustedOverlaps Overlap between pieces when using multiple Record routine and overlap is increased by tilt-foreshortening
XEdgeExpectedShifts Expected shift in X and Y in overlap zones between pieces in X direction when using multiple Record routine
YEdgeExpectedShifts Expected shift in X and Y in overlap zones between pieces in Y direction when using multiple Record routine

 

If an ".idoc" file is being constructed for a set of TIFF files, the essential elements are the "DataMode", "ImageSize", and "ImageSeries" entries, as well as a section header for each image.  The global "PixelSpacing" may be helpful in some contexts.  For a montage, the "Montage" entry must be present with the value 1, and there must be a "PieceCoordinates" entry in each section.  The X and Y values must fall on a regularly spaced grid, so that the coordinates can be analyzed to deduce the spacing and overlap between pieces unambiguously.  Not all pieces in a rectangle need to be present, and there need not be pieces at 0.  Pixel coordinates are in a right-handed system despite the ubiquitous inversion of images in Y.

If the alignment between montage pieces is known, this information can be placed in both the XedgeDxy/YedgeDxy and the XedgeDxyVS/YedgeDxyVS entries, so that the alignment can be used by SerialEM regardless of whether the Very Sloppy option is checked.  The entries are the amount that an "upper" piece, i.e., one to the right (for X) or above (for Y), is displaced from being aligned in the overlap zone.  This is the negative of the amount that the upper piece needs to be shifted to align the two pieces in the overlap zone.  The program uses the IMOD library routine findPieceShifts to solve for the best shifts to be applied to each piece to bring all of them into best alignment.  There is a fixed (but easily raised) limit to the size of montage for which this solution will be found, based on the nominal number of pieces in X and Y rather than the actual number.

Navigator Files

In SerialEM 3.6, the Navigator output was switched from one tab-delimited line per item to an autodoc format, with an option to output as XML instead.  XML or autodoc files can be read back in, as well as the tab-delimited files from earlier versions.  There are two lines of global data,
  AdocVersion = x.xx
  LastSavedAs = x\y.nav
where the second is the full absolute path and name of the file as it was last saved.  Then every item is in a section of type 'Item' and with its label string as the value.  The following table lists all of the keys that can occur in a section, the kind of value that is expected, and whether it is required, or if it is not required, what default value is set.

Key Type of value Default value, or Required CMapDrawItem member(s) Description
Color Integer Req mColor Color index (0-5)
StageXYZ Three Floats Req  (unless external) mStageX/Y/Z Stage position adjusted for current mag
NumPts Integer Req mNumPoints Number of points
Corner Integer 0 mCorner Flag for corner point
Draw Integer 1 mDraw Flag to draw
RegPt Integer 0 mRegPoint Registration point number
Regis Integer Req mRegistration Registration at which item exists
Type Integer Req mType Type: 0-2 for point, polygon, map
Note String blank mNote Note string
GroupID Integer 0 mGroupID ID of group thatitem belongs to
PolyID Integer 0 mPolygonID ID of polygon used to define supermontage
FitToPolygonID Integer 0 mFitToPolygonID ID of polygon that this montage map was fit to
Imported Integer 0 mImported Indicator of an imported map or point drawn on one
RegisteredToID Integer 0 mRegisteredToID For an imported map, ID of map that it was registered to for transforming
SuperMontXY Two Integers -1,-1 SuperMontX/Y Camera coordinate of a montage in supermontage
OrigReg Integer Regis mOriginalReg Original registration (default is Registration)
DrawnID Integer 0 mDrawnOnMapID ID of map point/polygon was drawn on
Flags Integer 0 mFlags Bit flags: bit 1 on if item was drawn on an FFT
BklshXY Two Floats 0,0 mBacklashX/Y Backlash when a montage was taken or associated with item
SamePosId Integer 0 mAtSamePosID Items with this matching were taken at same raw stage position
RawStageXY Two Floats -10000,-10000 mRawStageX/Y Raw stage position before adjustments
Acquire Integer 0 mAcquire Flag for acquiring
PieceOn Integer -1 mPieceDrawnOn Index of montage piece item was drawn on (piece_#_in_X * #_of_pieces_in_Y + piece_#_in_Y), or index of nearest piece if item is on a missing piece, in which case XYinPc will not be written to file.
XYinPc Two Floats -1,-1 mX/YinPiece X,Y coordinates in the montage piece, in right-handed pixels of stored montage
MapFile String Req if Map mMapFile Full or relative path of map file (see Read & Open)
MapID Integer Req if Map mMapID Unique ID (all items get one, not just maps)
FocusAxisPos Float -1.e8 mFocusAxisPos Stored position of Focus area on inter-area axis
LDAxisAngle Two integers 0,0 mRotateFocusAxis, mFocusAxisAngle First value: 1 to rotate inter-area axis from tilt axis, 0 not to; second value: angle of rotation
FocusOffsets Two Floats 0,0 mFocusX/Yoffset Offset of focus subarea in X and Y in unbinned pixels (Y inverted)
HoleArray Two integers 0,0 mNumX/Yholes # of positions in X and Y for multiple Record
SkipHoles List of integers none mSkipHolePos Pairs of X,Y indexes for positions to skip from  that array specified in HoleArray.  Positions are numbered from 0 at lower-left in stage coordinates 
HoleISXspacing Three Floats 0,0,0 mXHoleISSpacing X component of vectors from hole finding converted to image shift for use with multiple Records
HoleISYspacing Three Floats 0,0,0 mYHoleISSpacing Y component of hole vectors converted to image shift
TSstartEndAngles Two Floats -1.e8,-1.e8 mTSstartAngle, mTSendAngle Starting and ending angles of range for  tilt series item
TSbidirAngle Float -1.e8 mTSbidirAngle Bidirectional starting angle for tilt series item
TargetDefocus Float -1.e8 mTargetDefocus Target defocus for Acquire or tilt series item
FileToOpen String none mFileToOpen Name of file to open for Acquire or tilt series item
TSParamIndex Integer -1 mTSParamIndex Index of into parameter array for tilt series
MontParamIndex Integer -1 mMontParamIndex Index into array of parameters for montages to open
FilePropIndex Integer -1 mFilePropIndex Index into array of file options for file to open
MapMontage Integer Req if Map mMapMontage Flag that map is a montage
MapSection Integer Req if Map mMapSection Section number in file
MapBinning Integer Req if Map mMapBinning Binning at which map was taken, or of initial overview map image for montage
MapMagInd Integer Req if Map mMapMagInd Magnification index of map, or of non-map image a point or polygon was drawn on
MapCamera Integer Req if Map mMapCamera Camera index
MapScaleMat Four Floats Req if Map mMapScaleMat Stage to pixel scale matrix for drawing, based on pixels of initial map image.  For montage, needs to be adjusted by ratio of currently loaded map width/height to initial map width/height
MapWidthHeight Two Integers Req if Map mMapWidth/Height Size of initial map image at which scale matrix was defined
MapMinMaxScale Two Floats 0,0 mMapMin/MaxScale Min and max scale values of image when map was defined
MapFramesXY Two Integers 0,0 mMapFramesX/Y Number of montage frames when acquired
MontBinning Integer 0 mMontBinning Actual binning of montage used to make the map
MapExposure Float 0. mMapExposure Exposure time for original map images
MapSettling Float 0. mMapSettling Drift settling
ShutterMode Integer -1 mShutterMode Shutter mode
K2ReadMode Integer 0 mK2ReadMode Read mode for K2/K3 camera
MapSpotSize Integer 0 mMapSpotSize Spot size at which map was taken
MapIntensity Double 0 mMapIntensity Intensity at which map was taken
MapSlitIn Integer 0 mMapSlitIn Filter slit state when map was taken
MapSlitWidth Float -1. mMapSlitWidth Filter slit width when map was taken
RotOnLoad Integer 0 mRotOnLoad Flag to rotate when load
RealignedID Integer 0 mRealignedID ID of lower mag map with nearby realign error
RealignErrXY Two Floats 0,0 mRealignErrX/Y Final stage error of that realign operation
LocalErrXY Two Floats 0,0 mLocalRealiErrX/Y Error in second round of realign operation
RealignReg Integer 0 mRealignReg Original registration of that realign
ImageType Integer 0 mImageType Map file type 0-3 for MRC, IMOD, TIFF, IDOC
MontUseStage Integer -1 mMontUseStage 1 if montage was taken with stage, 0 if not, -1 if unknown
DefocusOffset Float 0. mDefocusOffset Defocus offset for map based on View images
K2ReadMode Integer 0 mK2ReadMode Read mode for K2/K3 camera
NetViewShiftXY Two Floats 0,0 mNetViewShiftX/Y Net IS offset for a View image map
MapAlpha Integer -999 mMapAlpha JEOL alpha value
ViewBeamShiftXY Two Floats 0,0 mViewBeamShiftX/Y Incremental beam shift to apply for a View map
ViewBeamTiltXY Two Floats 0,0 mViewBeamTiltX/Y Incremental beam tilt to apply for a View map
MapProbeMode Integer -1 mMapProbeMode Probe mode (0 for nano, 1 for micro)
MapLDConSet Integer -1 mMapLowDoseConSet Set # (0-4) used for map taken in low dose
MapTiltAngle Float -10000. mMapTiltAngle Tilt angle at which map was taken
PtsX NumPts Floats Req mPtX X stage cordinates of points
PtsY NumPts Floats Req mPtY Y stage cordinates of points
UserValueN String none mUserValueMap Arbitrary value set by script; N is # (currently1-8)
TSParam - a section containing tilt series parameters.  See TiltSeriesParam.h in the source code for a description of each item.
MontParam - a section containg montage parameters.  See MontageParam.h in the source code for a description of each item.
FileOptions - a section containing properties for opening a file.  See FileOptions.h in the source code in case this has descriptions.

Importing Externally Defined Navigator Items

It is possible to add items to a Navigator file whose positions are specified by image coordinates in a map and have SerialEM do the conversion to stage coordinates.  Instead of the StageXYZ entry, the item can have one of these entries:

The following points apply to externally defined items:

Detailed error reporting is available for items that have been read in successfully from the Navigator file.  A summary of the types of errors when processing the items is given in the log, and if Debug Output is set to 'n', then the type of error will be reported for each item, identified by its label.  However, there is no detail when the item is initially rejected either because it lacked a required entry or because there was an error (such as too few values on a line) reading data from the autodoc.

Settings and Calibration File Entries

This section is for lengthy entries to the settings or calibration file that sometimes need to be interpreted.  Entries will be added upon need or request.  All alpha values are numbered from 0, not 1.

Settings file entries:

LowDoseParameters:
There are three sets of parameters, for TEM, EFTEM, and STEM.
   low dose set number 0 to 4 for V F T R S, or negative of state number (numbered from 1)
   mag index or negative of camera length index
   spot size
   intensity
   axis offset in microns
   0-2 for regular/EFTEM/STEM or 0 for state
   filter slit in
   filter slit width
   energy loss
   zero loss flag
   beam X offset
   beam Y offset
   alpha on JEOL, -999 otherwise
   diffraction focus
   beam tilt X
   beam tilt Y
   probe mode
   dark field mode flag
   dark field tilt X

   dark field tilt Y

StateParameters:
   0 for non-low dose, -area # - 1 for low dose, 1 for old LD Record
   camera index
   mag index
   spot size
   intensity
   filter slit in
   filter energy loss
   filter slit width
   zero loss flag
   binning
   frame size in X
   frame size in Y
   exposure time
   drift settling
   shuttering
   read mode
   probe mode
   dose fractionation
   save frames
   processing/normalization
   align frames
   1 to use framealign, 2 to align in IMOD
   frame alignment parameter set index
   View read mode
   Focus read mode
   Trial read mode
   Preview read mode
   focus axis position
   alpha on JEOL, -999 otherwise
   defocus target
   low dose focus offset
   low dose shift offset X
   low dose shift offset Y
   flag that it is Mont-map

AutocenterParams:

   camera
  mag index
  spot size
  intensity
  binning
  exposure
  1 to use centroid
  probe mode
  1 to shift beam for centering
  beam shift in microns
  added X beam shift
  added Y beam shift

MultishotParams:

   Beam diameter for display
   Distance to off-center shots in main ring
   Number of shots in main ring
   Do center shot: -1 before, 0 none, 1 after
   Early return: 1 for last only, 2 for all, 3 for full sum on first
   Number of frames in early return
   Whether to save Record shot
   Additional delay after image shift
   Whether to use illuminated area for drawing diameter
   Whether to adjust beam tilt if possible
   1 to do pattern in hole, 2 to do multi-hole, 3 to do both
   Use the list of custom holes
   Factor by which to increase regular IS delay between holes
   X component of image shift vector in 'X' direction
   Y component of image shift vector in 'X' direction
   X component of image shift vector in 'Y' direction
   Y component of image shift vector in 'Y' direction
   Number of holes in 'X' direction
   Number of holes in 'Y' direction
   Mag at which regular pattern hole positions are measured
   Mag at which custom holes are defined
   Take cross pattern when it is 3x3
   Do a second ring of shots
   Number of shots in second ring
   Distance to off-center shots in second ring
   Tilt angle at which regular pattern was defined
   Tilt angle at which custom holes were defined
   Angle found in hole finder associated with regular array
 

Calibration file entries:

StandardLMFocus: This entry contains the recorded standard focus in LM or the eucentric focus is nonLM.
   Magnification index
   Focus value in microprobe, -999 if none
   Focus value in nanoprobe, -999 if none
   Magnification
Focus values are either absolute values between 0 and 1, or on JEOL, a large value that encodes coarse and fine focus components

FocusCalibration:
   mag index
   camera number
   Slope in X
   Slope in Y
   beam tilt
   number of points
   direction index
   probe mode
   alpha
   magnification

SpotIntensities:  There may be two tables if either nanoprobe or microprobe has two calibrations, one below crossover and one above crossover.  However, a table can have one condition done below crossover and the other above.  It all gets sorted out into separate tables internally.  Each row contains:
   Spot size
   Microprobe intensity ratio, intensity at which acquired, and crossover intensity
   Nanoprobe intensity ratio, intensity at which acquired, and crossover intensity

where a ratio of 0 indicates no measurement for that spot size.

BeamShiftCalibration: This calibration is a 2x2 transformation matrix from image shift to beam shift at the given magnification:
   mag index
   X change in beam shift per X change in image shift
   X change in beam shift per Y change in image shift
   Y change in beam shift per X change in image shift
   Y change in beam shift per Y change in image shift
   alpha
   probe mode
   1 to retain the calibration when a new one is done in the same mag range and beam conditions, 0 not to
   magnification

HighFocusMagCal and HighFocusISCal:
   spot size
   probe mode
   defocus
   intensity
   mag change
   rotation
   crossover
   aperture size
   Mag index (0 for mag cals)

ComaVsISCal:
   Mag index
   spot size
   probe mode
   alpha
   aperture size
   intensity
   X change in beam tilt needed per X change in image shift
   X change in beam tilt needed per Y change in image shift
   Y change in beam tilt needed per X change in image shift
   Y change in beam tilt needed per Y change in image shift
   X change in astigmatism needed per X change in image shift
   X change in astigmatism needed per Y change in image shift
   Y change in astigmatism needed per X change in image shift
   Y change in astigmatism needed per Y change in image shift

Stage Calibration and 2x2 transformation matrices

The stage calibration consists of 4 matrix terms, and like all such 2x2 transformation matrices, they are expressed as a change in output X value per change in X input value (xpx), change in X per change in Y (xpy), change in Y per change in X (ypyx), and change in Y per change in Y (ypy).  They are written to file in this order.   In this case the input value is the change in stage position in microns, and the output value is the negative of the amount that the image shifts in unbinned pixels, or the position of the new centered coordinate in the image relative to the original center.  This negative value is unfortunate in that it inverts the typical stage coordinate system relative to the underlying "specimen" coordinate system that has its X along the tilt axis.

The StageCalibration line has
   Mag index
   Camera number
   xpx
   xpy
   ypx
   ypy
   Absolute focus at which calibration taken (compound of coarse and fine values for JEOL)
   Magnification

Other Notes

Image Rotation

The 'specimen' coordinate system is a right-handed system with its X axis along the positive tilt axis.  (Looking toward the origin along the positive tilt axis, a positive tilt rotates the stage counterclockwise).  Image rotation is defined as the amount that the camera image has rotated CCW from the image at the specimen plane, i.e., the angle (CCW positive) from the positive X axis of the image to the positive tilt axis in the image.

If there is only one camera and there is no RotationAndPixel line with a calibrated absolute rotation, the rotation at one magnification is estimated from a fallback value, the sum of that mag's rotation value in the MagnificationTable, the GlobalExtraRotation, and the ExtraRotation for that camera.  If there are relative rotations in the table, the fallback is assigned to one mag as the absolute rotation, and this allows a rotation to be derived for the others.  This is an ambiguous situation because there is no way to know just which mag that fallback angle should apply to. 

Once there is a calibrated absolute angle, the MagnificationTable rotation, GlobalExtraRotation, and ExtraRotation become irrelevant for all the mags that are listed in the RotationAndPixel table and linked by relative rotations to a mag with an absolute rotation.  The relative rotation for a mag equals the rotation angle of that mag minus the rotation of the next lower mag; this relationship allows a rotation angle to be propagated outward from the mag with the calibrated absolute angle. If any of these mags have an image shift calibration, other mags in the same mag range that have an image shift calibration can also be assigned a rotation by assuming that image shift is invariant at the specimen within that mag range.  On a Thermo/FEI scope, the mag ranges are LM and nonLM; on a JEOL using image shift deflections there are other boundaries across which rotation cannot be transferred; on a JEOL scope using PLA, rotation cannot be transferred at all. The MagnificationTable rotations retain a fallback role for other mags that have not been assigned a rotation by one of these mechanisms; the rotation for such a mag is derived from the nearest mag with an assigned rotation by adding the difference between the MagnificationTable rotation of the mag in question and that of the nearby mag.

The situation is even more complicated when there is more than one camera at the same level, i.e., more than one non-energy filter camera or more than one energy filter camera, and one lacks a calibrated absolute angle.  The program will use image shift calibrations if possible to get an absolute angle from another camera.  All of the assignments described here are listed when the program is started with DebugOutput set to 'c'.  See also ShiftManager::PropagateRotations(). 

The image rotation value is intimately related to the stage calibration.  For a Thermo/FEI scope or JEOL scope without omega filter, an image rotation of 0 produces this pattern of image movement

  ^  image movement for positive Y stage move
  |
  |
  |
  |
  --------->   image movement for positive X stage move
The image rotation angle is the amount by which the axes of image movement are rotated counterclockwise relative to this pattern.  The image rotation (and the tilt axis angle) should equal  atan2(-ypx, -xpx) (from the X axis rotation) or atan2(-ypy, -xpy) - 90 ( from the Y axis rotation); the program averages these two estimates when it reports the rotation angle with a stage calibration. More common angles for Thermo/FEI scopes (except Krios) are near 90 degrees and the pattern of movement is
           ^  image movement for positive X stage move
           |
           |
           |
           |
  <---------   image movement for positive Y stage move

A typical stage calibration will have values like: -54   261   -262   -55, which indicates that an X movement produces predominantly a negative Y change in camera coordinate, while a Y movement produces a positive change in X coordinate.  This is inverted from the diagram because a positive shift of the image corresponds to a negative change in the (right-handed) camera coordinates of the centered point.  Similarly, a Krios or a JEOL F200 has an axis angle near 180 with a pattern

  <---------   image movement for positive X stage move
           |
           |
           |
           |
           V  image movement for positive Y stage move
and a typical calibration of 490   44   -67   464.

For a JEOL with omega filter with the tilt axis near horizontal (.e.g, 2200FS), the pattern is this:

  --------->   image movement for positive X stage move
  |
  |
  |
  |
  V  image movement for positive Y stage move
The X axis is inverted and this requires the property InvertStageXAxis to be set.  In this case the image rotation is estimated from atan2(ypx, xpx) and atan2(-ypy, -xpy) - 90.  These estimates are simply reported and never used directly by the program; they must be placed into the RotationAndPixel  lines to have any effect.  A typical stage calibration would be -799   -109   -104   823 (again, because of the inversion between direction of movement and sign of the matrix), and the program would report an angle around 180 from stage calibration once the InvertStageXAxis property is set.