Extra Header Module

The extraheader module contains functions for working with information in MRC file extended headers and with comparable information in metadata autodoc files. It does not have any functions for manipulating file headers.

MRC header members relevant to this are next, the total number of extra bytes, extType, a four-character stamp for the type of header, nint and nreal, which are the number of integers and reals per section in Agard/old FEI style headers, or the number of bytes and flags for SerialEM headers. Some of the functions here expect "modified" nint and nreal values for other kinds of headers, namely the negative of MRC_EXT_TYPE_... codes in mrcfiles.h and a possible version number. These modified values are returned directly when appropriate by iiuRetExtendedType, whereas mrcGetExtendedType just returns an MRC_EXT_TYPE_ value and the version.

If all images are being transferred from one file to another, then the extended header can be copied in its entirety with mrcCopyExtraHeader, iiuTransHeader, or iiuTransExtendedData. All fields will be managed appropriately.

If it is necessary to transfer data only for some sections, then the sequence of operations would be:

  1. Get the extent and type of the extended header from next>, nint> and nreal members and results from mrcGetExtendedType, or with iiuRetNumExtended and iiuRetExtendedType.
  2. Get the extended header into a buffer with mrcReadExtraHeader or iiuRetExtendedData.
  3. Determine the total bytes needed for the extended header from the number of output sections and the maximum size returned by getExtraHeaderMaxSecSize.
  4. Set up the output header before writing data by setting next and headerSize (to 1024 + next), transferring nint> and nreal if they are meaningful, and transferring the extended header type stamp with mrcCopyValidExtendedType; or with iiuAltNumExtended, iiuAltExtendedType if the incoming values of "nintOrBytes" and "nrealOrFlags" were non-negative, and iiuTransValidExtType.
  5. Copy data for individual sections from the input buffer to the output buffer with copyExtraHeaderSection.
  6. Write the extended header at the end with mrcWriteExtraHeader or iiuAltExtendedData.

Header to include: cfsemshare.h


int getExtraHeaderTilts(char *array, int numExtraBytes, int nbytes, int iflags, int nz, float * tilt, int *numTilts, int maxTilts, int *izPiece)
void get_extra_header_tilts(char *array, int *numExtraBytes, int *nbytes, int *iflags, int *nz, float *tilt, int *numTilts, int *maxTilts, int *izPiece)
int getExtraHeaderItems(char *array, int numExtraBytes, int nbytes, int iflags, int nz, int itype, float *val1, float *val2, int *numVals, int maxVals, int *izPiece)
void get_extra_header_items(char *array, int *numExtraBytes, int *nbytes, int *iflags, int *nz, int *itype, float *val1, float *val2, int *numVals, int *maxVals, int *izPiece)
double SEMshortsToFloat(short low, short ihigh)
int getMetadataItems(int indAdoc, int iTypeAdoc, int nz, int iTypeData, float *val1, float *val2, int *numVals, int *numFound, int maxVals, int *izPiece)
void get_metadata_items(int *indAdoc, int *iTypeAdoc, int *nz, int *iTypeData, float *val1, float *val2, int *numVals, int *numFound, int *maxVals, int *izPiece)
int getMetadataByKey(int indAdoc, int iTypeAdoc, int nz, const char *key, int ivalType, float *val1, float *val2, float *val3, char **valString, int *numVals, int *numFound, int maxVals, int *izPiece)
void get_metadata_by_key(int *indAdoc, int *iTypeAdoc, int *nz, char *key, int *ivalType, float *val1, float *val2, float *val3, char *valString, int *numVals, int *numFound, int *maxVals, int *izPiece, fortStrLen_t keySize, fortStrLen_t valSize)
int getExtraHeaderPieces(char *array, int numExtraBytes, int nbytes, int iflags, int nz, int *ixPiece, int *iyPiece, int *izPiece, int *numPieces, int maxPiece)
void get_extra_header_pieces(char *array, int *numExtraBytes, int *nbytes, int *iflags, int *nz, int *ixPiece, int *iyPiece, int *izPiece, int *numPieces, int *maxPiece)
int getMetadataPieces(int indAdoc, int adocType, int nz, int *ixPiece, int *iyPiece, int *izPiece, int maxPiece, int *numFound)
void get_metadata_pieces(int *indAdoc, int *adocType, int *nz, int *ixPiece, int *iyPiece, int *izPiece, int *maxPiece, int *numFound)
int getMetadataWeightingDoses(int indAdoc, int iTypeAdoc, int nz, int *izPiece, int bidirNumInvert, float *priorDose, float *secDose)
int getmetadataweightingdoses(int *indAdoc, int *iTypeAdoc, int *nz, int *izPiece, int *bidirNumInvert, float *priorDose, float *secDose)
void priorDosesFromImageDoses(float *secDose, int nz, int bidirNumInvert, float *priorDose)
void priordosesfromimagedoses(float *secDose, int *nz, int *bidirNumInvert, float *priorDose)
int getExtraHeaderValue(void *extHead, int offset, int type, unsigned char *bval, short *sval, int *ival, float *fval, double *dval)
int getextraheadervalue(void *extHead, int *offset, int *type, unsigned char *bval, short *sval, int *ival, float *fval, double *dval)
int getExtraHeaderSecOffset(void *extHead, int extSize, int numInt, int numReal, int izSect, int *offset, int *size)
int getextraheadersecoffset(void *extHead, int *extSize, int *numInt, int *numReal, int *izSect, int *offset, int *size)
int getExtraHeaderMaxSecSize(void *extHead, int extSize, int numInt, int numReal, int numSect, int *maxSize)
int getextraheadermaxsecsize(void *extHead, int *extSize, int *numInt, int *numReal, int *numSect, int *maxSize)
int copyExtraHeaderSection(void *extraIn, int sizeIn, void *extraOut, int sizeOut, int numInt, int numReal, int izSect, int *cumulBytesOut)
double getFeiExtHeadAngleScale(void *extHead)
double getfeiextheadanglescale(void *extHead)

int getExtraHeaderTilts(char *array, int numExtraBytes, int nbytes, int iflags, int nz, float * tilt, int *numTilts, int maxTilts, int *izPiece)

Returns tilt angles from an extra header written by SerialEM or in the Agard format.  It relies on the Z values of the sections in the file as listed in izpiece, which should be obtained first with getExtraHeaderPieces .
  array = array of extra header data
  numExtraBytes = number of bytes of data there
  nbytes = number of bytes per section
  iflags = flags for type of data present
  nz = number of sections or pieces
  tilt = array for tilt angles
  numTilts = # of tilt angles returned (or highest # if there are gaps)
  maxTilts = size of TILT array
  izPiece = Z value of each section in file  
Returns 1 for an error and sets error string with b3dError.

void get_extra_header_tilts(char *array, int *numExtraBytes, int *nbytes, int *iflags, int *nz, float *tilt, int *numTilts, int *maxTilts, int *izPiece)

Fortran wrapper to getExtraHeaderTilts. Exits with an ERROR: string upon error

int getExtraHeaderItems(char *array, int numExtraBytes, int nbytes, int iflags, int nz, int itype, float *val1, float *val2, int *numVals, int maxVals, int *izPiece)

Returns values of a defined type from an extra header written by SerialEM; or tilt angles from a header in the Agard format; or tilt angles, relative time stamps, or dose values (which are probably doses to the camera) from an FEI1 header.  It relies on the Z values of the sections in the file as listed in izpiece, which should be obtained first with getExtraHeaderPieces .
  array = array of extra header data
  numExtraBytes = number of bytes of data there
  nbytes = number of bytes per section
  iflags = flags for type of data present
  nz = number of sections or pieces
  itype = type of data to retrieve: 1 for tilt angle, 3 for stage position, 4 for magnification, 5 for intensity, 6 for exposure dose or dose from an FEI1 header, 10 for images times from the time stamps in an FEI1 header relative to the time for the first image, in seconds
  val1, val2 = arrays for one or two values to be returned
  numVals = # of values returned (or highest # if there are gaps)
  maxVals = size of val1 and val2 arrays
  izPiece = Z value of each section in file  
Returns 1 for an error and sets error string with b3dError.

void get_extra_header_items(char *array, int *numExtraBytes, int *nbytes, int *iflags, int *nz, int *itype, float *val1, float *val2, int *numVals, int *maxVals, int *izPiece)

Fortran wrapper to getExtraHeaderItems. Exits with an ERROR: string upon error

double SEMshortsToFloat(short low, short ihigh)

Converts the two short integers low and ihigh stored by SerialEM for a floating point number back into the number

int getMetadataItems(int indAdoc, int iTypeAdoc, int nz, int iTypeData, float *val1, float *val2, int *numVals, int *numFound, int maxVals, int *izPiece)

Returns values of a defined type from an image metadata file written by SerialEM or an HDF file in which such data have been incorporated.  It simply calls getMetadataByKey with a specific key defined by the value of itypeData.  See getMetadataByKey for a full description; the arguments special to this call are:
  iTypeData = type of data to retrieve: 1 for tilt angle, 3 for stage position, 4 for magnification, 5 for intensity, 6 for exposure dose, 7 for pixel size, 8 for defocus, 9 for exposure time
  val1, val2 = arrays for one or two values to be returned  
Returns 1 for an error and sets error string with b3dError.

void get_metadata_items(int *indAdoc, int *iTypeAdoc, int *nz, int *iTypeData, float *val1, float *val2, int *numVals, int *numFound, int *maxVals, int *izPiece)

Fortran wrapper to getMetadataItems. Exits with an ERROR: string upon error

int getMetadataByKey(int indAdoc, int iTypeAdoc, int nz, const char *key, int ivalType, float *val1, float *val2, float *val3, char **valString, int *numVals, int *numFound, int maxVals, int *izPiece)

Returns values of almost any available type from an image metadata file written by SerialEM or an HDF file in which such data have been incorporated.  The metadata file should be opened first with AdocOpenImageMetadata, or AdocGetImageMetaInfo should be called on the autodoc of an HDF file. The routine relies on the Z values of the sections in the file as listed in izPiece, which should be obtained first with getMetadataPieces.
  indAdoc = autodoc index of metadata file
  iTypeAdoc = type of metadata file: 1 for one file, 2 for series, 3 for other autodoc
  nz = number of sections or pieces
  key = Complete name string for data to retrieve (case-sensitive)
  ivalType = 1 for single integer, 2 for single float, 3 for two floats, 4 for three floats, or 0 for string
  val1, val2, val3 = arrays for one, two or three values to be returned.  val2 and val3 can be single floats if not needed, but when returning strings, val1 is needed because it will be set to 0 for every index where a string is found.
  valString = array for value strings to be returned.  This can be NULL unless ivalType is 0.  Specifically, this is an array of char * pointers; returned strings are duplicated into there and should be freed.
  numVals = # of values returned (or highest # if there are gaps)
  numFound = # of values actually found in metadata.  If the calling program wants to work with an incomplete collection of values, it should initialize the arrays before calling to a value distinct from possible actual values.
  maxVals = size of val1, val2, val3, and valString arrays
  izPiece = Z value of each section in file  
Returns 1 for an error and sets error string with b3dError.

void get_metadata_by_key(int *indAdoc, int *iTypeAdoc, int *nz, char *key, int *ivalType, float *val1, float *val2, float *val3, char *valString, int *numVals, int *numFound, int *maxVals, int *izPiece, fortStrLen_t keySize, fortStrLen_t valSize)

Fortran wrapper to getMetadataByKey. Exits with an ERROR: string upon error. valString can be a single character variable unless ivalType is 0, in which case it must be an array of character variables

int getExtraHeaderPieces(char *array, int numExtraBytes, int nbytes, int iflags, int nz, int *ixPiece, int *iyPiece, int *izPiece, int *numPieces, int maxPiece)

Returns piece coordinates from the extra header written by SerialEM
  array = array of extra header data
  numExtraBytes = number of bytes of data there
  nbytes = number of bytes per section
  iflags = flags for type of data present
  nz = number of pieces in the file
  ixPiece, iyPiece, izPiece = arrays in which coordinates are returned
  numPieces = number of coordinates returned (should equal nz)
  maxPiece = size of piece arrays  
Returns 1 for an error and sets error string with b3dError.

void get_extra_header_pieces(char *array, int *numExtraBytes, int *nbytes, int *iflags, int *nz, int *ixPiece, int *iyPiece, int *izPiece, int *numPieces, int *maxPiece)

Fortran wrapper to getExtraHeaderPieces. Exits with an ERROR: string upon error

int getMetadataPieces(int indAdoc, int adocType, int nz, int *ixPiece, int *iyPiece, int *izPiece, int maxPiece, int *numFound)

Returns piece coordinates from a metadata autodoc file written by SerialEM.  The file should be opened first with AdocOpenImageMetadata, or information about the autodoc in an HDF file should be obtained with AdocGetImageMetaInfo.
  indAdoc = index of autodoc
  adocType = type of metadata file: 1 for one file, 2 for image series, 3 for autodoc
  nz = number of pieces in the file
  ixPiece, iyPiece, izPiece = arrays in which coordinates are returned
  maxPiece = size of piece arrays
  numFound = number of sections piece coordinates were found for  
Returns 1 for an error and sets error string with b3dError.

void get_metadata_pieces(int *indAdoc, int *adocType, int *nz, int *ixPiece, int *iyPiece, int *izPiece, int *maxPiece, int *numFound)

Fortran wrapper to getMetadataPieces. Exits with an ERROR: string upon error

int getMetadataWeightingDoses(int indAdoc, int iTypeAdoc, int nz, int *izPiece, int bidirNumInvert, float *priorDose, float *secDose)

Returns dose information from a metadata autodoc file produced by SerialEM or an HDF file in which such data have been incorporated.  The metadata file should be opened first with AdocOpenImageMetadata, or AdocGetImageMetaInfo should be called on the autodoc of an HDF file. The data are assumed to be from a tilt series in which dose accumulates; thus a value for the cumulative dose before each image is returned in priorDose.  These prior doses are first sought in "PriorRecordDose" entries.  If those are not available for every image, then the routine analyzes the DateTime entries to determine the order of images.  If those are not available for every image, then the images are assumed to be in the standard order of a bidirectional tilt series if the value of bidirNumInvert is greater than 1.  Namely, views 1 through bidirNumInvert are assumed to be in the file in inverted order of acquisition, while the rest are assumed to be in order, and doses are summed for the images in that order.  If bidirNumInvert is less than 0 to indicate that the tilt series is in inverted order, then the order of images is taken to be from view bidirNumInvert + 1 to the end, then from bidirNumInvert to 1, and doses are summed for the images in that order.  If that value is not greater than 1 or less than 0, then the routine just assumes the images are all in order and the return value is -1.
The routine relies on the Z values of the sections in the file as listed in izPiece, which should probably just be filled with 0 to nz - 1 to obtain meaningful dose values for individual pieces of a montage.  Other arguments are:
  indAdoc = autodoc index of metadata file
  iTypeAdoc = type of metadata file: 1 for one file, 2 for series, 3 for other autodoc
  nz = number of sections or pieces for which data should be returned
  izPiece = Z value of each section in file  
Returns 1 for an error accessing the autodoc information, or 2 for insuffient ExposureDose entries, dose entries of 0, DateTime entries for only some sections, or bad month strings in the DateTime entries.  For any errors, it sets any error string with b3dError.

int getmetadataweightingdoses(int *indAdoc, int *iTypeAdoc, int *nz, int *izPiece, int *bidirNumInvert, float *priorDose, float *secDose)

Fortran wrapper for getMetadataWeightingDoses

void priorDosesFromImageDoses(float *secDose, int nz, int bidirNumInvert, float *priorDose)

Computes accumulated dose from nz image doses in secDoses and returns them in priorDose.  If bidirNumInvert is greater than 1, doses 1 to bidirNumInvert are summed in inverted order; if bidirNumInvert is less than 0, doses bidirNumInvert + 1 to the end are summed in order then doses 1 to bidirNumInvert are summed in inverted order; otherwise all image doses are summed in order.

void priordosesfromimagedoses(float *secDose, int *nz, int *bidirNumInvert, float *priorDose)

Fortran wrapper for priorDosesFromImageDoses

int getExtraHeaderValue(void *extHead, int offset, int type, unsigned char *bval, short *sval, int *ival, float *fval, double *dval)

Gets one value from an extended header in extHead, starting at the given offset and of the type specified by type.  type should 0 for a byte (unsigned char), 1 for a short integer, 2 for a float, 3 for an integer, or 4 for a double (these are EXT_HEAD_VALUE_ defines).  Values are copied by memcpy, not by casting; the caller is responsible for casting integers to unsigned if appropriate.  Returns 1 if type is out of range.

int getextraheadervalue(void *extHead, int *offset, int *type, unsigned char *bval, short *sval, int *ival, float *fval, double *dval)

Fortran wrapper for getExtraHeaderValue

int getExtraHeaderSecOffset(void *extHead, int extSize, int numInt, int numReal, int izSect, int *offset, int *size)

Returns the location and size of the extended header segment for section izSect, numbered from 0.  The full extended header should be supplied in extHead, its size in extSize, and either true nint and nreal header members in numInt and numReal for a SerialEM or Agard/old FEI style header, or the negative of the MRC_EXT_TYPE_ header type in numInt and a possible version number in numReal.  (These are the values returned by iiuRetExtendedType.) The byte offset for the section is returned in offset and the number of bytes in size.  The return value is one for an unknown extended header type or 2 if the header is not long enough to contain data for the given section.

int getextraheadersecoffset(void *extHead, int *extSize, int *numInt, int *numReal, int *izSect, int *offset, int *size)

Fortran wrapper for getExtraHeaderSecOffset

int getExtraHeaderMaxSecSize(void *extHead, int extSize, int numInt, int numReal, int numSect, int *maxSize)

Returns the maximum size in bytes of any extended header segment for the first numSect sections into maxSize.  Other arguments and the return value are as in getExtraHeaderSecOffset.

int getextraheadermaxsecsize(void *extHead, int *extSize, int *numInt, int *numReal, int *numSect, int *maxSize)

Fortran wrapper for getExtraHeaderMaxSecSize

int copyExtraHeaderSection(void *extraIn, int sizeIn, void *extraOut, int sizeOut, int numInt, int numReal, int izSect, int *cumulBytesOut)

Copies the extended header data for the section izSect from extraIn to extraOut. The sizes of the two arrays in bytes are in sizeIn and sizeOut, and numInt and numReal are either the nint and nreal members or the negative of a MRC_EXT_TYPE_ value and a version number.  The cumulative number of bytes copied is maintained in cumulBytesOut.  Returns 1 for unsupported type, 2 for input data not large enough to contain the section, or 3 for the output array not large enough for the copy.

double getFeiExtHeadAngleScale(void *extHead)

Returns the value by which angles in the FEI1 extended header in extHead need to be scaled; this is radians/degree for software versions before a bug was fixed.

double getfeiextheadanglescale(void *extHead)

Fortran wrapper for getFeiExtHeadAngleScale