Parallel writing to an HDF file requires a specific sequence of operations. When the file is first created, it needs to have at least one dataset (section) defined if not written; the routine iiuWriteDummySecToHDF or hdfWriteDummySection can be used to do this without having to write or supply data. When an existing file needs to be opened initially for writing, these are the steps:
Header to include: iimage.h or iiunit.h
int parWrtInitialize(const char *filename, int nxin, int nyin)
int parWrtProperties(int *allSec, int *linesBound, int *nfiles)
int parwrtproperties(int *allSec, int *linesBound, int *nfiles)
int parWrtSetCurrent(int index)
int parwrtsetcurrent(int *index)
void parWrtClose()
void parwrtclose()
int parallelWriteSlice(void *buf, FILE *fout, MrcHeader *hdata, int slice)
int parWrtToHDFchunk(ImodImageFile *iiFile, MrcHeader *hdata, void *buf, int section)
int parwrtgetregion(int *regionNum, char *filename, int *sections,
int *startLines, fortStrLen_t strlen)
int parWrtRecloseHDF(ImodImageFile *iiFile, MrcHeader *hdata)
int parWrtFlushBuffers(ImodImageFile *iiFile, MrcHeader *hdata)
int iiuParWrtInitialize(const char *filename, int iunitBound, int nxIn, int nyIn,
int nzIn)
int parwrtinitialize(char *filename, int *iunitBound, int *nxIn, int *nyIn, int *nzIn,
fortStrLen_t namelen)
void parWrtPosn(int iunit, int iz, int iy)
void parwrtposn(int *iunit, int *iz, int *iy)
int parWrtSec(int iunit, void *array)
int parwrtsec(int *iunit, void *array)
int parWrtLin(int iunit, void *array)
int parwrtlin(int *iunit, void *array)
int iiuParWrtSecPart(int iunit, void *array, int nxdim, int ixStart, int indX0,
int indX1, int iyStart, int iyEnd)
int iiuParWrtRecloseHDF(int iunit, int writeHeader)
int iiuparwrtreclosehdf(int *iunit, int *writeHeader)
void iiuWriteDummySecToHDF(int iunit)
void iiuwritedummysectohdf(int *iunit)
int iiuParWrtFlushBuffers(int iunit)
int iiuparwrtflushbuffers(int *iunit)
Initialize parallel writing to an image file by reading in the boundary
info file whose name is filename, and storing various values in static
variables. filename can be NULL or an empty string, in which case the count of
parallel writing files is incremented and this info file index is marked as having no
info file. nxin and nyin specify the dimensions of the images. Pass the negative
of nxin to indicate that the file is an HDF file, in which case the file specified
by filename will not be read and will simply be used as a lock file. The format of
the info file is to start with a line with these entries:
Version type nx numLines numFiles
version = 1 for now
type = 0 for chunks in Z, 1 for chunks in Y that require boundaries on each
section
nx = X dimension of image file
numLines = number of lines written at each boundary
numFiles = number of boundary files
For each boundary file, there are then two lines:
Name of boundary file
Boundary 1 section and starting line, boundary 2 section and starting line
Sections and lines are numbered from 0. For chunks in Z, the section number
should be -1 for no boundary (i.e., for boundary 1 of first chunk and
boundary 2 of last chunk). A line number of -1 indicates that the boundary
extends to the end of the section (thus, a setup script does not need to
know the size of the images in Y). This routine will convert a line number
of -1 to the appropriate starting line.
For chunks in Y, the section number is
ignored (and should be -1) and the line number should be -1 for no boundary.
Returns 1 for failure to open file, 2 for error reading file, 3 for
inappropriate values in header line of file, 4 for memory allocation errors,
5 for trying to open too many boundary info files for the array, or 6 for an error
opening the given file as a lock file for HDF.
Returns properties read from the current boundary info file. If the file not HDF, it returns allSec 1 if chunks are in Y, linesBound with the number of lines in each boundary, nfiles with the number of boundary files. If the file is HDF, the index to the lock file is returned in allSec, X size in linesBound, and Y size in nfiles. These values are all zero if there was no boundary info file at the current index. Returns 1 for parallel writing not initialized or -1 for an HDF file.
Fortran wrapper for parWrtProperties
Sets the index of the current boundary info file or file for parallel writing to index, numbered from 0. Nearly every function in this module applies to the current file set by this index, so if there are multiple files open for parallel writing, this function must be called before any operations on a different file from the previous one. Returns 1 for index out of range.
Fortran wrapper for parWrtSetCurrent, with index numbered from 1.
Closes any lock files opened for parallel writing
Fortran wrapper for ParWrtClose
Writes one Z slice of data at Z = slice from the buffer buf to file fout according to the header in hdata. If parallel writing has been initialized, lines will be written to a boundary file if appropriate. Returns errors from writing the slice with mrc_write_slice and also returns other non-zero values from opening the boundary file, writing its header, or writing to the file.
Writes data to one chunk of an HDF file Z = section from the buffer buf to file iiFile, accessing the header in hdata, and following the subarea and subarray information in the llx, urx, lly, ury, padLeft, and padRight members of iiFile. Returns errors from writing the slice with iiWriteSection and also returns other non-zero values from opening the boundary file, writing its header, or writing to the file.
Fortran-callable function to get the parameters of parallel writing region number regionNum (numbered from 1) and return the boundary file in filename and the sections and starting lines in arrays sections and startLines. Returns 2 if writing not initialized, or 1 if the region number is out of bounds. This is used by Fixboundaries.
Closes the ImodImageFile iiFile for an HDF file opened with iiFOpen and with header data in hdata. The file must be HDF and be the current parallel file. Returns 1 for error.
Writes out data in HDF buffer for the ImodImageFile iiFile opened with iiFOpen with header data in hdata. The file must be HDF and be the current parallel file. Returns 1 for error.
Initializes the parallel writing routines for use with unit-based I/O. The name of the boundary info file should be in filename, which can be empty. An available unit for opening image files should be in iunitBound, the unit number for the image file to be written in iunitOut and the dimensions of the image file in nxin, nyin, nzin. Returns the return value from the initialization routine parWrtInitialize
Fortran wrapper for iiuParWrtInitialize. Also returns 7 for failure to convert Fortran string.
Positions unit iunit for writing at section iz, line iy, numbered from 0.
Fortran wrapper for parWrtPosn
Writes an entire section from array into unit iunit at the current position and writes boundary lines if appropriate
Fortran wrapper for parWrtSec
Writes one line from array into unit iunit at the current location and writes it to a boundary file if appropriate
Fortran wrapper for parWrtLin
Writes data to one or more chunks of an HDF file from a potential subarray of array at the current line and section of the file open on unit iunit. The X dimension of array is given in nxdim, the starting and ending index at which to write the lines in X in indX0 and indX1, the starting index from which to take data in X in ixStart, and the starting and ending indices from which to take data in Y in iyStart and iyEnd (inclusive, numbered from 0). Returns errors from writing to the slice with iiuWriteSecPart and also returns other non-zero values from storing the data or reopening the file.
Closes the ImodImageFile that is open on unit iunit if it is an HDF file, and writes the header first if writeHeader is non-zero. Returns 1 if a file was closed, 0 if not, and exits on error.
Fortran wrapper for iiuParWrtRecloseHDF
Adds a dataset for section 0 to the HDF file open on unit iunit without writing any data. Exits on error.
Fortran wrapper for iiuWriteDummySecToHDF
Writes output buffers for the the current file open for parallel writing, which must be an HDF file. Returns 1 if it is not an HDF file or if there is an error writing data
Fortran wrapper for iiuParWrtFlushBuffers