manageshrmem(1) General Commands Manual manageshrmem(1)
NAME
manageshrmem - Set up shared memory as needed and runs a sequence of
processes
SYNOPSIS
manageshrmem options
DESCRIPTION
Manageshrmem provides a framework for running a sequence of image pro-
cessing programs and using shared memory for temporary, intermediate
files. When the IMOD system for file input/output is given a filename
starting with "/IMODShrMem_", it attempts to open an existing shared
memory segment with that name and does subsequent operations to or from
the shared memory. The calling programs do not need to do anything
special to use the shared memory files, and thus virtually all IMOD
programs can use this capability. Manageshrmem creates the shared mem-
ory areas when needed, runs each program in the sequence, and removes
each shared memory area when no longer needed. This approach is needed
in Windows to keep the shared memory in existence between processes.
POSIX shared memory calls are used in Linux and Mac.
To use this capability, Manageshrmem needs to be told the names of the
shared memory areas (files) to open, their sizes (which is contained in
the name), the processes to run, and which files are needed by each
process. The options below provide that information. The processes
can consist of the actual command to run a program, or a command to run
a command file with Submfg(1) or Vmstopy.
The shared memory module in the I/O system can read and write most
types of data without conversion, and can read in various data types as
floats and write floating point data back into various types. Reading
and writing without conversion is done with simple memory copies and is
not parallelized. The slower operations to read into or write from
floats are parallelized with OpenMP, so the penalty for using them is
not great. However, for ultimate efficiency it may be better to have
programs write intermediate files as floats.
OPTIONS
Manageshrmem uses the PIP package for input (see the manual page for
pip). Options can be specified either as command line arguments
(with the -) or one per line in a command file (without the -).
Options can be abbreviated to unique letters; the currently valid
abbreviations for short names are shown in parentheses.
-file (-f) OR -FileToCreate File name
Name of one shared memory file to create at some stage in the
processing, which must be in the form /IMODShrMem_sizeInKB_name,
where "sizeInKB" is the number of kilobytes (1024 byte units)
for the file, and "name" is arbitrary. The file size should
allow for the expected number of pixels times their data size
(e.g., 4 for floats), plus 2 KB for the MRC header structure to
be stored at the front. Image data is stored starting at byte
2048. (Successive entries accumulate)
-command (-c) OR -CommandToRun Text string
Command that writes to or reads from shared memory files or
needs to happen during the sequence. This must be the entire
command, which can include redirection of the output to a file,
and can even include piping. The commands are run in the order
that they are entered. (Successive entries accumulate)
-need (-n) OR -NeedFilesForCommand Multiple integers
Set of files needed by one command, numbered from 1. If this
option is entered at all, it must be entered for each command.
It must be entered if the number of files is not one less than
the number of commands. The default, when files do equal com-
mands minus 1, is for a simple chain of processing in which the
first file is needed by the first and second command, the second
is needed by the second and third, etc. It is possible to enter
a 0 to represent that no files are needed for a command. (Suc-
cessive entries accumulate)
-try (-tr) OR -TrySizesFirst Integer
Try creating the largest set of files needed at once before run-
ning any commands. This test is based on the actual sequence of
file usage specified with the options above. Enter -1 to simply
test the creation and exit, or 1 to run the commands after the
test succeeds. The files will be removed before starting the
first command, and created again only when needed.
-test (-te) OR -TestSizesInKilobytes Multiple integers
A set of file sizes in kilobytes to create together, to test
whether the maximum memory needed can be achieved. The program
will create the files, report if successful, remove them, and
exit.
-keep (-k) OR -ListOfFilesToKeep List of integer ranges
Indexes of files to keep when program ends, numbered from 1.
Comma-separated ranges may be entered. This option allows the
one to run this program but leave some files to be used by other
programs not run through the -command option. After they are
done, the -remove option can be used to remove the files. This
option will not work in Windows, where shared memory is removed
when no program still accesses it.
-remove (-r) OR -JustRemoveFiles
Remove the shared memory associated with the filenames entered
with the -file option then exit. This option allows one to
clean up the memory created when the program was first run with
the -keep option. Other options are ignored except -test, which
would supersede this option.
-help (-h) OR -Usage
Print help output
-StandardInput
Read parameter entries from standard input
EXAMPLE
With the 1K IMOD tutorial data set, this command would multiply data by
100 and output to floats, then multiply by 0.02 and output as integers.
manageshrmem -file /IMODShrMem_262146_BBa.st -com 'newstack -mode 2
-mult 100,0 BBa.st /IMODShrMem_262146_BBa.st' -com 'newstack -mode 1
-mult .02,0 /IMODShrMem_262146_BBa.st junk.mrc'
Note that the specified size is 4 * 1024 * 64 + 2 in kilobytes, to
allow for writing as floats, and to allow 2 KB for an MRC header struc-
ture to be written.
AUTHOR
David Mastronarde
On Linux, if this program is killed with SIGKILL, the shared memory
files will be left in /dev/shm. Use Imodkillgroup with the -e
option to kill with SIGTERM, which allows this program to unlink the
shared memory.
Email bug reports to mast at colorado dot edu.
IMOD 5.0.2 manageshrmem(1)