framewatcher(1) General Commands Manual framewatcher(1) NAME framewatcher - watch for command files or stacks to run with align- frames SYNOPSIS framewatcher [options] DESCRIPTION Framewatcher is a Python program that can watch for either command files or frame stacks appearing in a directory. The command files are expected to run the program Alignframes and to have extension .pcm, the extension on files produced by the SerialEMCCD plugin to DigitalMi- crograph. The program can modify a few of the parameters in such a file with the options below. The frame stacks must have extension .mrc or .tif. Output files with aligned sums can be placed in a different directory. Stacks and command files can be moved to another directory when they are finished. Whole tilt series can be processed when a com- mand file specifies the metadata (.mdoc) file for the tilt series stack. To process frame stacks, either a master command file for alignment must be supplied or the -nocom option must be entered; in either case, the program will watch only for stacks, not for command files. The master command file can run Alignframes, in which case the same parameters can be modified before running on each stack as when watch- ing for command files. Alternatively, the command file can run other software, in which case it should contain the text "INPUTFILE" and "OUTPUTFILE" wherever the input and output filenames are needed, respectively. The name of the stack will be substituted for "INPUT- FILE", and the name of the output file for the aligned sum, including the optional output directory, will be substituted for "OUTPUTFILE". A command file will be written with the root name of the stack plus the extension ".pcm". In either case, the command file is run with Vmstopy in the direc- tory where it is located and produces a log file there. If files are not being moved elsewhere when done, the command file is renamed with ".done" added at the end, which is how the program keeps from rerunning stacks or command files even if restarted. If no command file at all is run, ".done" will be added to the stack name itself unless it is moved elsewhere. An arbitrary command can be run before or after the command file is run (or a stack is moved or renamed, when there is no command file). Items such as the root name of the data can be substituted into the command, as described below. This may provide a more flexible way to run other alignment software than the master command file. The program works by looking in the directory for eligible files and running all of the ones that it sees. When it sees no files, it waits 5 seconds before scanning the directory again. When it see stacks, it runs all but the last one immediately, then waits until the last one (which typically would be the only one seen) has not changed for at least 15 seconds. Entrl Ctrl C to terminate the program. This will kill any running alignment operation. OPTIONS Framewatcher uses the PIP package for input (see the manual page for pip). Options can be specified either as command line arguments (with the -) or one per line in a command file (without the -). Options can be abbreviated to unique letters; the currently valid abbreviations for short names are shown in parentheses. -watch (-w) OR -WatchDirectory Text string Directory to watch for Alignframes command files, whose extension should be .pcm, or stacks, whose extension should be .mrc or .tif. -output (-o) OR -OutputDirectory Text string Directory in which to write aligned sums and other output. By default, files are written to the command file directory. -processed (-pr) OR -ProcessedDirectory Text string Directory into which frame stacks will be moved when they are finished being processed. If a relative path is entered, it will be treated as relative to the watched directory, not the current directory. When a command file to process a single frame file is done, the program will move the stack, command file, and log file into the directory. If this directory is on another filesystem, the files will be copied there and deleted from their current location. If the command file has options to use a gain reference file or a defect correction file, copies of these files will also be maintained in the processed directory. When a command file to process a whole tilt series is done, nothing is moved unless the last component of the path to the frame files is the same as the tilt series root name. SerialEM will write files in this way if the option to use the file name of the current open file in the name of the frame stack folder is selected in the Frame File Options dialog. In that case, the directory itself is moved to the processed directory, since it can be assumed not to contain other frame stacks to be pro- cessed. If this -processed option is not entered, the program will just rename each -master (-m) OR -MasterCommandFile File name Name of a command file to use if watching for frame stacks instead .pcm files. For each stack, the entries will be modi- fied appropriately, a file will be written with the root name of the stack plus ".pcm", and that file will be run. This master file should have either commands to run Alignframes (includ- ing lines with the options "InputFile" and "OutputImageFile") or entries INPUTFILE and OUTPUTFILE that will be substituted with the names of the frame stack and aligned sum file, respectively. If a file contains Alignframes options to apply a gain refer- ence or defect file, the specified files will be used without any checking for whether there is a newer file that applies to the current frames. Thus, be sure these entries are correct for the current session before using a command file with such entries. -nocom (-noc) OR -NoCommandFile Just watch for frame stacks without running a command file. This option cannot be entered with -master and must be accompa- nied by either -processed, -before, or -after; i.e., there must be something for the program to do with each stack. -age (-ag) OR -MinimumAgeOfStacks Floating point When watching for frame stacks, use this option to set the mini- mum time interval between the last detected change in the modi- fication time of the stack and starting to process the stack. The default is 15 seconds. -gpu (-g) OR -UseGPU Integer Use the GPU (graphical processing unit) for computations; enter 0 to use the best GPU on the system, or the number of a specific GPU (numbered from 1). This entry overrides whatever was origi- nally written into the command file. -binning (-bi) OR -BinningOfSum Integer Image reduction to apply to summed frames; this will override the value in the command files, which was set from the binning of the image returned to SerialEM. -reduce (-r) OR -ReducedSum Integer Amount to reduce image by, or target size, to get a reduced copy of the summed image. A value of 32 or less will be taken as the shrinkage factor, and a value greater than 32 will be taken as the target size to shrink too. In the latter case, the geomet- ric mean of the X and Y size will be divided by the target size and rounded to an integer. The output file will be named as the stack root name followed by "_redN.mrc", where N is the reduc- tion factor. The file will be placed in the same directory as the aligned sum unless a different destination is specified with the -thumb option. If the summed file is a tilt series, only the middle image will be used. -power (-po) OR -PowerSpectrum Integer Height for a JPEG file with a side-by-side reduced sum and reduced power spectrum. The summed image will be reduced by exactly the amount to give the specified height. The power spectrum will be computed on the full-sized summed image and reduced to the given height, so it shows all the frequencies in the summed image. The output file will be named as the stack root name followed by "_power.jpg". The file will be placed in the same directory as the aligned sum unless a different desti- nation is specified with the -thumb option. If the summed file is a tilt series, only the middle image will be used. -scale (-s) OR -ScaleSpectrum Two floats Background gray level near the edge of the power spectrum, and the relative diameter of the annulus around the center whose mean will be scaled to be white. The default gray level is 48, and the default diameter is 0.02. Making the diameter smaller will saturate less area at the center of the power spectrum; making it bigger may give more dynamic range to show Thon rings away from the center. -thumb (-thu) OR -ThumbnailDirectory Text string Directory into which the reduced sums or power spectra will be written. If this option is not entered, they will be placed in the same directory as the aligned sums. -before (-be) OR -RunCommandBefore Text string Command to run before running a command file, if any. The com- mand can include items enclosed in %{...} to be substituted with the actual values for the current frame stack. These items include: %{frameFile} - Frame stack, including original path when watching for .pcm files %{outputFile} - Aligned sum output file, including path %{gainReference} - Gain reference file, including path %{defectFile} - Defect file, including path %{rootName} - Root name of command file/stack %{outputDir} - Directory entered with -output, if any %{processedDir} - Directory entered with -processed, if any %{reducedDir} - Directory entered with -thumb, if any The substituted text will be enclosed in double quotes, unless it is a root name with no spaces included. Despite the quotes, it may be possible to concatenate these items with other text (it works in Linux). The output file will not exist when a com- mand is run before a command file. -after (-af) OR -RunCommandAfter Text string Command to run after running a command file, if any. The com- mand can include the items enclosed in %{...} just described. This command will be run after any moving or renaming of pro- cessed files, so the input stack may no longer exist by the same name. -none (-non) OR -TextForNone Text string Text to substitute in a command entered with -before or -after if the file or directory is not available for a particular item. The default is simply to remove the %{...} item if it has no value. Using a defined text such as "NONE" can make it easier for a script to parse the command, for example, because then each option entered to the script would always be followed by another token instead of a blank. -keep (-k) OR -KeepCommandLogs Keep log files with output from running commands before or after a command file. The default is to write a log then delete it if there is no error. -nice (-ni) OR -NicenessSetting Integer Priority at which to run jobs, a number between 1 and 19, where a higher value has lower priority. All values in this range are meaningfully different on Linux or Mac OS X, but they will be translated into a smaller selection of priority levels on Linux. The default is 15. -thread (-thr) OR -ThreadLimit Integer Limit on number of threads used for processing. This value will be used to set OMP_NUM_THREADS, which sets an upper limit to the number of threads used for operations that are multi-threaded with OpenMP (image reductions, FFTs). This thread number is already limited to the number of real cores when hyperthreading is enabled, but allowing that many threads may be counterproduc- tive when other multi-threaded operations are running. You can determine the number of physical and logical cores by entering "imodqtassist -t" in a terminal. -help (-h) OR -usage Print help output -StandardInput Read parameter entries from standard input EXAMPLES To run command files tha appear in the current directory with no optional modifications, enter framewatcher To watch for files in a directory "X:\MyFrames" and move them to a sub- directory "Processed" when done, enter (in a DOS terminal on Windows) framewatcher -watch X:\MyFrames -proc Processed where the directory would be entered as "X:/MyFrames" in a Cygwin win- dow. To make sums with image reduction by 2 and run on a specific GPU, say # 2, enter framewatcher -bin 2 -gpu 2 where you could enter "-gpu 0" to run on the best available GPU even if the command file does not specify using a GPU. AUTHOR David Mastronarde SEE ALSO alignframes Email bug reports to mast at colorado dot edu. IMOD 4.9.10 framewatcher(1)