framewatcher(1)             General Commands Manual            framewatcher(1)



NAME
       framewatcher - watch for command files or stacks to run with align-
       frames

SYNOPSIS
       framewatcher  [options]

DESCRIPTION
       Framewatcher is a Python program that can watch for either command
       files or frame stacks appearing in a directory.  The command files are
       expected to run the program Alignframes and to have extension .pcm,
       the extension on files produced by the SerialEMCCD plugin to DigitalMi-
       crograph.  The program can modify a few of the parameters in such a
       file with the options below.  The frame stacks must have extension .mrc
       or .tif.  Output files with aligned sums can be placed in a different
       directory.  Stacks and command files can be moved to another directory
       when they are finished.  Whole tilt series can be processed when a com-
       mand file specifies the metadata (.mdoc) file for the tilt series
       stack.

       To process frame stacks, either a master command file for alignment
       must be supplied or the -nocom option must be entered; in either case,
       the program will watch only for stacks, not for command files.  The
       master command file can run Alignframes, in which case the same
       parameters can be modified before running on each stack as when watch-
       ing for command files.  Alternatively, the command file can run other
       software, in which case it should contain the text "INPUTFILE" and
       "OUTPUTFILE" wherever the input and output filenames are needed,
       respectively.  The name of the stack will be substituted for "INPUT-
       FILE", and the name of the output file for the aligned sum, including
       the optional output directory, will be substituted for "OUTPUTFILE".  A
       command file will be written with the root name of the stack plus the
       extension ".pcm".

       In either case, the command file is run with Vmstopy in the direc-
       tory where it is located and produces a log file there.  If files are
       not being moved elsewhere when done, the command file is renamed with
       ".done" added at the end, which is how the program keeps from rerunning
       stacks or command files even if restarted.  If no command file at all
       is run, ".done" will be added to the stack name itself unless it is
       moved elsewhere.

       An arbitrary command can be run before or after the command file is run
       (or a stack is moved or renamed, when there is no command file).  Items
       such as the root name of the data can be substituted into the command,
       as described below.  This may provide a more flexible way to run other
       alignment software than the master command file.

       The program works by looking in the directory for eligible files and
       running all of the ones that it sees.  When it sees no files, it waits
       5 seconds before scanning the directory again.  When it see stacks, it
       runs all but the last one immediately, then waits until the last one
       (which typically would be the only one seen) has not changed for at
       least 15 seconds.

       Entrl Ctrl C to terminate the program.  This will kill any running
       alignment operation.

OPTIONS
       Framewatcher uses the PIP package for input (see the manual page for
       pip).  Options can be specified either as command line arguments
       (with the -) or one per line in a command file (without the -).
       Options can be abbreviated to unique letters; the currently valid
       abbreviations for short names are shown in parentheses.

       -watch (-w) OR -WatchDirectory      Text string
              Directory to watch for Alignframes command files, whose
              extension should be .pcm, or stacks, whose extension should be
              .mrc or .tif.

       -output (-o) OR -OutputDirectory    Text string
              Directory in which to write aligned sums and other output.  By
              default, files are written to the command file directory.

       -processed (-pr) OR -ProcessedDirectory       Text string
              Directory into which frame stacks will be moved when they are
              finished being processed.  If a relative path is entered, it
              will be treated as relative to the watched directory, not the
              current directory.  When a command file to process a single
              frame file is done, the program will move the stack, command
              file, and log file into the directory.  If this directory is on
              another filesystem, the files will be copied there and deleted
              from their current location.  If the command file has options to
              use a gain reference file or a defect correction file, copies of
              these files will also be maintained in the processed directory.
              When a command file to process a whole tilt series is done,
              nothing is moved unless the last component of the path to the
              frame files is the same as the tilt series root name.  SerialEM
              will write files in this way if the option to use the file name
              of the current open file in the name of the frame stack folder
              is selected in the Frame File Options dialog.  In that case, the
              directory itself is moved to the processed directory, since it
              can be assumed not to contain other frame stacks to be pro-
              cessed.  If this -processed option is not entered, the program
              will just rename each

       -master (-m) OR -MasterCommandFile       File name
              Name of a command file to use if watching for frame stacks
              instead .pcm files.  For each stack, the entries will be modi-
              fied appropriately, a file will be written with the root name of
              the stack plus ".pcm", and that file will be run.  This master
              file should have either  commands to run Alignframes or
              entries INPUTFILE and OUTPUTFILE that will be substituted with
              the names of the frame stack and aligned sum file, respectively.
              If a file contains Alignframes options to apply a gain refer-
              ence or defect file, the specified files will be used without
              any checking for whether there is a newer file that applies to
              the current frames.  Thus, be sure these entries are correct for
              the current session before using a command file with such
              entries.

       -nocom (-noc) OR -NoCommandFile
              Just watch for frame stacks without running a command file.
              This option cannot be entered with -master and must be accompa-
              nied by either -processed, -before, or -after; i.e., there must
              be something for the program to do with each stack.

       -age (-ag) OR -MinimumAgeOfStacks   Floating point
              When watching for frame stacks, use this option to set the mini-
              mum time interval between the last detected change in the modi-
              fication time of the stack and starting to process the stack.
              The default is 15 seconds.

       -gpu (-g) OR -UseGPU      Integer
              Use the GPU (graphical processing unit) for computations; enter
              0 to use the best GPU on the system, or the number of a specific
              GPU (numbered from 1).  This entry overrides whatever was origi-
              nally written into the command file.

       -binning (-bi) OR -BinningOfSum     Integer
              Image reduction to apply to summed frames; this will override
              the value in the command files, which was set from the binning
              of the image returned to SerialEM.

       -dtotal (-dt) OR -FixedTotalDose    Floating point
              Total dose for each set of frames, in electrons/square Angstrom.
              This entry will lead to a corresponding entry in the .pcm file,
              replacing an existing one if any, and remove entries for Dose-
              FileType or FixedFrameDoses.  This option cannot be entered with
              -dframe.

       -dframe (-df) OR -FixedFrameDoses   Floating point
              List of frame doses and numbers that applies to all sets of
              frames.  The list is a set of paired numbers: a dose in elec-
              trons/square Angstrom and the number of frames taken at that
              dose (e.g., enter "2,10,4,5" for 10 frames of 2 e/A^2 and 5
              frames of 4 e/A^2).  This entry will lead to a corresponding
              entry in the .pcm file, replacing an existing one if any, and
              remove entries for DoseFileType or FixedTotalDose' This option
              cannot be entered with -dtotal.

       -dprior (-dp) OR -InitialPriorDose       Floating point
              Dose applied before any of the images in each frame set were
              taken; this value will lead to a corresponding entry in the .pcm
              file, replacing an existing one if any.

       -volt (-v) OR -Voltage    Integer
              Microscope voltage in kV; this value must be either 200 or 300.
              This entry will lead to a corresponding entry in the .pcm file,
              replacing an existing one if any; if none is entered, the
              default in Alignframes is 300 kV.

       -reduce (-r) OR -ReducedSum    Integer
              Amount to reduce image by, or target size, to get a reduced copy
              of the summed image.  A value of 32 or less will be taken as the
              shrinkage factor, and a value greater than 32 will be taken as
              the target size to shrink too.  In the latter case, the geomet-
              ric mean of the X and Y size will be divided by the target size
              and rounded to an integer.  The output file will be named as the
              stack root name followed by "_redN.mrc", where N is the reduc-
              tion factor.  The file will be placed in the same directory as
              the aligned sum unless a different destination  is specified
              with the -thumb option.  If the summed file is a tilt series,
              only the middle image will be used.

       -power (-po) OR -PowerSpectrum      Integer
              Height for a JPEG file with a side-by-side reduced sum and
              reduced power spectrum.  The summed image will be reduced by
              exactly the amount to give the specified height.  The power
              spectrum will be computed on the full-sized summed image and
              reduced to the given height, so it shows all the frequencies in
              the summed image.  The output file will be named as the stack
              root name followed by "_power.jpg".  The file will be placed in
              the same directory as the aligned sum unless a different desti-
              nation is specified with the -thumb option.  If the summed file
              is a tilt series, only the middle image will be used.

       -scale (-s) OR -ScaleSpectrum       Two floats
              Background gray level near the edge of the power spectrum, and
              the relative diameter of the annulus around the center whose
              mean will be scaled to be white.  The default gray level is 48,
              and the default diameter is 0.02.  Making the diameter smaller
              will saturate less area at the center of the power spectrum;
              making it bigger may give more dynamic range to show Thon rings
              away from the center.

       -thumb (-thu) OR -ThumbnailDirectory     Text string
              Directory into which the reduced sums or power spectra will be
              written.  If this option is not entered, they will be placed in
              the same directory as the aligned sums.

       -before (-be) OR -RunCommandBefore       Text string
              Command to run before running a command file, if any.  The com-
              mand can include items enclosed in %{...} to be substituted with
              the actual values for the current frame stack.  These items
              include:
                %{frameFile} - Frame stack, including original path when
              watching for .pcm files
                %{outputFile} - Aligned sum output file, including path
                %{gainReference} - Gain reference file, including path
                %{defectFile} -  Defect file, including path
                %{rootName} - Root name of command file/stack
                %{outputDir} - Directory entered with -output, if any
                %{processedDir} - Directory entered with -processed, if any
                %{reducedDir} - Directory entered with -thumb, if any
               The substituted text will be enclosed in double quotes, unless
              it is a root name with no spaces included.  Despite the quotes,
              it may be possible to concatenate these items with other text
              (it works in Linux).  The output file will not exist when a com-
              mand is run before a command file.

       -after (-af) OR -RunCommandAfter    Text string
              Command to run after running a command file, if any.  The com-
              mand can include the items enclosed in %{...} just described.
              This command will be run after any moving or renaming of pro-
              cessed files, so the input stack may no longer exist by the same
              name.

       -none (-non) OR -TextForNone   Text string
              Text to substitute in a command entered with -before or -after
              if the file or directory is not available for a particular item.
              The default is simply to remove the %{...} item if it has no
              value.  Using a defined text such as "NONE" can make it easier
              for a script to parse the command, for example, because then
              each option entered to the script would always be followed by
              another token instead of a blank.

       -keep (-k) OR -KeepCommandLogs
              Keep log files with output from running commands before or after
              a command file.  The default is to write a log then delete it if
              there is no error.

       -nice (-ni) OR -NicenessSetting     Integer
              Priority at which to run jobs, a number between 1 and 19, where
              a higher value has lower priority.  All values in this range are
              meaningfully different on Linux or Mac OS X, but they will be
              translated into a smaller selection of priority levels on Linux.
              The default is 15.

       -thread (-thr) OR -ThreadLimit      Integer
              Limit on number of threads used for processing. This value will
              be used to set OMP_NUM_THREADS, which sets an upper limit to the
              number of threads used for operations that are multi-threaded
              with OpenMP (image reductions, FFTs).  This thread number is
              already limited to the number of real cores when hyperthreading
              is enabled, but allowing that many threads may be counterproduc-
              tive when other multi-threaded operations are running.  You can
              determine the number of physical and logical cores by entering
              "imodqtassist -t" in a terminal.

       -help (-h) OR -usage
              Print help output

       -StandardInput
              Read parameter entries from standard input


EXAMPLES
       To run command files tha appear in the current directory with no
       optional modifications, enter

          framewatcher

       To watch for files in a directory "X:\MyFrames" and move them to a sub-
       directory "Processed" when done, enter (in a DOS terminal on Windows)

          framewatcher -watch X:\MyFrames -proc Processed

       where the directory would be entered as "X:/MyFrames" in a Cygwin win-
       dow.

       To make sums with image reduction by 2 and run on a specific GPU, say #
       2, enter

          framewatcher -bin 2 -gpu 2

       where you could enter "-gpu 0" to run on the best available GPU even if
       the command file does not specify using a GPU.

AUTHOR
       David Mastronarde

SEE ALSO
       alignframes

       Email bug reports to mast at colorado dot edu.



IMOD                                4.10.10                    framewatcher(1)