serieswatcher(1) General Commands Manual serieswatcher(1)
NAME
serieswatcher - watch for stacks to run with batchruntomo
SYNOPSIS
serieswatcher [options]
DESCRIPTION
Serieswatcher is a Python program that can watch for tilt series stacks
to be created in a directory, and start an automated reconstruction
with Batchruntomo after the tilt series is done. It can handle sin-
gle-axis and dual-axis data sets; in the latter case it can either wait
until both axes are finished, or start the reconstruction when the
first axis is present and do another run to complete the reconstruction
when the second axis is finished. The program can also be used simply
to transfer stacks from one directory to another, with another copy of
the program used to run the reconstructions when it sees the stacks in
their new location.
To start reconstructions, the program needs a batch directive file and
a command file for running Batchruntomo. Although Etomo does not
yet have an interface for managing this program, its batch interface
can be used to prepare both of these files even without adding any
stacks to the table there. If you do add a stack, it will be ignored,
but doing so would enable you to set the two parameters that are speci-
fied only in the table: whether the sets are dual-axis, and whether
gold beads are on two surfaces. Otherwise, you would need to specify
these parameters with the -dual and -two options. The directives for
these parameters are located in the data set specific ".adoc" file, the
one whose name has the form "batchroot_dataset.adoc". Note that the
program detects whether each tilt series is a montage or not and man-
ages the montage entry in the batch directive file.
When you use the -project option to enter the root name of the batch
project created to specify parameters here, the program will access the
files from this project, including the ".ebt" project file, and use the
latter as a framework for creating a new batch project containing all
of the data sets that were run. You can open this project in Etomo and
use the table on the Run tab to examine logs or reconstructions or open
the individual data sets. As long as Serieswatcher is running, the
file will be named with the new batch root followed by ".ebt.active".
You should not open this file in Etomo, since both programs would then
be writing to it. Rather, make a copy with a different name and open
that in etomo. When Serieswatcher exits (including after a Ctrl-C
interrupt) it will rename an existing ".ebt" file to ".ebt~" and rename
the new file to be ".ebt".
When processing dual axis sets, the -match entry of what filenames to
match determines what will be done. If there is neither an "a" nor a
"b" before the extension, the program will watch for all matching
stacks. When it sees a first-axis stack it will create a data set and
start a reconstruction of that axis only. When the second-axis stack
appears, it will start a separate run to reconstruct that axis and fin-
ish combining. If the entry includes an "a" before the extension, then
the program will watch for first-axis stacks only, and just start the
first-axis reconstructions. A second run of this program would be
needed to finish the reconstructions with the second axis. If the
entry has a "b" before the extension, it will watch for second-axis
stacks only. If both stacks are present when the "b" stack is fin-
ished, it will start a full dual-axis reconstruction. If the "a" stack
is no longer present and the data set directory already exists with
sizable files, then it will start a reconstruction of the second axis
plus the combine. Thus, the "b" entry would be used either for the
second run of Serieswatcher after running it with an "a" entry, or to
make it wait for the second axis and process both axes in one run of
Batchruntomo. There is an important restriction whichever way it is
done: the first axis acquired must be named with "a", not "b".
In the general case, the program decides that a stack is finished
because it has not been modified for the amount of time specified by
the -age option, then it checks its tilt range or number of images to
see if it is above a minimum. If not, the program will retry periodi-
cally to see if it either starts changing again or passes the tests.
This process if much more reliable if a version of SerialEM is used
that creates a file with the stack filename plus extension ".openTS"
when it starts the tilt series, and deletes that file when the stack is
closed (version 3.7.4 or higher, or 3.8.0 beta from April 18, 2019 or
later). Also, as of IMOD 4.10.34, Alignframes also creates a ".openTS"
file. With such a signal, Serieswatcher can safely proceed right after
the ".openTS" disappears. If you do stop a tilt series after a small
number of tilts, close the file, and start a new series with the same
file name, the program should be able to reject the first stack and
start watching the new one when it sees it.
If you press Ctrl-C while any reconstructions are running, the program
will ask if you want to quit all processing immediately, or just exit
the program and allow any current data sets to finish.
OPTIONS
Serieswatcher uses the PIP package for input (see the manual page for
pip). Options can be specified either as command line arguments
(with the -) or one per line in a command file (without the -).
Options can be abbreviated to unique letters; the currently valid
abbreviations for short names are shown in parentheses.
-watch (-w) OR -WatchDirectory File name
Directory to watch for tilt series stacks. The default is the
current directory.
-deliver (-del) OR -DeliverToDirectory File name
Move each stack to this directory and process it in a subdirec-
tory. This entry is not needed if the batch command file has
the corresponding entry, but it does override the one in the
command file, if any. If the program is being used just to
deliver stacks, this option must be entered. Even when the
delivery is specified in the command file, the file is moved to
the specified directory by Serieswatcher, not Batchruntomo,
because Serieswatcher is more capable of moving files between
filesystems. Without an entry here or in the command file, data
sets will be processed in subdirectories of the watched direc-
tory, because Batchruntomo is run with its -make option.
-remote (-re) OR -RemoteDirectory File name
Path on remote machines to directory from which this program is
started. This entry is not needed if the batch command file has
the corresponding entry, but it does override the one in the
command file, if any. This would be necessary if you move a
batch command file from the place where it was generated with
such an entry.
-match (-m) OR -MatchPatternOrExt Text string
Filename pattern or extension to match. This entry can have
wild card characters for selecting filenames, namely * to match
multiple characters, ? to match a single character, and [...] to
match a single character matching one that is inside the brack-
ets, where ranges can be included (e.g., [cd0-9]. (If you enter
such a pattern at the command line, you need to enclose it in
quotes.) If no wild cards are included, a "*" will automati-
cally be added before the entry. Thus, an entry of ".st" will
make it watch for files with extension ".st". The default pat-
tern is ".mrc".
-age (-ag) OR -MinimumAgeOfStacks Floating point
Minimum time interval in seconds between the last detected
change in the modification time of a stack and starting to
process the stack when no ".openTS" file was ever seen. The
default is 300 seconds.
-opents (-o) OR -MinHoursIfOpenTSPresent Floating point
Minimum time interval in hours between the last detected change
in the modification time of the stack and starting to process
the stack when a ".openTS" is still present. The default is 17
hours. Enter a large number to prevent such stacks from ever
being processed.
-range (-ra) OR -MinimumTiltRange Floating point
Minimum range of tilt angles before accepting an apparently
mature stack for processing. The default is 40; enter a larger
number if there there is some chance that you will be restarting
data sets that failed with bigger tilt ranges than this.
-views (-v) OR -MinimumNumberOfViews Integer
If the tilt range cannot be determined, the program tests
whether the number of views in a tilt series stack meets this
minimum before accepting an apparently mature stack for process-
ing. The default is 12; enter a larger number if there are no
tilt angles in your stacks and there is some chance that you
will be restarting data sets that failed with bigger sizes than
this.
-project (-pr) OR -EtomoProjectRoot File name
Root name of starting Etomo project, including path if neces-
sary. When files are supplied in this way, the program will
make a complete batch project that can be run in Etomo. There
must be a ".com" and a ".ebt" file with this root name, and
there must be at least one ".adoc" file starting with the root
name. Specifically, if any ".adoc" file with the data set name
included, it will use the first such file that it finds and
ignore the master ".adoc" file without a data set name, other-
wise it will use the master ".adoc" file. Neither -com nor
-adoc can be entered with this option.
-com (-co) OR -CommandFile File name
Command file for running Batchruntomo. This file can be gener-
ated in the Etomo batch interface without adding any stacks,
allowing you to specify a directory to deliver, templates, and
the CPU and GPU resources and to have the remote directory set,
if any. If you do add a stack, you can also set whether the
data sets are dual axis and whether there is gold on two sur-
faces.
-adoc (-ad) OR -DirectiveFile File name
Batch directive file for processing stacks with Batchruntomo.
This option must be entered if a command file is.
-check (-ch) OR -CheckFile File name
File to check for quit and finish signals. The default is
serieswatcher.###.input in the directory where this program is
started, where ### is the process ID. A Q in this file will
make the program kill all of the current reconstructions then
exit; an F will make it allow those to finish then exit. In
either case, the .ebt.active file will be updated with an appro-
priate status for each set processed.
-dual (-du) OR -DualAxis Integer
1 for dual-axis data, 0 if not. This option is not needed if it
is specified in the batch directive file or a template file, but
if it is entered it will override any directives. The default
is 0.
-two (-t) OR -TwoSurfaces Integer
1 if beads are on two surfaces, 0 if not. This option is not
needed if it is specified in the batch directive file or a tem-
plate file, but if it is entered it will override any direc-
tives. The default is 0.
-cpus (-cp) OR -CPUMachineList Text string
Machines to use in parallel processing, or number of local
cores. Each machine name can be followed by a ":" (colon), fol-
lowed by the number of CPUs to use in that machine; e.g.,
druid:4,tubule:4. This entry is required if there is no CPUMa-
chineList entry in the batch command file, and it would override
one in the command file.
-gpus (-g) OR -GPUMachineList Text string
Machines to use for parallel processing with a GPU, or 1 for
local GPU. The list has the same format as the -G entry to Pro-
cesschunks, with multiple or specific GPUs specified by colon-
separated numbers after a machine name, e.g.,
druid:2,tubule:1:2. The GPUMachineList entry in the batch com-
mand file will be used if this option is not entered, and
-parallel (-pa) OR -ParallelRuns Integer
Number of data sets to process in parallel runs of Process-
chunks, with each data set assigned to a separate set of CPUs,
referred to as a "machine slot". When you select parallel runs,
the total cores in the CPU list will be divided as evenly as
possible among the machine slots. If there are multiple comput-
ers being used, this may result in some slots having all cores
on one machine and some having cores on more than one machine,
unless the number of cores per computer is a multiple of the
number being assigned to each slot. The collection of GPUs will
be managed by Gpuallocator, just as when running parallel
batch reconstructions with Splitbatch, and the entry to
Batchruntomo for -MaxGPUsInParallelBatch will determine the
maximum number granted to one run when it requests it.
-etomo (-e) OR -EtomoOptions Integer
This entry is a sum of flags useful to Etomo:
1 to skip renaming the active ".ebt" file at the end by remov-
ing ".active"
2 to skip cleaning out existing rows from the incoming ".ebt"
file
4 to start adding row numbers past the highest row number
found in the incoming .ebt file, even if existing rows are
cleaned out
-DebugMode (-Deb) OR -debug
Print debugging output
-help (-h) OR -usage
Print help output
-StandardInput
Read parameter entries from standard input
Examples
The following steps can be used to run this program with a minimum num-
ber of entries and make a new project that can be opened in Etomo:
1) Change directory to the location where the stacks will be cre-
ated.
2) Start Etomo and open the Batch Tomograms interface.
3) If data sets are to be processed in subdirectories of a different
directory, select Move all stacks to dataset directories under: and
select the directory with the chooser. Otherwise, it does not matter
which of the other two options are selected.
4) Select a starting directive file and template files if desired.
5) Change the root name if you want to; it determines the names of
the command and directive files that you will provide to Serieswatcher.
The command line below assumes the name was changed to "forWatcher".
6) If you want to add a stack in order to specify dual-axis data or
beads on two surfaces, switch to the Stack tab and do so.
7) Enter all parameters as usual on the Dataset Values tab.
8) On the Run tab, select the CPU and GPU resources to use as usual.
If you plan on parallel runs, select enough to be divided among the
runs. You can also specify a step to end after.
9) Close Etomo.
10) Run this program with:
serieswatcher -proj forWatcher
plus additional options as needed:
-dual and -two if appropriate and not specified in step 6.
-match if the stack extension is not ".mrc" or you want to specify a
more restrictive filename pattern.
-parallel to run multiple reconstructions in parallel.
-age with a different time since last modication if appropriate
(e.g., a shorter time if stacks are being moved to this watched direc-
tory by a Serieswatcher on the acquisition compution, or a longer time
if you do not have ".openTS" files).
AUTHOR
David Mastronarde
SEE ALSO
batchruntomo, processchunks
Email bug reports to mast at colorado dot edu.
IMOD 5.2.0 serieswatcher(1)