Multiple Grid Operations Dialog

This dialog provides for acquisition from multiple grids using an autoloader.  The operations can occur in three phases: acquisition of grid maps at low maginification, acquisition of medium magnification maps using the 'Mapping' parameters of the Navigator Acquire at Items dialog, and acquisition of final data with the 'Final Data' parameters of Acquire at Items.  Navigator files are managed throughout the process.  Information about the grids and some of the parameter settings are saved automatically in a 'session' file in the autodoc format, with a name of the form 'multiGrid_date_#.adoc', saved into the top working directory for all the files from the runs.  The dialog has controls in different sections that can be closed when not relevant, and a Close Right button at the bottom that can be used to show just the grid table and none of the sections on the right.  The +/- button at the upper left can be used to close up or reopen the whole dialog area.

In principle, the multiple grid capabilities provided here allow all phases to be done in sequence during one run on a set of grids.  The use of imaging states is crucial to being able to run more than one phase at a time with predictable results.  The dialog thus includes drop-down boxes with all of the states managed in the Imaging State dialogs.  For states with names, the boxes show the state index, the type of state, a 0 if the camera is not currently available, the magnification, and the name.   The type can be S, V, R, F, T, or - as in the Imaging States dialog, plus RM for a Low Dose state or -M for a non-Low Dose state that sets Mont-map parameters.  For states without names, the box shows the index, type, camera number or 0 if not available, magnification, and spot/probe mode.  For grid mapping, only one state can be selected, one directly related to the camera parameter set to be used for the montage.  For medium magnification mapping and final acquisition, four states can be selected because up to 5 states can be set in Low Dose and because one can imagine needing three for doing related tasks plus the primary acquisition.  If a non-Low Dose state is selected, only one state can be set.  The program will reject selections that are inconsistent or redundant, such as selecting a non-Low Dose state when acquiring with View or Search.  The controls in the Imaging State dialog for deleting, updating, or renaming an existing state are disabled while the multi-grid dialog is open.  If you need to change these settings, just close and reopen this dialog.

The first time that the procedure is run on a set of grids, the routine needs to collect an image similar to those which will be taken for the grid map, but with an unobstructed beam.  This is necessary to allow proper evaluation of whether a grid is too dark overall, and to pick optimal locations for finding eucentricity and for realigning to the grid map.  Thus, it has to unload a grid on the stage, take this image, and load the first grid.  When starting the procedure on later phases, or doing more grid maps after the reference counts have been measured, the routine will always start with the grid on the stage.

Before taking a grid map, the program will initially examine up to four candidate locations for running the coarse eucentricity to find a place that is not too dark.  If a suitable position is found, it runs the rough eucentricity routine so that the map will be taken at the eucentric height.  Otherwise, it will defer this operation until the map is done, using information gathered during the montage to pick a good location near the center if possible.  If the property MaximumTiltAngle is set to less than 8 degrees, the progam will skip doing eucentricity.

When a grid is reloaded after mapping, the multi-grid routine must be able to apply a stored shift from a Shift to Marker to the Navigator items, unless it sees that such a shift has already been applied.  A grid has a different center point and rotation every time it is reloaded, so whenever a grid is reloaded for the later phases of operation, it is realigned to the grid map (see Realign to Map below and the 'RealignReloadedGrid' script command).  Navigator points are transformed based on the shift and rotation. To give a better registration with stage positions at higher magnifications, a one-time Shift to Marker has to be applied to the grid map and any items already marked on it.  Thus, in general, such a shift needs to be stored before running the later phases.  If the Shift to Marker dialog is opened while this dialog is open, its radio buttons will be set appropriately for storing a shift.   The 'ShiftItemsByMicrons' script command can be used instead to apply a shift that will satisfy this requirement.

If you are doing Multiple Records from multiple holes during final data acquisition, there is another situation where a pre-existing transformation is important.  In this case, the hole vectors stored with a map after hole finding are used to set the current image shift vectors for points on that map.  They are also transformed automatically to work at the Record magnification if an appropriate transformation has been saved from the 'Step To and Adjust IS' procedure in the Multiple Record Setup dialog.

Managing operations on a single grid with no autoloader: This dialog may also be useful for managing the workflow when there is just one manually loaded grid.  When the property 'ScopeHasAutoloader 0' is present, the menu entry appears as 'Multiple Operations on Grid', the dialog is simplified to eliminate irrelevant controls, and autoloader-related operations are omitted from the sequence of steps in each phase.  File names are based only on the Prefix entry, so the name entry in the table is irrelevant.  Controls for setting states and parameters for individual grids are omitted.  There are two special requirements:

• Before running the first step, the grid map, you need to have an estimate of the number of counts in a blank image (beam only) taken with the same parameters and conditions as for the montage images (i.e., same exposure, binning, magnification, and beam conditions).  It does not need to be very accurate; an underestimate is better than an overestimate.
• When a run is started with medium-mag mapping or final acquisition using an existing grid map, the step for realigning to that map is skipped.  If you do remove and reinsert a grid, you would need to use the Realign to Map button before running operations.

Grid Setup and the Grid Table

The top section of the dialog has a table showing the grids that are in the autoloader, omitting any empty cartridges.  The first column has a set of checkboxes to select which grids will be included in a multi-grid operation, labeled by the order in which they would be run with the current selection of options.  The second column is a set of radio push-buttons for selecting one grid for which to do one of the operations in the row of buttons below the table.  The push-buttons are labeled with the slot number for Thermo/FEI scopes or the ID number for JEOL scopes.  Pressing a button with the Ctrl key held down select that grid and load its navigator file, if one exists.  The third column contains indicators of whether the grid has separate states or parameter settings and a status code showing how far the grid has been processed.  The fourth column has text boxes to allow you to see and edit the names for each grid.  The name of the currently loaded grid will be shown in bold text and its box will have a green border.  Once the names are fixed, a small note can be added in the text box below the table and will show up after the name here.

Grid Setup: the controls in ths section determine where files will be located and how they will be named. They are disabled and their values are locked in when the first run is started; the name text boxes are also disabled.  The controls and outputs here are:

Run Inventory

Use this button to run an inventory of the autoloader contents.  An inventory is needed to determine which slots are occupied.  On a JEOL scope, this is a relatively quick operation that must be run from SerialEM and populates an internal table of grid information.  On a Thermo/FEI scope, it is a time-consuming operation that need not be run from SerialEM and can be run just as well with the button in the Autoloader panel of the TEM User Interface.  The inventory can be stopped only from that panel, not by SerialEM.

Get Names

Use this button to set up the dialog table with a row for each available grid.  If there is a name in the microscope's autoloader interface, it will be used; otherwise the name will be 'Car' followed by the slot or ID number.  Spaces are replaced by underscores, and a variety of special characters are replaced by '@'.  The goal is to eliminate not just characters that are illegal in filenames, but also characters that can cause trouble when using scripting and shell commands to manipulate filenames.  When a name is edited, unacceptable characters are immediately replaced.  Properties could be added to provide more flexibility (retention of spaces, use of '^' instead of '@', or retention of characters that are legal in filenames). 

For Thermo/FEI scopes, it is not possible for SerialEM to determine if there is a grid on the stage unless SerialEM loaded it.  The program will ask you to enter the slot number of the grid on the stage, or 0 if there is none.  The entry box will default to an unoccupied slot that has a name, or an unoccupied slot surrounded by filled ones, but only if there is just one such slot.

Reset

Use this button to discard your name changes and revert to the names from the microscope or the default names provided.  The program will ask you to confirm this action if you have edited names.

Clear

Use this button to clear out all the names, which might be convenient if you are going to modify all of them. The program will ask you to confirm this action if you have edited names.

Prefix

Add an entry to this text box to have the root name for each individual grid start with the prefix.  Again, unacceptable characters will be replaced as you add them.  The prefix can include a preferred way to refer to the cartridges or grids.  Specifically, if the prefix ends with 'Car', 'car', 'Grid' or 'grid', or one of these followed by '_', then the slot or ID number will be appended after this without the default '_C'.

Append names

Select this option to have the names shown in the table added to the root name, after an underscore.

Root name

 This line shows the form of the root name, with '##' for the slot or ID number, and 'name' for the individual name.

Use subdirectories

Select this option to have files placed in a subdirectory for each grid, named by the root name.

Current dir/Session dir

Before any runs have been done, this line shows the program's current working directory; afterwards, it is labeled 'Session dir' and shows the location where this session's files are.

Set Current Dir

Use this button to set the program's current directory, the same as can be done through the File menu.

Grid Actions:

The first row of controls consists of:

Unlabeled Toggle All checkbox

Turning this checkbox on or off will turn the Run checkboxs on or off for all grids.

Set Order

Use this button to set a custom order for running the grids.  After pressing the button, press the selection push-button for each grid in turn until the order numbers afterthe checkboxes are the way you want them.  Press the button again if you are done before pressing all the selection buttons; the button will reset automatically if you do select every grid in turn.  Another way to set order is to  astart with all grids turned off, or only the first one turned on.  Press the button and turn on the checkbox for each grid in order.

Reset

Use this button to discard the custom run order amd return to the order defined by current option settings.  This button is enabled only when there is a custom order set.

Note

This edit box is enabled after the first run, when the names can no longer be edited.  It allows you to add a note for the grid selected by the push-buttons in the table.  The note will appear after the grid name in the edit box for that grid.

The second row of buttons below the table operate on the grid selected by the push-buttons in the table.

Open Nav

Use this button to open the Navigator file for the selected grid.  It is disabled if there is a no Navigator file or if it is already open.  If the current Navigator table has changes, you will be asked whether you want to save them.

Load Grid

Use this button to load the selected grid.  The program will also load the Navigator file if it exists.

Realign to Map

Use this button to run the routine to realign to the grid map for the selected grid, which is necessary after reloading the grid.  The program will analyze the image intensity information stored in the map file's '.mdoc'  to determine suitable locations for determining the alignment.  It will align to a central position and find the rotation angle that best aligns with the map.  Then it will move to 4 peripheral positions find the shift at each of those, assuming the rotation found initially.  From all of these alignment positions, it determines a more accurate rotation and shift, and uses them to transform the Navigator coordinates of the map and any items at the same registration.

Params

Press this button to open the Multiple Grid Settings Manager and store some separate settings for the currently selected grid.  Up to 5 kinds of settings can be stored: for final data acquisition, autocontouring, hole finding, multiple records, and some general settings that would typically be different for gold grids.  The Navigator acquisition parameters are stored in a separate file, named as the root name of the session file plus '_navAcq.txt'.  Other parameters are stored as arrays of values within the grid sections of the session autodoc file.  When a grid has any one of these kinds of parameters stored, the second character of the status indicator will be 'P' instead of '—'.

Logs

Press this button to open any log files from running Acquire at Items on the grid into secondary log windows.  Log files are managed by the routine, with a main log that is repeatedly reopened for top-level information, and a separate log for each Acquire at Items run.  If you save the log to a file before starting a run, that file will be retained as the main log; otherwise the log will be named based on the session file name.  Log files for each grid are in the subdirectories and named like the grid-level files, with '_MMM' or '_final' for medium mag or final runs, respectively.  Similarly to other situations where log files are automatically managed, the first log with a particular name will have no number attached, and further logs (in later runs) will be numbered with '-1', -'2', etc.  When you open log files with this button, their title bars will contain 'SECONDARY Log:' and the file name, and they will have a pale yellow background.  If you make changes in one of these logs, you can save it with the File menu item Save Secondary Log, which operates on the last secondary log that had focus.  If you press this button while some of the grid's logs are already open, the ones that are are open will be raised to the foreground and any that are not will be opened.

Acquire Grid Maps

This section of the dialog contains parameters for acquiring grid maps; some of them also apply when realigning to a reloaded grid.  Grid maps must be montages.

Use: LD Search  -  LD View  -  Current or set state
Set state

The radio buttons, possibly together with a selected state to be set, determine what type of image will be used to acquire the montage.  LD Search and LD View will result in taking the montage in Low Dose mode with Search or View, respectively.  If a state is selected in the drop-down box, it must be consistent, i.e., it must be a Low Dose S or V state. 

The situation is more complicated if Current or set state is selected. If Set state is also selected, the state picked in the drop-down box determines the acquisition.  If no state is to be set, then the current settings govern. The rules are:

• Setting a Low Dose S, V, R, or RM state will result in a map taken in the given state with Search, View, Record, or Mont-map respectively. 
• Setting a non-Low Dose state will make the map be taken outside of Low Dose with either Mont-map parameters for a '-M' state or Record otherwise.
• Outside of Low Dose with no state set, the montage will be taken at the current magnification with Record parameters, or with Mont-map parameters if 'Use Montage Mapping, not Record parameters' is set in the Montage Setup dialog. 
• In Low Dose mode with no state set and a currently defined area, it will be taken with Search or View if currently in one of those areas, or with Record or Mont-map if in another area, depwnding on whether 'Use Montage Mapping, not Record parameters' is set in the Montage Setup dialog. 
• In Low Dose mode with no area is defined yet, it will be taken with Search or View if 'Use Search in Low Dose mode' or 'Use View in Low Dose mode' are selected in the Montage Setup dialog, or with Mont-map if 'Use Montage Mapping, not Record parameters' is set in the Montage Setup dialog; otherwise the program will give an error when trying to set up the montage.

Remove objective aperture

Select this option to have the objective aperture removed before starting the grid maps.  The aperture will also be removed in later phases if grids are reloaded and maps have to be realigned to.  Apertures are always restored to their current state at the end of a run.

Select this option and fill in a size to have the condenser aperture changed to the given size before starting grid maps.  Again, this size will used when realigning after reloading a map.  For a JEOL with no 'ApertureSizes' property with actual condenser aperture sizes, this must be a size index between 1 and 4.

Montage: Full grid  -  N x N pieces

Use these radio buttons to select whether to do a full grid montage, the same as would be done with the Navigator - Montaging and Grids - Setup Full Montage menu item, or a montage with a specified number of pieces in X and Y.  In the latter case, fill in the number of pieces.

Use overlap of xx %

A smaller than usual overlap factor may be possible with low-magnification maps.  Select the check box and fill in a value to set a specific overlap; otherwise you would have to change it in the Montage Setup dialog.

Setup Montage

Press this button to check the montage settings in the Montage Setup dialog.  If you do not do this, the dialog will come up when you go to start the run.  Only a few parameters can actually be modified in the dialog.

Autocontour grid squares

Select this option to run the routine to autocontour grid squares after acquiring the grid map, using parameters set in the  Autocontour Grid Squares dialog. 

Setup

Press this button to open the Autocontour Grid Squares dialog and set parameters.  Opened this way, it will allow the group selections to be adjusted even though no contours exist.  You must select the set of groups to be converted to contours before starting the first run, unless you have used the autocontouring and selected groups previously in this run of the program and the Autocontouring dialog is still open.  The group selection is stored in the session file, so groups will already be defined if you start another run to take maps on other grids in the set.

Run script at end

Select this option to run a script after each grid map is finished, after autocontouring if any.  Pick the desired script in the drop-down box.

Acquire Medium Mag Maps

This section contains some controls governing the acThis section contains some controls governing the acquisition of medium-magnification maps, although other aspects are controlled through the Acquire at Items dialog.  Select the checkbox to include this phase of operations in the run.

Use: LD Search  -  LD View  -  Current or set state
States

The radio buttons, together with the pre-existing settings in the Montage Setup dialog and with states to be set, determine what kind of images are used for these maps.  The role of states is less than for grid mapping because simply selecting a View or Search state does not imply that it should be used for the maps. Four drop-down boxes are available for these states but only two are shown by default; the number shown can be set with the 'MulGridNumMMMcombos' property.  Here are the rules for what parameters are used for these maps:

•  Search will be used if LD Search is selected or if all of these conditions exist: Current or set state is selected, a Low Dose state is to be set or Low Dose mode is currently on, Polygon mont or nXn mont is selected, and 'Use Search parameters in Low Dose mode' is selected in the Montage Setup dialog.  Obviously, it is easier just to set the relevant radio button for the View and Search cases.
• Mont-map parameters will be used if neither of the above conditions apply, and if Current or set state is selected, Polygon mont or nXn mont is selected, and either a Mont-map state is selected or 'Use Montage Mapping, not Record parameters' is selected in the Montage Setup dialog.
• Otherwise, Record will be used.

Take: Image  -  Polygon mont  -  nXn mont

Use the buttons to select how the maps will be acquired:

• Image will acquire a single image instead of a montage.  The kind of image is determined by the selections just described.
• Polygon mont will fit a montage to each polygon set for acquisition.  'Reusable' polygon montage files will be used so that multiple maps of essentially the same size can be saved into the same file, resulting in fewer files.
• nXn mont will acquire montages with a fixed number of pieces in X and Y.

Setup

Use this button to check the parameters for polygon montages and set the number of pieces for fixed montages in the Montage Setup dialog.  A polygon montage is fit to a square of an arbitrary but hopefully representative size, to give you some idea of how big the actual montages will be. 

Set Mapping Parameters

Use this button to open the Navigator Acquire at Items dialog and set the Mapping parameters.  Some entries there, such as the type of single image to take in Low Dose mode, will be disabled because they are controlled by the multi-grid routine.  To do Multiple Records from multiple holes, you must select the option to 'Use map hole vectors for shifts'.  Currently, for this to work well, it will almost certainly require a transformation that was stored by running 'Step To and Adjust' after using hole vectors in the Multiple Record Setup dialog, and the transformation will need to be good for the current conditions.  The Acquire at Items dialog will indicate if a transformation exists.

Run Final Acquisition

Select the checkbox to include final data acquisition in the run.

States

The selected states must be consistent and determine what the parameters are for images taken through Acquire at Items, but do not affect the behavior of the multi-grid routine.  All four of the available drop-down boxes are shown by default, but the number can be set with the property 'MulGridNumFinalCombos'.

Set & show states for each grid

Select this option if you want to use different states for some of the grids.  When the option is first turned on, the current set of states will be assigned to the currently selected grid, if any.  While it is on, changing the state selections will change the set of states for the selected grid only.  When a different grid is selected, its set of states will be shown if it already has some defined; otherwise the currently shown set of states will be assigned to that grid.  Thus, the easiest way to give a set of grids the same set of separate states is:

• Select one of the grids out of that set
• Turn on the option
• Change to the desired states
• Select each other grid in the set, in turn
• Turn off the option

The first character in the status column will be 'S' instead of '—' when a grid has separate states.  Any grid without a separate state will have the global set of states (if any); i.e., there is no inheritance from one grid to the next.

If you just want to check what the states are without changing anything, turn on the option with no grid selected, select each grid with an 'S' status, and then turn off the option.

Revert to Global

Use this button to remove the separate states for the selected grid. 

Make frame directories under session directory/grid directories

If camera frames are saved during final acquisition, they will be placed in a separate subdirectory for each grid.  This option will be present if there is any camera available that can save frames in arbitrary locations (e.g., not for Falcon or DE cameras); select it if you are taking frames with such a camera and want the frames to be saved in subdirectories under the session or grid directories.   When the option is selected, and the actual camera being used allows it, frames will be saved in a directory named 'frames' under the grid directory if Use subdirectories is chosen, or in a directory 'gridRootName_frames' under the session directory if not.   Otherwise, frames will be saved in a directory 'gridRootName' under the currently defined frame folder for the camera if the camera allows that, or the frame folder will simply be defined as 'gridRootName' in a case where directories are allowed only directly under the frame storage location.

With regard to saving frames, the program will keep track of a sequential number separately for each grid.  If acquisition is run again on a grid, the numbering will pick up where it left off.  Also note that if the base name is used in frame stack or folder names, it will have the name of the grid added to it.

Set Final Data Parameters

Use this button to open the Navigator Acquire at Items dialog and set the Final Data parameters.  Here, only a few choices will be disabled.   Any type of main action can be chosen.  If you happen to want to take tilt series on some grids and other acquisition on the rest of the grids, first set up the other acquisition, then switch the main action to tilt series and set that up.  Leave the dialog with tilt series selected.   When running from the multi-grid routine, the program will automatically switch the acquisition to tilt series when there are items marked for tilt series and now marked for Acquire.

Running the Grids, Errors, Pausing, Stopping

Press Run All to run all the grids whose 'Run' box is checked.  Run Undone can be used to run only the phases that have not yet been done on the selected grids. It is enabled only if some, but not all, of the grids have already succeeded on the first phase selected to be run (e.g., if you select Acquire Grid Maps and Acquire Medium Mag Maps and some grids already have grid maps, they will start with medium mag maps).

If you have changed the camera subarea size for one of the parameter sets being used in a montage to be acquired since the montage was set up, the program will open the montage setup dialog for the affected montage, and may ask you if you want to use a subarea instead of the full frame when fitting the montage to an area.

For a run including medium mag mapping or final acquisition, the program will check the acquisition parameters for feasibility and consistency with the Navigator file contents to the extent possible.  Specifically, for each grid, it does most of the checks that are done before a regular Acquire at Items run (such as for the existence of template maps and labeled Navigator locations at which to perform specific tasks), and if Eucentricity by Focus is to be run, it verifies that a usable calibration is available.  If both medium mag mapping and final acquisition are being run, only the first stage can be checked, since the items to be acquired in the second stage do not exist yet.

The status codes will be updated as grids run and are as follows:
GM - Grid map is done
MM - Medium mag maps are done
AQ - Final acquisition is done
G-D - Grid map is done but grid is too dark to use
DA - Grid was judged too dark to use before taking a grid map (this does not occur with current logic)
FL - Something failed making the grid map or trying to set eucentricity
G-F - Grid map is done, something failed on a later step
M-F - Medium mag maps are done, something failed in final acquisition

An error in a task started by the multi-grid routine, except for Acquire at Items, will result in one of three actions: for some errors, it will go on to the next grid; for others, it will retry the operation once; for the rest, the whole run will be stopped.

An error in an operation performed by Acquire at Items will be treated as it usually is, with Acquire at Items either stopping or going on to the next item.  The multi-grid routine will be put into a suspended state.  Restarting Acquire at Items will unsuspend the multi-grid routine so that it can proceed normally when the acquisition finishes.

Pressing the STOP button also suspends the multi-grid routine as well as stopping whatever operation was occurring.  If the grid montage was stopped, starting a montage will unsuspend the multi-grid routine.

While operations are running, the button below Setup in the Camera & Macro Tools panel (usually labeled Preview or Montage) will be labeled PauseMG.  Pressing this button will suspend the multi-grid routine at the end of the current grid.  Whenever no operations are running and this routine is suspended, that button will be labeled End MGrid, and pressing it will terminate the routine, return the microscope to the initial conditions, and re-enable everything in the dialog.  The usual Resume button will be labeled ResumeM.  Pressing this after a pause will resume the multi-grid routine on the next grid.

Pressing ResumeM after any other kind of stop will bring up a message box asking whether you want to repeat the last operation in the multi-grid sequence, go on to the next operation, or skip to the next grid.  There are no good error checks on this choice yet, so for now you will have to exercise some judgement when doing anything except skipping to the next grid.  Running the grid montage, Acquire at Items, realign to the grid map, and eucentricity should be safe to redo.  Also note that the routine assumes that nothing has changed on the scope (e.g., states or apertures) that would prevent operations from continuing successfully.  If there is any doubt about this, it is safest to terminate with End MGrid, uncheck the grids done, and restart the run.