flattenwarp(1)                                                  flattenwarp(1)

       flattenwarp - compute warping transformations for flattening a volume

       flattenwarp  [options]  model_file  warping_output_file

       Flattenwarp produces transformations for flattening the material of
       interest in a volume, such as the sectioned material in a tomogram.  It
       works from two different kinds of IMOD models.  One kind would have
       contours drawn in X/Z planes along the surfaces or through the middle
       of the sectioned material.  The other kind would contained scattered
       points at the positions of gold beads on the surface of the section.
       The program uses the positions in these models to determine how to
       shift each part of the volume in Z to a common plane.  In addition, it
       finds the rotations needed to make the tangents to the surface be flat,
       so as to avoid a shearing distortion of the volume.  Finally, it solves
       for lateral shifts in X and Y that preserve the area locally after the
       Z shifts and rotations.  In the case of a bent sheet such as a cylin-
       der, these transformations should simply unbend the sheet.  If the sec-
       tion is dome-shaped, then flattening will inevitably distort it, and
       the solution here will expand the material radially by as much as it
       contracts tangentially.

       This program will work only on a volume where X/Z planes are parallel
       to the plane of the specimen and Z is depth through the specimen.  A
       tomogram generated by Tilt must be rotated or flipped into this ori-

       The program can perform two kinds of smoothing of the modeled posi-
       tions.  The first is performed automatically with contours when the
       spacing between points is small.  The contours are resampled in X at
       regular intervals of about 2/3 of the spacing between contours in Y.
       If there are enough nearby points to a point being sampled, the program
       fits a parabola to their Z values and measures the Z value from that;
       otherwise, it simply interpolates linearly between the two surrounding
       points.  This smoothing allows you to draw closely spaced points with-
       out having local jitters in position cause artifacts in the warping.

       The second kind of smoothing involves fitting a two-dimensional spline
       function called a thin plate spline (TPS) to all of the points.  This
       smoothing is optional but recommended for contour models and mandatory
       for scattered point models.  The TPS fitting minimizes a combination of
       the total energy needed to bend a stiff sheet into the final surface,
       and the sum of the deviations of points from the surface.  A parameter
       called lambda controls the smoothing by specifying the weight given to
       the bending energy.  In the implementation here, the mean deviation of
       the points from the surface increases roughly linearly with the loga-
       rithm of lambda.  Thus, the values entered with the -lambda option are
       the logarithm of the actual lambda parameters, small numbers typically
       in the range of 0-4.

       It is recommended that you try the TPS smoothing with a range of lambda
       values and look at the resulting surfaces to pick the right lambda
       value.  Simply run the program with a list of values for the -lambda
       option and use the -middle option to output a model with the smoothed
       surfaces.  Values at intervals of 0.5 will produce useful increments in
       smoothing.  Values of around 2 to 4 are needed for smoothing surfaces
       from contour models, whereas values around 0 to 2 are effective for
       smoothing scattered point models.  The model is ready to examine in
       3dmodv and has a Z-scale set to 10 so that it is easier to assess
       the features in Z.  There are two useful ways to look at the model: by
       examining the original contours or scattered points along with one or
       two surfaces at a time; or by examining the original and smoothed con-
       tours all at once.  The former seems to be more useful and is the
       default for the initial display; the model can be output in the latter
       form with the -contour options.  Objects are named by their log lambda
       values; open the Edit-Object List dialog to turn objects on and off
       conveniently.  The right amount of smoothing is one that eliminates
       small-scale bumps in the surface without causing consistent deviations
       of the surface from the contours or points.

       To prepare a contour model, open the volume in 3dmod and use Edit-
       Image-Flip to view the X/Z planes in the Zap window.  Use Edit-Object-
       Type to change the object to open contours, and select the option to
       start a new contour automatically at each new Z plane.  Start drawing
       contours to describe the location of the sectioned material.  There are
       three ways that this can be done: 1) with a pair of contours on an X/Z
       plane, one along the top and one along the bottom surface; 2) with a
       single contour through the middle of the section; or 3) with a single
       contour along one surface of the section.  You can use methods 1 and 2
       interchangeably in the same model since they both describe the location
       of the middle of the section.  If you use method 3, drawing one sur-
       face, you must use it for all the X/Z planes in your model.  Thus,
       before you begin, you need to decide whether to draw just one surface.

       To assess the consistency between successive contours in Y, you should
       turn on ghost contour display mode and use the "Near" setting for the
       distance, which is the default and will display contours from the near-
       est sections with contours.  Use the "g" hot key, or use Edit-Contour-
       Type to open the Surf/Cont/Point dialog and select other options under
       "Section ghost".

       If you do not apply the TPS smoothing, random variability in the Z
       heights of successive contours could lead to substantial local distor-
       tions.  To reduce this possibility, keep the contours well-spaced apart
       in Y.  The spacing between contours should not be much smaller than is
       needed to describe the changes in the surface.  The program will inter-
       polate linearly in Y between the contours that you draw, so if the Z
       height is changing linearly through a set of slices, you do not need to
       draw contours within that set of slices.  You can vary the spacing,
       making it smaller where needed.  Although the initial contour smoothing
       means you can use continuous drawing, do not follow every apparent bump
       in the surface of the section.

       You do not need to draw all the contours to the same extent in X, nor
       do you need to draw them over the full range of positions in Y.  The
       program will output warping transformations over the range of positions
       where the contours were drawn, and Warpvol will extrapolate the
       transforms outside that range.

       A scattered point model can be based on either the 3D fiducial model
       that is output from Tiltalign, or a model of bead positions from
       Findbeads3d.  Flattening with such models can save the time involved
       in modeling contours, but many gold bead models will not be suitable.
       Models with too few points, or points not widely enough distributed,
       are unlikely to give good results.  Models will generally need to be
       processed with Sortbeadsurfs before being input to Flattenwarp.
       That program can sort the beads from Findbeads3d into two objects,
       one for each surface.  It can also apply many changes that might be
       needed to make a 3D fiducial model correspond to the final tomogram
       that you are trying to flatten.  If you have a model from Find-
       beads3d(1), be sure to delete all the points that are not gold parti-
       cles (points below a threshold peak strength), and also select and
       delete points that obviously not on one surface or the other.

       For either kind of bead model, eliminating outlying points is also rec-
       ommended for preventing bumps in the fitted surface.  The "-criterion"
       option lets you enter a criterion for removing points with extreme
       deviation from the TPS fit; the criterion is similar to a standard
       deviation-based criterion and numbers in the range 2.5 to 4 should be
       useful.  The program will report how many points are removed; it should
       be safe to remove up to ~10% of the points.

       The program determines the shifts in X and Y by obtaining a least-
       squares solution to a massive set of linear equations.  The number of
       variables being solved for depends on the spacing between grid points
       at which the warping transformation is computed.  If a solution seems
       problematic, try increasing the spacing with the -spacing option
       described below.

       After you obtain the set of warping transformations, you can apply them
           warpvol -xf warp_file -same input_volume output_volume
       where "warp_file" is the output file from Flattenwarp.  If you know
       that you can reduce the size of Z in the flattened volume, say to
       "new_z", you can add the option "-size ,,new_z".

       Flattenwarp uses the PIP package for input (see the manual page for
       pip).  Options can be specified either as command line arguments
       (with the -) or one per line in a command file (without the -).
       Options can be abbreviated to unique letters; the currently valid
       abbreviations for short names are shown in parentheses.

       -input (-i) OR -InputFile      File name
              Input model file with contours or scattered points delineating

       -output (-ou) OR -OutputFile   File name
              Output file for warping transformations

       -patch (-p) OR -PatchOutputFile     File name
              Output text file for patch vectors that can be converted to a
              model with patch2imod.

       -middle (-m) OR -MiddleContourFile       File name
              Output model file for average contours through middle of sec-
              tion.  For an input model consisting of boundary contours, this
              model will contain a contour at each Y value, derived by smooth-
              ing and sampling from the original contour(s) at that Y value.
              For input consisting of scattered points, the scattered points
              object(s) will be copied to the output model.  In either case,
              the model will also contain contours based on whatever thin
              plate splines are fit to the data, one object per lambda value.
              The model has its Z-scale set to 10, which will greatly exagger-
              ate the Z changes and make it easier to see the bumps in the
              surface that might cause artifacts in the flattened valume.

       -binning (-b) OR -BinningOfTomogram      Two integers
              Binning in X/Y, and binning in Z, of image file that model was
              built on, relative to the tomogram to be warped.  Positions and
              vectors will be scaled up by this amount so that the transforma-
              tions can be validly applied to the unbinned tomogram.  For con-
              tours or scattered points modeled on a tomogram, this option is
              needed only if the tomogram file itself is binned down, not if
              an unbinned tomogram is read into 3dmod with binning.  For a
              fiducial model, the option is needed only if 1) the prealigned
              stack was binned relative to the tomogram being warped, and 2)
              the model was not rescaled to match the tomogram when it was
              processed with Sortbeadsurfs.

       -one (-on) OR -OneSurface
              Use this option to indicate that all contours are on one surface
              of the section rather than in the middle.  With this option,
              there cannot be any paired contours.

       -flip (-f) OR -FlipOption      Integer
              This option allows control over whether the Y and Z coordinates
              of the model are flipped.  By default, if the Z dimension (nz)
              of the modeled volume is greater than the Y dimension (ny), a
              contour model will be flipped, and a scattered point model will
              be rotated around the X axis.  For a scattered point model, this
              operation follows a flipping that might be needed to make the
              model match the native orientation of the volume.  Enter 1 to
              force flipping even if ny > nz and even for a scattered point
              model, or 0 to prevent rotation or flipping even if nz > ny, or
              2 to force rotation of a scattered point model even if ny > nz.

       -spacing (-sp) OR -WarpSpacingXandY      Two floats
              Spacing between warp positions in X and Y, in pixels in the
              tomogram that was modeled.  By default, with contour data, the
              program will try to make the spacing in X and Y be 4.5 and 3
              times smaller, respectively, than the minimum spacing between
              contours in Y.  With scattered point data, it will try to make
              the spacing in both X and Y be 3 times smaller than the average
              spacing between points.  In either case, if that will lead to
              too many patches, it will make the spacing bigger.  This option
              can be used to specify a spacing instead.  This may be needed if
              too fine a spacing leads to problems when solving the linear
              equations for the shifts.

       -lambda (-l) OR -LambdaForSmoothing      Multiple floats
              One or more values to control the amount of smoothing in the
              thin plate spline (TPS).  The values are the log of the lambda
              parameter.  If one value is entered, the program will go on to
              compute warping transforms.  If multiple values are entered in
              order to assess their effect, an output model must be specified
              with -middle, but no transforms will be computed and no output
              file is needed for them.  Values varying in steps of 0.5 will
              produce useful changes in smoothing.  Negative values are just
              as meaningful as positive ones.

       -criterion (-c) OR -CriterionForOutliers      Floating point
              Criterion for dropping scattered points from the fit for the
              thin plate spline.  The fit is done first with all points, the
              deviation from the spline surface is computed for each point,
              and the median absolute value of the difference from the median
              deviation is determined.  This value is the "normalized median
              absolute deviation", referred to as MADN, and is similar to a
              standard deviation.  Points whose difference from the median
              deviation, divided by the MADN, is greater than this criterion
              are eliminated and the spline is fit again.  Values of around
              2.5-3 will eliminate a useful number of points.

       -show (-sh) OR -ShowContours
              Set object properties in the output model so that contours will
              appear for all objects in 3dmodv.  By default, the properties
              are set so that all objects based on smoothing with a TPS will
              display a meshed surface instead of a spline, but only the first
              such object will be turned on.

       -restore (-r) OR -RestoreOrientation
              Put the output model back into the same orientation as the input
              model.  By default, it is left in an orientation so that it can
              be viewed in 3dmodv(3) with the Z scale applied to the correct
              axis.  You may need to use this option if you want to load this
              model onto a tomogram.

       -PID   Output process ID to standard error

              Read parameter entries from standard input

       David Mastronarde

       3dmod, warpvol, sortbeadsurfs

       Email bug reports to mast at colorado dot edu.

BL3DEMC                              4.7.3                      flattenwarp(1)