Boulder Laboratory for 3-Dimensional Electron Microscopy of Cells

NEWSTACK(1)                                                         NEWSTACK(1)

NAME
	newstack - Make a new mrc image stack.

SYNOPSIS
	newstack [options] <input file(s)>  <output file>

DESCRIPTION

  Newstack is a general stack editor to move images into, out of, or between
  stacks.  It can float the images to a common range or mean of density. It
  can bin images and apply a general linear transformations as well as a
  specified rotation or expansion. It can put the output into a smaller or
  larger array and independently recenter each image separately from the
  transformation.  Images can be taken from multiple input files and placed
  into multiple output files.
   
  Newstack uses the PIP package for input (see the manual page for pip)
  and can take input interactively only for options available before
  conversion to PIP, to maintain compatibility with old command files.  The
  options accepted by the old newst script are all accepted by newstack.
  Almost all options can be abbreviated to one or two letters.  The following
  options can be specified either as command line arguments (with the -) or
  one per line in a command file or parameter file (without the -):

 -input OR -InputFile   File name
    Input image file.  Input files may also be entered after all arguments on
    the command line, as long as an output file is the last name on the
    command line.  Files entered with this option will be processed before
    files at the end of the command line, if any.
    (Successive entries accumulate)

 -output OR -OutputFile   File name
    Output image file.  The last filename on the command line will also be
    treated as an output file, following any specified by this option.
    (Successive entries accumulate)

 -fileinlist OR -FileOfInputs   File name
    Name of file with list of input files and sections to read.  The format of
    the file is the number of input files on the first line, the first file to
    read on the next line, the list of sections to read on next line, next
    filename on next line, etc.  If this option is used, no input filenames
    may be specified with the -input option or following options on the
    command line.

 -fileoutlist OR -FileOfOutputs   File name
    Name of file with list of output files and number of sections to place in
    each.  The format of the file is the number of output files on the first
    line, the first file to write on the next line, number of sections to
    write on next line, next filename on next line, etc.  If this option is
    used, no output filenames may be specified with the -output option or as
    the last file on the command line.

 -split OR -SplitStartingNumber   Integer
    Starting number for a series of numbered output files, where one section
    will be placed in each file.  There must be only one output file name
    entered, and it will be taken as the root name for the output files.  A
    dot and the number will be appended to the root name for each file unless
    an extension is entered with the -append option.  The number of digits
    used for the number will be the same for all files so that they will list
    in order.

 -append OR -AppendExtension   Text string
    Extension to append to the filenames in a series of numbered files.  With
    this option, files will be named like "rootnnn.ext" instead of "root.nnn",
    where "root" is the root name entered as the output file name, and "ext"
    is the extension entered with this option.

 -secs OR -SectionsToRead   List of integer ranges
    List of sections to read from an input file, numbered from 0.  Ranges may
    be entered (e.g., 1-3,5,8), and / may be used to specify all sections in
    the file.  If multiple lists are entered, each one will be applied to the
    respective input file.  If no list is entered for a file, all sections
    will be read.
    (Successive entries accumulate)

 -skip OR -SkipSectionIncrement   Integer
    Increment for reading a regularly spaced subset of the sections in the
    list from the input file.  Enter 2 to take every other section, 3 to take
    every third section, etc.  Note that the increment is used to select from
    the input list, not from all the sections in the file.

 -numout OR -NumberToOutput   Multiple integers
    Number of sections to write to each output file, when there are multiple
    output files.  These numbers may be listed in sequence in one entry, or in
    several entries.
    (Successive entries accumulate)

 -replace OR -ReplaceSections   List of integer ranges
    List of section numbers at which to write images into an existing output
    file.  There must be only a single output file, its size in X and Y must
    be appropriate, and the given sections must already exist in the file. 
    The minimum and maximum density in the file header will be modified by the
    min and max of the sections being written, so they can become more extreme
    but not less extreme.  The mean density will not be changed.

 -blank OR -BlankOutput
    Output blank sections for nonexistent section numbers in input file,
    namely for section numbers less than 0 or more than the NZ - 1.  The
    sections are filled with the mean value of the input file or with a fill
    value if one is entered with the -fill option; this value is then scaled
    in the same way that actual data are.

 -size OR -SizeToOutputInXandY   Two integers
    The dimensions of the output image in X and Y.  The default size is the
    same as that of the first input file.

 -mode OR -ModeToOutput   Integer
    The storage mode of the output file; 0 for byte, 1 for 16-bit signed
    integer, 6 for 16-bit unsigned integer, or 2 for 32-bit floating point. 
    The default is the mode of the first input file.

 -bytes OR -BytesSignedInOutput   Integer
    This entry controls how bytes are stored in the output file and overrides
    both the default for this version of IMOD and the setting of the
    environment variable WRITE_MODE0_SIGNED.  Enter 0 for unsigned numbers or
    1 for signed numbers.  Unsigned output is needed to work with versions of
    IMOD before IMOD 4.2.23; signed output may be needed for compatibility
    with external software that has followed recent documentation of the MRC
    format.  Regardless of the representation in the file, bytes are read into
    IMOD programs as unsigned with a range of 0 to 255.

 -offset OR -OffsetsInXandY   Multiple floats
    X and Y center offsets for each section.  This is an offset to the center
    of the area from which image will be taken, so a positive offset in X will
    move images to the left.  If images are being binned, the entry should be
    in unbinned pixels.  Enter one X,Y pair to apply a single offset to all
    sections.
    (Successive entries accumulate)

 -applyfirst OR -ApplyOffsetsFirst
    Apply offsets in X and Y before transforming image, which will make a
    rotated image come from a predictable location.  The default is to apply
    offsets after, which shifts the transformed image instead.

 -xform OR -TransformFile   File name
    File with one or more linear transformations to apply to images

 -uselines OR -UseTransformLines   List of integer ranges
    A list of the line numbers of lines of transforms to use in the transform
    file (numbered from 0).  Ranges are allowed.  The default is for the line
    numbers to match the input section numbers.  To have a single transform
    applied to all of the sections, just enter a single number.
    (Successive entries accumulate)

 -onexform OR -OneTransformPerFile
    Apply one transform line for each input file.  The default is for the
    first transform to be applied to the first file, the second transform to
    the second file, etc., in which case the transform file must have at least
    one line per input file.  Otherwise, -uselines can be used to specify
    which transform to apply for each file, in which case at least one line
    must be specified per input file.

 -rotate OR -RotateByAngle   Floating point
    Rotate all images by the given angle in degrees, positive for a
    counterclockwise rotation.  Rotation is applied after binning and
    transformation, if any.  It can be combined with linear transforms but not
    with warping transforms.  If the rotation is by +90 or -90 degrees and no
    output size is specified, then the X and Y sizes of the image will be
    transposed so the the output file will show the whole rotated image.  For
    any other rotation angles, you will get the same output size as the input
    image unless you enter a new size explicitly.

 -expand OR -ExpandByFactor   Floating point
    Scale all images up in size by the given factor, greater than 1 to expand,
    or less than 1 to shrink with ordinary interpolation.  Scaling is applied
    after binning and transformation, if any.  It can be combined with linear
    transforms but not with warping transforms.  If no output size is
    specified, the size of the input image will be scaled appropriately so
    that the output will contain the entire image.

 -shrink OR -ShrinkByFactor   Floating point
    Scale all images down in size by the given factor, a value greater than 1,
    using antialias filtering.  This option should be used for large image
    reductions, because using ordinary interpolation will give aliasing
    artifacts that appear primarily as noise.  Even binning will give some
    aliasing.  Shrinking can be applied after binning but cannot be combined
    with rotation, general transformation, or distortion correction.  If no
    output size is specified, the size of the input image will be scaled
    appropriately so that the output will contain the entire image.

 -antialias OR -AntialiasFilter   Integer
    Type of antialiasing filter to use when reducing images with the -shrink
    option.  The available types of filters are: 
        1: Box - equivalent to binning 
        2: Blackman - fast but not as good at antialiasing as slower filters 
        3: Triangle - fast but smooths more than Blackman 
        4: Mitchell - good at antialiasing, smooths a bit 
        5: Lanczos 2 lobes - good at antialiasing, less smoothing than Mitchel
    
        6: Lanczos 3 lobes - slower, even less smoothing but more risk of
    ringing 
    The default is 5 for the Lanczos 2, which many people consider the best
    compromise among the various factors.

 -bin OR -BinByFactor   Integer
    Use binning to reduce images in size by the given factor.  Binning is
    applied before all other image transformations.  If no output size is
    specified, the size of the input image will be scaled appropriately so
    that the output will contain the entire image.

 -origin OR -AdjustOrigin
    Adjust the origin values in the image file header for changes in image
    size or the centering of a subarea.  With this adjustment, a model built
    on the input stack should be correctly located when loaded onto the output
    stack in 3dmod.  Model points will be correctly located in Z provided
    that a contiguous set of sections is output.  They will be correct in X
    and Y provided that the only operations on the data are a change in size
    of the output, binning with -bin, scaling with -expand, and shifting with
    -offset.

 -linear OR -LinearInterpolation
    Use linear instead of cubic interpolation to transform images.  Linear
    interpolation is more suitable when images are very noisy, but cubic
    interpolation will preserve fine detail better when noise is not an issue.
     Images are transformed when the -xform, -expand, -rotate, -distort, or
    -gradient option is entered.

 -float OR -FloatDensities   Integer
    Adjust densities of sections individually.  Enter 1 for each section to
    fill the data range, 2 to scale sections to common mean and standard
    deviation, 3 to shift sections to a common mean without scaling, or 4 to
    shift sections to a common mean then rescale the minimum and maximum
    densities to the Min and Max values specified with the -scale option.

 -contrast OR -ContrastBlackWhite   Two integers
    Rescale densities to match the contrast seen in 3dmod with the given
    black and white values.  This works properly only when the output file
    will be bytes.  It will not work if the data were loaded into 3dmod with
    intensity scaling; use mrcbyte in that case.

 -scale OR -ScaleMinAndMax   Two floats
    Rescale the densities of all sections by the same factors so that the
    original minimum and maximum density will be mapped to the Min and Max
    values that are entered.

 -fill OR -FillValue   Floating point
    Value to fill areas of the output image that have no image data.  The
    filling is done before intensity scaling, so the value in the filled areas
    will be modified by any scaling that is done.

 -multadd OR -MultiplyAndAdd   Two floats
    Rescale the densities of all sections by multiplying by the first entered
    value then adding the second value.  This option must be entered either
    once only, or once per input file.
    (Successive entries accumulate)

 -taper OR -TaperAtFill   Two integers
    To taper an output image at a border between real image and a filled area,
    just as is done with Mrctaper, enter this option with two values: 1)
    the distance over which to taper or 1 for a default distance, and 2) a 0
    for tapering outside or 1 for tapering inside the border.  The default
    distance is 1% of the mean of the X and Y sizes of the output image, but
    at least 16 pixels and no more than 127.  The program will exit with an
    error if there is not enough memory to hold the entire output image.

 -distort OR -DistortionField   File name
    Image distortion field file to use for undistorting images.  The
    undistortion is applied before any transformations.

 -imagebinned OR -ImagesAreBinned   Integer
    The current binning of the images, so that the distortion field can be
    applied correctly.  This entry is required unless the program can
    determine the binning unambiguously from the image size.

 -fields OR -UseFields   List of integer ranges
    A list of the distortion fields to apply for each section (numbered from
    0).  Ranges are allowed.  The default is for the field numbers to match
    the input section numbers, unless there is only one field in the file.  To
    have a single field applied to all of the sections, just enter a single
    number.  This option was added when the only way to warp images was with
    distortion field files but it is not needed with current warping files.
    (Successive entries accumulate)

 -gradient OR -GradientFile   File name
    File with magnification gradients to be applied for each image.  This
    should be a file listing the tilt angle, the percent magnification change
    per micron of Z height, and the degrees of rotation per micron of Z height
    for each image, such as is produced by Extractmaggrad.  The mag gradient
    correction is applied before a distortion field correction and before any
    transformations.

 -memory OR -MemoryLimit   Integer
    Use this entry to specify the amount of memory used by the program for its
    main array, in megabytes.  By default, the program will try to use enough
    memory to hold both an input image and an output image in floating point
    form (4 bytes per pixel).  Limiting the memory can keep it from using more
    memory than available but may result in it operating on the images in
    chunks.  The main array is used to hold both the input image (in whole or
    in part) and a transformed image (in whole or in part).

 -test OR -TestLimits   Two integers
    To test the code for reading in and binning images in chunks, enter limits
    for the total size of the working array, and for the size of the array
    used for binning.  The sizes are the number of 4-byte elements.

 -verbose OR -VerboseOutput   Integer
    1 for diagnostic output

 -param OR -ParameterFile   Parameter file
    Read parameter entries as keyword-value pairs from a parameter file.

 -help OR -usage
    Print help output

  -StandardInput
     Read parameter entries from standard input.

  If no command line arguments are entered, inputs are prompted interactively.
  The input is as follows; lines starting with IF are entered only if a
  particular option is selected.

  Number of input files, or -1 to enter list of files from a file

  IF you enter a number >0, then for each input file, enter two lines:
   	File name
   	list of section numbers to take from that file.  The first section is
   		numbered 0 and ranges may be entered (e.g. 0-4,5,7-9)

  OTHERWISE, IF you enter -1, then enter the name of the file with list of
   		files and sections.  The format of this file should be
     Number of input files
     File name
     list of section numbers (first is 0!!!), ranges allowed (e.g. 0-3,5,7-9)
     next file and list of sections, etc.

  Number of output files, or -1 to enter list of files from a file

  IF you enter 1, next enter the name of the single output file.

  OR IF you enter a number >1, then for each output file, enter two lines:
   	File name
   	Number of sections to put in that file

  OR IF you enter -1, then enter the name of the file with the list of files
   	and numbers of of sections.  The format of this file is:
     Number of output files
     File name
     number of sections to place in that file
     next file and number of sections, etc.

  The X and Y dimensions of the output file, or / to take the same
   	dimensions as the first input file

  The data mode of the output file, or / to take the same mode as the first
      input file.  Mode can be 0 for bytes, 1 for 16-bit integers, 2 for 
      32-bit real numbers, or 9 to 15 for 9 to 15 bit values.

  0 for no offsets, or 1 to apply X and Y offsets separately to each section,
      or -1 to apply the same X and Y offset to all sections

  IF offsets are selected, next specify the X and Y center offsets of
     each of the sections (they need not be on separate lines), or the single
     offset to apply to all sections.
     Hint: offsets of 50,50 will take the center of the output image
     from the upper right quadrant of the input image.

  0 for no transforms, or 1 to transform images using cubic interpolation, or
     2 to transform using bilinear interpolation

  IF transforming is selected, next specify the file for transforms, and
     on a separate line, a list of lines to use in the file (first is 0!!!).
     Ranges are allowed here, but all must be on one line.  To have a single
     transform applied to all of the sections, just enter a single number.

  0 for no floating of densities, 1 to float densities so that each section
       occupies the same RANGE of densities (the optimum range for the
	particular data mode), 2 to float so that all sections have the same
       MEAN and STANDARD DEVIATION of density (which requires the input
       sections to be read twice), -1 to specify a single scaling that will
       be applied equally to all sections, -2 to scale all sections by the
       same amount, determined by black and white contrast settings, in order
       to convert to bytes with stretched contrast, 3 to SHIFT sections to
       the same mean value without rescaling, or 4 to SHIFT sections to the 
       same mean then rescale all sections by the same specified scaling.
         
  IF single rescaling is selected with "-1", next specify the density values
       to which the minimum and maximum in the data file should be mapped, or
       just enter "/" to have them mapped to the full-scale range for the
       data mode, or enter "1,1" to override any scaling of data that would
       otherwise occur because of a change of mode.  This scaling method can
       be used to invert densities by entering, e.g., 255,0.

  IF scaling by black and white contrast settings is selected with -2, next
       enter the black and white settings (see mrcbyte).

  OR, IF shifting and rescaling was selected with "4", next specify the
       density values to which the new, shifted minimum and maximum values
       should be mapped, or just enter "/" to have them mapped to the full-
       scale range for the data mode.

ADDITIONAL NOTES

  The program will adjust the pixel spacing in the header of the output file
  if binning and/or the expansion or skrinkage options are used.  Models built
  on the input file in 3dmod should display correctly on a file scaled by
  these three options.  However, the image origin is not adjusted for images
  that are a subset of the input image unless the -origin option is entered as
  well.  Without this option, it would be necessary to shift a model manually
  in 3dmod to display it on a subset.

  When there is a change of modes, data are rescaled optimally in most
  cases. If floating is not selected, the scaling is by an amount that would
  map the range for the input data mode into the range for the output data
  mode (e.g., going from mode 1 to mode 2, it would scale data up by a factor
  of 128).  Generally, all sections will be rescaled by the same amount, thus
  preserving relative differences between sections.  However, if the output
  mode is 2 (reals), this kind of rescaling will not occur.  In general, if
  your input mode is real and you are outputing to bytes or integers, you need
  to specify some kind of floating, shifting, or scaling in order to get
  usable values.  These same kinds of automatic scalings upon change of mode
  will occur with shifting to a common mean (option 3), but here the program
  is more careful to apply the same scaling to all sections.  To disable the
  scaling upon change of mode, the use the option "-scale 1,1," or the "-1"
  floating option with interactive input.
         
  At the end, the program will report how many pixels have been truncated at
  the low or high limits of the output range.  If this happens after shifting
  with a specified output range, try again with a slightly smaller range.

  The transformation file entered with -xform can contain warping
  transformations as well as simple linear transforms.  The transforms can
  consist of either displacements at a set of control points, as output
  directly by Midas, or displacements on a regular grid, as produced by
  Xftoxg.  Such warping files contain information about the size of the
  images that were aligned and their pixel size.  The transforms will be
  adjusted for a difference between that pixel size and the pixel size of the
  image file being transformed.  They will also be extrapolated as necessary
  to cover an image area larger than that specified in the warping file;
  however, the program assumes that the input images are centered on the
  images with which the warping file was prepared.  The warping will be
  incorrect if this is not the case.

  Warping cannot be used with distortion corrections.  If this is necessary,
  consider using Blendmont, which can combine such operations.  The -expand
  and -rotate options also cannot be used with warping transformations because
  such transformation would occur after the warping while the linear transform
  included in a warping file occurs before the warping.  If these operations
  are necessary, you can avoid transforming the images twice by preparing a
  one-line text file with
     factor  0.  0.  factor  0.  0.     
  for expansion by the factor, or with
     cos(angle)  -sin(angle)  sin(angle)  cos(angle)  0.  0.
  for rotation by the angle.  Then use Xfproduct to multiply the warping
  transforms by the single linear transform, and transform the images with the
  product.

  Complex data (Fourier transforms) can be stacked but not otherwise  
  manipulated.  Don't try to change the size or center of the FFT's.

  Newstack will not work with color data (MRC data mode 16).  If your images
  came from color Tiff files and you do not need the color information, use
  the "-g" option when converting them with Tif2mrc.  If you do need to 
  preserve the color information, then you can split the color channels into
  three byte-mode files with "clip splitrgb", use Newstack to perform
  identical operations on each of the three files, and recombine the data into
  a color file with "clip joinrgb".
         
  The program attempts to allocate enough memory to hold an entire input
  image and output image in 4 bytes per pixel, but it can work on large images
  with less memory than this by reading, operating on, and writing strips of
  the image if necessary.  On a 32-bit system, if the memory allocation fails,
  it will try to work with just under 2 GB of memory.  Memory allocation will
  probably not fail on a 64-bit system.  In either case, system performance
  will suffer if the program allocates more than the amount of memory
  available, and it is up to the user to limit the memory as appropriate with
  the "-memory" option.  The ability to work on the image in strips is limited
  when images are rotated by large angles (30-90 degrees); in such cases the
  program will need at least enough memory to hold the whole input images.

  The program can manipulate and transform images bigger than 4 gigapixels,
  but distortion correction is currently limited to 2 gigapixel images.

  Each linear transformation in a transform file is specified by a line with 
  six numbers:
    A11 A12 A21 A22 DX DY
  where the coordinate (X, Y) is transformed to (X', Y') by:
    X' = A11 * (X - Xci) + A12 * (Y - Yci) + DX + Xco
    X' = A21 * (X - Xci) + A22 * (Y - Yci) + DY + Yco
  where (Xci, Yci) and (Xco, Yco) are the center coordinates of the input and
  output images, respectively.

HISTORY
	
	DNM (mast) for VAX    8/88
	5/4/89 changed to float to same mean and sd rather than mean and min.
	12/19/89 added full control of density scaling; fixed bug in
	interpolation that was generating half-pixel error half the time.
	4/19/90 added line-by-line prompts for file and section entry and
	added time/date stamp
	1/23/91: fixed so that it can correctly pad images and extract
	subsets of images with either center-shifting or transforms
	6/11/91: fixed bug if trying to output scaled real values les than 0
	Feb 94 : Ported to SGI by Paul Furcinitti.
	Jul 94 : PSF (Furcinitti) Fixed bug when writing byte files.
	11/16/94: DNM added shifting to mean, rationalized scaling somewhat
        9/24/01: changed to read and write in chunks if images are big
        12/24/03: converted to PIP input, added undistorting and binning
        June 2011: Added support for warping

SEE ALSO
	mrcbyte(1), tif2mrc, clip