Boulder Laboratory for 3-Dimensional Electron Microscopy of Cells
NEWSTACK(1) NEWSTACK(1)
NAME
newstack - Make a new mrc image stack.
SYNOPSIS
newstack [options] <input file(s)> <output file>
DESCRIPTION
Newstack is a general stack editor to move images into, out of, or between
stacks. It can float the images to a common range or mean of density. It
can bin images and apply a general linear transformations as well as a
specified rotation or expansion. It can put the output into a smaller or
larger array and independently recenter each image separately from the
transformation. Images can be taken from multiple input files and placed
into multiple output files.
Newstack uses the PIP package for input (see the manual page for pip)
and can take input interactively only for options available before
conversion to PIP, to maintain compatibility with old command files. The
options accepted by the old newst script are all accepted by newstack.
Almost all options can be abbreviated to one or two letters. The following
options can be specified either as command line arguments (with the -) or
one per line in a command file or parameter file (without the -):
-input OR -InputFile File name
Input image file. Input files may also be entered after all arguments on
the command line, as long as an output file is the last name on the
command line. Files entered with this option will be processed before
files at the end of the command line, if any.
(Successive entries accumulate)
-output OR -OutputFile File name
Output image file. The last filename on the command line will also be
treated as an output file, following any specified by this option.
(Successive entries accumulate)
-fileinlist OR -FileOfInputs File name
Name of file with list of input files and sections to read. The format of
the file is the number of input files on the first line, the first file to
read on the next line, the list of sections to read on next line, next
filename on next line, etc. If this option is used, no input filenames
may be specified with the -input option or following options on the
command line.
-fileoutlist OR -FileOfOutputs File name
Name of file with list of output files and number of sections to place in
each. The format of the file is the number of output files on the first
line, the first file to write on the next line, number of sections to
write on next line, next filename on next line, etc. If this option is
used, no output filenames may be specified with the -output option or as
the last file on the command line.
-split OR -SplitStartingNumber Integer
Starting number for a series of numbered output files, where one section
will be placed in each file. There must be only one output file name
entered, and it will be taken as the root name for the output files. A
dot and the number will be appended to the root name for each file unless
an extension is entered with the -append option. The number of digits
used for the number will be the same for all files so that they will list
in order.
-append OR -AppendExtension Text string
Extension to append to the filenames in a series of numbered files. With
this option, files will be named like "rootnnn.ext" instead of "root.nnn",
where "root" is the root name entered as the output file name, and "ext"
is the extension entered with this option.
-secs OR -SectionsToRead List of integer ranges
List of sections to read from an input file, numbered from 0. Ranges may
be entered (e.g., 1-3,5,8), and / may be used to specify all sections in
the file. If multiple lists are entered, each one will be applied to the
respective input file. If no list is entered for a file, all sections
will be read.
(Successive entries accumulate)
-skip OR -SkipSectionIncrement Integer
Increment for reading a regularly spaced subset of the sections in the
list from the input file. Enter 2 to take every other section, 3 to take
every third section, etc. Note that the increment is used to select from
the input list, not from all the sections in the file.
-numout OR -NumberToOutput Multiple integers
Number of sections to write to each output file, when there are multiple
output files. These numbers may be listed in sequence in one entry, or in
several entries.
(Successive entries accumulate)
-replace OR -ReplaceSections List of integer ranges
List of section numbers at which to write images into an existing output
file. There must be only a single output file, its size in X and Y must
be appropriate, and the given sections must already exist in the file.
The minimum and maximum density in the file header will be modified by the
min and max of the sections being written, so they can become more extreme
but not less extreme. The mean density will not be changed.
-blank OR -BlankOutput
Output blank sections for nonexistent section numbers in input file,
namely for section numbers less than 0 or more than the NZ - 1. The
sections are filled with the mean value of the input file or with a fill
value if one is entered with the -fill option; this value is then scaled
in the same way that actual data are.
-size OR -SizeToOutputInXandY Two integers
The dimensions of the output image in X and Y. The default size is the
same as that of the first input file.
-mode OR -ModeToOutput Integer
The storage mode of the output file; 0 for byte, 1 for 16-bit signed
integer, 6 for 16-bit unsigned integer, or 2 for 32-bit floating point.
The default is the mode of the first input file.
-bytes OR -BytesSignedInOutput Integer
This entry controls how bytes are stored in the output file and overrides
both the default for this version of IMOD and the setting of the
environment variable WRITE_MODE0_SIGNED. Enter 0 for unsigned numbers or
1 for signed numbers. Unsigned output is needed to work with versions of
IMOD before IMOD 4.2.23; signed output may be needed for compatibility
with external software that has followed recent documentation of the MRC
format. Regardless of the representation in the file, bytes are read into
IMOD programs as unsigned with a range of 0 to 255.
-offset OR -OffsetsInXandY Multiple floats
X and Y center offsets for each section. This is an offset to the center
of the area from which image will be taken, so a positive offset in X will
move images to the left. If images are being binned, the entry should be
in unbinned pixels. Enter one X,Y pair to apply a single offset to all
sections.
(Successive entries accumulate)
-applyfirst OR -ApplyOffsetsFirst
Apply offsets in X and Y before transforming image, which will make a
rotated image come from a predictable location. The default is to apply
offsets after, which shifts the transformed image instead.
-xform OR -TransformFile File name
File with one or more linear transformations to apply to images
-uselines OR -UseTransformLines List of integer ranges
A list of the line numbers of lines of transforms to use in the transform
file (numbered from 0). Ranges are allowed. The default is for the line
numbers to match the input section numbers. To have a single transform
applied to all of the sections, just enter a single number.
(Successive entries accumulate)
-onexform OR -OneTransformPerFile
Apply one transform line for each input file. The default is for the
first transform to be applied to the first file, the second transform to
the second file, etc., in which case the transform file must have at least
one line per input file. Otherwise, -uselines can be used to specify
which transform to apply for each file, in which case at least one line
must be specified per input file.
-rotate OR -RotateByAngle Floating point
Rotate all images by the given angle in degrees, positive for a
counterclockwise rotation. Rotation is applied after binning and
transformation, if any. It can be combined with linear transforms but not
with warping transforms. If the rotation is by +90 or -90 degrees and no
output size is specified, then the X and Y sizes of the image will be
transposed so the the output file will show the whole rotated image. For
any other rotation angles, you will get the same output size as the input
image unless you enter a new size explicitly.
-expand OR -ExpandByFactor Floating point
Scale all images up in size by the given factor, greater than 1 to expand,
or less than 1 to shrink with ordinary interpolation. Scaling is applied
after binning and transformation, if any. It can be combined with linear
transforms but not with warping transforms. If no output size is
specified, the size of the input image will be scaled appropriately so
that the output will contain the entire image.
-shrink OR -ShrinkByFactor Floating point
Scale all images down in size by the given factor, a value greater than 1,
using antialias filtering. This option should be used for large image
reductions, because using ordinary interpolation will give aliasing
artifacts that appear primarily as noise. Even binning will give some
aliasing. Shrinking can be applied after binning but cannot be combined
with rotation, general transformation, or distortion correction. If no
output size is specified, the size of the input image will be scaled
appropriately so that the output will contain the entire image.
-antialias OR -AntialiasFilter Integer
Type of antialiasing filter to use when reducing images with the -shrink
option. The available types of filters are:
1: Box - equivalent to binning
2: Blackman - fast but not as good at antialiasing as slower filters
3: Triangle - fast but smooths more than Blackman
4: Mitchell - good at antialiasing, smooths a bit
5: Lanczos 2 lobes - good at antialiasing, less smoothing than Mitchel
6: Lanczos 3 lobes - slower, even less smoothing but more risk of
ringing
The default is 5 for the Lanczos 2, which many people consider the best
compromise among the various factors.
-bin OR -BinByFactor Integer
Use binning to reduce images in size by the given factor. Binning is
applied before all other image transformations. If no output size is
specified, the size of the input image will be scaled appropriately so
that the output will contain the entire image.
-origin OR -AdjustOrigin
Adjust the origin values in the image file header for changes in image
size or the centering of a subarea. With this adjustment, a model built
on the input stack should be correctly located when loaded onto the output
stack in 3dmod. Model points will be correctly located in Z provided
that a contiguous set of sections is output. They will be correct in X
and Y provided that the only operations on the data are a change in size
of the output, binning with -bin, scaling with -expand, and shifting with
-offset.
-linear OR -LinearInterpolation
Use linear instead of cubic interpolation to transform images. Linear
interpolation is more suitable when images are very noisy, but cubic
interpolation will preserve fine detail better when noise is not an issue.
Images are transformed when the -xform, -expand, -rotate, -distort, or
-gradient option is entered.
-nearest OR -NearestNeighbor
Use nearest neighbor interpolation instead of cubic interpolation to
transform images. This method simply picks the nearest existing pixel
value instead of interpolating between surrounding values, so it can be
used when pixels have discrete or meaningful values that need to be
preserved. This option and -linear are mutually exclusive. Images are
transformed when the -xform, -expand, -rotate, -distort, or -gradient
option is entered.
-float OR -FloatDensities Integer
Adjust densities of sections individually. Enter 1 for each section to
fill the data range, 2 to scale sections to common mean and standard
deviation, 3 to shift sections to a common mean without scaling, or 4 to
shift sections to a common mean then rescale the minimum and maximum
densities to the Min and Max values specified with the -scale option.
-contrast OR -ContrastBlackWhite Two integers
Rescale densities to match the contrast seen in 3dmod with the given
black and white values. This works properly only when the output file
will be bytes. It will not work if the data were loaded into 3dmod with
intensity scaling; use mrcbyte in that case.
-scale OR -ScaleMinAndMax Two floats
Rescale the densities of all sections by the same factors so that the
original minimum and maximum density will be mapped to the Min and Max
values that are entered.
-fill OR -FillValue Floating point
Value to fill areas of the output image that have no image data. The
filling is done before intensity scaling, so the value in the filled areas
will be modified by any scaling that is done.
-multadd OR -MultiplyAndAdd Two floats
Rescale the densities of all sections by multiplying by the first entered
value then adding the second value. This option must be entered either
once only, or once per input file.
(Successive entries accumulate)
-taper OR -TaperAtFill Two integers
To taper an output image at a border between real image and a filled area,
just as is done with Mrctaper, enter this option with two values: 1)
the distance over which to taper or 1 for a default distance, and 2) a 0
for tapering outside or 1 for tapering inside the border. The default
distance is 1% of the mean of the X and Y sizes of the output image, but
at least 16 pixels and no more than 127. The program will exit with an
error if there is not enough memory to hold the entire output image.
-distort OR -DistortionField File name
Image distortion field file to use for undistorting images. The
undistortion is applied before any transformations.
-imagebinned OR -ImagesAreBinned Integer
The current binning of the images, so that the distortion field can be
applied correctly. This entry is required unless the program can
determine the binning unambiguously from the image size.
-fields OR -UseFields List of integer ranges
A list of the distortion fields to apply for each section (numbered from
0). Ranges are allowed. The default is for the field numbers to match
the input section numbers, unless there is only one field in the file. To
have a single field applied to all of the sections, just enter a single
number. This option was added when the only way to warp images was with
distortion field files but it is not needed with current warping files.
(Successive entries accumulate)
-gradient OR -GradientFile File name
File with magnification gradients to be applied for each image. This
should be a file listing the tilt angle, the percent magnification change
per micron of Z height, and the degrees of rotation per micron of Z height
for each image, such as is produced by Extractmaggrad. The mag gradient
correction is applied before a distortion field correction and before any
transformations.
-memory OR -MemoryLimit Integer
Use this entry to specify the amount of memory used by the program for its
main array, in megabytes. By default, the program will try to use enough
memory to hold both an input image and an output image in floating point
form (4 bytes per pixel), up to 15 GB. Limiting the memory can keep it
from using more memory than available but may result in it operating on
the images in chunks. Entering a value larger than 15000 will allow it to
use more memory than the default. The main array is used to hold both the
input image (in whole or in part) and a transformed image (in whole or in
part).
-test OR -TestLimits Two integers
To test the code for reading in and binning images in chunks, enter limits
for the total size of the working array, and for the size of the array
used for binning. The sizes are the number of 4-byte elements.
-verbose OR -VerboseOutput Integer
1 for diagnostic output
-param OR -ParameterFile Parameter file
Read parameter entries as keyword-value pairs from a parameter file.
-help OR -usage
Print help output
-StandardInput
Read parameter entries from standard input.
If no command line arguments are entered, inputs are prompted interactively.
The input is as follows; lines starting with IF are entered only if a
particular option is selected.
Number of input files, or -1 to enter list of files from a file
IF you enter a number >0, then for each input file, enter two lines:
File name
list of section numbers to take from that file. The first section is
numbered 0 and ranges may be entered (e.g. 0-4,5,7-9)
OTHERWISE, IF you enter -1, then enter the name of the file with list of
files and sections. The format of this file should be
Number of input files
File name
list of section numbers (first is 0!!!), ranges allowed (e.g. 0-3,5,7-9)
next file and list of sections, etc.
Number of output files, or -1 to enter list of files from a file
IF you enter 1, next enter the name of the single output file.
OR IF you enter a number >1, then for each output file, enter two lines:
File name
Number of sections to put in that file
OR IF you enter -1, then enter the name of the file with the list of files
and numbers of of sections. The format of this file is:
Number of output files
File name
number of sections to place in that file
next file and number of sections, etc.
The X and Y dimensions of the output file, or / to take the same
dimensions as the first input file
The data mode of the output file, or / to take the same mode as the first
input file. Mode can be 0 for bytes, 1 for 16-bit integers, 2 for
32-bit real numbers, or 9 to 15 for 9 to 15 bit values.
0 for no offsets, or 1 to apply X and Y offsets separately to each section,
or -1 to apply the same X and Y offset to all sections
IF offsets are selected, next specify the X and Y center offsets of
each of the sections (they need not be on separate lines), or the single
offset to apply to all sections.
Hint: offsets of 50,50 will take the center of the output image
from the upper right quadrant of the input image.
0 for no transforms, or 1 to transform images using cubic interpolation, or
2 to transform using bilinear interpolation
IF transforming is selected, next specify the file for transforms, and
on a separate line, a list of lines to use in the file (first is 0!!!).
Ranges are allowed here, but all must be on one line. To have a single
transform applied to all of the sections, just enter a single number.
0 for no floating of densities, 1 to float densities so that each section
occupies the same RANGE of densities (the optimum range for the
particular data mode), 2 to float so that all sections have the same
MEAN and STANDARD DEVIATION of density (which requires the input
sections to be read twice), -1 to specify a single scaling that will
be applied equally to all sections, -2 to scale all sections by the
same amount, determined by black and white contrast settings, in order
to convert to bytes with stretched contrast, 3 to SHIFT sections to
the same mean value without rescaling, or 4 to SHIFT sections to the
same mean then rescale all sections by the same specified scaling.
IF single rescaling is selected with "-1", next specify the density values
to which the minimum and maximum in the data file should be mapped, or
just enter "/" to have them mapped to the full-scale range for the
data mode, or enter "1,1" to override any scaling of data that would
otherwise occur because of a change of mode. This scaling method can
be used to invert densities by entering, e.g., 255,0.
IF scaling by black and white contrast settings is selected with -2, next
enter the black and white settings (see mrcbyte).
OR, IF shifting and rescaling was selected with "4", next specify the
density values to which the new, shifted minimum and maximum values
should be mapped, or just enter "/" to have them mapped to the full-
scale range for the data mode.
ADDITIONAL NOTES
The program will adjust the pixel spacing in the header of the output file
if binning and/or the expansion or skrinkage options are used. Models built
on the input file in 3dmod should display correctly on a file scaled by
these three options. However, the image origin is not adjusted for images
that are a subset of the input image unless the -origin option is entered as
well. Without this option, it would be necessary to shift a model manually
in 3dmod to display it on a subset.
When there is a change of modes, data are rescaled optimally in most
cases. If floating is not selected, the scaling is by an amount that would
map the range for the input data mode into the range for the output data
mode (e.g., going from mode 1 to mode 2, it would scale data up by a factor
of 128). Generally, all sections will be rescaled by the same amount, thus
preserving relative differences between sections. However, if the output
mode is 2 (reals), this kind of rescaling will not occur. In general, if
your input mode is real and you are outputing to bytes or integers, you need
to specify some kind of floating, shifting, or scaling in order to get
usable values. These same kinds of automatic scalings upon change of mode
will occur with shifting to a common mean (option 3), but here the program
is more careful to apply the same scaling to all sections. To disable the
scaling upon change of mode, the use the option "-scale 1,1," or the "-1"
floating option with interactive input.
At the end, the program will report how many pixels have been truncated at
the low or high limits of the output range. If this happens after shifting
with a specified output range, try again with a slightly smaller range.
The transformation file entered with -xform can contain warping
transformations as well as simple linear transforms. The transforms can
consist of either displacements at a set of control points, as output
directly by Midas, or displacements on a regular grid, as produced by
Xftoxg. Such warping files contain information about the size of the
images that were aligned and their pixel size. The transforms will be
adjusted for a difference between that pixel size and the pixel size of the
image file being transformed. They will also be extrapolated as necessary
to cover an image area larger than that specified in the warping file;
however, the program assumes that the input images are centered on the
images with which the warping file was prepared. The warping will be
incorrect if this is not the case.
Warping cannot be used with distortion corrections. If this is necessary,
consider using Blendmont, which can combine such operations. The -expand
and -rotate options also cannot be used with warping transformations because
such transformation would occur after the warping while the linear transform
included in a warping file occurs before the warping. If these operations
are necessary, you can avoid transforming the images twice by preparing a
one-line text file with
factor 0. 0. factor 0. 0.
for expansion by the factor, or with
cos(angle) -sin(angle) sin(angle) cos(angle) 0. 0.
for rotation by the angle. Then use Xfproduct to multiply the warping
transforms by the single linear transform, and transform the images with the
product.
Complex data (Fourier transforms) can be stacked but not otherwise
manipulated. Don't try to change the size or center of the FFT's.
Newstack will not work with color data (MRC data mode 16). If your images
came from color Tiff files and you do not need the color information, use
the "-g" option when converting them with Tif2mrc. If you do need to
preserve the color information, then you can split the color channels into
three byte-mode files with "clip splitrgb", use Newstack to perform
identical operations on each of the three files, and recombine the data into
a color file with "clip joinrgb".
The program attempts to allocate enough memory to hold an entire input
image and output image in 4 bytes per pixel, but it can work on large images
with less memory than this by reading, operating on, and writing strips of
the image if necessary. On a 32-bit system, if the memory allocation fails,
it will try to work with just under 2 GB of memory. Memory allocation will
probably not fail on a 64-bit system. In either case, system performance
will suffer if the program allocates more than the amount of memory
available, and it is up to the user to limit the memory as appropriate with
the "-memory" option. The ability to work on the image in strips is limited
when images are rotated by large angles (30-90 degrees); in such cases the
program will need at least enough memory to hold the whole input images.
The program can manipulate and transform images bigger than 4 gigapixels,
but distortion correction is currently limited to 2 gigapixel images.
Each linear transformation in a transform file is specified by a line with
six numbers:
A11 A12 A21 A22 DX DY
where the coordinate (X, Y) is transformed to (X', Y') by:
X' = A11 * (X - Xci) + A12 * (Y - Yci) + DX + Xco
X' = A21 * (X - Xci) + A22 * (Y - Yci) + DY + Yco
where (Xci, Yci) and (Xco, Yco) are the center coordinates of the input and
output images, respectively.
HISTORY
DNM (mast) for VAX 8/88
5/4/89 changed to float to same mean and sd rather than mean and min.
12/19/89 added full control of density scaling; fixed bug in
interpolation that was generating half-pixel error half the time.
4/19/90 added line-by-line prompts for file and section entry and
added time/date stamp
1/23/91: fixed so that it can correctly pad images and extract
subsets of images with either center-shifting or transforms
6/11/91: fixed bug if trying to output scaled real values les than 0
Feb 94 : Ported to SGI by Paul Furcinitti.
Jul 94 : PSF (Furcinitti) Fixed bug when writing byte files.
11/16/94: DNM added shifting to mean, rationalized scaling somewhat
9/24/01: changed to read and write in chunks if images are big
12/24/03: converted to PIP input, added undistorting and binning
June 2011: Added support for warping
SEE ALSO
mrcbyte(1), tif2mrc, clip