EMAN2/Programs/e2refinemulti

e2refinemulti

This is a variant of the standard e2refine program. This version performs so-called multi-model refinement. Rather than producing a single 3-D model based on a set of 2-D single particles, this program will create multiple 3-D models from a single set of 2-D particles. Several starting maps are provided, with (optionally) a different symmetry for each. Each particle will not only be classified based on orientation, but also based on which of the input models it best-matches. This process is iterative, so in theory, identical input models perturbed by a little random noise can be used as seeds, and eventually the refinement should find the greatest structural variations to base the different output maps on. However, if the input maps are nearly identical, this convergence could require many iterations. If the 'seed maps' are similar to the final expected structures, clearly the process will converge more quickly. Normally, after performing this multi-model refinement, in which particles are free to move between different output maps from iteration to iteration, the particles associated with each map would be separated into different 'sets', then normal single-model refinement would be performed on each of these particle sets, to achieve a more self-consistent result, and higher resolution.

The options for this program are basically identical to those for e2refine.py with 2 exceptions, instead of --model to specify the starting model, use --models= with a comma-separated list of inputs. --sym can be either a single symmetry for all models, or a comma separated list with the same number of symmetries as input models.

Command Line Arguments

General Options

	--version	bool	show program's version number and exit
-h	--help	bool	show this help message and exit
-c	--check	bool	Checks the contents of the current directory to verify that e2refine.py command will work - checks for the existence of the necessary starting files and checks their dimensions.
-v	--verbose	int	verbose level [0-9], higner number means higher level of verboseness

Options impacting the overall refinement

	--iter	int	The total number of refinement iterations to perform
	--models	string	A set of 3-D maps to seed the refinement. Comma separated list. The number of outputs will match the number of inputs.
	--input	string	The name of the image containing the particle data
	--usefilt	string	Note: some unresolved bugs may exist with this option (6/2011) Specify a particle data file that has been low pass or Wiener filtered. Has a one to one correspondence with your particle data. If specified will be used in projection matching routines, and elsewhere.
	--path	string	The name of a directory where results are placed. If not specified (suggested), will use a path of the form refine_xx.
	--mass	float	The mass of the particle in kilodaltons, used to run normalize.bymass. If unspecified nothing happens. Requires the --apix argument.
	--apix	float	The angstrom per pixel of the input particles. This argument is required if you specify the --mass argument. If unspecified, the convergence plot is generated using either the project apix, or an apix of 1.
	--sym	string	Symmetry to be imposed throughout: c<n>, d<n>, h<n>, tet, oct, icos. Omit this option or specify 'c1' for asymmetric reconstructions. Either a single symmetry may be specified for all output maps, or the same number of (comma separated) symmetries as maps.
	--lowmem	bool	Make limited use of memory when possible. Slight speed penalty.
-P	--parallel	string	Run in parallel, specify type:<option>=<value>:<option>:<value>. See EMAN2/Parallel

Options related to making projections

Options for comparing particles to projections

Options for classifying particles based on similarity matrix

Options for generating class-averages

Options related to 3-D Reconstruction of Class-averages and post-processing

--pad	int	To reduce Fourier artifacts, the model is typically padded by ~25% - only applies to Fourier reconstruction. Please read EMAN2/BoxSize
--recon	string	reconstructor to use. Main choices are 'fourier' or 'wiener_fourier'
--m3dkeep	float	The percentage of slices to keep in e2make3d.py
--m3dkeepsig	bool	Similar to classkeepsig above, changes the meaning of --m3dkeep to be in terms of standard deviations
--m3dsetsf	bool	Filters the final 3-D map to match the precomputed structure factor (stored in the project database). Normally used with a --m3dpostprocess=filter.lowpass.* option
--m3diter	int	The number of times the 3D reconstruction should be iterated. 2 and 3 are the only valid values. 2 is faster and normally has sufficient accuracy.
--m3dpreprocess	string	Normalization processor applied before 3D reconstruction
--m3dpostprocess	string	Post processor to be applied to the 3D volume once the reconstruction is completed
--automask3d	string	The 5 parameters of the mask.auto3d processor, applied after 3D reconstruction. These parameters are, in order: isosurface threshold,radius,nshells and ngaussshells. From e2proc3d.py you could achieve the same thing using --process=mask.auto3d:threshold=1.1:radius=30:nshells=5:ngaussshells=5. e2help.py processors -v2 for more information on mask.auto3d.

The refinement results will normally go into directories called multi_XX rather than refine_XX. The output files from e2refinemulti are identical to those produced by e2refine, except instead of having threed_YY files for each iteration, threed_YY_ZZ files are produced representing iteration number and map number within that iteration. The projections, similarity matrix, classification matrix and class-averages are all combined into a single file for all output maps. Specifically, all projections for map #1 appear first, followed by all projections for map #2, etc., with the class-averages in an identical order. This is a very different structure than used in EMAN1's multirefine program, which separated maps into separate subdirectories. In addition to the class-averages following the same order as the projections, each class-average will have a 'model-id' header item identifying which map that average is associated with. This item is used by e2make3d.py to perform 3-D reconstructions on the appropriate subset of class-averages for each output map.

The most common usage pattern for e2refinemulti.py is to run it until pseudoconvergence is achieved, similar to single-model refinement. Once this is achieved (observed by looking at the map convergence FSC curves), generally the particles will be split into groups and refined independently using e2refine. This process is still less automated than it could be , but it is at least fairly straightforward:

Select Evaluate Particle Sets in the workflow or run e2evalparticles.py
Use the range selection option to select the class-averages associated with a particular map
Generate a new set of particles associated with each map
Run an independent single-model refinement on each of these sets.
Run e2eotest to determine the resolution of each of these single-model refinements
for additional verification that the results you have obtained are result, a so-called cross validation test is a good idea. In this test, you take the refined map from particle subset #1 and use it to seed a single-model refinement of particle subset #2, etc. If the structure converges to the structure originally produced for that subset despite the strong bias towards the 'wrong' structure, this is strong evidence that the extracted subset of particles truly does represent a different structure.

Note: at present e2eotest.py cannot be run directly on the results of an e2refinemulti run. Resolution can only be assessed after the subsequent single-model refinements are complete.

	--projector	string	Projector to use. 'standard' is the default
	--orientgen	string	The orientation generation argument for e2project3d.py. Typically something like: --orientgen=eman:delta=2.0:inc_mirror=0

--twostage	int	Optionally run a faster 2-stage similarity matrix, ~5-30x faster, almost identical results. Value specifies shrink factor for first stage, typ 1-3
--shrink	int	Optionally shrink the input particles by an integer amount prior to computing similarity scores. For speed. If used with --twostage, this specifies the second stage shrink factor.
--simcmp	string	The name of a comparator to be used in comparing the aligned images
--simalign	string	The name of an aligner to use prior to comparing the images
--simaligncmp	string	Name and options for a comparator to use in first stage alignment for classification
--simralign	string	The name and parameters of the second stage aligner which refines the results of the first alignment. Currently this is either not specified or is 'refine'.
--simraligncmp	string	The name and parameters of the comparator used by the second stage aligner. Default is dot.
--simmask	string	A file containing a single 0/1 image to apply as a mask before comparison but after alignment
--prefilt	bool	Filter each reference (c) to match the power spectrum of each particle (r) before alignment and comparison

--classiter	int	The number of iterations to perform. Default is 1. Larger values reduce model/noise bias, but slightly decrease resolution.
--classcmp	string	The name and parameters of the comparitor used to generate similarity scores, when class averaging. Default is 'dot:normalize=1'
--classalign	string	If doing more than one iteration, this is the name and parameters of the aligner used to align particles to the previous class average.
--classaligncmp	string	This is the name and parameters of the comparator used by the fist stage aligner Default is dot.
--classralign	string	The second stage aligner which refines the results of the first alignment in class averaging. Currently this is either not specified or is 'refine'.
--classraligncmp	string	The comparator used by the second stage aligner in class averageing. Default is dot:normalize=1.
--classaverager	string	The averager used to generate the class averages. Default is 'mean'.
--classkeep	float	The fraction of particles to keep in each class, based on the similarity score generated by the --cmp argument (see also --classkeepsig).
--classkeepsig	bool	Change the keep ('--keep') criterion from fraction-based to sigma-based. eg - with this set, 1.0 would correspond to discarding particles more than 1 standard deviation from the mean
--classnormproc	string	Normalization processor and options applied during class averaging. Typically 'normalize.edgemean'
--classrefsf	bool	This will impose the 1-D structure factor of each model projection onto the corresponding class-average to improve its filtration. This is an alternative to Wiener filtration if the map resolution is regulated.
--classautomask	bool	Experimental. This will apply a 2-D automask to the class-average during iterative alignment for better accuracy. The final class averages are unmasked.