eman2:e2tomosmall
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
eman2:e2tomosmall [2025/06/18 10:56] – steveludtke | eman2:e2tomosmall [2025/08/05 11:06] (current) – [Particle extraction (~2 min (a few manual) - hours (a couple of thousand))] steveludtke | ||
---|---|---|---|
Line 50: | Line 50: | ||
* Once the options are set, press **Launch** | * Once the options are set, press **Launch** | ||
+ | <note tip> | ||
When working with your own data: | When working with your own data: | ||
- | + | | |
- | | + | |
* %%__%% (double underscore) is also reserved for describing modified versions of the same file, and should not be used in your image filenames. | * %%__%% (double underscore) is also reserved for describing modified versions of the same file, and should not be used in your image filenames. | ||
* If your tilt series isn't a single stack file, but is many individual images instead, you will need to use **Generate tiltseries** to build an image stack | * If your tilt series isn't a single stack file, but is many individual images instead, you will need to use **Generate tiltseries** to build an image stack | ||
+ | </ | ||
===== Tiltseries Alignment and Tomogram Reconstruction (~10 min Small) ===== | ===== Tiltseries Alignment and Tomogram Reconstruction (~10 min Small) ===== | ||
Line 85: | Line 86: | ||
{{http:// | {{http:// | ||
+ | <note tip> | ||
When working with your own data: | When working with your own data: | ||
* We strongly recommend reconstructing at most 5-10 tilt series to start. Then completing the handedness check in the next step before returning to this step and reconstructing the full set of tilt series! This will allow you to specify the correct //tltax// value. | * We strongly recommend reconstructing at most 5-10 tilt series to start. Then completing the handedness check in the next step before returning to this step and reconstructing the full set of tilt series! This will allow you to specify the correct //tltax// value. | ||
Line 95: | Line 97: | ||
* It can also be helpful with thin ice to specify a **clipz** value to generate thinner tomograms (perhaps 64 or 96 for a 1k tomogram). | * It can also be helpful with thin ice to specify a **clipz** value to generate thinner tomograms (perhaps 64 or 96 for a 1k tomogram). | ||
* There is a new // | * There is a new // | ||
+ | </ | ||
===== Handedness Check ===== | ===== Handedness Check ===== | ||
Line 136: | Line 139: | ||
* Launch | * Launch | ||
+ | <note tip> | ||
When working with your own data: | When working with your own data: | ||
* The first two options, //dfrange// and //psrange// indicate the defocus and phase shift range to search. It is critical that the actual defocus be within the search range (obviously). They take the format of “start, end, step”, so “2, 5, .1” will search defocus from 2 to 5 um with a step size of 0.1. Units for phase shift is degrees. | * The first two options, //dfrange// and //psrange// indicate the defocus and phase shift range to search. It is critical that the actual defocus be within the search range (obviously). They take the format of “start, end, step”, so “2, 5, .1” will search defocus from 2 to 5 um with a step size of 0.1. Units for phase shift is degrees. | ||
* For images taken with volta phase plate, we usually have **dfrange** of “0.2, | * For images taken with volta phase plate, we usually have **dfrange** of “0.2, | ||
+ | </ | ||
- | //Note: this program is only estimating CTF parameters, taking tilt into account. It is not performing any phase-flipping corrections on whole tomograms. CTF correction is performed later as a per-particle process. This process requires metadata determined during tilt-series alignment, so it cannot be used with tomograms reconstructed using other software packages.// | + | < |
- | + | This program is only estimating CTF parameters, taking tilt into account. It is not performing any phase-flipping corrections on whole tomograms. CTF correction is performed later as a per-particle process. This process requires metadata determined during tilt-series alignment, so it cannot be used with tomograms reconstructed using other software packages.// | |
- | // | + | </note> |
+ | < | ||
+ | In >=2022 snapshots of EMAN2 it is possible after CTF correction to return to the 3-D reconstruction step and produce CTF corrected whole tomograms, but this does nothing useful when following the EMAN2 pipeline. If you wish to compare EMAN2 tomograms with other software doing CTF correction, this could potentially be useful// | ||
+ | </ | ||
===== Tomogram reconstruction evaluation ===== | ===== Tomogram reconstruction evaluation ===== | ||
{{http:// | {{http:// | ||
Line 162: | Line 169: | ||
* The image at the top is the central slice through the tomogram | * The image at the top is the central slice through the tomogram | ||
* the //show2d// button displays the selected tomogram in a slice-wise view. | * the //show2d// button displays the selected tomogram in a slice-wise view. | ||
- | * //!ShowTilts// shows the corresponding raw tilt series | + | * // |
* Please note that most tomograms include some out-of-plane tilt (the actual rotation isn't a simple tilt along a single axis), which is taken into account during alignment. This may make it visually appear that the tilt series alignment is not as robust as it actually is. | * Please note that most tomograms include some out-of-plane tilt (the actual rotation isn't a simple tilt along a single axis), which is taken into account during alignment. This may make it visually appear that the tilt series alignment is not as robust as it actually is. | ||
* //Boxer// opens the 3D particle picker | * //Boxer// opens the 3D particle picker | ||
- | * //!PlotLoss// will plot the fiducial error for each tilt | + | * // |
- | * //!PlotCtf// plot the defocus and phase shift at the center of each tilt image | + | * //PlotCtf// plot the defocus and phase shift at the center of each tilt image |
* // | * // | ||
* You can adjust //X Col// and //Y Col// in the plot control panel (middle click the plot). The columns represent: | * You can adjust //X Col// and //Y Col// in the plot control panel (middle click the plot). The columns represent: | ||
Line 191: | Line 198: | ||
This is the easiest approach by far, and is basically a single step process, but is only available in recent (2022+) snapshots of EMAN2. | This is the easiest approach by far, and is basically a single step process, but is only available in recent (2022+) snapshots of EMAN2. | ||
* You should easily be able to select ~500 particles per tomogram, potentially more, but 500 is more than sufficient for the tutorial | * You should easily be able to select ~500 particles per tomogram, potentially more, but 500 is more than sufficient for the tutorial | ||
- | * For instructions, | ||
- | * When complete, skip ahead to the next major section **Particle Extraction** | ||
+ | {{https:// | ||
+ | |||
+ | - Subtomogram Averaging → Convnet based auto-boxing | ||
+ | * label: **ribo** | ||
+ | * gpuid: **-1** //if you don't have a GPU, otherwise use default// | ||
+ | - **Launch** | ||
+ | - Four windows will appear: " | ||
+ | - The " | ||
+ | - Click on tomob. This will cause a window showing the central section of the tomogram to appear. You will need this a lot, so move it somewhere where it is accessible, but doesn' | ||
+ | - Next to the " | ||
+ | - The goal is to select a few (~5 - 20) regions with a ribosome particle well centered in the middle of the circle (which will appear when you click). | ||
+ | * Each time you click, the corresponding region will appear in the (Positive) window | ||
+ | * Holding shift and clicking will remove a reference already selected. | ||
+ | * Using the up/down arrows in the tomogram view will allow you to move up and down through the Z slices of the tomogram. | ||
+ | - Once you have selected 5 or more well centered particles, Change "Good References" | ||
+ | - Use the arrow keys to move " | ||
+ | * Do **not** select very high contrast " | ||
+ | * Instead, select regions of background which contain nothing at all, or weak features which don't look like a particle. | ||
+ | * Again, select 5 or more of these background regions, which will appear in the (Negative) window. | ||
+ | - Once you're happy with your references, hit the **Train** button. | ||
+ | - Bring the terminal window where you launched the project manager to the front. As it trains you will see a series of lines like: " | ||
+ | - Assuming the number decreases to a reasonable value (<1.0). You probably got a pretty decent result. | ||
+ | - Change "Bad References" | ||
+ | - You should see some particles appear in both the tomogram (you may need to move up and down) and the (particles) window. The table will tell you how many were found. | ||
+ | - If the particles look ok, but the number found is significantly less than the ~500 you should be able to get, try reducing " | ||
+ | - If you are satisfied with the results on this Tomogram, press **Save** to save the network, then press **ApplyAll** to box out all 3 Tomograms. | ||
+ | - When complete, skip ahead to the next major section **Particle Extraction** | ||
+ | |||
+ | < | ||
+ | See: [[EMAN2: | ||
+ | </ | ||
==== Tomogram annotation (GPU recommended) ==== | ==== Tomogram annotation (GPU recommended) ==== | ||
- | {{http:// | + | This is an older strategy using the deep-learning based tomogram annotation program to find particles. It still works, but the new deep-learning picker in the section above will generally work better. |
+ | |||
+ | {{https:// | ||
* Since the tutorial data set is purified ribosomes, this step can be skipped for the tutorial data, and you can move on to template-based particle picking. For cells or other types of complex specimens, tomogram annotation can be used to more easily distinguish locations of different types of objects. | * Since the tutorial data set is purified ribosomes, this step can be skipped for the tutorial data, and you can move on to template-based particle picking. For cells or other types of complex specimens, tomogram annotation can be used to more easily distinguish locations of different types of objects. | ||
Line 246: | Line 284: | ||
* If you did the previous optional annotation step above, you will be able to see the selected particles here, and if you like, manually update them. | * If you did the previous optional annotation step above, you will be able to see the selected particles here, and if you like, manually update them. | ||
- | ===== Particle extraction (~2 min (a few manual) - hours (a couple of thousand)) ===== | + | ===== Particle extraction (~2 min (a few manual) - over an hour (a couple of thousand, large tutorial)) ===== |
Note that this step will be vastly different resource-wise if you are only extracting a few manually selected particles for purposes of later template matching or if you already selected hundreds of particles from each tomogram. | Note that this step will be vastly different resource-wise if you are only extracting a few manually selected particles for purposes of later template matching or if you already selected hundreds of particles from each tomogram. | ||
+ | |||
+ | < | ||
+ | Depending on how you selected particles you may have a significant number of particles which contain fiducials or artifacts from the fiducials (bright spots). Having these in your particle data can cause significant problems, and could lead to getting bad initial models and thus bad refinements. For a small data set like this, it might be a good idea to open the manual boxer for each tomogram and delete any particles with fiducials in them before completing this step (particle extraction). | ||
+ | </ | ||
The reduced 1k x 1k (or 2k) tomograms are used only as a reference to identify the location of the objects to be averaged. Now that we have particle locations, the software returns to the original tilt-series, | The reduced 1k x 1k (or 2k) tomograms are used only as a reference to identify the location of the objects to be averaged. Now that we have particle locations, the software returns to the original tilt-series, | ||
Line 257: | Line 299: | ||
* set // | * set // | ||
* If you had the correct size in the previous step this should be the same as leaving the default -1 | * If you had the correct size in the previous step this should be the same as leaving the default -1 | ||
- | * enter the label you used when picking particles (" | + | * enter the label you used when picking particles (" |
* //threads// = value for your machine | * //threads// = value for your machine | ||
* Launch | * Launch | ||
Line 266: | Line 308: | ||
* This will generate particle sets, which are virtual particle stacks that consist of particles with the same label from different tomograms. | * This will generate particle sets, which are virtual particle stacks that consist of particles with the same label from different tomograms. | ||
- | For your own data | + | <note tip> |
+ | When working with your own data: | ||
* If the box size is correct when you select particles from the GUI, you can leave // | * If the box size is correct when you select particles from the GUI, you can leave // | ||
* If your particles are deeply buried in other densities, using a bigger //padtwod// may help, but doing so may significantly increase the memory usage and slow down the process. | * If your particles are deeply buried in other densities, using a bigger //padtwod// may help, but doing so may significantly increase the memory usage and slow down the process. | ||
* With CTF information present, it generally does not hurt to check //wiener//, which filters the 2D particles by SSNR before reconstructing them in 3D. | * With CTF information present, it generally does not hurt to check //wiener//, which filters the 2D particles by SSNR before reconstructing them in 3D. | ||
* Specify a binning factor in //shrink// to produce downsampled particles if your memory/ | * Specify a binning factor in //shrink// to produce downsampled particles if your memory/ | ||
+ | </ | ||
===== Initial model generation (10 - 60 min) ===== | ===== Initial model generation (10 - 60 min) ===== | ||
While intuitively it seems like (since the particles are already in 3-D) the concept of an " | While intuitively it seems like (since the particles are already in 3-D) the concept of an " | ||
Line 309: | Line 351: | ||
Once it gets past 3-4 iterations, you can use the browser to look in // | Once it gets past 3-4 iterations, you can use the browser to look in // | ||
- | For your own data: | + | Note that while the program converges pretty quickly, it won't always get the correct structure, particularly if you have some bad particles (like fiducials) included in the particle set. It is important to look at the resulting starting map and make sure it looks at least vaguely like you expect. If not you may wish to try running this step again. This will produce // |
+ | |||
+ | <note tip> | ||
+ | When working with your own data: | ||
* If your particle has known // | * If your particle has known // | ||
* setting //shrink// to something in the range of 2-4 will make the runtime faster but, depending on the shape, could potentially cause problems. | * setting //shrink// to something in the range of 2-4 will make the runtime faster but, depending on the shape, could potentially cause problems. | ||
* using more than the minimal 30-50 particles is fine. If you have a very large set of selected particles, go ahead and use them all. This will not slow the process down at all, since it's stochastic. | * using more than the minimal 30-50 particles is fine. If you have a very large set of selected particles, go ahead and use them all. This will not slow the process down at all, since it's stochastic. | ||
* it is critical that the full sampling box size of the extracted particles divided by //shrink// be divisible by 2. If not, the program will crash. | * it is critical that the full sampling box size of the extracted particles divided by //shrink// be divisible by 2. If not, the program will crash. | ||
+ | </ | ||
==== Old Initial Model Generator ==== | ==== Old Initial Model Generator ==== | ||
- | This section is the older program, which is still functional, but not recommended. Skip this if you already used the new version above. | + | This section is the older program, which is still functional, but not recommended. |
For the tutorial tilt-series: | For the tutorial tilt-series: | ||
Line 329: | Line 374: | ||
* You can terminate the job as soon as // | * You can terminate the job as soon as // | ||
- | For your own data: | + | When working with your own data: |
* If your particle has known // | * If your particle has known // | ||
* The symmetry you specify will not be imposed on the map unless you also check // | * The symmetry you specify will not be imposed on the map unless you also check // | ||
Line 358: | Line 402: | ||
* note that this process stores 3-D particle locations in the appropriate info/* files, but does not extract particles from the micrographs | * note that this process stores 3-D particle locations in the appropriate info/* files, but does not extract particles from the micrographs | ||
- | For your own data: | + | <note tip> |
+ | When working with your own data: | ||
* Frankly, in most cases the deep learning based picker will do a better job. For cells, you may consider the full Tomogram Annotation tool. This program works with highly downsampled tomograms and references for speed. | * Frankly, in most cases the deep learning based picker will do a better job. For cells, you may consider the full Tomogram Annotation tool. This program works with highly downsampled tomograms and references for speed. | ||
* Picking a good //vthr// and a reasonable //nptcl// maximum may take a little trial and error. Test on 1-3 tomograms before the full run. | * Picking a good //vthr// and a reasonable //nptcl// maximum may take a little trial and error. Test on 1-3 tomograms before the full run. | ||
+ | </ | ||
==== Particle extraction (ONLY if not done for full data above) ==== | ==== Particle extraction (ONLY if not done for full data above) ==== | ||
If you already extracted a complete set of particles (not just the few initial references) above, you don't need to repeat it again here. | If you already extracted a complete set of particles (not just the few initial references) above, you don't need to repeat it again here. | ||
Line 387: | Line 432: | ||
===== New integrated refinement program ===== | ===== New integrated refinement program ===== | ||
- | There is a new refinement program which implements both traditional subtomogram averaging and subtilt refinement in a single program. This is an alternative to the next two major sections | + | There is a new refinement program which implements both traditional subtomogram averaging and subtilt refinement in a single program. This is a highly recommended |
+ | |||
+ | // | ||
==== Small (large below) ==== | ==== Small (large below) ==== | ||
Line 444: | Line 491: | ||
Skip the next 2 sections about the old refinement. Other notes are below that. | Skip the next 2 sections about the old refinement. Other notes are below that. | ||
+ | |||
===== Old Subtomogram refinement (~1 hr/ | ===== Old Subtomogram refinement (~1 hr/ | ||
- | {{http:// | + | {{https:// |
- | As an alternative to the new integrated tool above, the older pair of programs is still available. You shouldn' | + | |
+ | **This section is ONLY for use if you skipped | ||
This step performs a conventional iterative subtomogram averaging using the full set of particles. Typically it will achieve resolutions in the 15-25 A range with a reasonable number of particles. As it involves 3-D alignment of the full set of particles multiple times, it takes a significant amount of compute time. Higher resolutions are achieved in the next stage after this (subtilt refinement). | This step performs a conventional iterative subtomogram averaging using the full set of particles. Typically it will achieve resolutions in the 15-25 A range with a reasonable number of particles. As it involves 3-D alignment of the full set of particles multiple times, it takes a significant amount of compute time. Higher resolutions are achieved in the next stage after this (subtilt refinement). | ||
Line 470: | Line 519: | ||
===== Old Subtilt refinement (~9 hr/ | ===== Old Subtilt refinement (~9 hr/ | ||
{{http:// | {{http:// | ||
- | This is the second half of the old refinement strategy. It is conceptually similar to the t,p and r iterations in the newer integrated program above. | + | **This is the second half of the old refinement strategy.** It is conceptually similar to the t,p and r iterations in the newer integrated program above. |
With the results of a good subtomogram alignment/ | With the results of a good subtomogram alignment/ | ||
Line 492: | Line 541: | ||
===== Refinement evaluation (optional) ===== | ===== Refinement evaluation (optional) ===== | ||
- | {{http:// | + | {{https:// |
* **Analysis and Visualization -> Evaluate SPT Refinements** | * **Analysis and Visualization -> Evaluate SPT Refinements** |
eman2/e2tomosmall.1750244211.txt.gz · Last modified: by steveludtke