Contents

  1. Binary Installation Instructions v2.31
    1. All Systems
    2. Mac OS X and Linux Workstations (not clusters)
    3. Linux Clusters
        1. Use system OpenMPI
        2. Rebuild your own OpenMPI
    4. Windows
      1. Native Win7/10 64 bit
      2. Windows 10 - Linux/Bash shell
    5. Using the GPU

Binary Installation Instructions v2.31

All Systems

Official releases are in reverse chronological order. The Continuous Build is rebuilt any time a developer makes a change they think users should have access to. It is normally reasonably stable, and will contain the latest pre-publication features. Alternatively, the highest numbered version will contain a stable and tested release. Windows10 users please see the instructions below before downloading.

Mac OS X and Linux Workstations (not clusters)

The neural network code in EMAN2 works best on GPUs, which are available only on the Linux installations. It can still run on Mac, but will be quite slow.

  1. If you have previously installed EMAN2:
    • Please remove or rename any existing installed EMAN2 folder you might have.
    • Please remove any existing EMAN2 entries from PATH.
    • LD_LIBRARY_PATH, DYLD_LIBRARY_PATH, PYTHONPATH and PYTHONHOME are NO LONGER USED, and should be removed if you have them set.
    • If you have any of these shell variables set for use with other software, it may be necessary to remove those settings as well. If the tests below fail after installation, this is the first thing to check.
  2. Download eman2.X.MacOS.sh/eman2.X.linux64.sh.
  3. Run: 
       1 bash <path-to-EMAN2-installer>
    • You will be prompted for a location to install EMAN2. Note that you cannot rename this folder after installation! You must reinstall if you wish to move the installation.
    • You will be asked if you want to initialize EMAN2.
    • EMAN2 initialization will add a block of code to your .profile/.bashrc file. After initialization open your .profile/.bashrc file and confirm that a block like the following has been added 
         1 # added by EMAN2 2.31 installer
   2 # >>> conda init >>>
   3 # !! Contents within this block are managed by 'conda init' !!
   4 ...
   5 ...
   6 ...
   7 unset __conda_setup
   8 # <<< conda init <<<
   9
    If you have any other similar looking blocks before the last one, it might be a good idea to remove them to avoid any potential conflicts.
    • If you use a different shell, such as tcsh or zsh, you may need to edit the appropriate file yourself.
    • You should not normally need to reinstall OpenMPI. The copy of OpenMPI/Pydusa now distributed with the binaries should work on Macs/Linux workstations in most cases.
    • Don't forget to restart your shell if you changed the .profile/.bashrc or other scripts.
    • The changes made in .profile/.bashrc will ensure updating PATH, so that python/conda/anaconda in the installed EMAN2 package are found correctly. Users who want to use another python version, like the system version, can run 'conda deactivate' which will remove EMAN2 and its python version from PATH.

    • Mac users: If you don't understand what the .profile instructions are talking about, this may help: https://stackoverflow.com/questions/7501678/set-environment-variables-on-mac-os-x-lion

    • Mac users (bash only): On MacOS, this modifies ~/.profile. If you have a ~/.bash_profile startup file, ~/.profile will not be read. A simple solution is to source .profile within .bash_profile. Add 

         1 source ~/.profile

      to ~/.bash_profile.

  4. Linux users: The new neural network based routines are much faster running on a GPU. If you have an NVidia graphics card, see Using the GPU section below.

  5. Run these programs to see if the install worked: 
       1 e2version.py
   2 e2speedtest.py
   3 e2display.py
   4 e2proc2d.py :64:64:1 test.hdf --process mask.sharp:outer_radius=24
   5 e2display.py test.hdf
  6. If you have problems with any of these programs
    • The first thing to check is whether you have PYTHONPATH, PYTHONHOME, LD_LIBRARY_PATH or DYLD_LIBRARY_PATH set in your shell. While used in previous versions of EMAN, variables are no longer used, and in some cases may interfere with Anaconda. If they are set to make some other software package work, but they interfere with the programs above, you will have to unset them, and set them only when you need the other software.

    • Check PATH in your shell and make sure you have an entry that points to your installation, before any other python entries. The entry might look like like '<path-to-EMAN2-installation>/condabin' or similar. Probably, best not to have any other python entries at all in PATH.

    • Specifically, if only the last command fails and you are using an Nvidia graphics card, it is likely caused by a graphics card driver incompatibility. Updating the Nvidia driver usually fixes the problem. On recent Ubuntu systems, running apt-get install nvidia-current works. On other systems, you may need to follow the installation guide from Nvidia.

  7. You will find that when you open a new shell, you will see (base) added to your command prompt. This indicates that Anaconda, the environment EMAN2 now uses for distribution, is active, and you can run EMAN2/SPARX/SPHIRE commands.

    • If this causes issues for other software, you can type conda deactivate and EMAN2 commands will no longer work (but any software that doesn't like Anaconda will work).

    • Alternatively, you can type conda config --set auto_activate_base False, which will prevent Anaconda from being activated automatically when you open new shells. In that case you will need to do a conda activate before running EMAN2/SPARX/SPHIRE commands.

    MacOS Big Sur Users

    The new Big Sur MacOS release (fall 2020) broke some things in Python, which are only now (Dec 2020) getting fixed. These fixes have not yet made their way into Anaconda. If you experience an error similar to "Unable to load OpenGL library", this quick fix works when building from source and may also work for binary installs:

    You need to go into your miniconda3 or anaconda3 folder and find this file (the path may be slightly different on your install, but you should be able to find it):

    ~/anaconda3/envs/eman2/lib/python3.7/site-packages/OpenGL/platform/ctypesloader.py

    Around line 80 you should find a line like:

    fullName = util.find_library( name )

    change this line to:

    fullName = '/System/Library/Frameworks/' + name + '.framework/' + name

    (important to use 8 spaces before fullName, not a <tab>)

Linux Clusters

Use system OpenMPI

Most Linux clusters will have at least one OpenMPI installation on the cluster. In some cases there may be more than one, and you may have to select a "module" to get the correct one. It is also critical that OpenMPI be compiled with the --disable-dlopen option. If you don't understand this statement, please consult with your cluster sysadmin.

  1. Remove the OpenMPI we provided: 
    conda remove openmpi --force
  2. Make sure that the correct OpenMPI for your cluster is in your path. You should be able to run 'mpicc' and get a message like 'gcc: no input files'.
  3. Rebuild Pydusa using the system installed OpenMPI. 
    export GIT_PYDUSA_BRANCH=v20180515
conda build <path-to-EMAN2-directory>/recipes/pydusa --numpy 1.13 --no-test

    • Warning: If you see an error after this process like: 

      Can't build /home/stevel/EMAN2/recipes/fftw-mpi due to environment creation error:
Downloaded bytes did not match Content-Length
  url: http://www.fftw.org/fftw-3.3.6-pl1.tar.gz
  target_path: /home/stevel/EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz
  Content-Length: 4179807
  downloaded bytes: 208916

      this means the fftw download failed. You will need to re-run this step, but first, delete the failed download :  rm EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz 

    • Warning: If you see an error during this process like: 

           configure: error: could not find mpi library for --enable-mpi
      this means the fftw build failed. On our system, this happened due to a libuuid library mismatch in combination with the omnipath cluster architecture. To workaround this situation try: 
          cd <path-to-EMAN2-directory>/lib
    rm libuuid.so.1
    ln -s /lib64/libuuid.so.1.3.0 libuuid.so.1 (Try  'ldconfig -p | grep libuuid'  to find the correct libuuid location and version)
    cd -
      Run this step again and revert this change after the whole installation process is complete. 
          cd <path-to-EMAN2-directory>/lib
    rm libuuid.so.1
    ln -s libuuid.so.1.0.0 libuuid.so.1
    cd -
  4. Finally, install the compiled Pydusa: 
    conda install fftw-mpi --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults
conda install 'pydusa=1.15=np113*' --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults

Rebuild your own OpenMPI

This option insures that --disable-dlopen is used when compiling OpenMPI, but may lack some system-specific optimizations provided by your sysadmin.

  1. Remove the OpenMPI we provided: 
    conda remove openmpi --force
  2. Rebuild OpenMPI.

    It is important to have the hwloc library available on the system. It might be available with 'module load hwloc'. It is also important to have the libraries of the used queuing system available (slurm, Torque/PBS, SGE, ...). 

    conda build <path-to-EMAN2-directory>/recipes/openmpi -c defaults -c conda-forge -c conda-forge/label/cf201901
conda install openmpi --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults
  3. Rebuild Pydusa using the rebuilt OpenMPI: 
    export GIT_PYDUSA_BRANCH=v20180515
conda build <path-to-EMAN2-directory>/recipes/pydusa --numpy 1.13 --no-test

    • Warning: If you see an error after this process like: 

      Can't build /home/stevel/EMAN2/recipes/fftw-mpi due to environment creation error:
Downloaded bytes did not match Content-Length
  url: http://www.fftw.org/fftw-3.3.6-pl1.tar.gz
  target_path: /home/stevel/EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz
  Content-Length: 4179807
  downloaded bytes: 208916

      this means the fftw download failed. You will need to re-run this step, but first, delete the failed download :  rm EMAN2/conda-bld/src_cache/fftw-3.3.6.tar.gz 

    • Warning: If you see an error during this process like: 

           configure: error: could not find mpi library for --enable-mpi
      this means the fftw build failed. On our system, this happened due to a libuuid library mismatch in combination with the omnipath cluster architecture. To workaround this situation try: 
          cd <path-to-EMAN2-directory>/lib
    rm libuuid.so.1
    ln -s /lib64/libuuid.so.1.3.0 libuuid.so.1 (Try  'ldconfig -p | grep libuuid'  to find the correct libuuid location and version)
    cd -
      Run this step again and revert this change after the whole installation process is complete. 
          cd <path-to-EMAN2-directory>/lib
    rm libuuid.so.1
    ln -s libuuid.so.1.0.0 libuuid.so.1
    cd -
  4. Finally, install the compiled Pydusa: 
    •   conda install fftw-mpi --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults
  conda install 'pydusa=1.15=np113*' --force-reinstall --override-channels -c file://<path-to-EMAN2-directory>/conda-bld -c defaults

Windows

We are finally able to provide 64 bit Windows binaries for EMAN2, however, please see the Windows 10- Linux/Bash shell option below for what may be a better alternative. Notes:

Native Win7/10 64 bit

  1. Download eman2.X.win64.exe.
  2. Launch the installer.

    1. Select Installation Type: Just Me

    2. Choose Installation Location: Select a location with NO space in path

    3. Advanced Installation Options: Don't add EMAN2 to PATH environment variable.

  3. Open Anaconda Prompt by clicking Windows Start Menu -> Anaconda2 (64-bit) -> Anaconda Prompt.

  4. In most cases you will want to install: Python Launcher.

  5. Run these programs to see if the install worked: 
    e2version.py
e2speedtest.py
e2display.py
e2proc2d.py :64:64:1 test.hdf --process mask.sharp:outer_radius=24

Windows 10 - Linux/Bash shell

Windows 10 includes an embedded Ubuntu Linux environment. It is possible to run the EMAN2 Linux binaries within this Win10 environment, but you will need to install some additional dependencies to do so. Also, you will effectively be running at a Linux command prompt, so you will have to become a bit familiar with Linux to do this, but it does avoid installing an additional operating system on your machine.

  1. Enable "Windows Subsystem for Linux" and install Ubuntu or any other distribution that you are familiar with, https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/.

  2. Install OpenGL. 
    sudo apt-get update
sudo apt-get install libsm-dev libxrender-dev build-essential libgl1-mesa-dev mesa-utils mesa-common-dev libglu1-mesa libglu1-mesa-dev
sudo apt-get autoremove

  3. Install Xming X Server for Windows.

    • Don't forget the fonts and Mesa (OpenGL) modules! If it seems to work, but the letters are black boxes, or you have other visual artifacts, the problem is probably with OpenGL support.
  4. Set environment variables. 
    export DISPLAY=:0
glxinfo | grep OpenGL
export KMP_AFFINITY=disabled # per https://github.com/Microsoft/BashOnWindows/issues/785#issuecomment-238079769

  5. Download and install eman2.X.linux64.sh.

  6. Start X Server before running eman2 programs.
  7. Run these programs to see if the install worked: 
    e2version.py
e2speedtest.py
e2display.py
e2proc2d.py :64:64:1 test.hdf --process mask.sharp:outer_radius=24

Using the GPU

Currently, the GPU is only used for neural network operations in tomogram annotation and in particle picking. It provides a ~10 fold or more speed up in neural network training. The new GPU developments are currently based on TensorFlow. From about 2006-2012 EMAN2 had its own internal CUDA code, which could be compiled into the C++ library. This has been deprecated, and likely no longer works, though the code is still present. We are working on a new GPU support strategy moving forward.

Many machines will have CUDA installed already, and if CUDA is an appropriate version, this should work fine with the TensorFlow version shipped with EMAN2. However, if you are running newer versions of CUDA there may be problems. You can test compatibility quickly with: 

# Make sure you have your environment set to run EMAN2 programs
e2version.py
# The above command should work and return your current version. If it does, then run:
python -c "import tensorflow"

If this command does not return an error, then you should be able to run deep learning software within EMAN2. If it does raise an error, then you will need to debug the problem: 

apt-get install nvidia-cuda-toolkit

conda remove tensorflow-gpu tensorflow-gpu-base
pip install cython
pip install tensorflow
# read any messages carefully, if there are errors you may need other installations

EMAN2/Install/BinaryInstallAnaconda/2.31 (last edited 2020-12-29 02:15:12 by TunayDurmaz)