Installation

Introduction

These are instructions for installing the "mxlib" c/c++ library. This is my personal library which I have built up over many years. Along the way I have tried to keep it organized and documented. I have installed it on several different linux systems (running CentOS or Ubuntu). Installation on Mac O/S is also possible using Conda.

The below instructions were updated in June 2022 after a fresh install on Ubuntu 22.04.

Basic Dependencies

This is the list of the other libraries and code-bases mxlib depends on. I usually install these from scratch to ensure maximum optimization (though I have little to no quantitative evidence that it matters), but you can probably manage most of these from the O/S package manager. You will need to make sure to install the dev packages.

Alternatively, see the next section for quickstart instructions using the conda package manager (part of miniconda or Anaconda) on Linux or macOS.

1. You need a c++14 compatible compiler with OpenMP support.
   • CentOS 7: you will need to install the devtoolset package (see https://www.softwarecollections.org/en/scls/rhscl/devtoolset-7/):

     $ sudo yum install centos-release-scl
     $ sudo yum install devtoolset-7

     To enable it, use scl enable devtoolset-7 bash. To enable it for a user add this to .bashrc:

     source /opt/rh/devtoolset-7/enable

2. You need the GNU Scientific Library (GSL) (this is probably obtainable from the linux distro package manager, but I recommend building yourself)
3. You need the boost c++ library (this is probably obtainable from the linux distro package manager, fine to install with apt or yum).
   • On Ubuntu you can install libboost-all-dev.
4. You need the cfitsio FITS library.
5. You need a BLAS and LAPACK.
   
   • MKL: the intel Math Kernel Library (MKL) is probably the highest performance option for Intel CPUs. It is free (as-in-beer) and straightforward to install. See my notes on installing MKL.
     
   • ATLAS: Another option is ATLAS, see my notes on installing ATLAS. ATLAS has been successfully used on a recent AMD machine by Jacob Trzaska at UA.
6. You need the FFTW library, with at least version 3.3.5. (In principle the MKL fftw interface could be used, but this does not yet (Aug 2017) have the "make_planner_thread_safe" routines and only works in float and double.)
   
7. You need the Eigen matrix library (http://eigen.tuxfamily.org/index.php). I am currently on 3.4.0 for mxxlib::dev.
   • It is best to use cmake to configure Eigen and install it. This will ensure that the package config is updated.

The relevant header files for each library need to be accessible to the compiler, either in a standard location or in one you specify when compiling your programs.

Installing dependencies from conda

The conda package manager contains packages for all of mxlib's external dependencies (not including vendored libraries, shipped under source/vendor/). This means setting up a correct build environment on macOS or Linux – even in a cluster environment, where you may not have root access – is straightforward. (To do: CUDA support with conda.)

Once you have installed conda (either miniconda or Anaconda is fine), you will create and activate an environment to use mxlib with.

$ conda create -y -n mxlib_env

The install process is tested with conda-forge packages, which you may not be using by default. (Installing them in the mxlib_env environment won't cause conflicts with any other environments you may have.)

$ conda activate mxlib_env
$ bash ./setup/conda_setup_Linux.sh
# or, on macOS:
$ bash ./setup/conda_setup_macOS.sh

You may see the message: "To make your changes take effect please reactivate your environment." That would be "conda deactivate" followed by "conda activate mxlib_env".

If all went well, you should now be able to build mxlib:

$ make install PREFIX=$CONDA_PREFIX NEED_CUDA=no

Now mxlib headers and libraries are in $CONDA_PREFIX/include and $CONDA_PREFIX/lib and should be found by default as long as "conda activate mxlib_env" is always run after you open a new terminal and before you compile things that use mxlib.

Git Version Tracking

Mxlib includes a systems to track its git version, using the SHA1 hash. This, in principle, allows any results to be tagged with the version of the library used to generate them. The goal is to always be able to restore the code to a given point so that previous results can be reproduced. The library will issue a warning at compile time if there are any modifications to it that aren't yet committed, meaning that they aren't trackable.

Warning

To avoid getting this warning due to makefile modifications, the only files you should edit during installation are the local/Common.mk, local/MxLib.mk, and local/mxApp.mk. Do not edit these in the mk folder.

Configuring and Building mxlib

1. clone mxlib from https://github.com/jaredmales/mxlib
2. If your install prefix is different from $HOME, you will want to invoke make with make PREFIX=/path/to/libraries (where /path/to/libraries is the path to the folder containing lib and include folders for things like levmar and Eigen.) Alternatively, set that variable permanently following the steps below.
3. If you need to customize the build (e.g. switch the BLAS or FFT library used or modify paths), run make setup. This will create three Makefiles in local/ for you to customize.
4. To configure mxlib edit the files in the local/ directory only. Some options are:
   • Any variable set with ?= in the files in mk/ can be overridden in the corresponding local/ one. The files in local/ are not tracked in git and editing them will not cause compile warnings.
   • If you want to change the install path from $HOME to something else, set PREFIX= in local/Common.mk.
   • USE_BLAS_FROM can be set to mkl, ATLAS, or Accelerate (macOS only). You may also need to override BLAS_INCLUDES, BLAS_LDFLAGS, and/or BLAS_LDLIBS to set paths and flags. (See mk/MxApp.mk for examples.)
5. Build mxlib by running make at the top level.
   
6. Install mxlib with make install. sudo may be required depending on the PREFIX. Possibly sudo -E.

Build Environment

Once mxlib is built and installed, set the $MXMAKEFILE variable in your shell (and/or shell profile files):

export MXMAKEFILE=/path/to/mxlib/mk/MxApp.mk

You can then build a single .cpp program, let's say "myProgram.cpp", with the following line

make -B -f $MXMAKEFILE t=myProgram 

To install it, you need to give make a hint about what is to be installed. The command is

make -f $MXMAKEFILE TARGET=myProgram install

This installs the myProgram binary to $PREFIX/bin (which is $HOME/bin unless you have set PREFIX in one of the local/ Makefiles).

For more complicated applications, you can create a local Makefile which specifies additional objects with OTHER_OBJS, and finally includes MxApp.mk.