Building Airbus solver stack manually

1. About
2. Preparing build environment
3. Building and installing

1. About

The present document keeps the track of the steps to take when compiling the Airbus software stack manually, e. g. without relying on the Guix ¹ package manager.

2. Preparing build environment

We decide to place package sources in $HOME/src and use $HOME/bin as the destination folder for the Airbus packages binaries. By default, we install their dependencies system-wide under /usr/local to simplify subsequent development environment set-up.

mkdir -p $HOME/src && mkdir -p $HOME/bin

Although, for machines where global installation is not an option, we detail also the steps for installing the dependencies in $HOME/bin.

3. Building and installing

The build and installation steps of the Airbus solver framework vary depending on the target Linux distribution and the ability to issue commands as super-user.

3.1. Personal computer running a Debian-based distribution

In this section, we provide the steps for an installation with administrative privileges on a personal computer running a Debian-based Linux distribution.

Some of the required packages can be installed directly using the Aptitude package manager.

sudo apt install git build-essential pkg-config python-dev flex bison zlib1g \
     zlib1g-dev libhwloc5 libhwloc-dev libjemalloc2 libjemalloc-dev wget

3.1.1. Intel(R) libraries and compilers

The linear system solvers we rely on use the Intel(R) Math Kernel Library. Moreover, we build our packages using the Intel(R) MPI implementation as well as their C/C++ and Fortran compilers.

Note that, this section is based on the official tutorial ² and the following description may change independently of the present document.

At first, we need to include Intel(R) repositories into the local Aptitutde configuration.

echo "deb https://apt.repos.intel.com/oneapi all main" | \
  sudo tee /etc/apt/sources.list.d/oneAPI.list

Then, we have to import corresponding GPG keys.

cd /tmp
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
rm GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB

Now, we can update the local package information and install the required Intel(R) packages. Be aware, that the installation takes circa 7 GiB of disk space.

sudo apt update
sudo apt install intel-oneapi-mkl intel-oneapi-mkl-devel \
     intel-oneapi-compiler-fortran \
     intel-oneapi-compiler-dpcpp-cpp-and-cpp-classic intel-oneapi-mpi-devel

Finally, we have to modify the .bashrc in such a way as to always set the environment variable containing the path to the Intel(R) license file. In this case, the file is called NCOM_L___XXXX-XXXXXXXX.lic and is stored under $HOME/src.

echo "export INTEL_LICENSE_FILE=$HOME/src/NCOM_L___XXXX-XXXXXXXX.lic" \
     >> $HOME/.bashrc

Also, to ensure the libraries and compilers are available permanently, it is convenient to use .bashrc to source the shell script provided by Intel(R) for setting up the necessary environment variables such as LD_LIBRARY_PATH and so on.

echo "source /opt/intel/oneapi/setvars.sh" >> $HOME/.bashrc

3.1.2. CMake

The version of CMake available in the Aptitude repositories of Debian is not compatible with the latest Intel(R) MPI. Therefore, we need to download a more recent release, at least the version 3.19.3, of CMake from the official source and install it independently from the package manager. The installation directory is set to /usr/local.

Note that, by the time of writing this tutorial, the latest version of CMake available was 3.19.6.

cd /tmp
wget https://github.com/Kitware/CMake/releases/download/v3.19.6/cmake-3.19.6-Linux-x86_64.sh
chmod +x cmake-3.19.6-Linux-x86_64.sh
sudo ./cmake-3.19.6-Linux-x86_64.sh --prefix=/usr/local/ --exclude-subdir
rm cmake-3.19.6-Linux-x86_64.sh

3.1.3. StarPU

Although the Debian repositories provide the StarPU package, the latter is not build using the required configuration. Therefore, we have to build StarPU from sources by ourselves.

At first, we download the latest version from the official website and extract the sources into $HOME/src/starpu.

cd $HOME/src
mkdir starpu
wget https://files.inria.fr/starpu/starpu-1.3.7/starpu-1.3.7.tar.gz
tar -xvzf starpu-1.3.7.tar.gz -C starpu --strip-components 1
rm starpu-1.3.7.tar.gz

Then, we configure the build from $HOME/src/starpu.

cd $HOME/src/starpu
./configure --enable-fast --enable-shared --disable-static --disable-cuda \
--disable-opencl --enable-maxcpus=48 --enable-maxbuffers=4 \
--disable-build-examples --disable-build-doc --disable-starpu-top \
--disable-gcc-extensions --disable-fortran --disable-mlr

Lastly, we can build and install.

make -j8
sudo make install

3.1.4. ScaLAPACK

For the ScaLAPACK library, we need explicitly the version 2.0.2.

cd $HOME/src
mkdir scalapack
wget http://www.netlib.org/scalapack/scalapack-2.0.2.tgz
tar -xvzf scalapack-2.0.2.tgz -C scalapack --strip-components 1
rm scalapack-2.0.2.tgz

ScaLAPACK uses CMake build system requiring a separate build directory. So, we create a directory named build inside of the source folder of the library prior to configuring the build.

cd $HOME/src/scalapack
mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=ON ..

Finally, we build and install.

make -j8
sudo make install

3.1.5. Scotch

We use the version 6.0.4 of the Scotch library.

cd $HOME/src
mkdir scotch
wget https://gforge.inria.fr/frs/download.php/file/34618/scotch_6.0.4.tar.gz
tar -xvzf scotch_6.0.4.tar.gz -C scotch --strip-components 1
rm scotch_6.0.4.tar.gz

We configure the build using one of the provided and slightly modified Makefile.inc files.

cd $HOME/src/scotch/src
sed 's/mpicc//g;s/-DIDXSIZE64/-fPIC/g' \
    Make.inc/Makefile.inc.x86-64_pc_linux2 > Makefile.inc

Then, we build the library using the make command.

make -j8
make -j8 esmumps

Before installing, we convert produced static library files into a single dynamic library file.

cd $HOME/src/scotch/lib
rm libscotcherrexit.a
gcc -shared -Wl,-z,defs -Wl,--whole-archive lib*.a -Wl,--no-whole-archive -lz \
    -lm -lrt -pthread -o libscotch.so
rm *.a

Finally, we install the library manually.

cd $HOME/src/scotch
sudo cp -a include/*scotch*.h /usr/local/include
sudo cp -a lib/libscotch.so /usr/local/lib

3.1.6. MUMPS

In the case of the MUMPS solver, we need the version 5.1.2.

cd $HOME/src
mkdir mumps
wget http://mumps.enseeiht.fr/MUMPS_5.1.2.tar.gz
tar -xvzf MUMPS_5.1.2.tar.gz -C mumps --strip-components 1
rm MUMPS_5.1.2.tar.gz

Then, we navigate to the source folder

cd $HOME/mumps

and use the Makefile.inc below to configure the build. The file must be placed at $HOME/src/mumps/Makefile.inc.

LPORDDIR = $(topdir)/PORD/lib/
IPORD    = -I$(topdir)/PORD/include/
LPORD    = -L$(LPORDDIR) -lpord
ORDERINGSF  = -Dpord -Dscotch
ORDERINGSC  = $(ORDERINGSF)
LORDERINGS = $(LMETIS) $(LPORD) $(LSCOTCH)
IORDERINGSF = $(ISCOTCH)
IORDERINGSC = $(IMETIS) $(IPORD) $(ISCOTCH)
PLAT    =
LIBEXT  = .a
OUTC    = -o 
OUTF    = -o 
RM = /bin/rm -f
CC = mpiicc
FC = mpiifort
FL = mpiifort
AR = ar vr 
RANLIB  = echo
LAPACK = -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core
SCALAP = -L$(MKLROOT)/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64
INCPAR =
LIBPAR = $(SCALAP) $(LAPACK)
INCSEQ = -I$(topdir)/libseq
LIBSEQ  = $(LAPACK) -L$(topdir)/libseq -lmpiseq
LIBBLAS  = -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread
LIBBLAS += -lmkl_core 
LIBOTHERS = -lpthread
CDEFS   = -DAdd_
OPTF    = -g -fPIC -Ofast -DBLR_MT -DMUMPS_USE_BLAS2 -DGEMMT_AVAILABLE -qopenmp
OPTL    = -O -nofor_main -qopenmp
OPTC    = -g -fPIC -Ofast -DBLR_MT -DMUMPS_USE_BLAS2 -DGEMMT_AVAILABLE -qopenmp
INCS = $(INCPAR)
LIBS = 
LIBSEQNEEDED =

At this point, we can build MUMPS for each type of arithmetic.

make -j8 -e c
make -j8 -e s
make -j8 -e z
make -j8 -e d

Finally, we convert produced static library files into a single dynamic library file

cd $HOME/src/mumps/lib
ifort -g -qopenmp -shared -mkl=parallel -Wl,-z,defs -lscotch -lscalapack \
      -lmpifort -lmpi -Wl,--whole-archive *.a -Wl,--no-whole-archive -o \
      libmumps.so
rm *.a

and perform the installation manually.

cd $HOME/src/mumps
sudo cp lib/*.so /usr/local/lib
sudo cp include/*.h /usr/local/include

3.1.7. PaStiX

We take the latest release of version 5, concretely 5.2.3.

cd $HOME/src
mkdir pastix
wget https://gforge.inria.fr/frs/download.php/file/36212/pastix_5.2.3.tar.bz2
tar -xvf pastix_5.2.3.tar.bz2 -C pastix --strip-components 1
rm pastix_5.2.3.tar.bz2

After extraction, we navigate to the source folder

cd $HOME/pastix/src

and use the config.in below to configure the build. The file must be placed at $HOME/src/pastix/src/config.in.

HOSTARCH    = i686_pc_linux
VERSIONBIT  = _64bit
EXEEXT      =
OBJEXT      = .o
LIBEXT      = .a
CCPROG      = icc -fPIC
CFPROG      = ifort -fPIC
CF90PROG    = ifort
CXXPROG     = icpc
MCFPROG     = mpiifort -std90
CF90CCPOPT  = -fpp
CCFOPT      = -Ofast -fPIC -DMULT_SMX -DFORCE_NO_CUDA
CCFDEB      = -g3
CXXOPT      = -O3
NVCCOPT     = -O3
LKFOPT      =
MKPROG      = make
MPCCPROG    = mpiicc -Wall
MPCXXPROG   = mpiicpc -Wall
CPP         = cpp
ARFLAGS     = ruv
ARPROG      = ar
EXTRALIB    = -lifcore -lm -lrt
CTAGSPROG   = ctags
VERSIONMPI  = _mpi
VERSIONSMP  = _smp
VERSIONSCH  = _static
VERSIONINT  = _int
VERSIONPRC  = _double
VERSIONFLT  = _complex
VERSIONORD  = _scotch
ROOT          = /usr/local
INCLUDEDIR    = ${ROOT}/include
LIBDIR        = ${ROOT}/lib
BINDIR        = ${ROOT}/bin
PYTHON_PREFIX = ${ROOT}
SHARED=1
SOEXT=.so
SHARED_FLAGS =  -shared -Wl,-soname,__SO_NAME__
CCFDEB       := ${CCFDEB} -fPIC
CCFOPT       := ${CCFOPT} -fPIC
CFPROG       := ${CFPROG} -fPIC
VERSIONINT  = _int32
CCTYPES     = -DINTSIZE32
CCTYPESFLT  =
CCTYPESFLT := $(CCTYPESFLT) -DPREC_DOUBLE
VERSIONFLT  = _complex
CCTYPESFLT := $(CCTYPESFLT) -DTYPE_COMPLEX
CCPASTIX   := $(CCPASTIX) -DPASTIX_FUNNELED
CCPASTIX   := $(CCPASTIX) -DCUDA_SM_VERSION=20
NVCCOPT    := $(NVCCOPT) -arch sm_20
CCPASTIX   := $(CCPASTIX) -DMEMORY_USAGE
CCPASTIX   := $(CCPASTIX) -DSTATS_SOPALIN
SCOTCH_HOME ?= /usr/local
SCOTCH_INC ?= $(SCOTCH_HOME)/include
SCOTCH_LIB ?= $(SCOTCH_HOME)/lib
CCPASTIX   := $(CCPASTIX) -I$(SCOTCH_INC) -DWITH_SCOTCH
HWLOC_INC  ?= /usr/include/hwloc/
HWLOC_LIB  ?= /usr/lib/x86_64-linux-gnu/hwloc/
CCPASTIX   := $(CCPASTIX) -I$(HWLOC_INC) -DWITH_HWLOC
EXTRALIB   := $(EXTRALIB) -L$(HWLOC_LIB) -lhwloc
EXTRALIB   := $(EXTRALIB) -lpthread
BLAS_HOME=$(MKLROOT)
BLASLIB =  -L$(BLAS_HOME) -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
FOPT      := $(CCFOPT)
FDEB      := $(CCFDEB)
CCHEAD    := $(CCPROG) $(CCTYPES) $(CCFOPT)
CCFOPT    := $(CCFOPT) $(CCTYPES) $(CCPASTIX)
CCFDEB    := $(CCFDEB) $(CCTYPES) $(CCPASTIX)
NVCCOPT   := $(NVCCOPT) $(CCTYPES) $(CCPASTIX)
MAKE     = $(MKPROG)
CC       = $(MPCCPROG)
CFLAGS   = $(CCFOPT) $(CCTYPESFLT)
FC       = $(MCFPROG)
FFLAGS   = $(CCFOPT)
LDFLAGS  = $(EXTRALIB) $(BLASLIB)
CTAGS    = $(CTAGSPROG)

Finally, we build and install PaStiX. Notice that, we have to enter the super-user's prompt before installing. Using sudo does not work in this case because we need to preserve some environment variables.

make -j8
su
make install

3.1.8. Chameleon

We begin by cloning the Git repository of Chameleon, checking out the required commit and initializing the submodules.

cd $HOME/src
git clone https://gitlab.inria.fr/solverstack/chameleon.git
cd chameleon
git checkout b1d809179e2a8de168996483858ec5438672e082
git submodule update --init

Then, we create a separate build folder in the source directory before calling CMake to configure the build.

mkdir build
cd build
cmake -DCHAMELEON_USE_MPI=ON -DBLA_VENDOR=Intel10_64lp -DBUILD_SHARED_LIBS=ON \
      -DCMAKE_Fortran_COMPILER=ifort -DCMAKE_C_COMPILER=icc ..

Eventually, we build and install.

make -j8
sudo make install

3.1.9. hmat

At the beginning, we clone the corresponding Git repository and initialize the hmat-oss submodule. Note that, the repository is accessible only from the Airbus company network or from Inria VPN.

cd $HOME/src
git clone https://aseris-lab.info/airbus/hmat.git
cd hmat
git submodule update --init

Then, we create a separate build folder in the source directory before calling CMake to configure the build.

mkdir build
cd build
cmake -DHMAT_JEMALLOC=ON -DHMAT_EXPORT_BUILD_DATE=ON \
      -DCMAKE_INSTALL_PREFIX=$HOME/bin/hmat ..

Finally, we build and install into $HOME/bin/hmat.

make -j8
make install

3.1.10. mpf

We are currently developing the mpf package. Therefore, we need to use our development branch of the latter.

cd $HOME/src
git clone https://aseris-lab.info/airbus/mpf.git
cd mpf
git checkout -b mf/devel origin/mf/devel

Then, we create a separate build folder in the source directory before calling CMake to configure the build. Also, we set the build type to Debug in order to preserve debugging symbols in the output binaries.

mkdir build
cd build
cmake -DENABLE_PASTIX:BOOL=ON -DENABLE_CHAMELEON=TRUE \
      -DCMAKE_BUILD_TYPE="Debug" -DCMAKE_INSTALL_PREFIX=$HOME/bin/mpf \
      -DCMAKE_PREFIX_PATH="$HOME/bin/hmat" ..

Finally, we build and install into $HOME/bin/mpf.

make -j8
make install

3.1.11. scab

Just as in the case of mpf, we use our development branch of scab.

cd $HOME/src
git clone https://aseris-lab.info/airbus/scab.git
cd scab
git checkout -b mf/devel origin/mf/devel

Next, we create a separate build folder in the source directory before calling CMake to configure the build and we set the build type to Debug.

mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$HOME/bin/scab -DCMAKE_BUILD_TYPE="Debug" \
      -DCMAKE_PREFIX_PATH="$HOME/bin/hmat;$HOME/bin/mpf" ..

Finally, we build and install into $HOME/bin/scab.

make -j8
make install

3.2. occigen super-computer

In this section, we detail the build and installation process on the occigen super-computer relying a module management system. As the /home file system on the platform accepts only a limited amount of files, we shall put the sources in /scratch.

The RedHat Linux installed on occigen comes pre-installed with nearly all of the essential system tools and libraries except jemalloc required by hmat. Then, the module management system provides other core dependencies of the Airbus solver stack such as the Intel(R) compilers and libraries.

We use the following command to prepare the environment for manual build of the remaining packages.

module load intel/19.4 intelmpi/2019.4.243 mkl/19.4 cmake/3.16.1

Note that, accessing Internet from the cluster's network is very limited. Therefore, we need to download the source tarballs and clone the Git repositories on our personal computer and transfer them to occigen using scp.

Note that, the Airbus Git repositories are accessible only from the company network or from Inria VPN. Also the call to the scp command below may vary depending on the configuration of the computer.

mkdir /tmp/soft
cd /tmp/soft
wget https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2
wget https://files.inria.fr/starpu/starpu-1.3.7/starpu-1.3.7.tar.gz
wget http://www.netlib.org/scalapack/scalapack-2.0.2.tgz
wget https://gforge.inria.fr/frs/download.php/file/34618/scotch_6.0.4.tar.gz
wget http://mumps.enseeiht.fr/MUMPS_5.1.2.tar.gz
wget https://gforge.inria.fr/frs/download.php/file/36212/pastix_5.2.3.tar.bz2
git clone https://gitlab.inria.fr/solverstack/chameleon.git
cd chameleon
git checkout b1d809179e2a8de168996483858ec5438672e082
git submodule update --init
cd ..
git clone https://aseris-lab.info/airbus/hmat.git
cd hmat
git submodule update --init
cd ..
git clone https://aseris-lab.info/airbus/mpf.git
cd mpf
git checkout -b mf/devel origin/mf/devel
cd ..
git clone https://aseris-lab.info/airbus/scab.git
cd scab
git checkout -b mf/devel origin/mf/devel
cd ..
scp -r * occigen:~/src
cd ..
rm -rf soft

After the transfer, we log in to occigen and move the sources from $HOME/src to $SCRATCHDIR/src.

ssh occigen
mv $HOME/src $SCRATCHDIR/src

3.2.1. jemalloc

We need to use the version 4.5.0 of jemalloc.

cd $SCRATCHDIR/src
mkdir jemalloc
tar -xvf jemalloc-4.5.0.tar.bz2 -C jemalloc --strip-components 1
rm jemalloc-4.5.0.tar.bz2

We configure the build from $SCRATCHDIR/src/jemalloc.

cd $SCRATCHDIR/src/jemalloc
./configure --prefix=$HOME/bin/jemalloc

Then, we can build and install into $HOME/bin/jemalloc.

make -j8
make install

3.2.2. StarPU

At first, we extract the sources into $SCRATCHDIR/src/starpu.

cd $SCRATCHDIR/src
mkdir starpu
tar -xvzf starpu-1.3.7.tar.gz -C starpu --strip-components 1
rm starpu-1.3.7.tar.gz

Then, we configure the build from $SCRATCHDIR/src/starpu.

cd $SCRATCHDIR/src/starpu
HWLOC_CFLAGS="-I/opt/software/occigen/tools/hwloc/1.11.0/intel/17.0/include" \
HWLOC_LIBS="-lhwloc" ./configure --enable-fast --enable-shared \
--disable-static --disable-cuda --disable-opencl --enable-maxcpus=48 \
--enable-maxbuffers=4 --disable-build-examples --disable-build-doc \
--disable-starpu-top --disable-gcc-extensions --disable-fortran --disable-mlr \
--prefix=$HOME/bin/starpu

Lastly, we can build and install into $HOME/bin/starpu.

make -j8
make install

3.2.3. ScaLAPACK

For the ScaLAPACK library, we need explicitly the version 2.0.2.

cd $SCRATCHDIR/src
mkdir scalapack
tar -xvzf scalapack-2.0.2.tgz -C scalapack --strip-components 1
rm scalapack-2.0.2.tgz

ScaLAPACK uses CMake build system requiring a separate build directory. So, we create a directory named build inside of the source folder of the library prior to configuring the build.

cd $SCRATCHDIR/src/scalapack
mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=$HOME/bin/scalapack ..

Finally, we build and install into $HOME/bin/scalapack.

make -j8
make install

3.2.4. Scotch

We use the version 6.0.4 of the Scotch library.

cd $SCRATCHDIR/src
mkdir scotch
tar -xvzf scotch_6.0.4.tar.gz -C scotch --strip-components 1
rm scotch_6.0.4.tar.gz

We configure the build using one of the provided and slightly modified Makefile.inc files.

cd $SCRATCHDIR/src/scotch/src
sed 's/mpicc//g;s/-DIDXSIZE64/-fPIC/g' \
    Make.inc/Makefile.inc.x86-64_pc_linux2 > Makefile.inc

Then, we build the library using the make command.

make -j8
make -j8 esmumps

Before installing, we convert produced static library files into a single dynamic library file.

cd $SCRATCHDIR/src/scotch/lib
rm libscotcherrexit.a
gcc -shared -Wl,-z,defs -Wl,--whole-archive lib*.a -Wl,--no-whole-archive -lz \
    -lm -lrt -pthread -o libscotch.so
rm *.a

Finally, we install the library manually into $HOME/bin/scotch.

cd $SCRATCHDIR/src/scotch
mkdir -p $HOME/bin/scotch/include $HOME/bin/scotch/lib
cp -a include/*scotch*.h $HOME/bin/scotch/include
cp -a lib/libscotch.so $HOME/bin/scotch/lib

3.2.5. MUMPS

In the case of the MUMPS solver, we need the version 5.1.2.

cd $SCRATCHDIR/src
mkdir mumps
tar -xvzf MUMPS_5.1.2.tar.gz -C mumps --strip-components 1
rm MUMPS_5.1.2.tar.gz

Then, we navigate to the source folder

cd $SCRATCHDIR/mumps

and use the Makefile.inc below to configure the build. The file must be placed at $SCRATCHDIR/src/mumps/Makefile.inc.

SCOTCHDIR  = ${HOME}/bin/scotch
ISCOTCH    = -I$(SCOTCHDIR)/include
LSCOTCH    = -L$(SCOTCHDIR)/lib -lesmumps -lscotch -lscotcherr
LPORDDIR = $(topdir)/PORD/lib/
IPORD    = -I$(topdir)/PORD/include/
LPORD    = -L$(LPORDDIR) -lpord
ORDERINGSF  = -Dpord -Dscotch
ORDERINGSC  = $(ORDERINGSF)
LORDERINGS = $(LMETIS) $(LPORD) $(LSCOTCH)
IORDERINGSF = $(ISCOTCH)
IORDERINGSC = $(IMETIS) $(IPORD) $(ISCOTCH)
PLAT    =
LIBEXT  = .a
OUTC    = -o 
OUTF    = -o 
RM = /bin/rm -f
CC = mpiicc
FC = mpiifort
FL = mpiifort
AR = ar vr 
RANLIB  = echo
LAPACK = -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core
SCALAP = -L$(MKLROOT)/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64
INCPAR =
LIBPAR = $(SCALAP) $(LAPACK)
INCSEQ = -I$(topdir)/libseq
LIBSEQ  = $(LAPACK) -L$(topdir)/libseq -lmpiseq
LIBBLAS  = -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread
LIBBLAS += -lmkl_core 
LIBOTHERS = -lpthread
CDEFS   = -DAdd_
OPTF    = -g -fPIC -Ofast -DBLR_MT -DMUMPS_USE_BLAS2 -DGEMMT_AVAILABLE -qopenmp
OPTL    = -O -nofor_main -qopenmp
OPTC    = -g -fPIC -Ofast -DBLR_MT -DMUMPS_USE_BLAS2 -DGEMMT_AVAILABLE -qopenmp
INCS = $(INCPAR)
LIBS = 
LIBSEQNEEDED =

At this point, we can build MUMPS for each types of arithmetic.

make -j8 -e c
make -j8 -e s
make -j8 -e z
make -j8 -e d

Finally, we convert produced static library files into a single dynamic library file

cd $SCRATCHDIR/src/mumps/lib
ifort -g -qopenmp -shared -mkl=parallel -Wl,-z,defs -I$HOME/bin/scotch/include \
      -L$HOME/bin/scotch/lib -lscotch -I$HOME/bin/scalapack/include \
      -L$HOME/bin/scalapack/lib -lscalapack -lmpifort -lmpi \
      -Wl,--whole-archive *.a -Wl,--no-whole-archive -o libmumps.so
rm *.a

perform the installation manually into $HOME/bin/mumps.

cd $SCRATCHDIR/src/mumps
mkdir -p $HOME/bin/mumps/include $HOME/bin/mumps/lib
cp -a lib/*.so $HOME/bin/mumps/lib
cp -a include/*.h $HOME/bin/mumps/include

3.2.6. PaStiX

We take the latest release of version 5, concretely 5.2.3.

cd $SCRATCHDIR/src
mkdir pastix
tar -xvf pastix_5.2.3.tar.bz2 -C pastix --strip-components 1
rm pastix_5.2.3.tar.bz2

After extraction, we navigate to the source folder

cd $SCRATCHDIR/pastix/src

and use the config.in below to configure the build. The file must be placed at $SCRATCHDIR/src/pastix/src/config.in.

HOSTARCH    = i686_pc_linux
VERSIONBIT  = _64bit
EXEEXT      =
OBJEXT      = .o
LIBEXT      = .a
CCPROG      = icc -fPIC
CFPROG      = ifort -fPIC
CF90PROG    = ifort
CXXPROG     = icpc
MCFPROG     = mpiifort -std90
CF90CCPOPT  = -fpp
CCFOPT      = -Ofast -fPIC -DMULT_SMX -DFORCE_NO_CUDA
CCFDEB      = -g3
CXXOPT      = -O3
NVCCOPT     = -O3
LKFOPT      =
MKPROG      = make
MPCCPROG    = mpiicc -Wall
MPCXXPROG   = mpiicpc -Wall
CPP         = cpp
ARFLAGS     = ruv
ARPROG      = ar
EXTRALIB    = -lifcore -lm -lrt
CTAGSPROG   = ctags
VERSIONMPI  = _mpi
VERSIONSMP  = _smp
VERSIONSCH  = _static
VERSIONINT  = _int
VERSIONPRC  = _double
VERSIONFLT  = _complex
VERSIONORD  = _scotch
ROOT          = ${HOME}/bin/pastix
INCLUDEDIR    = ${ROOT}/include
LIBDIR        = ${ROOT}/lib
BINDIR        = ${ROOT}/bin
PYTHON_PREFIX = ${ROOT}
SHARED=1
SOEXT=.so
SHARED_FLAGS =  -shared -Wl,-soname,__SO_NAME__
CCFDEB       := ${CCFDEB} -fPIC
CCFOPT       := ${CCFOPT} -fPIC
CFPROG       := ${CFPROG} -fPIC
VERSIONINT  = _int32
CCTYPES     = -DINTSIZE32
CCTYPESFLT  =
CCTYPESFLT := $(CCTYPESFLT) -DPREC_DOUBLE
VERSIONFLT  = _complex
CCTYPESFLT := $(CCTYPESFLT) -DTYPE_COMPLEX
CCPASTIX   := $(CCPASTIX) -DPASTIX_FUNNELED
CCPASTIX   := $(CCPASTIX) -DCUDA_SM_VERSION=20
NVCCOPT    := $(NVCCOPT) -arch sm_20
CCPASTIX   := $(CCPASTIX) -DMEMORY_USAGE
CCPASTIX   := $(CCPASTIX) -DSTATS_SOPALIN
SCOTCH_HOME ?= ${HOME}/bin/scotch
SCOTCH_INC ?= $(SCOTCH_HOME)/include
SCOTCH_LIB ?= $(SCOTCH_HOME)/lib
CCPASTIX   := $(CCPASTIX) -I$(SCOTCH_INC) -DWITH_SCOTCH
HWLOC_INC  ?= /opt/software/occigen/tools/hwloc/1.11.0/intel/17.0/include/hwloc
HWLOC_LIB  ?= /opt/software/occigen/tools/hwloc/1.11.0/intel/17.0/lib
CCPASTIX   := $(CCPASTIX) -I$(HWLOC_INC) -DWITH_HWLOC
EXTRALIB   := $(EXTRALIB) -L$(HWLOC_LIB) -lhwloc
EXTRALIB   := $(EXTRALIB) -lpthread
BLAS_HOME=$(MKLROOT)
BLASLIB =  -L$(BLAS_HOME) -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
FOPT      := $(CCFOPT)
FDEB      := $(CCFDEB)
CCHEAD    := $(CCPROG) $(CCTYPES) $(CCFOPT)
CCFOPT    := $(CCFOPT) $(CCTYPES) $(CCPASTIX)
CCFDEB    := $(CCFDEB) $(CCTYPES) $(CCPASTIX)
NVCCOPT   := $(NVCCOPT) $(CCTYPES) $(CCPASTIX)
MAKE     = $(MKPROG)
CC       = $(MPCCPROG)
CFLAGS   = $(CCFOPT) $(CCTYPESFLT)
FC       = $(MCFPROG)
FFLAGS   = $(CCFOPT)
LDFLAGS  = $(EXTRALIB) $(BLASLIB)
CTAGS    = $(CTAGSPROG)

Finally, we build and install PaStiX into $HOME/bin/pastix.

make -j8
make install

3.2.7. Chameleon

We create a separate build folder in the source directory before calling CMake to configure the build.

cd $SCRATCHDIR/src/chameleon
mkdir build
cd build
cmake -DCHAMELEON_USE_MPI=ON -DBLA_VENDOR=Intel10_64lp -DBUILD_SHARED_LIBS=ON \
      -DCMAKE_Fortran_COMPILER=ifort -DCMAKE_C_COMPILER=icc \
      -DCMAKE_PREFIX_PATH="$HOME/bin/starpu" \
      -DCMAKE_INSTALL_PREFIX=$HOME/bin/chameleon ..

Eventually, we build and install.

make -j8
make install

3.2.8. hmat

We create a separate build folder in the source directory before calling CMake to configure the build.

cd $SCRATCHDIR/src/hmat
mkdir build
cd build
cmake -DHMAT_JEMALLOC=ON -DHMAT_EXPORT_BUILD_DATE=ON \
      -DCMAKE_INSTALL_PREFIX=$HOME/bin/hmat \
      -DCMAKE_PREFIX_PATH="$HOME/bin/jemalloc;$HOME/bin/starpu" ..

Then, we build and install into $HOME/bin/hmat.

make -j8
make install

3.2.9. mpf

We create a separate build folder in the source directory before calling CMake to configure the build. Also, we set the build type to Debug in order to preserve debugging symbols in the output binaries.

cd $SCRATCHDIR/src/mpf
mkdir build
cd build
cmake -DENABLE_PASTIX:BOOL=ON -DENABLE_CHAMELEON=TRUE \
      -DCMAKE_BUILD_TYPE="Debug" -DCMAKE_INSTALL_PREFIX=$HOME/bin/mpf \
      -DCMAKE_PREFIX_PATH="$HOME/bin/hmat;$HOME/bin/scalapack;$HOME/bin/mumps;$HOME/bin/scotch;$HOME/bin/pastix;$HOME/bin/chameleon" \
      ..

Finally, we build and install into $HOME/bin/mpf.

make -j8
make install

3.2.10. scab

We create a separate build folder in the source directory before calling CMake to configure the build and we set the build type to Debug.

cd $SCRATCHDIR/src/scab
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$HOME/bin/scab -DCMAKE_BUILD_TYPE="Debug" \
      -DCMAKE_PREFIX_PATH="$HOME/bin/hmat;$HOME/bin/mpf;$HOME/bin/chameleon" ..

Then, we build and install into $HOME/bin/scab.

make -j8
make install

3.2.11. Final touch

At the end of the process, we add the following line to our .bashrc on occigen in order to make the dependency libraries available in the environment permanently.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/bin/scotch/lib:$HOME/bin/mumps/lib:$HOME/bin/pastix/lib:$HOME/bin/scalapack/lib

3.3. PlaFRIM platform

In this section, we detail the build and installation process on the PlaFRIM testbed using a module management system.

The CentOS Linux installed on PlaFRIM comes pre-installed with nearly all of the essential system tools and libraries except jemalloc required by hmat. Then, the module management system provides other core dependencies of the Airbus solver stack.

We use the following command to prepare the environment for manual build of the remaining packages.

module load build/cmake/3.15.3 mpi/openmpi/4.1.1 linalg/mkl/2020_update4 \
       compiler/gcc/10.3.0 hardware/hwloc/2.5.0

We can acquire most of the source tarballs directly from the platform by using the wget command.

mkdir -p ~/src
cd ~/src
wget https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2
wget https://files.inria.fr/starpu/starpu-1.3.7/starpu-1.3.7.tar.gz
wget https://gforge.inria.fr/frs/download.php/file/34618/scotch_6.0.4.tar.gz
wget https://gforge.inria.fr/frs/download.php/file/36212/pastix_5.2.3.tar.bz2
git clone https://gitlab.inria.fr/solverstack/chameleon.git
cd chameleon
git checkout b1d809179e2a8de168996483858ec5438672e082
git submodule update --init

Given that PlaFRIM can not access some public servers nor Airbus Git repositories directly for security reasons, we have to clone them first on an another computer having access to the VPN of Inria and transfer the clones to PlaFRIM afterwards.

The scp command may vary depending on the SSH configuration of the computer used to acquire the Airbus sources.

cd /tmp
wget http://www.netlib.org/scalapack/scalapack-2.0.2.tgz
wget http://mumps.enseeiht.fr/MUMPS_5.1.2.tar.gz
git clone https://aseris-lab.info/airbus/hmat.git
cd hmat
git submodule update --init
cd ..
git clone https://aseris-lab.info/airbus/mpf.git
cd mpf
git checkout -b mf/devel origin/mf/devel
cd ..
git clone https://aseris-lab.info/airbus/scab.git
cd scab
git checkout -b mf/devel origin/mf/devel
cd ..
scp -r hmat mpf scab scalapack-2.0.2.tgz MUMPS_5.1.2.tar.gz plafrim:~/src
rm -rf hmat mpf scab

3.3.1. jemalloc

We need to use the version 4.5.0 of jemalloc.

cd ~/src
mkdir jemalloc
tar -xvf jemalloc-4.5.0.tar.bz2 -C jemalloc --strip-components 1
rm jemalloc-4.5.0.tar.bz2

We configure the build from $HOME/src/jemalloc.

cd ~/src/jemalloc
./configure --prefix=$HOME/bin/jemalloc

Then, we can build and install into $HOME/bin/jemalloc.

make -j8
make install

3.3.2. StarPU

At first, we extract the sources into $HOME/src/starpu.

cd $HOME/src
mkdir starpu
tar -xvzf starpu-1.3.7.tar.gz -C starpu --strip-components 1
rm starpu-1.3.7.tar.gz

Then, we configure the build from $HOME/src/starpu.

cd $HOME/src/starpu
./configure --enable-fast --enable-shared --disable-static --disable-cuda \
            --disable-opencl --enable-maxcpus=96 --enable-maxbuffers=4 \
            --disable-build-examples --disable-build-doc --disable-starpu-top \
            --disable-gcc-extensions --disable-fortran --disable-mlr \
            --prefix=$HOME/bin/starpu

Lastly, we can build and install into $HOME/bin/starpu.

make -j8
make install

3.3.3. ScaLAPACK

For the ScaLAPACK library, we need explicitly the version 2.0.2.

cd $HOME/src
mkdir scalapack
tar -xvzf scalapack-2.0.2.tgz -C scalapack --strip-components 1
rm scalapack-2.0.2.tgz

ScaLAPACK uses CMake build system requiring a separate build directory. So, we create a directory named build inside of the source folder of the library prior to configuring the build.

cd $HOME/src/scalapack
mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=$HOME/bin/scalapack ..

Finally, we build and install into $HOME/bin/scalapack.

make -j8
make install

3.3.4. Scotch

We use the version 6.0.4 of the Scotch library.

cd $HOME/src
mkdir scotch
tar -xvzf scotch_6.0.4.tar.gz -C scotch --strip-components 1
rm scotch_6.0.4.tar.gz

We configure the build using one of the provided and slightly modified Makefile.inc files.

cd $HOME/src/scotch/src
sed 's/mpicc//g;s/-DIDXSIZE64/-fPIC/g' \
    Make.inc/Makefile.inc.x86-64_pc_linux2 > Makefile.inc

Then, we build the library using the make command.

make -j8
make -j8 esmumps

Finally, we install the library manually into $HOME/bin/scotch.

cd $HOME/src/scotch
mkdir -p $HOME/bin/scotch/include $HOME/bin/scotch/lib
cp -a include/*scotch*.h include/esmumps.h $HOME/bin/scotch/include
cp -a lib/*.a $HOME/bin/scotch/lib

3.3.5. MUMPS

In the case of the MUMPS solver, we need the version 5.1.2.

cd $HOME/src
mkdir mumps
tar -xvzf MUMPS_5.1.2.tar.gz -C mumps --strip-components 1
rm MUMPS_5.1.2.tar.gz

Then, we navigate to the source folder

cd $HOME/src/mumps

and use the Makefile.inc below to configure the build. The file must be placed at $HOME/src/mumps/Makefile.inc.

SCOTCHDIR  = ${HOME}/bin/scotch
ISCOTCH    = -I$(SCOTCHDIR)/include
LSCOTCH    = -L$(SCOTCHDIR)/lib -lesmumps -lscotch -lscotcherr
LPORDDIR = $(topdir)/PORD/lib/
IPORD    = -I$(topdir)/PORD/include/
LPORD    = -L$(LPORDDIR) -lpord
ORDERINGSF  = -Dpord -Dscotch
ORDERINGSC  = $(ORDERINGSF)
LORDERINGS = $(LMETIS) $(LPORD) $(LSCOTCH)
IORDERINGSF = $(ISCOTCH)
IORDERINGSC = $(IMETIS) $(IPORD) $(ISCOTCH)
PLAT    =
LIBEXT  = .a
OUTC    = -o 
OUTF    = -o 
RM = /bin/rm -f
CC = mpicc
FC = mpifort
FL = mpifort
AR = ar vr 
RANLIB  = echo
LAPACK = -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core
SCALAP = -L$(MKLROOT)/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64
INCPAR = -I/cm/shared/modules/generic/mpi/openmpi/4.1.1-psm2/include
LIBPAR = $(SCALAP) $(LAPACK)
INCSEQ = -I$(topdir)/libseq
LIBSEQ  = $(LAPACK) -L$(topdir)/libseq -lmpiseq
LIBBLAS  = -L$(MKLROOT)/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread
LIBBLAS += -lmkl_core 
LIBOTHERS = -lpthread
CDEFS   = -DAdd_
OPTF    = -g -fPIC -Ofast -DBLR_MT -DMUMPS_USE_BLAS2 -DGEMMT_AVAILABLE -fopenmp
OPTL    = -O -nofor_main -fopenmp
OPTC    = -g -fPIC -Ofast -DBLR_MT -DMUMPS_USE_BLAS2 -DGEMMT_AVAILABLE -fopenmp
INCS = $(INCPAR)
LIBS = 
LIBSEQNEEDED =

At this point, we can build MUMPS for each types of arithmetic.

make -j8 -e c
make -j8 -e s
make -j8 -e z
make -j8 -e d

Finally, we perform the installation manually into $HOME/bin/mumps.

cd $HOME/src/mumps
mkdir -p $HOME/bin/mumps/include $HOME/bin/mumps/lib
cp -a lib/*.a $HOME/bin/mumps/lib
cp -a include/*.h $HOME/bin/mumps/include

3.3.6. PaStiX

We take the latest release of version 5, concretely 5.2.3.

cd $HOME/src
mkdir pastix
tar -xvf pastix_5.2.3.tar.bz2 -C pastix --strip-components 1
rm pastix_5.2.3.tar.bz2

After extraction, we navigate to the source folder

cd $HOME/src/pastix/src

and use the config.in below to configure the build. The file must be placed at $HOME/src/pastix/src/config.in.

HOSTARCH    = i686_pc_linux
VERSIONBIT  = _64bit
EXEEXT      =
OBJEXT      = .o
LIBEXT      = .a
CCPROG      = gcc -fPIC
CFPROG      = gfortran -fPIC
CF90PROG    = gfortran
CXXPROG     = icpc
MCFPROG     = mpifort -std=f95
CF90CCPOPT  = -fpp
CCFOPT      = -Ofast -fPIC -DMULT_SMX -DFORCE_NO_CUDA
CCFDEB      = -g3
CXXOPT      = -O3
NVCCOPT     = -O3
LKFOPT      =
MKPROG      = make
MPCCPROG    = mpicc -Wall
MPCXXPROG   = mpic++ -Wall
CPP         = cpp
ARFLAGS     = ruv
ARPROG      = ar
EXTRALIB    = -lifcore -lm -lrt
CTAGSPROG   = ctags
VERSIONMPI  = _mpi
VERSIONSMP  = _smp
VERSIONSCH  = _static
VERSIONINT  = _int
VERSIONPRC  = _double
VERSIONFLT  = _complex
VERSIONORD  = _scotch
ROOT          = ${HOME}/bin/pastix
INCLUDEDIR    = ${ROOT}/include
LIBDIR        = ${ROOT}/lib
BINDIR        = ${ROOT}/bin
PYTHON_PREFIX = ${ROOT}
SHARED=1
SOEXT=.so
SHARED_FLAGS =  -shared -Wl,-soname,__SO_NAME__
CCFDEB       := ${CCFDEB} -fPIC
CCFOPT       := ${CCFOPT} -fPIC
CFPROG       := ${CFPROG} -fPIC
VERSIONINT  = _int32
CCTYPES     = -DINTSIZE32
CCTYPESFLT  =
CCTYPESFLT := $(CCTYPESFLT) -DPREC_DOUBLE
VERSIONFLT  = _complex
CCTYPESFLT := $(CCTYPESFLT) -DTYPE_COMPLEX
CCPASTIX   := $(CCPASTIX) -DPASTIX_FUNNELED
CCPASTIX   := $(CCPASTIX) -DCUDA_SM_VERSION=20
NVCCOPT    := $(NVCCOPT) -arch sm_20
CCPASTIX   := $(CCPASTIX) -DMEMORY_USAGE
CCPASTIX   := $(CCPASTIX) -DSTATS_SOPALIN
SCOTCH_HOME ?= ${HOME}/bin/scotch
SCOTCH_INC ?= $(SCOTCH_HOME)/include
SCOTCH_LIB ?= $(SCOTCH_HOME)/lib
CCPASTIX   := $(CCPASTIX) -I$(SCOTCH_INC) -DWITH_SCOTCH
HWLOC_INC  ?= /cm/shared/dev/modules/generic/apps/hardware/hwloc/hwloc-2.5.0/install/include/hwloc
HWLOC_LIB  ?= /cm/shared/dev/modules/generic/apps/hardware/hwloc/hwloc-2.5.0/install/lib
CCPASTIX   := $(CCPASTIX) -I$(HWLOC_INC) -DWITH_HWLOC
EXTRALIB   := $(EXTRALIB) -L$(HWLOC_LIB) -lhwloc
EXTRALIB   := $(EXTRALIB) -lpthread
BLAS_HOME=$(MKLROOT)
BLASLIB =  -L$(BLAS_HOME) -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
FOPT      := $(CCFOPT)
FDEB      := $(CCFDEB)
CCHEAD    := $(CCPROG) $(CCTYPES) $(CCFOPT)
CCFOPT    := $(CCFOPT) $(CCTYPES) $(CCPASTIX)
CCFDEB    := $(CCFDEB) $(CCTYPES) $(CCPASTIX)
NVCCOPT   := $(NVCCOPT) $(CCTYPES) $(CCPASTIX)
MAKE     = $(MKPROG)
CC       = $(MPCCPROG)
CFLAGS   = $(CCFOPT) $(CCTYPESFLT)
FC       = $(MCFPROG)
FFLAGS   = $(CCFOPT)
LDFLAGS  = $(EXTRALIB) $(BLASLIB)
CTAGS    = $(CTAGSPROG)

Finally, we build and install PaStiX into $HOME/bin/pastix.

make -j8
make install

3.3.7. Chameleon

We create a separate build folder in the source directory before calling CMake to configure the build.

cd $HOME/src/chameleon
mkdir build
cd build
cmake -DCHAMELEON_USE_MPI=ON -DBLA_VENDOR=Intel10_64lp -DBUILD_SHARED_LIBS=ON \
      -DCMAKE_PREFIX_PATH="$HOME/bin/starpu" \
      -DCMAKE_INSTALL_PREFIX=$HOME/bin/chameleon ..

Eventually, we build and install.

make -j8
make install

3.3.8. hmat

We create a separate build folder in the source directory before calling CMake to configure the build.

cd $HOME/src/hmat
mkdir build
cd build
cmake -DHMAT_JEMALLOC=ON -DHMAT_EXPORT_BUILD_DATE=ON \
      -DCMAKE_INSTALL_PREFIX=$HOME/bin/hmat \
      -DCMAKE_PREFIX_PATH="$HOME/bin/jemalloc;$HOME/bin/starpu" ..

Then, we build and install into $HOME/bin/hmat.

make -j8
make install

3.3.9. mpf

cd $HOME/src/mpf
mkdir build
cd build
cmake -DENABLE_PASTIX:BOOL=ON -DENABLE_CHAMELEON=TRUE \
      -DCMAKE_BUILD_TYPE="Debug" -DCMAKE_INSTALL_PREFIX=$HOME/bin/mpf \
      -DCMAKE_PREFIX_PATH="$HOME/bin/hmat;$HOME/bin/scalapack;$HOME/bin/mumps;$HOME/bin/scotch;$HOME/bin/pastix;$HOME/bin/chameleon" \
      -DSCOTCH_LIBRARIES="-L$HOME/bin/scotch/lib -lesmumps -lscotch -lscotcherr" \
      -DSCALAPACK_LIBRARIES="-L$HOME/bin/scalapack/lib -lscalapack"
      ..

Finally, we build and install into $HOME/bin/mpf.

make -j8
make install

3.3.10. scab

We create a separate build folder in the source directory before calling CMake to configure the build and we set the build type to Debug.

cd $HOME/src/scab
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$HOME/bin/scab -DCMAKE_BUILD_TYPE="Debug" \
      -DCMAKE_PREFIX_PATH="$HOME/bin/hmat;$HOME/bin/mpf;$HOME/bin/chameleon" ..

Then, we build and install into $HOME/bin/scab.

make -j8
make install

3.3.11. Final touch

At the end of the process, we add the following line to our .bashrc on occigen in order to make the dependency libraries available in the environment permanently.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/bin/scotch/lib:$HOME/bin/mumps/lib:$HOME/bin/pastix/lib:$HOME/bin/scalapack/lib

Footnotes:

https://guix.gnu.org

https://software.intel.com/content/www/us/en/develop/articles/installing-intel-oneapi-toolkits-via-apt.html

Building Airbus solver stack manually

Table of Contents

1. About

2. Preparing build environment

3. Building and installing

3.1. Personal computer running a Debian-based distribution

3.1.1. Intel(R) libraries and compilers

3.1.2. CMake

3.1.3. StarPU

3.1.4. ScaLAPACK

3.1.5. Scotch

3.1.6. MUMPS

3.1.7. PaStiX

3.1.8. Chameleon

3.1.9. hmat

3.1.10. mpf

3.1.11. scab

3.2. occigen super-computer

3.2.1. jemalloc

3.2.2. StarPU

3.2.3. ScaLAPACK

3.2.4. Scotch

3.2.5. MUMPS

3.2.6. PaStiX

3.2.7. Chameleon

3.2.8. hmat

3.2.9. mpf

3.2.10. scab

3.2.11. Final touch

3.3. PlaFRIM platform

3.3.1. jemalloc

3.3.2. StarPU

3.3.3. ScaLAPACK

3.3.4. Scotch

3.3.5. MUMPS

3.3.6. PaStiX

3.3.7. Chameleon

3.3.8. hmat

3.3.9. mpf

3.3.10. scab

3.3.11. Final touch

Footnotes: