spack_dolfinx.md

Install Instructions

Clone spack:

git clone https://github.com/spack/spack.git

Selecting Modules

#!/bin/bash

module purge
module load gcc/11
module load rhel8/slurm
module load dot
module load intel/mkl/2020.4
module load intel/impi/2020.4/gnu

# load tools
module load intel/tools/itac/2020.4
module load intel/tools/vtune/2020.4

Using the System's MPI

source ./spack/share/spack/setup-env.sh
spack config edit packages

and save this:

packages:
  intel-mpi:
    externals:
    - spec: [email protected]
      modules:
      - intel/impi/2020.4/gnu
    buildable: False
  intel-mkl:
    externals:
    - spec: [email protected]
      modules:
      - intel/mkl/2020.4
    buildable: False
  all:
    providers:
      mpi: [intel-mpi]
      blas: [intel-mkl]

Spack enviroment

Create new spack enviroment

source ./spack/share/spack/setup-env.sh
spack env create fenicsx-env
spack env activate fenicsx-env
echo "  concretization: together" >> ./spack/var/spack/environments/fenicsx-env/spack.yaml

Install dolfinx on enviroment

Install dolfinx@main

spack add py-fenics-dolfinx@main ^fenics-dolfinx@main ^py-fenics-ffcx@main ^py-fenics-basix@main ^fenics-basix@main ^intel-mpi ^intel-mkl
spack install

Post Build

Make sure to use latest cmake:

spack load cmake

Trying with:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
packages:
  intel-mpi:
    externals:
    - spec: [email protected]
      modules:
      - intel/mpi/32/2020/4.304
    buildable: False
  all:
    providers:
      mpi: [intel-mpi]

spack add py-fenics-dolfinx cflags="-O3" fflags="-O3" ^[email protected] ^[email protected]