Installation Methods - Dedalus Conda

Overview

There are two primary approaches to installing Dedalus with Conda:

conda-forge packages: Pre-built binaries for standard configurations
Custom build scripts: Flexible builds with system-specific library linking

This guide helps you choose the right method for your use case.

conda-forge Installation

What It Provides

The Dedalus feedstock on conda-forge provides pre-built packages with:

Dedalus v2 (latest stable release)
OpenMPI support from conda-forge
Parallel HDF5 built against OpenMPI
FFTW from conda-forge
NumPy/SciPy with OpenBLAS

Supported Platforms

Linux (x86_64)
macOS (x86_64)

The conda-forge builds support both MPICH and OpenMPI variants. The default installation uses OpenMPI.

Installation Command

conda create -y -n dedalus2
conda activate dedalus2
conda install -y -c conda-forge dedalus

When to Use conda-forge

Choose conda-forge installation when:

Quick Setup

You need the fastest path to a working installation without configuration

Standard Hardware

You’re on a laptop or workstation with no special library requirements

Development Work

You’re developing Dedalus-based applications and need a stable baseline

Teaching

You’re setting up environments for students or workshops

Limitations

conda-forge builds cannot link to custom system libraries. If your cluster requires specific MPI, FFTW, or HDF5 installations, use the custom build scripts instead.

Custom Build Scripts

What They Provide

The custom build scripts (conda_install_dedalus2.sh and conda_install_dedalus3.sh) offer:

Custom MPI linking: Use system-installed MPI (MPICH, Intel MPI, Cray MPICH, etc.)
Custom FFTW: Link to hardware-optimized FFTW builds
Custom HDF5: Use cluster-specific parallel HDF5 installations
BLAS selection: Choose between OpenBLAS or Intel MKL
Development versions: Install Dedalus v3 from the master branch

Configuration Options

The scripts expose configuration variables at the top of each file:

# Conda environment name
CONDA_ENV="dedalus2"

# Install OpenMPI from conda (1) or use custom MPI (0)
INSTALL_MPI=1
# export MPI_PATH=/path/to/custom/mpi

# Install FFTW from conda (1) or use custom FFTW (0)
INSTALL_FFTW=1
# export FFTW_PATH=/path/to/custom/fftw

# Install HDF5 from conda (1) or use custom HDF5 (0)
INSTALL_HDF5=1
# export HDF5_DIR=/path/to/custom/hdf5
# export HDF5_MPI="ON"

# BLAS options: "openblas" or "mkl"
BLAS="openblas"

# Python version
PYTHON_VERSION="3.12"

When to Use Custom Builds

Choose custom build scripts when:

HPC Clusters

You’re on a supercomputer with optimized system libraries

Custom MPI

You need to use Intel MPI, Cray MPICH, or other MPI implementations

Performance Tuning

You want to use Intel MKL or hardware-specific FFTW builds

Development

You’re testing Dedalus v3 or contributing to Dedalus development

Comparison Matrix

Feature	conda-forge	Custom Build Scripts
Installation Speed	Fast (pre-built)	Slower (builds Dedalus)
Configuration	None required	Extensive options
MPI Support	OpenMPI only	Any MPI implementation
FFTW	conda-forge version	System or conda-forge
HDF5	conda-forge parallel	System or conda-forge
BLAS	OpenBLAS	OpenBLAS or Intel MKL
Dedalus Versions	v2 stable only	v2 stable or v3 development
Platform Support	Linux, macOS x86	Linux, macOS x86, macOS ARM
Cluster Optimization	Limited	Full control

Detailed Configuration Examples

Example 1: Full conda-forge Stack

The simplest configuration uses all conda-forge packages:

conda_install_dedalus2.sh

INSTALL_MPI=1
INSTALL_FFTW=1
INSTALL_HDF5=1
BLAS="openblas"

This is functionally similar to the conda-forge installation but gives you control over Python version and BLAS library.

Example 2: Custom MPI with conda-forge FFTW/HDF5

On clusters with optimized MPI but standard FFTW/HDF5:

conda_install_dedalus2.sh

INSTALL_MPI=0
export MPI_PATH=/opt/intel/mpi/latest

INSTALL_FFTW=1
INSTALL_HDF5=1
BLAS="openblas"

When using custom MPI, mpicc must be in your PATH or explicitly set via MPI_PATH.

Example 3: Full Custom Stack (HPC)

On HPC systems with all libraries provided by modules:

conda_install_dedalus2.sh

INSTALL_MPI=0
export MPI_PATH=/opt/cray/pe/mpt/default

INSTALL_FFTW=0
export FFTW_PATH=/opt/fftw/3.3.10

INSTALL_HDF5=0
export HDF5_DIR=/opt/hdf5/1.14.0
export HDF5_MPI="ON"

BLAS="mkl"

This configuration:

Uses Cray MPI from the system
Links to hardware-optimized FFTW
Uses system parallel HDF5
Selects Intel MKL for linear algebra

When using custom libraries, ensure they are compatible with each other. For example, parallel HDF5 must be built against the same MPI implementation you’re using.

Example 4: Intel MKL with OpenMPI

For workstations with Intel processors:

conda_install_dedalus2.sh

INSTALL_MPI=1
INSTALL_FFTW=1
INSTALL_HDF5=1
BLAS="mkl"

This uses Intel MKL for optimized performance on Intel CPUs while keeping other dependencies from conda-forge.

When using MKL, the build script automatically sets FFTW_STATIC=1 to statically link FFTW and avoid symbol conflicts with MKL.

Installation Workflow Details

Script Execution Flow

The custom build scripts follow this process:

Validation

Check that the conda base environment is active and required paths are set for custom libraries

Environment Creation

Create a new conda environment with the specified name. On Apple Silicon, configure x86_64 emulation if needed.

Channel Configuration

Set conda-forge as the primary channel with strict priority:

conda config --add channels conda-forge
conda config --set channel_priority strict

Base Python Stack

Install Python, pip, wheel, setuptools, and Cython from conda-forge

BLAS and Numerical Libraries

Install NumPy and SciPy with selected BLAS implementation:

# For OpenBLAS
conda install "libblas=*=*openblas" numpy scipy

# For Intel MKL
conda install "libblas=*=*mkl" numpy scipy

MPI Installation

Either install OpenMPI from conda-forge or build mpi4py against custom MPI:

# conda-forge OpenMPI
conda install compilers openmpi openmpi-mpicc mpi4py

# Custom MPI
export PATH=${MPI_PATH}/bin:${PATH}
pip install --no-cache mpi4py

FFTW Installation

Install FFTW from conda-forge or use system FFTW via environment variable

HDF5 and h5py

Install parallel or serial HDF5 based on MPI configuration:

# Parallel (with conda MPI)
conda install "hdf5=*=mpi*" "h5py=*=mpi*"

# Serial
conda install "hdf5=*=nompi*" "h5py=*=nompi*"

# Custom parallel HDF5
CC=mpicc pip install --no-cache --no-binary=h5py h5py

Additional Dependencies

Install remaining Python packages:

conda install docopt matplotlib

Dedalus Installation

Build and install Dedalus with MPI compiler:

# Dedalus v2 (from PyPI)
CC=mpicc pip install --no-cache --no-build-isolation "dedalus==2.*"

# Dedalus v3 (from GitHub master)
CC=mpicc pip install --no-cache --no-build-isolation \
  http://github.com/dedalusproject/dedalus/zipball/master/

Environment Configuration

Set threading variables to prevent oversubscription:

conda env config vars set OMP_NUM_THREADS=1
conda env config vars set NUMEXPR_MAX_THREADS=1

Advanced Scenarios

Installing Development Versions via conda-forge

You can use the conda-forge base and install development Dedalus:

# Create environment with dependencies
conda create -y -n dedalus2
conda activate dedalus2
conda install -y -c conda-forge dedalus c-compiler cython "h5py=*=mpi*" setuptools

# Remove conda-forge Dedalus and install from GitHub
conda uninstall -y --force dedalus
CC=mpicc pip3 install --no-cache --no-build-isolation \
  http://github.com/dedalusproject/dedalus/zipball/v2_master/

This approach is used in the GitHub Actions test workflows.

Testing Both Installation Methods

The repository includes GitHub Actions workflows that test both methods nightly:

.github/workflows/test_conda_builds.yml

jobs:
  v2_from_conda_forge:
    # Tests pure conda-forge installation
    
  v2_custom:
    # Tests custom build script
    
  v2_master_via_conda_forge:
    # Tests development version on conda-forge base
    
  v3_master_via_conda_forge:
    # Tests Dedalus v3 development

These workflows ensure both installation methods remain functional across Ubuntu and macOS.

Troubleshooting Installation Issues

Library Incompatibility

If you encounter errors about incompatible libraries:

# Check MPI consistency
which mpicc
ldd $(which mpicc)

# Check HDF5 was built with parallel support
h5pcc -showconfig | grep -i parallel

# Verify mpi4py uses correct MPI
python3 -c "import mpi4py; print(mpi4py.get_config())"

FFTW Linking Issues

If FFTW is not found during Dedalus installation:

# Verify FFTW_PATH points to correct location
ls ${FFTW_PATH}/lib/libfftw3*

# Check for both single and double precision
ls ${FFTW_PATH}/lib/libfftw3.* 
ls ${FFTW_PATH}/lib/libfftw3f.*

Environment Conflicts

Ensure clean Python environment:

# These should be unset or empty
echo $PYTHONPATH
echo $PYTHONNOUSERSITE

# Check no ~/.local packages interfere
python3 -c "import sys; print(sys.path)"

The build scripts automatically set:

export PYTHONNOUSERSITE=1  # Ignore ~/.local
unset PYTHONPATH            # Clear Python path

Performance Considerations

BLAS Selection

OpenBLAS:

Open source and widely compatible
Good performance on most hardware
Default choice for portability

Intel MKL:

Optimized for Intel processors
Better performance on Intel CPUs
May require static FFTW linking

Benchmark your specific workload to determine which BLAS provides better performance. The difference varies by problem size and hardware.

MPI Implementation

Different MPI implementations have different performance characteristics:

OpenMPI: Good default choice, wide hardware support
MPICH: Alternative with different communication strategies
Intel MPI: Optimized for Intel fabrics and processors
Cray MPICH: Optimized for Cray supercomputers

Use the MPI implementation recommended by your HPC center for optimal interconnect performance.

Summary

conda-forge

Best for: Laptops, workstations, quick setupInstallation: conda install -c conda-forge dedalusTime: < 5 minutes

Custom Scripts

Best for: HPC clusters, custom libraries, developmentInstallation: bash conda_install_dedalus2.shTime: 10-30 minutes

Both methods produce fully functional Dedalus installations. Choose based on your environment and requirements.

Get Started

Build Scripts

Configuration

Advanced

Documentation Index

​Overview

​conda-forge Installation

​What It Provides

​Supported Platforms

​Installation Command

​When to Use conda-forge

Quick Setup

Standard Hardware

Development Work

Teaching

​Limitations

​Custom Build Scripts

​What They Provide

​Configuration Options

​When to Use Custom Builds

HPC Clusters

Custom MPI

Performance Tuning

Development

​Comparison Matrix

​Detailed Configuration Examples

​Example 1: Full conda-forge Stack

​Example 2: Custom MPI with conda-forge FFTW/HDF5

​Example 3: Full Custom Stack (HPC)

​Example 4: Intel MKL with OpenMPI

​Installation Workflow Details

​Script Execution Flow

​Advanced Scenarios

​Installing Development Versions via conda-forge

​Testing Both Installation Methods

​Troubleshooting Installation Issues

​Library Incompatibility

​FFTW Linking Issues

​Environment Conflicts

​Performance Considerations

​BLAS Selection

​MPI Implementation

​Summary

conda-forge

Custom Scripts

Build docs developers (and LLMs) love

Overview

conda-forge Installation

What It Provides

Supported Platforms

Installation Command

When to Use conda-forge

Limitations

Custom Build Scripts

What They Provide

Configuration Options

When to Use Custom Builds

Comparison Matrix

Detailed Configuration Examples

Example 1: Full conda-forge Stack

Example 2: Custom MPI with conda-forge FFTW/HDF5

Example 3: Full Custom Stack (HPC)

Example 4: Intel MKL with OpenMPI

Installation Workflow Details

Script Execution Flow

Advanced Scenarios

Installing Development Versions via conda-forge

Testing Both Installation Methods

Troubleshooting Installation Issues

Library Incompatibility

FFTW Linking Issues

Environment Conflicts

Performance Considerations

BLAS Selection

MPI Implementation

Summary