A blog about Damien Merenne

Managing projects environments with Direnv and Micromamba 🔗

Feb 11, 2025, filled in development, shell, tech

Managing project development and build environments nowadays can be complicated, between different operating systems, different and of course incompatible tool versions, the different development languages and platforms. After a lot of iteration, trying to use some or a mix of Docker, Conan, Pip, Homebrew, Nix or native package managers, I think we might have found a solution that's working for us. We set up on Direnv and Micromamba to manage the basic tooling and bootstrap our environments.

Direnv allows us to configure the shell environment per directory. When entering a directory containing a .envrc file, it will load it, export some variables, setup virtual environments for Python or Ruby if needed and do custom bootstrapping tasks like downloading vendor binary tools.

We combine Direnv with Micromamba to download tools inside the project's directory, without cluttering the user's system. Everything is self-contained in the project directory. It can download and install multiple Python versions, CMake, the Java virtual machine and lot's of other tools. We configure Direnv so that it knows about the Micromamba environment. It will bootstrap the environment or update it automatically if it changes. Here is a typical example of a Micromamba environment definition file and a matching Direnv .envrc.

            
name: my-env
channels:
- conda-forge
dependencies:
- just
- python 3.12.*
- libprotobuf 3.20.*
- terraform
- awscliv2
- pip:
  - -r requirements.txt
  - -r requirements-dev.txt

A micromamba environment definition file

            
# Bootstrap micromamba direnv support
source_url https://gitlab.com/-/snippets/4803021/raw/main/direnv-micromamba.sh sha256-v9hWwL8HsCOdN+f8HBSijmXbGTgpPAxQJcUgPaYku3A=

# Setup and activate the micromamba environment.
use micromamba my-environment my-env.txt

# Setup some variable
export AWS_PROFILE=my-aws-profile

# Use a per project bash history file
export HISTFILE=$PWD/.bash_history

# Inject some path into the Python environment
path_add PYTHONPATH "${PWD}"

# Allow user to customize the environment using a local direnv file
source_env_if_exists .envrc.local

# Bootstrap project
if [ ! -d "my-dir" ]; then
  bootstrap.sh
fi

A direnv .envrc file

The first lines tell Direnv to download and check the sha256 of the Micromamba integration script that's used next. The use micromamba lines calls Micromamba to setup the environment my-environment that is defined in my-env.txt in the .micromamba-my-environment directory and activates it. It will also watch the environment definition file so that the environment is rebuilt if the file changes. As you can see, Micromamba supports installing pip packages in it's own environment. While Direnv supports setting up a Python virtual environment, it doesn't support running pip out of the box.

Here is the Direnv/Micromamba integration code snippet.

            
# Define `use micromamba` direnv command that installs micromamba if needed,
# bootstrap and activates an environment given micromamba environment definition
# files.
use_micromamba() {
    ENV="$1"
    shift
    FILES="${@:-${ENV}.env.yml}"
    PREFIX="${PWD}/.micromamba-${ENV}"
    SHA="${PREFIX}/environment.sha256sum"
    BINDIR="${PWD}/.bin"
    PATH_add "${PWD}/.bin"

    # Install micromamba if needed
    if ! which micromamba >/dev/null; then
        echo "direnv: installing micromamba in ${PWD}/.bin"
        mkdir -p "${BINDIR}"
        INSTALL_MICROMAMBA="$(fetchurl https://raw.githubusercontent.com/mamba-org/micromamba-releases/main/install.sh sha256-7+/a3LEvnYPcDGoUNWrrtzenAxFuc6xMTQpRfdZiBTg=)"
        INIT_YES=no BIN_FOLDER="${BINDIR}" bash < "${INSTALL_MICROMAMBA}" >/dev/null 2>&1
        PATH="${PATH}"
        if ! which micromamba >/dev/null; then
            echo could not find micromamba, ensure $BIN_FOLDER is in your PATH
            exit 1
        fi
    fi

    # Hook micromamba into the shell
    eval "$(micromamba shell hook --shell posix)"

    # Regenerate micromamba environment when the environment definitions are changed.
    watch_file ${FILES}

    # If already installed, check if the environment yaml definition file has been updated
    if [ ! -e "${SHA}" ] || ! sha256sum --quiet -c "${SHA}" >/dev/null 2>&1; then
        echo "direnv: micromamba $ENV environment updated, recreating environment"
        rm -fr "${PREFIX}"
    fi

    # Create the environment if needed
    if [ ! -d ${PREFIX} ]; then
        echo "direnv: creating micromamba ${ENV} environment, it might take a while"
        micromamba --quiet create --prefix "${PREFIX}" $(for i in $FILES; do echo --file $i; done) --yes || return 1
        sha256sum "${FILES}" > "${SHA}"
    fi

    # Activate micromamba environment
    micromamba activate "${PREFIX}" || return 1
}

Micromamba integration script with direnv