Development with Nix

Nix provides the recommended development environment for NativeLink. It ensures reproducible builds, matches the CI environment, and ships almost all development tooling in a single flake.

Why Nix?

NativeLink uses Nix because it provides:

Reproducibility - Bit-for-bit identical builds across machines
CI Parity - Same environment locally and in continuous integration
Complete Tooling - All development tools in one place
Declarative Setup - Environment defined in flake.nix
Cross-Platform - Works on Linux, macOS, and WSL2

While Nix is optional, it’s highly recommended as it lets you reproduce most of CI locally.

Installation

Prerequisites

Install Nix with Flakes

Use the experimental nix installer which enables flakes by default:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install

This installer automatically:

Installs Nix
Enables flakes and nix-command features
Configures your shell

For more information, see the Nix Flakes Wiki.

Install direnv (Highly Recommended)

direnv automatically loads the Nix environment when you enter the repository:

nix profile install nixpkgs#direnv

Hook direnv into your shell:

Bash
Zsh
Fish

echo 'eval "$(direnv hook bash)"' >> ~/.bashrc

echo 'eval "$(direnv hook zsh)"' >> ~/.zshrc

echo 'direnv hook fish | source' >> ~/.config/fish/config.fish

See the direnv hook documentation for other shells.

Restart Your Terminal

After installing direnv and adding the hook, restart your terminal or source your shell configuration:

source ~/.bashrc  # or ~/.zshrc, etc.

Allow direnv in Repository

When you cd into the nativelink repository, you’ll see a message asking you to run:

direnv allow

This activates the Nix development environment.

If you don’t use direnv, you’ll need to manually run nix develop every time you enter the repository, switch branches, or make changes to Nix files.

Verify Installation

Check Nix Environment

Verify the environment is active:

env | grep NIX

You should see several *NIX_* environment variables.

Check Available Tools

# Bazel (via bazelisk)
bazel --version

# Rust toolchain
rustc --version
cargo --version

# Development tools
pre-commit --version
git-cliff --version

Development Shell

Included Tools

The Nix development shell (defined in flake.nix:453-560) includes:

git - Version control
pre-commit - Git hooks for code quality
git-cliff - Changelog generation
buck2 - Alternative build system

See flake.nix:454-520 for the complete list.

Shell Hook

When entering the development shell, several setup scripts run automatically (see flake.nix:522-559):

Pre-commit Hooks

Generates .pre-commit-config.yaml symlink:

${config.pre-commit.installationScript}

LRE Configuration

Generates lre.bazelrc for local remote execution:

${config.lre.installationScript}

NativeLink Cache

Generates nativelink.bazelrc for cache access:

${config.nativelink.installationScript}

Platform-Specific Config

Generates platform configs:

nixos.bazelrc (on NixOS)
darwin.bazelrc (on macOS)

Environment Variables

Sets development variables:

export CC=clang
export PULUMI_K8S_AWAIT_ALL=true
export PLAYWRIGHT_BROWSERS_PATH=${pkgs.playwright-driver.browsers}

Building with Nix

Build Commands

Build NativeLink

# Build for your current platform
nix build

# Binary is in result/bin/
./result/bin/nativelink --help

Build for Specific Platform

# Linux x86_64 (musl, static)
nix build .#nativelink-x86_64-linux

# Linux aarch64 (musl, static)
nix build .#nativelink-aarch64-linux

# macOS (native compilation only)
nix build .#nativelink-aarch64-darwin  # On ARM Mac
nix build .#nativelink-x86_64-darwin   # On Intel Mac

See flake.nix:174-179 for platform definitions.

Build Container Image

# Build OCI container image
nix build .#nativelink-image

# Load into Docker
docker load < result

# Image config at flake.nix:198-225

Build with Coverage

# Generate code coverage report
nix build .#nativelinkCoverageForHost

# View coverage report
open result/index.html

Uses nightly Rust with cargo-llvm-cov (see flake.nix:293-308).

Build Internals

NativeLink’s Nix build uses crane for Rust projects:

Build Arguments
Dependency Caching
Cross-Compilation

Common build arguments (see flake.nix:98-155):

# Target architecture (musl on Linux)
CARGO_BUILD_TARGET = "x86_64-unknown-linux-musl";

# Static linking on Linux
CARGO_BUILD_RUSTFLAGS = "-C target-feature=+crt-static";

# Custom linker (mold on Linux, lld elsewhere)
CARGO_TARGET_*_LINKER = linkerPath;

# Dependencies
buildInputs = [cacert] ++ darwin.frameworks;
nativeBuildInputs = [bash mold/lld];

Nix builds dependencies separately for efficient caching:

# Build dependencies first
cargoArtifactsFor = p: (craneLibFor p).buildDepsOnly (commonArgsFor p);

# Then build NativeLink
nativelinkFor = p: (craneLibFor p).buildPackage (
  (commonArgsFor p) // { cargoArtifacts = cargoArtifactsFor p; }
);

See flake.nix:158-165.

Cross-compilation using Nix package sets:

# x86_64 Linux (musl)
pkgs.pkgsCross.musl64

# aarch64 Linux (musl)
pkgs.pkgsCross.aarch64-multiplatform-musl

# macOS doesn't support cross-compilation
# Must build natively

See flake.nix:167-179.

Testing with Nix

Run Tests

Unit Tests

# Run via Bazel in Nix shell
bazel test :unit_tests

# Or run all tests
bazel test //...

Integration Tests

# Build integration test packages
nix build .#buildstream-with-nativelink-test
nix build .#mongo-with-nativelink-test
nix build .#buck2-with-nativelink-test

# Run the tests
./result/bin/test

See flake.nix:364-375 for test definitions.

Doc Tests

bazel test :doctests

Pre-commit Hooks

Pre-commit hooks are configured via the Nix flake (see flake.nix:409-414):

# Run all hooks
pre-commit run -a

# Run specific hook
pre-commit run rustfmt
pre-commit run clippy

Hooks are defined in tools/pre-commit-hooks.nix and include:

rustfmt (formatting)
clippy (linting)
vale (documentation)
Various config generators

Development Workflow

Typical Workflow

Enter Repository

cd nativelink
# direnv automatically loads environment
# Or manually: nix develop

Make Changes

Edit source files using your preferred editor.

Build and Test

# Quick iteration with Cargo
cargo build
cargo test

# Or with Bazel for CI parity
bazel build //:nativelink
bazel test //...

Check Code Quality

# Run pre-commit hooks
pre-commit run -a

# Fix formatting
cargo fmt --all
bazel run --config=rustfmt @rules_rust//:rustfmt

Run Locally

cargo run -- nativelink-config/examples/basic_cas.json5

Using Bazel in Nix

The Nix shell provides a Bazel wrapper (see flake.nix:455-458):

# Wrapper unsets TMPDIR for Bazel compatibility
bazel = pkgs.writeShellScriptBin "bazel" ''
  unset TMPDIR TMP
  exec ${pkgs.bazelisk}/bin/bazelisk "$@"
'';

This ensures Bazel works correctly in the Nix environment.

Advanced Features

Local Remote Execution (LRE)

Nix provides LRE toolchains automatically (see flake.nix:416-425):

# LRE environment variables
Lre.Env = [
  # C++ toolchain (Linux only)
  lre-cc.meta.Env
  # Rust toolchain
  lre-rs.meta.Env
];

# Platform prefix
Lre.prefix = "linux" or "macos";

LRE config is generated in lre.bazelrc by the shell hook.

Custom Packages

The flake exports several custom packages (see flake.nix:336-398):

nativelink-worker-*

Worker images for different toolchains:

lre-cc (C++ toolchain)
lre-rs (Rust toolchain)
lre-java (Java toolchain)
siso-chromium (Chromium builds)

Integration Tests

buildstream-with-nativelink-test
mongo-with-nativelink-test
buck2-with-nativelink-test
rbe-toolchain-with-nativelink-test

Utilities

generate-bazel-rc
generate-stores-config
update-module-hashes
local-image-test

Build any package:

nix build .#package-name

Flake Apps

The flake provides runnable apps (see flake.nix:325-334):

# Run NativeLink directly
nix run github:TraceMachina/nativelink -- config.json5

# Run native CLI
nix run .#native

Documentation Generation

Generate documentation using the docs command:

# In Nix shell
docs

# This runs the docs script from flake.nix:190
# Opens browser with generated documentation

The docs package is defined at tools/docs.nix.

Troubleshooting

direnv not loading environment

Check direnv status:

direnv status

Ensure you ran direnv allow in the repository.If issues persist, manually enter the shell:

nix develop

Nix build fails with hash mismatch

Update flake inputs:

nix flake update

Or rebuild without cache:

nix build --rebuild

Out of disk space in /nix/store

Collect garbage to free space:

# Delete unreferenced packages
nix-collect-garbage

# Delete old generations and collect garbage
nix-collect-garbage -d

# Check space usage
du -sh /nix/store

macOS: Cannot find C++ compiler

Install Xcode Command Line Tools:

xcode-select --install

The Nix environment doesn’t ship a full C++ toolchain.

Nix commands hang or are very slow

Check Nix daemon status:

# Check if daemon is running
systemctl status nix-daemon  # On Linux with systemd

# Restart daemon
sudo systemctl restart nix-daemon

Or rebuild Nix database:

nix-store --verify --check-contents

Platform-Specific Notes

Linux

Builds use musl for static linking
LRE fully supported with C++ and Rust toolchains
Uses mold linker for fast linking (see flake.nix:120)

macOS

No cross-compilation support (must build natively)
LRE uses Rust toolchain only (C++ not yet supported)
Requires Xcode Command Line Tools
Uses lld linker (see flake.nix:121)
Darwin-specific frameworks included automatically (see flake.nix:512-516)

NixOS

Special PATH configuration for LRE (see flake.nix:426-452)
Generates nixos.bazelrc with correct binary paths
Fully integrated with system Nix configuration

Resources

Nix Manual

Official Nix documentation

Nix Flakes

Flakes reference and guide

crane

Nix library for Rust projects

direnv

Environment switcher for shells

Next Steps

Development with Bazel

Learn Bazel build system features

Development with Cargo

Use Cargo for fast iteration

Development

Community

​Why Nix?

​Installation

​Prerequisites

​Verify Installation

​Development Shell

​Included Tools

​Shell Hook

​Building with Nix

​Build Commands

​Build Internals

​Testing with Nix

​Run Tests

​Pre-commit Hooks

​Development Workflow

​Typical Workflow

​Using Bazel in Nix

​Advanced Features

​Local Remote Execution (LRE)

​Custom Packages

nativelink-worker-*

Integration Tests

Utilities

​Flake Apps

​Documentation Generation

​Troubleshooting

​Platform-Specific Notes

​Linux

​macOS

​NixOS

​Resources

Nix Manual

Nix Flakes

crane

direnv

​Next Steps

Development with Bazel

Development with Cargo

Build docs developers (and LLMs) love

Why Nix?

Installation

Prerequisites

Verify Installation

Development Shell

Included Tools

Shell Hook

Building with Nix

Build Commands

Build Internals

Testing with Nix

Run Tests

Pre-commit Hooks

Development Workflow

Typical Workflow

Using Bazel in Nix

Advanced Features

Local Remote Execution (LRE)

Custom Packages

Flake Apps

Documentation Generation

Troubleshooting

Platform-Specific Notes

Linux

macOS

NixOS

Resources

Next Steps