Testing Guide - Redox OS

This guide covers testing procedures for Redox OS, including running tests locally, CI configuration, and automated testing.

Continuous Integration

Redox uses GitLab CI for automated testing on all merge requests targeting the master branch.

CI Pipeline Stages

Lint Stage

The lint stage ensures code quality and style compliance.

rustfmt
cargo-test

Checks Rust code formatting:

fmt:
  image: "rust:trixie"
  stage: lint
  script:
    - rustup component add rustfmt
    - cargo fmt -- --check

Run locally:

rustup component add rustfmt
cargo fmt -- --check

Runs unit tests:

cargo-test:
  image: "rust:trixie"
  stage: lint
  script:
    - cargo test --locked

Run locally:

cargo test --locked

Test Stage

The test stage builds images and validates packaging.

Image Build
Package Validation

Builds system images:

img:
  image: "redoxos/redox-base"
  stage: test
  script:
    - |
      export PATH="$HOME/.cargo/bin:$PATH" &&
      (curl "https://sh.rustup.rs" -sSf | sh -s -- -y --default-toolchain stable --profile minimal ) &&
      cargo install cbindgen &&
      PODMAN_BUILD=0 SKIP_CHECK_TOOLS=1 REPO_BINARY=1 FSTOOLS_NO_MOUNT=1 COOKBOOK_VERBOSE=false make ci-img IMG_TAG=$CI_COMMIT_REF_NAME

Run locally:

make ci-img IMG_TAG=local-test

Tests packaging for all supported architectures:

pkg:
  image: "rust:trixie"
  stage: test
  script:
    - |
      export PATH="$HOME/.cargo/bin:$PATH" PODMAN_BUILD=0 &&
      make CONFIG_NAME=ci SKIP_CHECK_TOOLS=1 repo-tree ARCH=x86_64 &&
      make CONFIG_NAME=ci SKIP_CHECK_TOOLS=1 repo-tree ARCH=i586 &&
      make CONFIG_NAME=ci SKIP_CHECK_TOOLS=1 repo-tree ARCH=aarch64 &&
      make CONFIG_NAME=ci SKIP_CHECK_TOOLS=1 repo-tree ARCH=riscv64gc

CI Workflow Rules

The CI pipeline runs automatically when:

workflow:
  rules:
    - if: '$CI_COMMIT_BRANCH == "master" && $CI_PROJECT_NAMESPACE == "redox-os"'
    - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'

Commits to master branch in the redox-os namespace
Merge requests targeting master

Local Testing

Pre-commit Testing

Before creating a merge request, run these tests locally:

Format check

cargo fmt -- --check

Auto-fix formatting:

cargo fmt

Run unit tests

cargo test --locked

Build test

make all

QEMU test run

make qemu

Verify the system boots and basic functionality works.

Configuration-based Testing

Redox provides several test configurations:

tests.toml
os-test.toml
auto-test.toml

Standard testing configuration:

# Configuration for testing
include = ["server.toml"]

[general]
filesystem_size = 10000  # MiB
prompt = false           # Non-interactive

[packages]
redox-tests = {}
benchmarks = {}

Build and test:

make all CONFIG_NAME=tests
make qemu CONFIG_NAME=tests

OS-level testing configuration:

# Build and run OS tests
make CONFIG_NAME=os-test unmount
rm -f "build/$(ARCH)/os-test/harddrive.img"
make CONFIG_NAME=os-test qemu gpu=no

Extract test results:

make CONFIG_NAME=os-test mount
cp -rv build/$(ARCH)/os-test/filesystem/usr/share/os-test/html ./
cp -v build/$(ARCH)/os-test/filesystem/usr/share/os-test/os-test.json ./
make CONFIG_NAME=os-test unmount

Automated test configuration for CI:

make CONFIG_NAME=auto-test all
make CONFIG_NAME=auto-test qemu

CI Image Building

The CI system builds standard images for all architectures:

Build CI Images

make ci-img IMG_TAG=v0.8.0

This builds three configurations:

server - Server configuration
desktop - Desktop configuration
demo - Demo configuration

Images are created in build/img/$(ARCH)/ with the naming format:

redox_<config>_<tag>_harddrive.img.zst
redox_<config>_<tag>_livedisk.iso.zst

A SHA256SUM file is also generated for verification.

Customize Image Naming

# Change the separator
make ci-img IMG_TAG=v0.8.0 IMG_SEPARATOR=-

# Omit the tag
make ci-img IMG_TAG= IMG_SEPARATOR=

# Change output directory
make ci-img IMG_DIR=custom/output/path

Individual Configuration Builds

# Build just server configuration
make server

# Build desktop configuration
make desktop

# Build demo configuration
make demo

These targets match the filesystem config file names and create images in the IMG_DIR.

OS Testing

Run comprehensive OS-level tests:

make ci-os-test

Clean previous test

make CONFIG_NAME=os-test unmount
rm -f "build/$(ARCH)/os-test/harddrive.img"

Run tests in QEMU

make CONFIG_NAME=os-test qemu gpu=no

Extract test results

make CONFIG_NAME=os-test mount
mkdir -p build/os-test/$(ARCH)
cp -rv build/$(ARCH)/os-test/filesystem/usr/share/os-test/html build/os-test/$(ARCH)
cp -v build/$(ARCH)/os-test/filesystem/usr/share/os-test/os-test.json build/os-test/$(ARCH)
tar -czf build/os-test/$(ARCH)/out.tar.gz -C build/$(ARCH)/os-test/filesystem/usr/share/os-test out
make CONFIG_NAME=os-test unmount

Test Recipes

Redox includes several test recipes:

Hello World Tests

Multi-language hello world tests in recipes/tests/hello-redox/files/:

test.c - C test
test.cpp - C++ test
test.go - Go test
test.java - Java test
test.js - JavaScript test
test.lua - Lua test
test.py - Python test
test.rs - Rust test
test.zig - Zig test

Automated Testing

The auto-test recipe group includes an Ion script for automated testing:

recipes/groups/auto-test/auto-test.ion

Testing Specific Components

Recipe Testing
Kernel Testing
Driver Testing
relibc Testing

Test a specific recipe:

# Clean and rebuild
make cr.recipe-name

# Build with tests
make r.recipe-name

# Push to image and test
make crp.recipe-name
make qemu

Test kernel changes:

# Clean and rebuild kernel
make cr.kernel

# Update initfs
# See: https://doc.redox-os.org/book/coding-and-building.html#how-to-update-initfs

# Rebuild system
make rebuild

# Test in QEMU
make qemu

Test driver changes:

# Build with debug symbols
REPO_DEBUG=1 make cr.driver-name

# Update initfs if needed

# Test with logging
make qemu serial=yes qemu_serial_logfile=driver-test.log

Test relibc changes:

# Clean relibc (recipe and sysroot)
make c.relibc

# Rebuild
make r.relibc

# Rebuild dependent recipes
make rebuild

# Test
make qemu

Automated Testing in CI

Toolchain Testing

Build and package the toolchain:

make ci-toolchain

This creates:

gcc-install.tar.gz
relibc-install.tar.gz
rust-install.tar.gz
clang-install.tar.gz
SHA256SUM

In build/toolchain/$(HOST_TARGET)/$(TARGET)/

Multi-Architecture Testing

Test packaging for all architectures:

for arch in x86_64 i586 aarch64 riscv64gc; do
  make CONFIG_NAME=ci SKIP_CHECK_TOOLS=1 repo-tree ARCH=$arch
done

Regression Testing

Identify regression

Note the commit where the issue started.

Use git bisect

git bisect start
git bisect bad HEAD
git bisect good <known-good-commit>

Test each commit

make clean
make all
make qemu
# Test for the issue
git bisect good  # or bad

Find the culprit

Git bisect will identify the problematic commit.

Performance Testing

Benchmark Configuration

Use the tests configuration with benchmarks:

make all CONFIG_NAME=tests
make qemu CONFIG_NAME=tests

The tests.toml includes the benchmarks package.

Timing Builds

# Time a full build
time make all

# Time a recipe build
time make r.recipe-name

# Time with podman bootstrap
time bash -e podman_bootstrap.sh

Test Coverage

Test coverage for Rust code can be generated using cargo-tarpaulin or cargo-llvm-cov.

# Install tarpaulin
cargo install cargo-tarpaulin

# Generate coverage report
cargo tarpaulin --out Html --output-dir coverage

Testing Best Practices

Before Submitting MR

Format code

cargo fmt

Run tests

cargo test --locked

Build and test

make cr.component-name
make qemu

Test on multiple configs

make all CONFIG_NAME=server
make all CONFIG_NAME=desktop

Testing Cross-Platform

Test your changes on all supported architectures:

# x86_64 (default)
make all ARCH=x86_64
make qemu ARCH=x86_64

# i586 (32-bit x86)
make all ARCH=i586
make qemu ARCH=i586

# aarch64 (ARM64)
make all ARCH=aarch64
make qemu ARCH=aarch64

# riscv64gc (RISC-V)
make all ARCH=riscv64gc
make qemu ARCH=riscv64gc

Testing with Different Settings

Vary QEMU settings to catch edge cases:

# Different CPU counts
make qemu QEMU_SMP=1
make qemu QEMU_SMP=4
make qemu QEMU_SMP=8

# Different memory sizes
make qemu QEMU_MEM=512
make qemu QEMU_MEM=2048
make qemu QEMU_MEM=4096

# Different disk types
make qemu disk=nvme
make qemu disk=ata
make qemu disk=virtio

# With/without KVM
make qemu kvm=yes
make qemu kvm=no

Troubleshooting Tests

Tests Fail Locally But Pass in CI

Ensure your build environment matches CI (use podman)
Check for uncommitted changes
Verify dependencies are up-to-date: make pull
Clean and rebuild: make clean && make all

Tests Pass Locally But Fail in CI

Check CI logs for specific errors
Verify all files are committed
Ensure no local-only configuration
Test with PODMAN_BUILD=0 like CI does

Intermittent Test Failures

May be timing-related (try kvm=no)
Check for race conditions
Test with different QEMU_SMP values
Review serial logs: make qemu serial=yes qemu_serial_logfile=test.log

Additional Resources

CI Configuration

View the complete CI configuration

Developer FAQ

Common testing questions

Debugging Guide

Debug failing tests

Build System Reference

Complete build system documentation

Next Steps

Contributing Guide

Review contribution guidelines

Development Workflow

Learn the development process

Get Started

Build System

Architecture

Components

Development

Hardware

​Continuous Integration

​CI Pipeline Stages

​CI Workflow Rules

​Local Testing

​Pre-commit Testing

​Configuration-based Testing

​CI Image Building

​Build CI Images

​Customize Image Naming

​Individual Configuration Builds

​OS Testing

​Test Recipes

​Hello World Tests

​Automated Testing

​Testing Specific Components

​Automated Testing in CI

​Toolchain Testing

​Multi-Architecture Testing

​Regression Testing

​Performance Testing

​Benchmark Configuration

​Timing Builds

​Test Coverage

​Testing Best Practices

​Troubleshooting Tests

​Additional Resources

CI Configuration

Developer FAQ

Debugging Guide

Build System Reference

​Next Steps

Contributing Guide

Development Workflow

Build docs developers (and LLMs) love

Continuous Integration

CI Pipeline Stages

CI Workflow Rules

Local Testing

Pre-commit Testing

Configuration-based Testing

CI Image Building

Build CI Images

Customize Image Naming

Individual Configuration Builds

OS Testing

Test Recipes

Hello World Tests

Automated Testing

Testing Specific Components

Automated Testing in CI

Toolchain Testing

Multi-Architecture Testing

Regression Testing

Performance Testing

Benchmark Configuration

Timing Builds

Test Coverage

Testing Best Practices

Troubleshooting Tests

Additional Resources

Next Steps