The Linux kernel ships with an extensive suite of tools for catching bugs before they reach users. These range from a simple style checker to sophisticated runtime sanitizers that can detect concurrency bugs and memory corruption. Using them systematically is expected of anyone submitting non-trivial patches. The tools fall into four broad categories: style checking, static analysis, dynamic analysis (sanitizers), and testing frameworks.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/DeelerDev/linux/llms.txt
Use this file to discover all available pages before exploring further.
Style checking
Validate patch formatting before submission to avoid reviewer friction.
Static analysis
Analyze source code at compile time to catch type errors, locking issues, and common patterns.
Dynamic analysis
Detect memory errors, data races, and undefined behavior in a running kernel.
Testing frameworks
Write and run structured unit tests and system-level tests against the kernel.
Style checker
checkpatch.pl
scripts/checkpatch.pl checks patches (or source files) for violations of the kernel coding style. It reports three levels of messages: ERROR (very likely wrong), WARNING (requires careful review), and CHECK (requires thought, enabled with --strict).
Static analysis tools
Static analysis tools examine source code at compile time without running it. They catch classes of bugs that the compiler itself does not warn about.sparse
Sparse is the kernel’s primary static analysis tool. It performs type-checking that the C compiler cannot do, including:- Verifying that
__userpointers are not dereferenced in kernel context - Detecting endianness mismatches (
__be32vs__le32) - Checking for misuse of
__rcu-annotated pointers - Flagging lock annotation violations
C=1 to your make invocation:
C=2 to check all files, not just those that were recompiled.
Coccinelle
Coccinelle (make coccicheck) applies semantic patches — scripts that describe patterns to find and fix — across the entire source tree. It is used for mass API migrations and for finding common programming mistakes. Unlike sparse or smatch, Coccinelle produces patch output directly, not just warnings.
Dynamic analysis tools
Dynamic analysis tools instrument the running kernel to detect errors as they occur. They are configured viaKconfig and must be enabled before building the test kernel.
KASAN — Kernel Address Sanitizer
KASAN detects out-of-bounds memory accesses and use-after-free errors. Enable it withCONFIG_KASAN=y. When a violation occurs, KASAN prints a detailed report including the stack trace, the accessed address, and the allocation/free history for that memory region.
KCSAN — Kernel Concurrency Sanitizer
KCSAN detects data races — concurrent accesses to the same memory location where at least one access is a write and there is no synchronization. Enable it withCONFIG_KCSAN=y.
KCSAN reports are probabilistic — a data race that exists may not always be reported in a single run. Running tests multiple times or under stress conditions improves detection rate.
KFENCE — Kernel Electric Fence
KFENCE is a low-overhead sampling-based memory safety tool that can be enabled in production kernels. Unlike KASAN, it does not instrument every memory access; instead, it protects a random sample of allocations. This makes it suitable for catching use-after-free and out-of-bounds bugs in deployed systems.UBSAN — Undefined Behavior Sanitizer
UBSAN detects C undefined behavior at runtime, including integer overflows, out-of-bounds array accesses, and misaligned pointer dereferences.kmemleak
kmemleak scans the kernel’s memory allocations and reports objects that are allocated but never freed and no longer reachable. It is useful for tracking down memory leak bugs.lockdep
lockdep is a locking correctness validator. It tracks every lock acquisition and release in the kernel and verifies that:- Locks are always acquired in the same order (to prevent deadlocks)
- Interrupt safety assumptions are consistent across all paths
- Lock classes are used correctly
CONFIG_PROVE_LOCKING=y. Any code with non-trivial locking should be tested with lockdep enabled before submission.
Testing frameworks
KUnit — Kernel Unit Testing Framework
KUnit is the kernel’s in-kernel unit testing framework. Tests are written in C, live inside the kernel, and have access to internal functions and structures that are not exposed to userspace. This makes it well-suited to “white box” testing of individual functions and code paths. KUnit tests run either at boot (built-in) or as loadable modules. Results are reported in KTAP format in the kernel log. Thekunit.py script can run tests under QEMU or User Mode Linux and parse results automatically:
kselftest — Kernel Self Tests
kselftest provides a collection of userspace test scripts and programs undertools/testing/selftests/. Unlike KUnit, kselftest tests run in userspace and interact with the kernel through system calls, sysfs, procfs, and device files. This makes them suitable for testing whole-kernel features and their userspace interfaces.
All new system calls should be accompanied by kselftest tests.
Tracing and debugging tools
ftrace
ftrace is the kernel’s built-in function tracer. It can trace function calls, measure latency, and record events with minimal overhead. Access it through thetracefs filesystem:
trace-cmd and kernelshark.
perf
perf provides hardware performance counter support and software event profiling. It can profile entire systems, individual processes, or specific kernel functions:
KGDB — Kernel Debugger
KGDB allows source-level debugging of the Linux kernel using gdb over a serial connection or network. Enable it withCONFIG_KGDB=y and connect a gdb instance to the target machine. KGDB is particularly useful for debugging early boot or interrupt handler code where printk-based debugging is impractical.
Recommended development kernel config
For routine development, enabling the followingKconfig options is recommended:
Memory and locking debug options
Memory and locking debug options
Stack and compiler warnings
Stack and compiler warnings
Fault injection
Fault injection
The fault injection framework can force memory allocations to fail at a configurable rate, helping you test error-handling paths that would otherwise never execute:
