Use this file to discover all available pages before exploring further.
Clorch ships a set of self-contained example files that cover the full range of the library — from a ten-line defmodel to a Llama-3-style chat loop with KV caching and a multi-GPU distributed training harness with AMP and gradient accumulation. Each example is a runnable Clojure namespace; load it in a REPL or evaluate it top-to-bottom. The snippets below are taken directly from the source files.
Simple Model
Minimal defmodel with one training step
PyTorch Basics Tutorial
Tensors, datasets, optimization, and model save/load
Autograd Tutorial
Gradient computation and computation-graph behavior
Synthetic Training
End-to-end training loop on generated data
Modern Llama
RoPE, GQA, SwiGLU, and incremental KV caching
NanoChat
Compact Llama-style training, checkpointing, and chat generation
Source:examples/simple.cljThe simplest possible end-to-end demonstration of Clorch. It defines a two-layer MLP using the defmodel macro, constructs a fake batch, runs a forward pass, computes MSE loss, and performs one Adam optimizer step — all inside a with-torch scope to manage native memory correctly.defmodel accepts constructor arguments, a binding vector of registered sub-modules, and a forward form. Registered fields participate automatically in nn/parameters, nn/to, and state dictionaries.
Source:examples/pytorch_basics_tutorial.cljA direct Clojure port of the official PyTorch “Learn the Basics” tutorial. It demonstrates tensor creation, a custom dataset backed by the data/dataset protocol, a multi-layer perceptron trained over multiple epochs with SGD and cross-entropy loss, and model serialization via torch/save and torch/load.The training loop pattern — zero-grad → forward → loss → backward → step — matches PyTorch exactly and is idiomatic for all Clorch training code.
(defn train-epoch! [dataloader model loss-fn optimizer] (doseq [[batch-idx {:keys [data target]}] (map-indexed vector dataloader)] (torch/with-torch (let [target (torch/reshape target [(torch/size data 0)]) pred (nn/forward model data) loss (loss-fn pred target)] (optim/zero-grad optimizer) (autograd/backward loss) (optim/step optimizer) (when (= 0 (mod (inc batch-idx) 2)) (printf " Loss: %.6f [%d/%d]\n" (torch/item-float loss) (* (inc batch-idx) (torch/size data 0)) (data/get-size (:dataset dataloader))))))))
The dataset is built using data/dataset with :size and :get-item callbacks, returning {:data … :target …} maps that the dataloader batches automatically:
Source:examples/autograd_tutorial.cljA port of Sebastian Raschka’s Automatic Differentiation Made Easy tutorial. It covers scalar and tensor gradients, calling backward to populate .grad fields, reading gradients with autograd/grad, and iterating over tensor slices with torch/tseq.
Wrap autograd examples in with-torch to ensure intermediate tensors are released. The autograd/grad call returns the accumulated .grad tensor — do not retain it outside the scope unless you call torch/retain! explicitly.
Source:examples/synthetic.cljAn end-to-end training demonstration on procedurally generated multi-class data. It creates Gaussian clusters in 10-dimensional space, wraps them in a tensor-dataset, runs a two-layer ReLU MLP with SGD and cross-entropy loss for several epochs, and prints the epoch-averaged loss. It also demonstrates explicit cleanup of native resources with data/cleanup-data!.
Source:examples/modern_llama.cljDemonstrates a single Llama-style transformer block built from Clorch’s LLM primitives: nn/GroupedQueryAttention for grouped-query attention (fewer K/V heads than Q heads), nn/SwiGLU for the feed-forward block, nn/rmsnorm for pre-norm, and torch/precompute-rope-freqs + torch/apply-rope for rotary position embeddings. The example shows both a regular forward pass and an incremental KV-cache forward pass where a new token is appended to a prefix.
(nn/defmodel ModernLlamaBlock [dim n-heads n-kv-heads context-len drop-rate] [sa (nn/GroupedQueryAttention dim n-heads n-kv-heads context-len drop-rate) ffwd (nn/SwiGLU dim (* 4 dim)) ln1 (nn/rmsnorm dim) ln2 (nn/rmsnorm dim)] (forward [input] (let [{:keys [x mask freqs kv-cache]} (if (map? input) input {:x input}) x (torch/add x (nn/forward sa {:x (nn/forward ln1 x) :mask mask :freqs freqs :kv-cache kv-cache})) x (torch/add x (nn/forward ffwd (nn/forward ln2 x)))] x)))
The KV-cache forward pass shows how to pass an atom as :kv-cache and slice pre-computed RoPE frequencies to the exact token positions being processed:
Source:examples/nanochat.cljA compact, single-device Llama-3-style chat demo inspired by Karpathy’s NanoChat. It tokenizes a text corpus with jtokkit, trains a small Llama model (2 layers, 4 attention heads, 1 K/V head, 128-dimensional embeddings) using Adam with gradient clipping, saves a checkpoint, and supports interactive chat with streaming token generation.The Llama model is built with defmodel nesting LlamaBlock records:
The generation loop samples from F/softmax probabilities, maintains a sliding context window, and optionally retains KV-cache tensors across steps using torch/retain!:
nanochat requires the-verdict.txt in the working directory for training. It falls back gracefully to untrained weights for the chat function if no checkpoint exists.
Source:examples/distributed_training.cljA production-pattern multi-GPU training example demonstrating NCCL process groups, synchronous DistributedDataParallel, clorch.amp autocast with both :float16 (dynamic scaling) and :bfloat16, gradient accumulation with ddp/no-sync, and rank-zero checkpoint saving via dist/save-checkpoint!.The train-worker function is the per-rank entrypoint. It is passed rank, world-size, process-group, and args by the launcher:
Launch workers from the coordinator with run-local!, passing device indices and training hyperparameters:
(defn run-local! "Launches one training worker JVM per CUDA device and waits for completion." [devices & [options]] (let [job (dist/launch! {:nproc-per-node (count devices) :devices devices :main 'distributed-training/train-worker :args (or options {})})] (dist/await-job! job) {:status (dist/job-status job) :logs (into {} (map (fn [rank] [rank (dist/job-logs job rank)])) (range (count devices)))}))
Distributed training requires two or more NVIDIA GPUs, CUDA 13.1, cuDNN 9, and NCCL 2. Set CLORCH_FORCE_GPU=1 and LD_LIBRARY_PATH before starting the JVM. See the README for the full host setup checklist.
Source:examples/bayesian_linear_regression_mcmc.cljDemonstrates probabilistic inference using clorch.distributions. It generates 120 synthetic observations from a known linear model, defines a log-posterior combining Normal priors on weights and bias with a Gaussian likelihood, and samples the posterior using random-walk Metropolis-Hastings. Multiple independent chains are run, with R-hat convergence diagnostics computed across chains.
(defn log-posterior [model {:keys [w b log-sigma]}] (let [sigma (Math/exp (double log-sigma)) lp-prior (+ (reduce + (map (fn [wj] (t/item-float (dist/log-prob (dist/normal 0.0 5.0) wj))) w)) (t/item-float (dist/log-prob (dist/normal 0.0 5.0) b)) (t/item-float (dist/log-prob (dist/normal 0.0 1.0) log-sigma))) _ (set-model-params! model {:w w :b b}) mu (nn/forward model x-t) lp-like (t/item-float (t/sum (dist/log-prob (dist/normal mu sigma) y-t)))] (+ lp-prior lp-like)))
Each Metropolis-Hastings step proposes a new state by adding Gaussian jitter, computes log-alpha, and accepts or rejects based on a uniform draw:
Source:examples/einsum_edsl.cljShows how to use clorch.einsum’s ein macro — a declarative Clojure eDSL for tensor contractions. Index variables are declared with declare, then used directly in ein expressions without string notation. The macro supports matrix-vector products, outer products, traces, and scalar-scaled contractions.
The ein eDSL uses Clojure’s symbolic dispatch — index variables like i and j must be declared at the top of the namespace before they appear in ein expressions.