This guide walks you from a single tensor all the way to a complete training loop that runs on either CPU or CUDA. Each step builds on the previous one, so by the end you will have seen every core Clorch concept in a working context. All examples run against the real library — nothing is mocked or simplified.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/antlobach/clorch/llms.txt
Use this file to discover all available pages before exploring further.
Start by requiring the two core namespaces.
clorch.torch provides tensor factory functions and arithmetic; clorch.autograd provides gradient computation.Create a scalar tensor, raise it to the third power, and compute the gradient. Because
x = 2.0, the derivative of x³ is 3x² = 12.0:(def x (t/tensor [2.0] {:requires-grad true}))
(def y (t/pow x 3))
(autograd/backward y)
(autograd/grad x)
;; => [12.0]
(def w (t/randn [3 4] {:requires-grad true}))
(def z (t/sum (t/mul w w)))
(autograd/backward z)
;; Gradient shape matches w: [3 4]
(t/size (autograd/grad w))
;; => [3 4]
Autograd records every operation on tensors whose
:requires-grad is true. Call autograd/backward once on the scalar loss, then read each parameter’s gradient with autograd/grad. Call optim/zero-grad (or nn/zero-grad) before the next backward pass to clear accumulated gradients.nn/sequential composes layers into a forward pipeline. The result is a plain Clojure vector, so it is trivially inspectable and composable:(require '[clorch.nn :as nn])
(def model
(nn/sequential
(nn/linear 10 20)
(nn/relu)
(nn/linear 20 1)))
;; Run a batch of 4 samples through the model
(t/size (nn/forward model (t/randn [4 10])))
;; => [4 1]
Standard layers available out of the box include
nn/linear, nn/conv1d, nn/conv2d, nn/lstm, nn/gru, nn/embedding, nn/batchnorm1d, nn/layernorm, nn/rmsnorm, nn/dropout, and many activation functions (nn/relu, nn/gelu, nn/silu, nn/tanh, nn/sigmoid, and more).nn/defmodel defines a named model as a Clojure record. The constructor arguments configure the model, the binding vector registers modules or parameters, and the forward form defines execution. Registered fields automatically participate in nn/parameters, nn/to, and state dictionaries.(require '[clorch.nn.functional :as F])
(nn/defmodel CustomMLP [in hidden out]
[l1 (nn/linear in hidden)
l2 (nn/linear hidden out)]
(forward [x]
(nn/forward l2 (F/relu (nn/forward l1 x)))))
(def custom-model (CustomMLP 10 32 1))
(t/size (nn/forward custom-model (t/randn [4 10])))
;; => [4 1]
Inspect layer shapes and parameter counts with
nn/summary. Pass the model and an input shape vector:----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Linear [4 32] 352
Linear [4 1] 33
CustomMLPRecord [4 1] 385
================================================================
Total params: 385
Trainable params: 385
Non-trainable params: 0
----------------------------------------------------------------
Here is a slightly larger example from the
simple.clj example that shows a two-layer defmodel inside a complete forward pass:(nn/defmodel MyModel [in out]
[l1 (nn/linear in 16)
l2 (nn/linear 16 out)]
(forward [x]
(let [h (nn/forward l1 x)
h (F/relu h)]
(nn/forward l2 h))))
(def model (MyModel 10 3))
(t/size (nn/forward model (t/randn [4 10])))
;; => [4 3]
A complete supervised training loop ties together an optimizer, a loss function, a backward pass, and an optimizer step. Wrap each allocating batch step in
t/with-torch to release intermediate tensors immediately:(require '[clorch.optim :as optim])
(def model
(nn/sequential
(nn/linear 10 4)
(nn/relu)
(nn/linear 4 3)))
(def optimizer (optim/sgd (nn/parameters model) :lr 0.01))
(nn/train model true)
(dotimes [epoch 3]
(let [losses (atom [])]
;; Simulate a mini-batch loop
(dotimes [_ 5]
(t/with-torch
(let [x (t/randn [8 10])
target (t/rand-int 0 3 [8] {:dtype :int64})
logits (nn/forward model x)
loss (F/cross-entropy logits target)]
(optim/zero-grad optimizer)
(autograd/backward loss)
(optim/step optimizer)
(swap! losses conj (t/item-float loss)))))
(printf "Epoch %d | Avg loss: %.4f%n"
(inc epoch)
(/ (apply + @losses) (count @losses)))))
(require '[clorch.data :as data])
(def train-ds (data/tensor-dataset x-train y-train))
(def train-loader (data/dataloader train-ds :batch-size 16 :shuffle true))
(doseq [epoch (range 5)]
(doseq [{:keys [data target]} train-loader]
(t/with-torch
(let [logits (nn/forward model data)
loss (F/cross-entropy logits target)]
(optim/zero-grad optimizer)
(autograd/backward loss)
(optim/step optimizer)))))
Always return a JVM scalar (e.g., the result of
t/item-float) or nil from a t/with-torch block. Returning a raw tensor is safe only when you immediately bind it outside the scope with t/retain!. Keeping models and optimizers alive outside with-torch is the normal pattern — only the per-batch intermediate tensors need scoped cleanup.Resolve a device once at startup, then place models and input tensors explicitly. Device selection does not happen automatically — you choose a device and place everything on it:
(require '[clorch.cuda :as cuda])
;; Resolve device at startup
(def device (if (cuda/available?) :cuda :cpu))
;; Move model to device
(def model-on-device (nn/to model device))
;; Create input directly on the target device
(def input (t/randn [32 10] {:device device}))
;; Forward pass — model and input are on the same device
(t/size (nn/forward model-on-device input))
;; => [32 1]
nn/to recursively moves every parameter and buffer in the model tree, including nested defmodel records and nn/sequential vectors. t/randn, t/tensor, t/zeros, t/ones, and all other factory functions accept a :device option.cuda/available? returns false on CPU-only builds regardless of hardware. Use CLORCH_FORCE_GPU=1 to request the CUDA backend explicitly when the automatic detection does not select it. See the Installation page for system-package requirements and the full environment-variable reference.A self-contained device-aware training snippet that mirrors the
pytorch_basics_tutorial.clj example:(def device (if (cuda/available?) :cuda :cpu))
(def model
(nn/to
(nn/sequential
(nn/linear (* 28 28) 512)
(nn/relu)
(nn/linear 512 512)
(nn/relu)
(nn/linear 512 10))
device))
(def optimizer (optim/sgd (nn/parameters model) :lr 1e-3))
(defn train-step! [data target]
(t/with-torch
(let [data (nn/to data device)
target (nn/to target device)
pred (nn/forward model data)
loss (F/cross-entropy pred target)]
(optim/zero-grad optimizer)
(autograd/backward loss)
(optim/step optimizer)
(t/item-float loss))))
Next Steps
Tensors & Operations
Full reference for tensor creation, dtypes, arithmetic, reductions, slicing with
ix, and shape manipulation.Neural Networks
Complete layer reference, custom models with
defmodel, summaries, and state-dict checkpointing.Optimizers
SGD, Adam, AdamW, RMSprop, and Adagrad with learning-rate scheduling.
Distributed Training
Launch NCCL worker processes, wrap models in DDP, use AMP and gradient accumulation, and write rank-zero checkpoints.
Memory Management
Understand native allocation scopes,
with-torch, retain!, release!, and how to avoid memory leaks in long REPL sessions.Examples
Browse the full example directory: PyTorch basics, autograd tutorial, synthetic training, modern Llama, NanoChat, and distributed CUDA training.