Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/antlobach/clorch/llms.txt

Use this file to discover all available pages before exploring further.

Automatic differentiation is the mechanism that makes gradient-based learning practical. Clorch’s clorch.autograd namespace is a thin, idiomatic wrapper around LibTorch’s native reverse-mode autograd engine — the same engine that powers PyTorch. Every tensor operation that touches a gradient-tracked tensor is recorded in a dynamic computation graph. Calling autograd/backward on any scalar node in that graph walks the graph in reverse and accumulates gradients at all leaf tensors. The entire process is transparent to the caller; you write forward math as ordinary function calls and gradients appear automatically. 
(require '[clorch.torch    :as t]
         '[clorch.autograd :as autograd])

​
Enabling Gradient Tracking

A tensor only participates in the computation graph when created with {:requires-grad true}. Tensors created without this flag are treated as constants — they pass values through without recording any operations. 
;; Leaf tensor — gradients will accumulate here
(def x (t/tensor [2.0] {:requires-grad true}))

;; Intermediate node — graph edge is created automatically
(def y (t/pow x 2))  ;; y = x²
You can also enable tracking on an existing tensor retroactively: 
(autograd/set-requires-grad x true)
(autograd/set-requires-grad x false) ;; disable again
Only floating-point tensors support requires-grad. Setting it on an integer tensor throws a LibTorch error.

​
Computing Gradients

​
backward

Call autograd/backward on any scalar-valued tensor to trigger reverse-mode accumulation. After the call, the accumulated gradient is available at every tracked leaf via autograd/grad. 
;; y = x², dy/dx = 2x
(def x (t/tensor [2.0] {:requires-grad true}))
(def y (t/pow x 2))

(autograd/backward y)
(autograd/grad x)  ;; → [4.0]  (2 × 2.0)

​
grad

autograd/grad simply reads the .grad field of the underlying tensor. It returns a tensor (not a scalar), so use t/item-float when you need a JVM number: 
(t/item-float (autograd/grad x))  ;; 4.0

​
Worked examples

;; f(x) = x², f'(x) = 2x
(def x (t/tensor [3.0] {:requires-grad true}))
(def y (t/pow x 2))

(autograd/backward y)
(autograd/grad x)
;; → [6.0]   (2 × 3.0)

​
Detaching from the Graph

autograd/detach returns a new tensor that shares storage with the original but has no gradient history. Use it when you need the current numeric value of a tensor for a side-effect (logging, a metric, a threshold check) without polluting the computation graph. 
(def x (t/tensor [2.0] {:requires-grad true}))
(def y (t/pow x 2))

;; Detach before computing statistics that must not be differentiable
(def y-detached (autograd/detach y))

;; y-detached has the same values as y
(t/item-float y-detached)  ;; 4.0

;; But gradients do not flow through y-detached
detach creates a view of the original storage. Mutating the detached tensor in-place will affect the original. Prefer (t/clone (autograd/detach y)) if you need an independent copy.

​
Disabling Gradient Tracking with no-grad

The autograd/no-grad macro wraps a block of code in a LibTorch NoGradGuard, which prevents any tensor operation inside the block from being recorded in the computation graph. This is essential for:
  • Inference / validation — avoids allocating intermediate activation nodes, cutting memory by roughly half for a typical forward pass.
  • Metric and loss logging — you want numbers, not graph nodes.
  • Parameter updates — weight tensors should not track optimizer arithmetic.
(autograd/no-grad
  (let [pred (nn/forward model x)]
    (calculate-accuracy pred targets)))
Nested no-grad blocks are safe. The guard is re-entrant and restored correctly when the block exits, even on exception.
Every inference call in production should be wrapped in no-grad. Forgetting it can cause memory to grow unboundedly because the graph accumulates across calls.

​
Manual Gradient Control

autograd/set-requires-grad gives you fine-grained control over which tensors are tracked. The most common use case is freezing part of a model for transfer learning: 
;; Freeze every parameter in a sub-module
(doseq [p (nn/parameters encoder)]
  (autograd/set-requires-grad p false))

;; Later, unfreeze for fine-tuning
(doseq [p (nn/parameters encoder)]
  (autograd/set-requires-grad p true))

​
Training Loop Integration

A canonical gradient update has three steps: zero accumulated gradients from the previous iteration, run the forward pass and call backward, then apply the optimizer step. All three belong in a single with-torch scope per batch so that intermediate tensors are released deterministically. 
(require '[clorch.torch          :as t]
         '[clorch.nn             :as nn]
         '[clorch.nn.functional  :as F]
         '[clorch.autograd       :as autograd]
         '[clorch.optim          :as optim])

(let [model     (create-model)
      optimizer (optim/adam (nn/parameters model))]
  (doseq [{:keys [data target]} dataloader]
    (let [loss-value
          (t/with-torch
            ;; 1. Zero gradients from the previous batch
            (optim/zero-grad optimizer)
            ;; 2. Forward pass + backward
            (let [prediction (nn/forward model data)
                  loss       (F/cross-entropy prediction target)]
              (autograd/backward loss)
              ;; 3. Parameter update
              (optim/step optimizer)
              ;; Return a JVM scalar so no tensor escapes the scope
              (t/item-float loss)))]
      (println "Loss:" loss-value))))
1

Zero gradients

optim/zero-grad clears .grad on every tracked parameter. Without this step, gradients accumulate across batches.
2

Backward pass

autograd/backward traverses the computation graph from the scalar loss to every leaf, accumulating ∂loss/∂param at each parameter tensor.
3

Optimizer step

optim/step reads the accumulated gradients and updates each parameter according to the optimizer rule (SGD, Adam, AdamW, etc.).
4

Return a scalar

The with-torch scope sees (t/item-float loss) — a plain JVM Float — as its final result. No tensors are retained across iterations, keeping native memory bounded. See the Memory Management page for a complete explanation of why this matters.

​
Autograd API Reference

autograd/backward

Computes reverse-mode gradients for all tracked leaves reachable from the scalar tensor argument. Modifies .grad fields in place.

autograd/grad

Returns the accumulated gradient tensor at a leaf. Returns nil before backward is called or if the tensor has no gradient.

autograd/detach

Returns a view of the tensor with no graph history. Safe for metrics and logging. Does not copy storage.

autograd/no-grad

Macro that disables graph recording for the duration of its body. Mandatory for inference loops and validation steps.

autograd/set-requires-grad

Enables or disables gradient accumulation on an existing tensor. Use to freeze/unfreeze model parameters.

Build docs developers (and LLMs) love

Get started for freeTalk to us