Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/antlobach/clorch/llms.txt

Use this file to discover all available pages before exploring further.

Loss functions measure the discrepancy between a model’s predictions and the ground-truth targets, producing a scalar tensor whose gradient drives parameter updates. All Clorch loss functions live in clorch.nn.functional and return regular LibTorch tensors that participate in the autograd graph. 
(require '[clorch.nn.functional :as F])

​
Regression Losses

​
Mean Squared Error

The standard choice for continuous regression targets. Penalises large errors quadratically. 
(F/mse-loss input target)

​
L1 Loss (Mean Absolute Error)

Sums absolute differences rather than squared differences. More robust to outliers than MSE. 
(F/l1-loss input target)

​
Smooth L1 Loss (Huber Loss)

Combines the best of MSE and L1: behaves like MSE for small errors (smooth gradient near zero) and like L1 for large errors (bounded gradient, resistant to outliers). Commonly used in object detection. 
(F/smooth-l1-loss input target)
When you are unsure whether your dataset contains outliers, prefer smooth-l1-loss over plain mse-loss. It converges reliably in both clean and noisy settings.

​
Classification Losses

​
Cross Entropy

Combines log-softmax and negative log-likelihood in a single numerically stable operation. Expects raw logits (before any softmax), not probabilities. 
;; logits: [batch, num-classes]   targets: [batch] int64
(F/cross-entropy logits targets)

​
NLL Loss — Negative Log Likelihood

Expects log-probabilities as input (e.g. the output of F/log-softmax). Useful when you want to decouple the softmax step from the loss. 
(F/nll-loss (F/log-softmax logits -1) targets)

​
Binary Cross Entropy

For binary classification where each output is an independent probability in (0, 1). Input must already be passed through a sigmoid. 
;; input: probabilities in (0, 1) — apply sigmoid first
(F/bce-loss (F/sigmoid logits) target)

​
BCE with Logits

Numerically more stable than applying sigmoid then BCE separately. Accepts raw logits directly. 
;; input: raw logits — sigmoid + BCE fused in one kernel
(F/bce-with-logits-loss logits target)
Always prefer F/bce-with-logits-loss over (F/bce-loss (F/sigmoid x) target). The fused version avoids floating-point overflow that can occur when exponentiating very large logits.

​
Usage in a Training Loop

Loss tensors retain the full autograd graph until you call autograd/backward. The canonical pattern is: 
(require '[clorch.nn :as nn]
         '[clorch.nn.functional :as F]
         '[clorch.optim :as optim]
         '[clorch.autograd :as autograd]
         '[clorch.torch :as t])

(defn train-step [model optimizer x y]
  ;; 1. Clear gradients from the previous iteration
  (optim/zero-grad optimizer)
  ;; 2. Forward pass
  (let [pred (nn/forward model x)
        ;; 3. Compute loss
        loss (F/cross-entropy pred y)]
    ;; 4. Backpropagate
    (autograd/backward loss)
    ;; 5. Update parameters
    (optim/step optimizer)
    ;; 6. Extract a plain JVM float before the scope closes
    (t/item-float loss)))
Loss tensors — and all intermediate activations — hold native memory. Wrap each training iteration in t/with-torch (or an equivalent scope) so that non-root tensors are released after the optimizer step. See the Memory Management page for the canonical training loop pattern.

​
Autograd Graph and Retention

Every call to a loss function produces a tensor whose .requires_grad is true as long as at least one input tensor requires gradients. Calling autograd/backward on that tensor computes and accumulates gradients into every leaf parameter.
By default LibTorch frees the intermediate computation graph after a single backward call. If you need to call backward more than once (e.g. for second-order gradients or gradient accumulation with manual graph retention), pass the :retain-graph true option:
(autograd/backward loss :retain-graph true)
Retaining the graph keeps all intermediate activations in native memory until you release them explicitly, so use this only when necessary.

Build docs developers (and LLMs) love

Get started for freeTalk to us