Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/antlobach/clorch/llms.txt

Use this file to discover all available pages before exploring further.

Clorch executes tensor operations in C++ through LibTorch, bypassing the JVM for all heavy numerical compute. The JVM is responsible for orchestration — building computation graphs, dispatching operations, and managing control flow — while actual matrix multiplications, convolutions, and reductions run entirely in native code. Understanding where the JVM boundary sits is the key to writing fast Clorch programs and diagnosing unexpected slowdowns or memory growth.

​
Performance Design

​
Native C++ Execution

When you call (t/add tensor 1.0), Clorch invokes a native LibTorch operation through JavaCPP’s JNI bridge. The JVM does not touch the tensor data; the operation executes in the same C++ runtime that backs PyTorch. This means:
  • Matrix multiplications on CPU use BLAS/MKL/OpenBLAS (whichever LibTorch is linked against).
  • CUDA operations execute asynchronously on GPU kernels.
  • Autograd graph construction is handled in C++.
The JVM overhead per operation is a small, fixed JNI call cost — negligible when operations involve large tensors.

​
Vectorized Operations vs Clojure Loops

The most common performance mistake is iterating over tensor elements in Clojure instead of using vectorized tensor operations. Each iteration of a Clojure loop that touches individual tensor elements crosses the JNI boundary, defeating the purpose of native execution. 
;; FAST: a single C++ call, regardless of tensor size
(t/add tensor 1.0)

;; SLOW: crosses the JNI boundary for every element
(mapv #(+ % 1.0) (t/tseq tensor))
For a tensor with one million elements, the vectorized form is typically thousands of times faster. Always express computation as tensor operations rather than element-wise Clojure transformations.
When you find yourself writing mapv, reduce, or doseq over t/tseq, look for a tensor operation that achieves the same result. The clorch.torch and clorch.nn.functional namespaces cover arithmetic, reductions, activations, pooling, normalization, and most common operations.

​
Contiguous Tensors and view

Some shape manipulation operations like t/view require the tensor to be contiguous in memory. After operations like t/transpose or t/ix (slicing), a tensor may become non-contiguous — its elements are valid but laid out with non-unit strides. 
;; This may fail after a transpose
(t/view (t/transpose tensor 0 1) [batch-size -1])

;; Fix: make contiguous first
(-> tensor
    (t/transpose 0 1)
    (.contiguous)
    (t/view [batch-size -1]))
If you encounter an error like "input tensor must be contiguous", add .contiguous before the view call.

​
Using autograd/no-grad for Inference Speed

During inference, the autograd engine builds a computation graph to support backward. When you only need forward-pass results, disable graph construction with autograd/no-grad. This eliminates significant memory allocation and overhead. 
(require '[clorch.autograd :as autograd])

;; SLOW for inference: builds autograd graph unnecessarily
(nn/forward model x)

;; FAST for inference: no graph, lower memory, higher throughput
(autograd/no-grad
  (nn/forward model x))
nn/generate wraps its entire generation loop in autograd/no-grad automatically. Wrap your own evaluation and generation code similarly.

​
JNI Overhead Guidance

Each JNI call has a small but non-zero fixed cost (typically tens of microseconds). This cost is negligible when amortized over large tensors, but it compounds when you make many small calls in a tight loop.

​
Batch Your Operations

;; High overhead: N separate JNI calls
(doseq [i (range n)]
  (t/add (t/ix tensor i) 1.0))

;; Low overhead: one JNI call for the whole tensor
(t/add tensor 1.0)

​
Keep Long-Lived Objects Outside Loops

Models, optimizers, and large tensors that persist across iterations should be created once outside the training loop. Creating them inside with-torch scopes or loop bodies triggers unnecessary native allocation and deallocation. 
;; Correct: model created once
(def model (nn/to my-model :cuda))
(def optimizer (optim/adamw (nn/parameters model) :lr 3e-4))

(doseq [batch dataloader]
  (t/with-torch
    ;; Only per-batch tensors are allocated here
    (train-step model optimizer batch)))

​
Memory Profiling

LibTorch allocates tensors in native (non-JVM) memory. The JVM garbage collector cannot see or apply pressure to this memory. RSS (Resident Set Size) is the correct metric for tracking total process memory — it includes both JVM heap and native allocations.

​
Using the Built-in Profiler Script

Clorch includes a profiling script that monitors JVM heap and native RSS simultaneously: 
clojure -M test/profiler.clj
This script runs a workload and prints heap and RSS metrics at each step, making it easy to spot trends.

​
Manual RSS Monitoring

Use ps to track the RSS of your Clojure process: 
# Get PID of the running process
pgrep -f "clojure"

# Poll RSS every second
watch -n 1 "ps -o rss -p <PID>"
Alternatively, use /proc/<PID>/status for more detail: 
grep VmRSS /proc/<PID>/status

​
Diagnosing RSS Growth

If RSS grows steadily while the JVM heap remains stable, you likely have a native memory leak. The two most common causes:

​
Missing with-torch Scope

Tensors allocated inside a training loop accumulate if no with-torch scope releases them. Wrap each batch’s allocations: 
;; Each iteration releases intermediate tensors when the scope exits
(doseq [batch dataloader]
  (let [loss-value
        (t/with-torch
          (let [loss (train-step model optimizer batch)]
            (t/item-float loss)))]
    (println "Loss:" loss-value)))
Return only JVM scalars (numbers, booleans, strings) from with-torch. If a tensor must escape the scope, call t/retain! on it explicitly before the scope closes, then t/release! when you are done with it.

​
Leaked Native Pointers

Long-lived REPL sessions can accumulate tensors bound to Vars or atoms that are never released. Periodically check that your atoms and def values do not hold references to tensors that should have been freed.
If RSS grows indefinitely across training epochs while the JVM heap remains flat, the most likely cause is a missing with-torch scope around per-batch allocation. Add the scope and verify that RSS stabilizes after the first epoch.

​
Performance Checklist

ConcernCheck
VectorizationAre you using tensor ops instead of element-wise Clojure loops?
No-grad at inferenceIs every evaluation loop wrapped in autograd/no-grad?
Contiguous memoryDo you call .contiguous before view after transpose/slice?
Long-lived objectsAre models and optimizers defined outside the training loop?
Batch operationsAre you grouping small tensor calls rather than calling one at a time?
Memory scopesIs each batch’s allocation wrapped in t/with-torch?
RSS stabilityDoes RSS stabilize after the first epoch in a multi-epoch run?

Build docs developers (and LLMs) love

Get started for freeTalk to us