Skip to main content

Language-Assisted Feature Transformation

Transform visual features using natural language guidance for enhanced anomaly detection. LAFT leverages vision-language models to guide feature transformations through concept subspaces.

Get StartedAPI Reference

Quick Start

Get up and running with LAFT in minutes

1

Install LAFT

Install LAFT and its dependencies using pip:
pip install torch torchvision open_clip_torch scikit-learn
Clone the repository and install requirements:
git clone https://github.com/yuneg11/LAFT.git
cd LAFT
pip install -r requirements.txt
2

Load a CLIP model

Import LAFT and load a pre-trained CLIP model:
import laft
import torch

torch.set_grad_enabled(False)  # Disable autograd to prevent OOM

# Load CLIP model and preprocessing transform
model, transform = laft.load_clip("ViT-B-16-quickgelu:dfn2b")
3

Define concept prompts

Create text prompts that describe the concept you want to guide or ignore:
# Get prompts for a specific dataset and guidance type
prompts = laft.prompts.get_prompts("color_mnist", "number")

# Encode text prompts into features
text_features = model.encode_text(prompts["all"])
4

Transform image features

Apply language-assisted transformation to your image features:
# Encode images
image_features = model.encode_image(images)

# Construct concept subspace from prompt pairs
pair_diffs = laft.prompt_pair(text_features)
concept_basis = laft.pca(pair_diffs, n_components=24)

# Transform features
guided_features = laft.inner(image_features, concept_basis)
ignored_features = laft.orthogonal(image_features, concept_basis)
The inner() function projects features onto the concept subspace, while orthogonal() removes the concept’s influence.

Key Features

Powerful tools for language-guided anomaly detection

Language-Guided Transformations

Use natural language to guide feature transformations through CLIP embeddings and concept subspaces

Concept Subspace Construction

Build robust concept representations using PCA on prompt pair differences

Flexible Projections

Project features onto or orthogonal to concept subspaces with inner() and orthogonal() operations

Comprehensive Metrics

Evaluate anomaly detection with AUROC, AUPRC, FPR95, and optimal threshold selection

Multi-Domain Datasets

Built-in support for semantic (MNIST, Waterbirds, CelebA) and industrial datasets (MVTec AD, VisA)

Prompt Engineering

Pre-configured prompt templates and utilities for different guidance types

Explore Documentation

Learn more about LAFT’s capabilities

Core Concepts

Understand how LAFT works under the hood

Datasets

Work with semantic and industrial anomaly datasets

Evaluation

Measure and compare anomaly detection performance

Ready to Get Started?

Follow our quickstart guide to build your first language-assisted anomaly detector in minutes

Get Started Now