LECO: Concept Erasing

LECO (Low-rank adaptation for Erasing COncepts from diffusion models) trains a LoRA that modifies what a diffusion model generates — without any training images. All training signals come from the difference between the model’s own noise predictions on different text prompts.

LECO is well suited for safety and content filtering use cases. You can erase unwanted styles, subjects, or content from a model’s outputs without touching any training data or model weights directly. The resulting LoRA can be selectively applied or removed at inference time.

What LECO can do

Concept erasing

Remove a specific style or concept from generated images. Example: erase the “van gogh” painting style so the model stops producing Van Gogh-like outputs.

Concept enhancing

Strengthen a specific attribute. Example: make “detailed” more pronounced so the model consistently produces higher-detail outputs when prompted.

Slider LoRA

Create a bidirectional attribute slider. Example: a LoRA where positive weights produce “long hair” and negative weights produce “short hair”.

LECO vs standard LoRA

	Standard LoRA	LECO
Training data	Image dataset required	No images needed
Configuration	Dataset TOML	Prompt TOML
Training target	U-Net and/or Text Encoder	U-Net only
Training unit	Epochs and steps	Steps only
Checkpointing	Per-epoch or per-step	Per-step only (`--save_every_n_steps`)

Scripts

Script	Models
`train_leco.py`	Stable Diffusion 1.x and 2.x
`sdxl_train_leco.py`	Stable Diffusion XL

Prompt TOML configuration

Instead of a dataset TOML, LECO uses a prompt TOML file that defines the concepts and directions to train. Two formats are supported.

Format 1: Original LECO format

Use [[prompts]] sections for direct control over each training pair:

[[prompts]]
target = "van gogh"
positive = "van gogh"
unconditional = ""
neutral = ""
action = "erase"
guidance_scale = 1.0
resolution = 512
batch_size = 1
multiplier = 1.0
weight = 1.0

Field	Required	Default	Description
`target`	Yes	—	The concept modified by the LoRA
`positive`	No	same as `target`	The “positive direction” prompt for building the training target
`unconditional`	No	`""`	Unconditional / negative prompt
`neutral`	No	`""`	Neutral baseline prompt
`action`	No	`"erase"`	`"erase"` to remove the concept; `"enhance"` to strengthen it
`guidance_scale`	No	`1.0`	Scale for target construction — higher = stronger effect
`resolution`	No	`512`	Training resolution (int or `[height, width]`)
`batch_size`	No	`1`	Latent samples per training step for this prompt
`multiplier`	No	`1.0`	LoRA strength multiplier during training
`weight`	No	`1.0`	Loss weight for this prompt pair

Format 2: Slider target format (ai-toolkit style)

Use [[targets]] sections to define slider-style LoRAs. Each target automatically expands into bidirectional training pairs:

guidance_scale = 1.0
resolution = 1024
neutral = ""

[[targets]]
target_class = "1girl"
positive = "1girl, long hair"
negative = "1girl, short hair"
multiplier = 1.0
weight = 1.0

Top-level fields (guidance_scale, resolution, neutral, batch_size) serve as defaults for all targets.

Field	Required	Description
`target_class`	Yes	The base class/subject prompt
`positive`	No*	Prompt for the positive direction of the slider
`negative`	No*	Prompt for the negative direction of the slider
`multiplier`	No	LoRA strength multiplier
`weight`	No	Loss weight

* At least one of positive or negative must be provided.

Multiple neutral prompts

You can specify multiple neutral prompts to improve generalization across different contexts:

guidance_scale = 1.5
resolution = 1024
neutrals = ["", "photo of a person", "cinematic portrait"]

[[targets]]
target_class = "person"
positive = "smiling person"
negative = "expressionless person"

You can also load neutrals from a text file (one prompt per line):

neutral_prompt_file = "neutrals.txt"

[[targets]]
target_class = ""
positive = "high detail"
negative = "low detail"

Three use case examples

1. Concept erasing — remove Van Gogh style

Erase a specific artistic style from the model’s outputs by training against its own predictions of that style.

[[prompts]]
target = "van gogh"
positive = "van gogh"
unconditional = ""
neutral = ""
action = "erase"
guidance_scale = 1.5
resolution = 512
batch_size = 2
multiplier = 1.0
weight = 1.0

After training, applying this LoRA reduces the model’s tendency to produce Van Gogh-like brushwork and color palettes even when prompted.

2. Concept enhancing — strengthen 'detailed'

Strengthen an attribute so the model more consistently applies it.

[[prompts]]
target = "detailed"
positive = "highly detailed, intricate details"
unconditional = ""
neutral = ""
action = "enhance"
guidance_scale = 2.0
resolution = 512
batch_size = 2
multiplier = 1.0
weight = 1.0

After training, applying this LoRA with a positive weight makes the model produce more detailed outputs.

3. Slider LoRA — short hair ↔ long hair

Create a bidirectional slider between two attributes. Apply positive weights to go one direction, negative weights to go the other.

guidance_scale = 1.0
resolution = 512

[[targets]]
target_class = "1girl"
positive = "1girl, long hair"
negative = "1girl, short hair"
multiplier = 1.0
weight = 1.0

At inference, use weight 1.0 for long hair and weight -1.0 for short hair.

Key arguments

LECO-specific arguments

Argument	Description
`--prompts_file`	Required. Path to the prompt TOML configuration file
`--max_denoising_steps`	Number of partial denoising steps per training iteration. Default: `40`
`--leco_denoise_guidance_scale`	CFG guidance scale during the denoising pass. Default: `3.0`

Understanding the two guidance_scale parameters

LECO has two separate guidance scale controls:

--leco_denoise_guidance_scale (command-line): Controls CFG strength during the partial denoising pass that generates intermediate latents. Higher values produce more prompt-adherent latents for the training signal.
guidance_scale (in the TOML file): Controls the magnitude of the concept offset when constructing the training target. Higher values produce a stronger erase/enhance effect. This can be set per-prompt or per-target.

If your results are too subtle, increase the TOML guidance_scale (try 1.5 to 3.0).

LoRA network arguments

Argument	Description
`--network_dim`	LoRA rank. Typical range: `4`–`16`. Higher = more expressive but larger file.
`--network_alpha`	LoRA alpha for learning rate scaling. Common: half of `--network_dim`.
`--network_dropout`	Dropout rate for LoRA layers (optional).
`--network_weights`	Load pretrained LoRA weights to resume training.

Training parameters

Argument	Description
`--max_train_steps`	Total steps. Range for LECO: `300`–`2000`. `--max_train_epochs` is not supported.
`--learning_rate`	Typical range: `1e-4` to `1e-3`.
`--optimizer_type`	`AdamW8bit`, `AdamW`, `Lion`, `Adafactor`, etc.
`--save_every_n_steps`	Save intermediate checkpoints every N steps. `--save_every_n_epochs` is not supported.
`--gradient_checkpointing`	Reduces VRAM usage. Recommended, especially at higher resolutions.
`--sdpa`	Use Scaled Dot-Product Attention. Recommended.

Recommended starting settings

Parameter	SD 1.x / 2.x	SDXL
`--network_dim`	`4`–`8`	`8`–`16`
`--learning_rate`	`1e-4`	`1e-4`
`--max_train_steps`	`300`–`1000`	`500`–`2000`
`resolution` (TOML)	`512`	`1024`
`guidance_scale` (TOML)	`1.0`–`2.0`	`1.0`–`3.0`
`batch_size` (TOML)	`1`–`4`	`1`–`4`

Training commands

SD 1.x / 2.x
SDXL

accelerate launch --mixed_precision bf16 train_leco.py \
  --pretrained_model_name_or_path="model.safetensors" \
  --prompts_file="prompts.toml" \
  --output_dir="output" \
  --output_name="my_leco" \
  --network_dim=8 \
  --network_alpha=4 \
  --learning_rate=1e-4 \
  --optimizer_type="AdamW8bit" \
  --max_train_steps=500 \
  --max_denoising_steps=40 \
  --mixed_precision=bf16 \
  --sdpa \
  --gradient_checkpointing \
  --save_every_n_steps=100

accelerate launch --mixed_precision bf16 sdxl_train_leco.py \
  --pretrained_model_name_or_path="sdxl_model.safetensors" \
  --prompts_file="slider.toml" \
  --output_dir="output" \
  --output_name="my_sdxl_slider" \
  --network_dim=8 \
  --network_alpha=4 \
  --learning_rate=1e-4 \
  --optimizer_type="AdamW8bit" \
  --max_train_steps=1000 \
  --max_denoising_steps=40 \
  --mixed_precision=bf16 \
  --sdpa \
  --gradient_checkpointing \
  --save_every_n_steps=200

Using the trained model

The trained LoRA .safetensors file is saved to --output_dir. Load it in AUTOMATIC1111, ComfyUI, or any Diffusers-based tool just like a standard LoRA. For slider LoRAs:

Apply a positive weight (e.g., 0.5 to 1.5) to move in the positive direction.
Apply a negative weight (e.g., -0.5 to -1.5) to move in the negative direction.

Tuning the effect strength

If the trained LoRA has a weak or unnoticeable effect:

Increase guidance_scale in the TOML (e.g., from 1.0 to 2.0). This is the most direct adjustment.
Increase multiplier in the TOML (e.g., from 1.0 to 2.0).
Increase --max_denoising_steps for more refined intermediate latents.
Increase --max_train_steps to train longer.
Apply the LoRA with a higher weight at inference time.

Getting Started

Dataset Preparation

LoRA Training

Fine-tuning & Other Methods

Inference & Utilities

LECO: Concept Erasing

What LECO can do

Concept erasing

Concept enhancing

Slider LoRA

LECO vs standard LoRA

Scripts

Prompt TOML configuration

Format 1: Original LECO format

Format 2: Slider target format (ai-toolkit style)

Multiple neutral prompts

Three use case examples

Key arguments

Recommended starting settings

Training commands

Using the trained model

Tuning the effect strength

​What LECO can do

Concept erasing

Concept enhancing

Slider LoRA

​LECO vs standard LoRA

​Scripts

​Prompt TOML configuration

​Format 1: Original LECO format

​Format 2: Slider target format (ai-toolkit style)

​Multiple neutral prompts

​Three use case examples

​Key arguments

​Recommended starting settings

​Training commands

​Using the trained model

​Tuning the effect strength

What LECO can do

LECO vs standard LoRA

Scripts

Prompt TOML configuration

Format 1: Original LECO format

Format 2: Slider target format (ai-toolkit style)

Multiple neutral prompts

Three use case examples

Key arguments

Recommended starting settings

Training commands

Using the trained model

Tuning the effect strength