Stable Audio
Stable Audio generates high-quality music and sound effects from text descriptions.
Architecture
Model Configuration:- Audio model: dit1.0
- Latent format: StableAudio1 (64 channels)
- Sigma range: 0.03 to 500.0
- Sample rate: 44.1kHz
- DiffusionModel: DiT-based audio generator
- VAE: Audio compression/decompression
- Text Encoder: T5-based (SAT5 variant)
- Seconds Embedders: Start time and duration conditioning
Text Encoder
Model: SAT5 (Stable Audio T5)- Tokenizer: SAT5Tokenizer
- Purpose: Encodes text prompts for audio generation
Usage
Creating Empty Latent:- Channels: 64
- Length: Based on duration (44.1kHz / 2048 compression)
- Device: Intermediate device (for memory efficiency)
Audio Workflow Nodes
Loading Audio:Audio Processing
Trimming:Model Files Location
Examples
ACE Step
Architecture
Model Configuration:- Audio model: ace
- Latent format: ACEAudio
- Shift: 3.0
- Memory usage factor: 0.5
- Supported dtypes: BF16, FP32
- Sample rate: 44.1kHz (ACE Step 1.0) or 48kHz (ACE Step 1.5)
ACE Step 1.0
Text Encoder:- Custom ACE T5 model
- Tokenizer: AceT5Tokenizer
- Shape: [batch, 8, 16, length]
- Length: int(seconds * 44100 / 512 / 8)
ACE Step 1.5
Enhanced Features:- Higher sample rate (48kHz)
- More musical controls
- Audio reference support
- LLM-based audio code generation
- Shape: [batch, 64, length]
- Length: round(seconds * 48000 / 1920)
- en (English), ja (Japanese), zh (Chinese), es (Spanish)
- de (German), fr (French), pt (Portuguese), ru (Russian)
- it (Italian), nl (Dutch), pl (Polish), tr (Turkish)
- vi (Vietnamese), cs (Czech), fa (Persian), id (Indonesian)
- ko (Korean), uk (Ukrainian), hu (Hungarian), ar (Arabic)
- sv (Swedish), ro (Romanian), el (Greek)
- All major and minor keys
- Format: “[C/C#/D/Eb/E/F/F#/G/Ab/A/Bb/B] [major/minor]”
Usage Tips
ACE Step 1.0:- Use descriptive tags (genre, mood, instruments)
- Lyrics are optional but improve structure
- Adjust lyrics_strength to balance music vs. vocals
- Set BPM and time signature for rhythmic coherence
- Choose appropriate key scale for mood
- Enable generate_audio_codes for best quality
- Disable audio codes when using reference audio
- Adjust temperature for more/less variation
- Use CFG scale to control prompt adherence
Model Files Location
Examples
Audio Format Details
Waveform Tensor Format
Structure:- Mono: channels = 1
- Stereo: channels = 2
- 44.1kHz: Standard audio CD quality (Stable Audio, ACE 1.0)
- 48kHz: Professional audio standard (ACE 1.5)
Latent Compression
Stable Audio:- Temporal compression: ~2048x (44.1kHz → ~21.5Hz latent rate)
- Channel expansion: 1-2 channels → 64 latent channels
- Temporal compression: 512 * 8 = 4096x
- Spatial/channel layout: [8, 16, length]
- Temporal compression: 1920x (48kHz → 25Hz latent rate)
- Channel expansion: 1-2 channels → 64 latent channels
Performance Optimization
Memory Management
For Long Audio:- Use
VAEDecodeAudioTiledinstead ofVAEDecodeAudio - Adjust tile_size and overlap based on VRAM
- Typical settings: tile_size=512, overlap=64
- Stable Audio (47 seconds): ~4-6GB
- ACE Step 1.0 (120 seconds): ~6-8GB
- ACE Step 1.5 (120 seconds): ~8-10GB
Speed Optimization
Preprocessing:- Use FP16/BF16 for VAE when possible
- Enable
--fp16-vaeflag
- Start with shorter durations for testing
- Use fewer sampling steps initially
- ACE 1.5: Disable generate_audio_codes for faster preview
Quality Settings
Export Formats: Highest Quality:- FLAC lossless
- Full sample rate (44.1/48kHz)
- MP3 V0 (variable bitrate, ~245kbps)
- Opus 192k
- MP3 128k
- Opus 96k or 128k
- ComfyUI automatically normalizes audio to prevent clipping
- Output is divided by std * 5.0 (capped at 1.0)
Advanced Workflows
Music Production Pipeline
- Generate base track (ACE Step with tags and BPM)
- Trim to exact length (TrimAudioDuration)
- Adjust levels (AudioAdjustVolume)
- Apply EQ (AudioEqualizer3Band)
- Export (SaveAudio with FLAC or high-quality MP3)
Sound Design
- Generate variations (batch_size > 1)
- Mix multiple outputs (AudioMerge with “add” or “mean”)
- Layer effects (AudioConcat)
- Process channels separately (SplitAudioChannels → process → JoinAudioChannels)
Podcast/Voice Workflow
- Record or load voice (RecordAudio or LoadAudio)
- Trim silence (TrimAudioDuration)
- Normalize volume (AudioAdjustVolume)
- Add music bed (ACE Step + AudioMerge)
- Export for web (SaveAudioOpus)