Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/anil-matcha/open-generative-ai/llms.txt

Use this file to discover all available pages before exploring further.

The processLipSync function generates audio-driven video output using one of 9 dedicated lip sync models. It supports two input modes: portrait image mode (portrait image + audio → animated talking video) and video mode (existing video + audio → lip-synced video). The function uses the same submit-and-poll pattern as other Muapi.ai generation functions, with a 900-attempt polling window to accommodate longer video processing times.

​
processLipSync(apiKey, params)

​
apiKey
string
required
Your Muapi.ai API key.
​
params.model
string
required
Lip sync model ID. Available values:
  • infinitetalk-image-to-video — portrait image + audio
  • wan2.2-speech-to-video — portrait image + audio
  • ltx-2.3-lipsync — portrait image + audio, supports 480p/720p/1080p
  • ltx-2-19b-lipsync — portrait image + audio, supports 480p/720p/1080p
  • sync-lipsync — video + audio
  • latentsync-video — video + audio
  • creatify-lipsync — video + audio
  • veed-lipsync — video + audio
  • infinitetalk-video-to-video — video + audio
​
params.audio_url
string
required
URL of the audio file to drive the lip sync animation. Supported formats: MP3, WAV, M4A.
​
params.image_url
string
URL of the portrait image to animate. Required for image-based models (infinitetalk-image-to-video, wan2.2-speech-to-video, ltx-2.3-lipsync, ltx-2-19b-lipsync).
​
params.video_url
string
URL of the source video to apply lip sync to. Required for video-based models (sync-lipsync, latentsync-video, creatify-lipsync, veed-lipsync, infinitetalk-video-to-video).
​
params.prompt
string
Optional prompt to guide motion style. Forwarded when the model’s hasPrompt flag is enabled.
​
params.resolution
string
Output resolution for models that support it: 480p, 720p, or 1080p. LTX models support all three; Infinite Talk and Wan 2.2 Speech support 480p and 720p.
​
params.seed
number
Seed for reproducible generation. Pass -1 or omit for random.
​
params.onRequestId
(requestId: string) => void
Optional callback invoked after submission with the request_id.
Returns: Promise<{ url: string, outputs: string[], status: string }> The url field is the generated lip sync video URL.

​
Examples

Portrait image mode (image + audio → talking video): 
import { processLipSync } from 'studio';

const result = await processLipSync('your-api-key', {
  model: 'ltx-2.3-lipsync',
  image_url: 'https://example.com/portrait.jpg',
  audio_url: 'https://example.com/speech.mp3',
  resolution: '720p',
  prompt: 'natural head movement, subtle expression'
});

console.log(result.url); // URL of the generated talking video
Video mode (existing video + audio → lip-synced video): 
import { processLipSync } from 'studio';

const result = await processLipSync('your-api-key', {
  model: 'sync-lipsync',
  video_url: 'https://example.com/source-video.mp4',
  audio_url: 'https://example.com/new-audio.mp3',
  onRequestId: (id) => console.log('Request ID:', id)
});

console.log(result.url);
High-resolution portrait with LTX 19B: 
const result = await processLipSync('your-api-key', {
  model: 'ltx-2-19b-lipsync',
  image_url: 'https://example.com/portrait-hd.jpg',
  audio_url: 'https://example.com/narration.wav',
  resolution: '1080p'
});

​
Model Capabilities

ModelInputResolutionsPrompt
Infinite TalkPortrait image + audio480p, 720pOptional
Wan 2.2 Speech to VideoPortrait image + audio480p, 720pOptional
LTX 2.3 LipsyncPortrait image + audio480p, 720p, 1080pOptional
LTX 2 19B LipsyncPortrait image + audio480p, 720p, 1080pOptional
Sync LipsyncVideo + audio——
LatentSyncVideo + audio——
Creatify LipsyncVideo + audio——
Veed LipsyncVideo + audio——
Infinite Talk V2VVideo + audio480p, 720pOptional
For best results with portrait image mode, use a front-facing portrait with a clear face and neutral background. For video mode, ensure the source video has a visible speaking face for the model to track.

Build docs developers (and LLMs) love

Get started for freeTalk to us