Skip to main content
POST
Smart Captions (Manual Sessions)
This endpoint generates AI-powered captions for manual session slides with full narrative context. Unlike session playback captions, these support longer formats (50-400 characters), custom instructions, and maintain continuity with surrounding slides.

Authentication

Requires user authentication via session.

Credit Cost

Feature Key: smart_caption
Default Cost: 10 credits per caption
Credits are deducted after successful caption generation using best-effort async deduction. If the deduction fails, the caption is still returned to the user.

Request

string
required
The image reference for caption generation. Supports:
  • Data URLs (data:image/jpeg;base64,...)
  • HTTP/HTTPS URLs (public image URLs)
Security:
  • Localhost and private IP ranges are blocked (SSRF protection)
  • Maximum size: 20MB (estimated for data URLs)
  • Only static images (JPEG, PNG, WebP)
object
required
Narrative context for caption generation.
string
The title of the manual session.
number
required
The 0-based index of the current slide.
number
required
The total number of slides in the session.
string[]
Array of captions from previous slides (up to last 3). Used to maintain narrative flow.
string[]
Array of captions from upcoming slides (up to next 3). Used to avoid contradictions.
string
Optional creator guidance/instructions to influence the caption style or content.
string
Session mode indicator:
  • new_session - Writing fresh copy for new session
  • existing_session - Continuing established narrative

Request Example

Response

string
The generated caption text (50-400 characters).Characteristics:
  • Maintains narrative continuity with previous/next slides
  • Incorporates user steering when provided
  • Respects session stage (new vs. existing)
  • Uses second-person imperative mood for JOI
  • Avoids meta-references (no “slide X” mentions)

Success Response

Error Responses

string
Error message describing what went wrong.
boolean
Set to true if OPENROUTER_API_KEY is not configured.
string
Error code for programmatic handling:
  • MODEL_IMAGE_UNSUPPORTED - Selected model doesn’t support images

Error Examples

Implementation Details

OpenRouter Integration

Model Selection:
  • Primary: google/gemini-2.5-pro (configurable via OPENROUTER_MODEL_ID)
  • Fallback: google/gemini-2.5-flash-lite
  • Both use Gemini safety_settings with BLOCK_NONE thresholds
Retry Logic:
  • 2 attempts max
  • Retries on CONTENT_POLICY_REJECTION and EMPTY_RESPONSE_CONTENT_FILTERED
  • 400ms delay between retries

Prompt Engineering

Priority Order:
  1. Narrative continuity (surrounding captions + session mode)
  2. Session creator guidance (userSteering)
  3. Image context (light consistency anchor only)
Instruction Template:
Generation Parameters:

SSRF Protection

Blocked Hosts:
  • localhost, 127.0.0.1, ::1, 0.0.0.0
  • Private IP ranges: 10.x.x.x, 192.168.x.x, 172.16-31.x.x
  • Link-local: 169.254.x.x
Allowed Protocols:
  • http: and https: only
  • Data URLs validated for size (20MB max)

Image Validation

Supported:
  • JPEG (image/jpeg)
  • PNG (image/png)
  • WebP (image/webp)
Rejected:
  • Animated GIFs (.gif, image/gif)
  • Videos (.mp4, .webm, video/*)
  • Files over 20MB

Credit Deduction

Timing:
  • Credits checked before generation via requireCredits middleware
  • Deduction happens after response sent (best-effort async)
  • If deduction fails, caption is still returned to user
Failure Handling:
Pricing Configuration:
  • Feature key: smart_caption
  • Default: 10 credits
  • Admin-configurable via /api/admin/pricing

Usage Notes

Manual Editor Only: This endpoint is specifically designed for the Manual Session Editor. For automated session playback, use /api/generate-caption which is free and optimized for short captions.
Narrative Context: Providing previousCaptions and nextCaptions significantly improves caption quality by maintaining narrative flow. Always include the last 3 and next 3 captions when available.
User Steering: The userSteering parameter is treated as high-priority preference. Be specific about tone, structure, or themes you want incorporated.
Session Stage: Use new_session when creating fresh sessions to set coherent narrative tone. Use existing_session when editing imported sessions to align with established voice.