Skip to main content

Overview

The Content Judge is a safety layer that screens user queries before scene generation. It uses an LLM to classify queries as safe, sensitive, or harmful, helping maintain educational standards while allowing historically significant but mature content.
The Judge is always available but requires an OPENROUTER_API_KEY to function. Queries are blocked if the Judge is unavailable.

ContentJudge Class

Configuration

Environment Variables

Screening Queries

The primary method is screen(), which returns a verdict:

Return Values

Moderation Prompt

The Judge uses a carefully designed prompt:

Design Principles

  1. Educational first: Prioritize historical significance over comfort
  2. Context-aware: Distinguish between educational and exploitative content
  3. Transparent: Always provide a reason for the verdict

Example Classifications

Approve

Sensitive

Reject

Integration with JobManager

The Judge is typically invoked at the start of scene generation:

Response Parsing

The Judge handles markdown-wrapped JSON:
If JSON parsing fails or the LLM returns an invalid verdict, the Judge defaults to "reject" for safety.

Logging

All screening decisions are logged:
Example log output:

Error Handling

Fail-Safe Behavior

The Judge always defaults to rejection on errors:
  • API failures
  • Timeout
  • Invalid JSON
  • Unknown verdict values
This ensures the system errs on the side of caution.

Performance

  • Timeout: 60 seconds per query
  • Model: Gemini 2.0 Flash (fast inference)
  • Cost: ~0.0001perquery(1000requests=0.0001 per query (1000 requests = 0.10)
Screening adds ~1-2 seconds of latency to each generation request. This is acceptable for the safety benefits.

Customization

To use a different model or adjust the prompt:
For stricter moderation, modify JUDGE_PROMPT in app/workers/judge.py to adjust classification criteria.