BioScan Museo TTS WebSocket API: QR Detection Stream

The WebSocket endpoint ws://localhost:8010/ws/qr-stream provides a persistent, low-latency channel for streaming JPEG frames from a camera and receiving species information the moment a QR code is recognised. Unlike the HTTP frame endpoints, the connection stays open across multiple frames — the server simply discards frames where no QR is detected and closes cleanly after the first successful detection.

WS /ws/qr-stream

Protocol: WebSocket (RFC 6455)
Authentication: ?key=<TTS_API_KEY> query parameter — validated immediately after the connection is accepted, before any frame processing begins.

Connection URL

ws://localhost:8010/ws/qr-stream?key=<TTS_API_KEY>&style=<style>

Query Parameters

key

string

required

Your TTS_API_KEY. Checked before the first frame is processed. An invalid or missing key triggers an error message followed by a close with code 1008 (Policy Violation).

style

string

Narration style for the text field in the qr_found message. One of ficha, narrativo, corto. Defaults to ficha. An invalid value triggers an error message and close 1011.

Frame Format

After the connection is established, the client sends binary messages containing raw JPEG bytes. Each message must be a single complete JPEG image — partial or multi-frame messages are not supported. The server runs the same four-stage QR detection pipeline used by the HTTP endpoints:

Color — detection on original BGR frame
Grayscale — single-channel conversion
Otsu threshold — binarisation
2× upscale — cubic interpolation on grayscale

Frames that yield no QR code are silently skipped; the server keeps receiving the next binary message.

Every frame received via the WebSocket is saved to DEBUG_FRAMES_DIR with a timestamped filename (e.g. incoming_ws_1712345678.jpg). The last received frame is also always accessible at /debug/last-frame. Clear or rotate DEBUG_FRAMES_DIR periodically in production to avoid unbounded disk usage.

Server → Client Messages

The server only ever sends JSON text messages. There are two possible message types.

`error`

Sent when authentication fails or an unhandled exception occurs during frame processing. The connection is closed by the server immediately after sending this message.

{
  "type": "error",
  "detail": "unauthorized"
}

Close Code	Meaning
`1008`	Policy violation — authentication failure
`1011`	Internal error — unexpected exception during processing

`qr_found`

Sent once when a valid QR code is successfully detected and the species payload is resolved. After sending this message the server closes the connection cleanly with code 1000.

{
  "type": "qr_found",
  "found": true,
  "qr_id": "qr-condor-001",
  "species_id": "condor-001",
  "text": "Este animal llamado Cóndor Andino tiene como nombre científico Vultur gryphus...",
  "fields": {
    "species_id": "condor-001",
    "qr_id": "qr-condor-001",
    "common_name": "Cóndor Andino",
    "scientific_name": "Vultur gryphus",
    "description": "El cóndor andino es el ave voladora más grande del mundo.",
    "habitat": "los Andes de Sudamérica",
    "diet": "carroña",
    "curiosities": ["Puede planear más de 5 horas sin batir las alas"]
  },
  "decode_method": "grayscale"
}

type

string

Always "qr_found".

found

boolean

Always true in this message type.

qr_id

string

Raw QR code value decoded from the frame.

species_id

string

Canonical species ID resolved from the QR index.

text

string

Narration text for the species in the requested style.

fields

object

Full set of cleaned species fields: species_id, qr_id, common_name, scientific_name, description, habitat, diet, curiosities.

decode_method

string

Detection strategy that succeeded: color, grayscale, threshold_otsu, or upscaled_gray.

The WebSocket connection closes after the first successful QR detection. To scan a second exhibit, the client must open a new connection. This is by design — it prevents stale frame data from triggering duplicate detections after an exhibit has already been identified.

Connection Lifecycle

Client                              Server
  │                                   │
  │── WS Upgrade (key, style) ───────►│  Accept connection
  │                                   │  Validate key & style
  │◄── 1008 close (if auth fails) ────│
  │                                   │
  │── binary JPEG frame ─────────────►│  Run QR detection
  │                                   │  (no QR found — keep listening)
  │── binary JPEG frame ─────────────►│  Run QR detection
  │                                   │  QR found ✓
  │◄── { type: "qr_found", ... } ─────│
  │◄── close 1000 ────────────────────│
  │                                   │

JavaScript Client Example

const WS_URL = 'ws://localhost:8010/ws/qr-stream?key=YOUR_TTS_API_KEY&style=ficha';

const ws = new WebSocket(WS_URL);
ws.binaryType = 'arraybuffer';

ws.onopen = () => {
  console.log('Connected — streaming frames…');
  streamFrames();
};

ws.onmessage = (event) => {
  const msg = JSON.parse(event.data);

  if (msg.type === 'qr_found') {
    console.log('Species detected:', msg.species_id);
    console.log('Narration text:', msg.text);
    // Play pre-generated audio:
    const audioUrl = `/tts/by-qr/${msg.qr_id}?key=YOUR_TTS_API_KEY&style=ficha`;
    new Audio(audioUrl).play();
  }

  if (msg.type === 'error') {
    console.error('TTS WS error:', msg.detail);
  }
};

ws.onclose = (event) => {
  console.log(`Connection closed — code ${event.code}`);
  // Reconnect after a short delay for the next exhibit scan
  if (event.code === 1000) {
    setTimeout(() => reconnect(), 1500);
  }
};

// Send JPEG frames from a <video> element via canvas
function streamFrames() {
  const video = document.getElementById('camera');
  const canvas = document.createElement('canvas');
  canvas.width = video.videoWidth;
  canvas.height = video.videoHeight;
  const ctx = canvas.getContext('2d');

  const interval = setInterval(() => {
    if (ws.readyState !== WebSocket.OPEN) {
      clearInterval(interval);
      return;
    }
    ctx.drawImage(video, 0, 0);
    canvas.toBlob((blob) => {
      blob.arrayBuffer().then((buf) => ws.send(buf));
    }, 'image/jpeg', 0.85);
  }, 200); // send ~5 fps
}

Every JPEG frame sent to ws/qr-stream is persisted to DEBUG_FRAMES_DIR on disk. In a production deployment with continuous streaming this will consume significant storage. Set DEBUG_FRAMES_DIR to a tmpfs mount, add a periodic cleanup cron job, or reduce frame rate to minimise I/O. The /debug/last-frame HTTP endpoint always reflects the most recent frame regardless of the cleanup policy.

Error Reference

Close Code	`type`	`detail`	Cause
`1008`	`error`	`"unauthorized"`	`key` param missing or does not match `TTS_API_KEY`
`1011`	`error`	`"invalid_style"`	`style` param not one of `ficha`, `narrativo`, `corto`
`1011`	`error`	`"species_audio_not_found"`	QR detected but not in `_qr_index.json`
`1011`	`error`	`"species_meta_not_found"`	Index entry exists but `meta.json` is missing
`1011`	`error`	exception message	Unexpected server-side error during frame processing
`1000`	(no message after `qr_found`)	—	Server closes cleanly after the first successful QR detection

Flask App API

TTS Service API

BioScan Museo TTS WebSocket API: QR Detection Stream

WS /ws/qr-stream

Connection URL

Query Parameters

Frame Format

Server → Client Messages

`error`

`qr_found`

Connection Lifecycle

JavaScript Client Example

Error Reference

Build docs developers (and LLMs) love

Flask App API

TTS Service API

Documentation Index

​WS /ws/qr-stream

​Connection URL

​Query Parameters

​Frame Format

​Server → Client Messages

​error

​qr_found

​Connection Lifecycle

​JavaScript Client Example

​Error Reference

Build docs developers (and LLMs) love

WS /ws/qr-stream

Connection URL

Query Parameters

Frame Format

Server → Client Messages

`error`

`qr_found`

Connection Lifecycle

JavaScript Client Example

Error Reference