WannaCut Render Engine: Three.js Preview and Export Pipeline

WannaCut uses a unified Three.js WebGL render engine that powers both the real-time 10 FPS preview player and the final video export. There is no separate “offline renderer” — the same drawFrame function handles both paths. During export, an offscreen WebGLRenderer with a WebGLRenderTarget renders each frame at full project resolution, and the result is piped to Rust’s FFmpeg assembler. This page documents the renderBridge interface, the preview loop, the three-phase export pipeline, the pixel-space coordinate system, and the speed ramp math.

renderBridge Interface

renderBridge.tsx is the bridge between the React editor and the Three.js render engine. It exports two interfaces and two async functions.

`RenderEngineContext`

The context object passed to drawFrame on every preview tick or export frame render. It gives the render function read-only access to the current editor state without depending on React.

export interface RenderEngineContext {
  time: number;
  projectConfig: any;
  currentProjectPath: string;
  sceneRef: React.MutableRefObject<THREE.Scene | null>;
  rendererRef: React.MutableRefObject<THREE.WebGLRenderer | null>;
  cameraRef: React.MutableRefObject<THREE.OrthographicCamera | null>;
  topClips: React.MutableRefObject<any[]>;
  groupsRef: React.MutableRefObject<Map<string, THREE.Group>>;
  getInterpolatedValueWithFades: (time: number, clip: any, prop: string) => any;
  invoke: any;
  topAudios: React.MutableRefObject<any[]>;
  isPlaying?: boolean;
  settingsFolder?: string;
}

Field	Purpose
`time`	Current playhead time in seconds (composition time).
`projectConfig`	Project settings including `width`, `height`, `fps`, `backgroundColor`.
`currentProjectPath`	Absolute path to the project directory. Used to resolve asset paths.
`sceneRef`	The shared Three.js `Scene`. The render function adds/removes `THREE.Group` objects here.
`rendererRef`	The active `WebGLRenderer`. During preview this is the canvas renderer; during export it is the offscreen renderer.
`cameraRef`	The camera ref for the active renderer. Note: The interface type is declared as `THREE.OrthographicCamera	null`, but` App.tsx`actually instantiates a`THREE.PerspectiveCamera` (fov=45) in pixel-space coordinates. The interface type definition is inaccurate relative to runtime usage.
`topClips`	Clips currently visible at `time`, sorted by track render order (frontmost first). Updated at 10 FPS by `updatePreview()`.
`groupsRef`	Map from clip ID → `THREE.Group`. Allows the render function to update or remove scene objects by clip ID.
`getInterpolatedValueWithFades`	Keyframe interpolation function that evaluates opacity, volume, zoom, position, rotation3d, or speed at a given time for a given clip, including fade-in/out.
`invoke`	Tauri `invoke` reference, passed in so the render function can call `get_video_frame` or other commands without importing Tauri directly.
`topAudios`	Audio clips active at `time`. During export this is always empty (`[]`) to prevent audio initialization from interfering with the `OfflineAudioContext`.
`isPlaying`	`false` during export. The render function uses this to skip audio sync.
`settingsFolder`	Path to the WannaCut settings folder. Used by the render engine to resolve font paths.

`ExportOptions`

Options passed to exportVideo() when the user triggers an export.

export interface ExportOptions {
  targetPath: string;
  fps: number;
  projectConfig: { width: number; height: number };
  currentProjectPath: string;
  clips: any[];
  sceneRef: React.MutableRefObject<THREE.Scene | null>;
  rendererRef: React.MutableRefObject<THREE.WebGLRenderer | null>;
  cameraRef: React.MutableRefObject<any>;
  groupsRef: React.MutableRefObject<Map<string, THREE.Group>>;
  getInterpolatedValueWithFades: (time: number, clip: any, prop: string) => any;
  settingsFolder?: string;
  onProgress?: (percent: number) => void;
  onError?: (msg: string) => void;
}

Field	Purpose
`targetPath`	Absolute output path for the final MP4, chosen by the user via the native save dialog.
`fps`	Export frame rate. Must match `projectConfig.fps`.
`onProgress`	Callback receiving a `0–100` percent value as each phase completes. Drives the export progress bar in the UI.
`onError`	Callback receiving an error message string if any phase fails. Resets `renderStatus` to `'idle'`.

`getDrawFrameFunction()`

Asynchronously loads the render function from the private submodule at ./Render/Render. If the submodule is unavailable, it falls back to a no-op.

export async function getDrawFrameFunction() {
  try {
    return professionalDrawFrame;
  } catch (e) {
    console.warn("Error loading main engine.");
    return async (ctx: any) => { /* fallback no-op */ };
  }
}

This function is called once on mount in App.tsx and the result is stored in drawFrameEngine (useRef):

useEffect(() => {
  const loadEngine = async () => {
    const engine = await getDrawFrameFunction();
    drawFrameEngine.current = engine;
  };
  loadEngine();
}, []);

The actual draw implementation (professionalDrawFrame) lives in src/Render/Render.tsx, a private submodule not included in the public repository. The public renderBridge.tsx interface is stable and unchanged regardless of the render engine version.

Preview Rendering

The preview renders at a throttled 10 FPS to balance visual quality with GPU/CPU headroom during editing. The playhead itself moves at full requestAnimationFrame rate (≈60 Hz) via currentTimeRef, but drawFrame is only called when enough time has elapsed.

Preview Loop

const FPS_target    = 10;
const frameInterval = 1000 / FPS_target; // 100 ms

useEffect(() => {
  updatePreview(currentTime);  // 1. Filter clips visible at currentTime
  updateAudio();               // 2. Filter audio clips for HTMLAudioElement sync

  const now = performance.now();
  if (now - lastDrawTimeRef.current >= frameInterval) {
    newDrawFrame();            // 3. Call drawFrameEngine with RenderEngineContext
    lastDrawTimeRef.current = now;
  }
}, [currentTime, clips, drawFrameEngine.current]);

`updatePreview(currentTime)`

Filters the full clips array to find clips that are active at currentTime (i.e., clip.start <= time <= clip.start + clip.duration) and are not audio-only. The result is sorted by track render order (video tracks first, then audio; within type sorted by track ID) and stored in topClips.current.

`newDrawFrame(time?)`

Calls drawFrameEngine.current with a full RenderEngineContext:

const newDrawFrame = async (time: number | null = null) => {
  if (drawFrameEngine.current) {
    await drawFrameEngine.current({
      time: time ?? currentTime,
      projectConfig,
      currentProjectPath,
      sceneRef, rendererRef, cameraRef,
      topClips, groupsRef,
      getInterpolatedValueWithFades,
      invoke, settingsFolder,
      topAudios, isPlaying
    });
  }
};

When isPlaying is false (paused), newDrawFrame is still called on every scrub (seekTo) so the canvas always reflects the playhead position.

Export Pipeline

The export is a three-phase sequential process. Progress is reported from 0% to 100% via the onProgress callback, and the full pipeline is implemented in exportVideo() in renderBridge.tsx.

Phase 1: Video Frames  (0% → 70%)
┌──────────────────────────────────────────────────────────┐
│ offscreenRenderer = new THREE.WebGLRenderer(...)         │
│ renderTarget      = new THREE.WebGLRenderTarget(W, H)    │
│                                                          │
│ For each frame (0 .. totalFrames - 1):                   │
│   t = frameIdx / fps                                     │
│   offscreenRenderer.setRenderTarget(renderTarget)        │
│   drawFrame(RenderEngineContext { time: t, ... })        │
│   offscreenRenderer.render(scene, camera)                │
│   readRenderTargetPixels() → Uint8Array (RGBA)           │
│   flipVertical(buffer, W, H)  ← correct WebGL Y-flip    │
│   auxCtx.putImageData(...)                               │
│   invoke('save_export_frame', { frameIndex, pngBase64 }) │
│   onProgress( (frameIdx / totalFrames) * 70 )            │
└──────────────────────────────────────────────────────────┘

Phase 2: Audio Mix  (70% → 85%)
┌──────────────────────────────────────────────────────────┐
│ audioClips = clips.filter(non-muted video/audio clips)   │
│ renderAudioOffline(audioClips, duration, ...)            │
│   → OfflineAudioContext                                  │
│   → applies volume KFs, speed, audio effects            │
│   → saves mixed WAV to <projectPath>/export_audio.wav    │
│ onProgress(85)                                           │
└──────────────────────────────────────────────────────────┘

Phase 3: FFmpeg Assembly  (85% → 100%)
┌──────────────────────────────────────────────────────────┐
│ invoke('assemble_exported_video', {                      │
│   projectPath, targetPath,                               │
│   fps, duration, width, height                           │
│ })                                                       │
│ Rust: FFmpeg combines PNG sequence + WAV → MP4           │
│ Progress emitted via Tauri 'export-progress' events      │
│ onProgress(100)                                          │
└──────────────────────────────────────────────────────────┘

Phase 1 — Video Frame Rendering

A fresh THREE.WebGLRenderer and WebGLRenderTarget are created for the export (separate from the preview renderer to avoid disrupting the UI):

const offscreenRenderer = new THREE.WebGLRenderer({
  antialias: false, alpha: false,
  powerPreference: "high-performance",
  preserveDrawingBuffer: true,
});
offscreenRenderer.setSize(W, H, false);

const renderTarget = new THREE.WebGLRenderTarget(W, H, {
  format: THREE.RGBAFormat,
  type: THREE.UnsignedByteType,
  colorSpace: THREE.SRGBColorSpace,
});

For each frame, drawFrame is called with isPlaying: false (audio sync disabled) and topAudios: { current: [] } (empty audio list). After rendering, the pixel buffer is read back, vertically flipped, encoded as PNG via an auxiliary <canvas>, and sent to Rust via save_export_frame.

Phase 2 — Offline Audio Mixing

renderAudioOffline() is called with all non-muted audio/video clips and uses the Web Audio API’s OfflineAudioContext to produce the mixed audio WAV. It replicates the exact same audio effect chain used during preview (pitch, alien, microphone, volume keyframes, speed ramps), ensuring the exported audio is identical to what the user hears while editing. Using OfflineAudioContext means the mixing runs faster than real time — a 60-second timeline might complete Phase 2 in a few seconds.

Phase 3 — FFmpeg Assembly

The Rust assemble_exported_video command runs FFmpeg with the PNG sequence from <projectPath>/export_frames/ and the WAV from <projectPath>/export_audio.wav, producing the final MP4 at the target path. Progress is emitted as Tauri events that the frontend listens for:

useEffect(() => {
  const unlisten = listen<number>('export-progress', (event) => {
    setRenderPercent(event.payload);
  });
  return () => { unlisten.then(f => f()); };
}, []);

Vertical Flip Utility

WebGL’s coordinate origin is at the bottom-left, but image conventions (and video) use top-left. After reading pixels from the render target, flipVertical corrects this:

function flipVertical(buffer: Uint8Array, width: number, height: number): Uint8Array {
  const rowSize = width * 4;
  const result  = new Uint8Array(buffer.length);
  for (let y = 0; y < height; y++) {
    const srcRow = (height - 1 - y) * rowSize;
    const dstRow = y * rowSize;
    result.set(buffer.subarray(srcRow, srcRow + rowSize), dstRow);
  }
  return result;
}

Coordinate System

The Three.js camera is configured to map canvas pixel coordinates directly to world units, so no coordinate transformation is needed between the editor’s clip position values and the rendered output.

const fov  = 45;  // degrees
const zPos = (projectConfig.height / 2) / Math.tan((fov * Math.PI) / 360);

camera.position.set(projectConfig.width / 2, -projectConfig.height / 2, zPos);
camera.lookAt(projectConfig.width / 2, -projectConfig.height / 2, 0);

This places the camera so that:

Top-left corner of the canvas = world coordinates (0, 0, 0)
Bottom-right corner = world coordinates (width, -height, 0)
A clip with position = { x: 100, y: 200 } appears exactly 100 px from the left and 200 px from the top

The Y axis is negated because Three.js uses a right-handed coordinate system (Y up), but screen/video conventions use Y down. The camera compensates by being positioned at (W/2, -H/2, zPos) and looking at (W/2, -H/2, 0).

The vertical flip applied after readRenderTargetPixels() is separate from this Y-axis negation. The camera handles the visual coordinate mapping; the flip handles the raw pixel buffer’s reversed row order in WebGL’s memory layout.

Keyframe Interpolation

getInterpolatedValueWithFades is the core function for reading animated clip properties at any point in time. It is passed into the render engine context and called for every clip on every frame.

How it works

Default values: If a clip has no keyframes for a property, the default is returned (opacity: 1.0, volume: 0 dB, zoom: 1.0, position: {x:0, y:0}, speed: 1.0).
Boundary clamping: Times before the first keyframe return the first keyframe’s value; times after the last return the last keyframe’s value.
Linear interpolation (LERP): Between two adjacent keyframes, the value is linearly interpolated based on the normalized progress (t - t0) / (t1 - t0).
Object types: For position ({x, y}) and rotation3d ({rot, rot3d}), each component is interpolated independently.
Fade-in / Fade-out: Applied as a post-processing step. For opacity, fade uses a linear multiplier. For volume (in dB), the fade uses 20 * log10(fadeModifier) to achieve a perceptually natural fade curve. Silence threshold is -100 dB.

Speed Ramp Math

Speed keyframes allow each clip to have a variable playback rate over time — from 0.1× slow motion to 10× fast forward, with arbitrary curves drawn using many keyframe points.

Value Range Mapping

All speed keyframes store real speed values (e.g. 0.5 = half speed, 2.0 = double speed). The timeline UI uses a normalized 0–1 slider that maps to the 0.1×–10× range:

// 0→0.5 maps to 0.1×→1× (slow motion half)
// 0.5→1 maps to 1×→10× (fast forward half)
export const converterSpeed = (value: number): number => {
  const v = Math.max(0, Math.min(1, value));
  if (v <= 0.5) {
    return 0.1 + (v / 0.5) * (1.0 - 0.1);
  } else {
    return 1.0 + ((v - 0.5) / 0.5) * (10.0 - 1.0);
  }
};

Time Mapping Functions

Speed ramps require converting between two time domains: composition time (position on the timeline) and media time (position in the original footage). The relationship is:

mediaTime = ∫₀^compositionTime speed(t) dt

For linear interpolation between keyframes, this integral is computed using the trapezoid rule (exact for linear functions):

// For segment (t0, v0) → (t1, v1):
// area = (v_at_from + v_at_to) / 2 * span

Composition time → Media time (used to know which frame of the original footage to show):

export function compositionToMediaTime(
  compositionTime: number,
  speedKfs: SpeedKf[]
): number

Walks each speed segment and accumulates the area under the speed(t) curve using the trapezoid rule. Constant extrapolation is used before the first and after the last keyframe. Media time → Composition time (inverse, used to anchor non-speed keyframes):

export function mediaToCompositionTime(
  mediaTime: number,
  speedKfs: SpeedKf[]
): number

Uses a 64-iteration binary search on compositionToMediaTime (which is monotonically increasing) to find the inverse within 0.0001 s tolerance.

Keyframe Remapping After Speed Changes

When speed keyframes change, all other keyframe types (opacity, volume, zoom, position, rotation3d) must be remapped so they remain anchored to their original footage position rather than drifting on the composition timeline:

export function remapKeyframesToSpeed(
  keyframes: { opacity?: any[]; volume?: any[]; rotation3d?: any[];
               position?: any[]; zoom?: any[]; speed?: any[] },
  speedKfs: SpeedKf[]
): typeof keyframes

Each non-speed keyframe stores its originalTime (the media-domain time it was created at). After a speed change, remapKeyframesToSpeed uses mediaToCompositionTime(kf.originalTime, speedKfs) to recalculate the keyframe’s new composition-domain time, keeping the effect visually locked to the same moment in the footage.

Timeline Duration Recalculation

When speed keyframes are added, removed, or dragged, the clip’s visual duration on the timeline is recalculated:

const newDuration = mediaToCompositionTime(clip.originalduration, speedPoints);

This answers: “How much composition time does the original footage span at this speed curve?” A clip running at 2× throughout will appear half as wide on the timeline as the same clip at 1×.

Technical Reference

WannaCut Render Engine: Three.js Preview and Export Pipeline

renderBridge Interface

`RenderEngineContext`

`ExportOptions`

`getDrawFrameFunction()`

Preview Rendering

Preview Loop

`updatePreview(currentTime)`

`newDrawFrame(time?)`

Export Pipeline

Phase 1 — Video Frame Rendering

Phase 2 — Offline Audio Mixing

Phase 3 — FFmpeg Assembly

Vertical Flip Utility

Coordinate System

Keyframe Interpolation

How it works

Speed Ramp Math

Value Range Mapping

Time Mapping Functions

Keyframe Remapping After Speed Changes

Timeline Duration Recalculation

Build docs developers (and LLMs) love

Technical Reference

Documentation Index

​renderBridge Interface

​RenderEngineContext

​ExportOptions

​getDrawFrameFunction()

​Preview Rendering

​Preview Loop

​updatePreview(currentTime)

​newDrawFrame(time?)

​Export Pipeline

​Phase 1 — Video Frame Rendering

​Phase 2 — Offline Audio Mixing

​Phase 3 — FFmpeg Assembly

​Vertical Flip Utility

​Coordinate System

​Keyframe Interpolation

​How it works

​Speed Ramp Math

​Value Range Mapping

​Time Mapping Functions

​Keyframe Remapping After Speed Changes

​Timeline Duration Recalculation

Build docs developers (and LLMs) love

renderBridge Interface

`RenderEngineContext`

`ExportOptions`

`getDrawFrameFunction()`

Preview Rendering

Preview Loop

`updatePreview(currentTime)`

`newDrawFrame(time?)`

Export Pipeline

Phase 1 — Video Frame Rendering

Phase 2 — Offline Audio Mixing

Phase 3 — FFmpeg Assembly

Vertical Flip Utility

Coordinate System

Keyframe Interpolation

How it works

Speed Ramp Math

Value Range Mapping

Time Mapping Functions

Keyframe Remapping After Speed Changes

Timeline Duration Recalculation