Realtime CSV Importer

Overview

This Next.js example project demonstrates how to build a CSV importer with live progress updates. When a user uploads a CSV file, a Trigger.dev background task downloads and parses it, then fans out row validation across multiple child tasks. Progress is streamed back to the UI in real time using Trigger.dev Realtime. Tech stack:

Next.js — frontend and API routes
UploadThing — handles CSV file uploads
Trigger.dev Realtime — streams progress updates back to the browser

GitHub repo

View the Realtime CSV Importer repo

Fork this repo to use it as a starting point for your own CSV import feature.

Demo video

How it works

Task architecture

The processing logic is split across two tasks: csvValidator (parent task)

Receives the uploaded file URL
Downloads and parses the CSV
Splits rows into batches
Uses batch.triggerAndWait to dispatch all rows to the child task in parallel
Updates its own run metadata with overall progress as child tasks complete

handleCSVRow (child task)

Receives a single CSV row
Validates the email address (simulated in this example)
Reports progress back to the parent using metadata.parent

src/trigger/csv.ts

import { task, batch, metadata } from "@trigger.dev/sdk";
import Papa from "papaparse";

export const handleCSVRow = task({
  id: "handle-csv-row",
  run: async (payload: { row: Record<string, string>; index: number; total: number }) => {
    // Simulate email validation
    const email = payload.row["email"] ?? "";
    const isValid = /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email);

    // Update parent task progress
    metadata.parent.set("progress", {
      processed: payload.index + 1,
      total: payload.total,
    });

    return { row: payload.row, valid: isValid };
  },
});

export const csvValidator = task({
  id: "csv-validator",
  run: async (payload: { fileUrl: string }) => {
    // Download the file
    const response = await fetch(payload.fileUrl);
    const csvText = await response.text();

    // Parse the CSV
    const { data } = Papa.parse<Record<string, string>>(csvText, { header: true });

    metadata.set("status", { progress: 0, total: data.length, label: "Starting..." });

    // Fan out row processing
    const results = await batch.triggerAndWait(
      data.map((row, index) =>
        handleCSVRow.trigger({ row, index, total: data.length })
      )
    );

    const validRows = results.runs.filter((r) => r.ok && r.output.valid).length;

    return { total: data.length, valid: validRows, invalid: data.length - validRows };
  },
});

Streaming progress to the frontend

The useRealtimeRun hook subscribes to the parent task’s run and exposes its metadata, which includes the live progress values:

src/hooks/useRealtimeCSVValidator.ts

import { useRealtimeRun } from "@trigger.dev/react-hooks";
import { csvValidator } from "../trigger/csv";

export function useRealtimeCSVValidator(runId: string, accessToken: string) {
  const { run } = useRealtimeRun<typeof csvValidator>(runId, { accessToken });

  const progress = run?.metadata?.progress ?? 0;
  const total = run?.metadata?.total ?? 0;
  const label = run?.metadata?.label ?? "";

  return { run, progress, total, label };
}

The CSVProcessor component renders a progress bar driven by this hook:

src/components/CSVProcessor.tsx

import { useRealtimeCSVValidator } from "../hooks/useRealtimeCSVValidator";

export function CSVProcessor({ runId, accessToken }: { runId: string; accessToken: string }) {
  const { progress, total, label, run } = useRealtimeCSVValidator(runId, accessToken);

  const percentage = total > 0 ? Math.round((progress / total) * 100) : 0;

  return (
    <div>
      <p>{label}</p>
      <div className="w-full bg-gray-200 rounded-full h-2.5">
        <div
          className="bg-blue-600 h-2.5 rounded-full transition-all"
          style={{ width: `${percentage}%` }}
        />
      </div>
      <p>{percentage}% — {progress} of {total} rows processed</p>
      {run?.status === "COMPLETED" && (
        <p>Done! {run.output?.valid} valid rows, {run.output?.invalid} invalid.</p>
      )}
    </div>
  );
}

Key concepts used

Concept	How it’s used
`batch.triggerAndWait`	Fan out row processing to child tasks and wait for all to complete
`metadata.set`	Store progress state on the run so the frontend can read it
`metadata.parent`	Allow child tasks to update the parent task’s metadata
`useRealtimeRun`	Subscribe to the run from the frontend and receive live updates

Learn more

Trigger.dev Realtime

Learn how to subscribe to runs and stream data to your frontend

React hooks

The full reference for Trigger.dev’s React hooks

Run metadata

Store and update arbitrary data on a run, readable from anywhere

Batch triggering

Trigger multiple tasks at once and wait for all results

Guides

Example Projects

Example Tasks

Realtime CSV Importer

Overview

GitHub repo

View the Realtime CSV Importer repo

Demo video

How it works

Task architecture

Streaming progress to the frontend

Key concepts used

Learn more

Trigger.dev Realtime

React hooks

Run metadata

Batch triggering

​Overview

​GitHub repo

View the Realtime CSV Importer repo

​Demo video

​How it works

​Task architecture

​Streaming progress to the frontend

​Key concepts used

​Learn more

Trigger.dev Realtime

React hooks

Run metadata

Batch triggering

Overview

GitHub repo

Demo video

How it works

Task architecture

Streaming progress to the frontend

Key concepts used

Learn more