Polling Pattern for Async Jobs

Auto-generate your docs

The CGIAR Risk Intelligence Tool uses an asynchronous job pattern for long-running AI operations like prompt previews, document parsing, gap detection, risk analysis, and report generation.

Architecture

The async processing flow works as follows:

Implementation Pattern

Step 1: Initiate Job

Call an endpoint that creates an async job (e.g. POST /api/admin/prompts/preview). The endpoint returns immediately with:

202 Accepted status
jobId (UUID) to poll
status: "PROCESSING"

const response = await fetch('https://api.example.com/api/admin/prompts/preview', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${token}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    systemPrompt: 'You are an expert agricultural risk analyst...',
    userPrompt: 'Identify gaps in market risk coverage.',
    variables: { category_1: 'Market Risk' }
  })
});

const { data } = await response.json();
// data = { jobId: '550e8400-e29b-41d4-a716-446655440000', status: 'PROCESSING' }

Step 2: Poll Job Status

Poll GET /api/jobs/:id every 3 seconds until status is COMPLETED or FAILED.

function pollJobStatus(jobId: string, token: string): Promise<Job> {
  return new Promise((resolve, reject) => {
    const interval = setInterval(async () => {
      try {
        const response = await fetch(`https://api.example.com/api/jobs/${jobId}`, {
          headers: { 'Authorization': `Bearer ${token}` }
        });
        
        const job = await response.json();
        
        if (job.status === 'COMPLETED') {
          clearInterval(interval);
          resolve(job);
        } else if (job.status === 'FAILED') {
          clearInterval(interval);
          reject(new Error(job.error || 'Job failed'));
        }
        // If PENDING or PROCESSING, continue polling
      } catch (error) {
        clearInterval(interval);
        reject(error);
      }
    }, 3000); // Poll every 3 seconds
  });
}

// Usage
try {
  const job = await pollJobStatus(data.jobId, token);
  console.log('Job completed:', job.result);
} catch (error) {
  console.error('Job failed:', error);
}

Step 3: Handle Results

Once the job status is COMPLETED, extract the result field:

if (job.status === 'COMPLETED') {
  // For AI_PREVIEW jobs:
  const aiOutput = job.result.output;
  
  // For PARSE_DOCUMENT jobs:
  const extractedText = job.result.text;
  const metadata = job.result.metadata;
  
  // For GAP_DETECTION jobs:
  const gaps = job.result.gaps;
  
  // For RISK_ANALYSIS jobs:
  const riskScores = job.result.scores;
  
  // For REPORT_GENERATION jobs:
  const reportUrl = job.result.reportUrl;
}

Retry Logic

Jobs automatically retry up to 3 times on failure:

attempts field tracks retry count
maxAttempts is set to 3 by default
After 3 failed attempts, status becomes FAILED

Timeout Recommendations

Client-Side Timeout

Implement a maximum polling duration to avoid infinite loops:

function pollJobStatus(
  jobId: string, 
  token: string,
  maxDuration: number = 120000 // 2 minutes
): Promise<Job> {
  return new Promise((resolve, reject) => {
    const startTime = Date.now();
    
    const interval = setInterval(async () => {
      if (Date.now() - startTime > maxDuration) {
        clearInterval(interval);
        reject(new Error('Polling timeout exceeded'));
        return;
      }
      
      try {
        const response = await fetch(`https://api.example.com/api/jobs/${jobId}`, {
          headers: { 'Authorization': `Bearer ${token}` }
        });
        
        const job = await response.json();
        
        if (job.status === 'COMPLETED') {
          clearInterval(interval);
          resolve(job);
        } else if (job.status === 'FAILED') {
          clearInterval(interval);
          reject(new Error(job.error || 'Job failed'));
        }
      } catch (error) {
        clearInterval(interval);
        reject(error);
      }
    }, 3000);
  });
}

Expected Durations

Job Type	Typical Duration
`AI_PREVIEW`	3-10 seconds
`PARSE_DOCUMENT`	5-30 seconds (depends on file size)
`GAP_DETECTION`	10-30 seconds
`RISK_ANALYSIS`	15-60 seconds (7 categories analyzed)
`REPORT_GENERATION`	20-90 seconds

Best Practices

1. Exponential Backoff (Optional)

For production use, consider exponential backoff if the job is taking longer than expected:

let pollInterval = 3000; // Start at 3 seconds
let attempt = 0;

const interval = setInterval(async () => {
  attempt++;
  
  // After 10 attempts (30 seconds), slow down polling
  if (attempt > 10) {
    clearInterval(interval);
    // Switch to slower polling
    pollInterval = 10000; // 10 seconds
  }
  
  // ... fetch job status
}, pollInterval);

2. User Feedback

Show progress indicators while polling:

const statuses = {
  PENDING: { message: 'Job queued...', icon: '⏳' },
  PROCESSING: { message: 'AI model processing...', icon: '🤖' },
  COMPLETED: { message: 'Complete!', icon: '✅' },
  FAILED: { message: 'Job failed', icon: '❌' }
};

// Update UI with current status
const { message, icon } = statuses[job.status];

3. Error Handling

Handle different failure scenarios:

if (job.status === 'FAILED') {
  if (job.error?.includes('ThrottlingException')) {
    // Bedrock rate limit hit — suggest retry later
  } else if (job.error?.includes('ValidationException')) {
    // Invalid input — don't retry
  } else {
    // Unknown error — allow manual retry
  }
}

4. Cancellation

Allow users to cancel polling (jobs continue running in background):

let pollController: AbortController | null = null;

function startPolling(jobId: string) {
  pollController = new AbortController();
  
  const interval = setInterval(async () => {
    if (pollController?.signal.aborted) {
      clearInterval(interval);
      return;
    }
    // ... fetch job status
  }, 3000);
}

function cancelPolling() {
  pollController?.abort();
}

React Hook Example

import { useState, useEffect } from 'react';

type Job = {
  id: string;
  status: 'PENDING' | 'PROCESSING' | 'COMPLETED' | 'FAILED';
  result?: any;
  error?: string;
};

export function useJobPolling(jobId: string | null) {
  const [job, setJob] = useState<Job | null>(null);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);
  
  useEffect(() => {
    if (!jobId) return;
    
    setLoading(true);
    setError(null);
    
    const interval = setInterval(async () => {
      try {
        const response = await fetch(`/api/jobs/${jobId}`);
        const data = await response.json();
        
        setJob(data);
        
        if (data.status === 'COMPLETED' || data.status === 'FAILED') {
          clearInterval(interval);
          setLoading(false);
          
          if (data.status === 'FAILED') {
            setError(data.error || 'Job failed');
          }
        }
      } catch (err) {
        clearInterval(interval);
        setLoading(false);
        setError('Failed to fetch job status');
      }
    }, 3000);
    
    return () => clearInterval(interval);
  }, [jobId]);
  
  return { job, loading, error };
}

// Usage in component
function MyComponent() {
  const [jobId, setJobId] = useState<string | null>(null);
  const { job, loading, error } = useJobPolling(jobId);
  
  const handlePreview = async () => {
    const response = await fetch('/api/admin/prompts/preview', {
      method: 'POST',
      body: JSON.stringify({ /* ... */ })
    });
    const { data } = await response.json();
    setJobId(data.jobId);
  };
  
  return (
    <div>
      <button onClick={handlePreview}>Preview Prompt</button>
      
      {loading && <p>Processing... ({job?.status})</p>}
      {error && <p className="error">{error}</p>}
      {job?.status === 'COMPLETED' && (
        <pre>{JSON.stringify(job.result, null, 2)}</pre>
      )}
    </div>
  );
}

Job Scoping

Jobs are scoped to the user who created them:

Job records include createdById field
GET /api/jobs/:id only returns jobs created by the authenticated user
Attempting to access another user’s job returns 404

Database Schema

Jobs are stored in the jobs table with the following fields:

model Job {
  id          String    @id @default(uuid())
  type        JobType   // AI_PREVIEW, PARSE_DOCUMENT, etc.
  status      JobStatus @default(PENDING)
  input       Json      // serialized request payload
  result      Json?     // output from worker
  error       String?   // error message if FAILED
  attempts    Int       @default(0)
  maxAttempts Int       @default(3)
  createdById String
  createdAt   DateTime  @default(now())
  startedAt   DateTime?
  completedAt DateTime?
  updatedAt   DateTime  @updatedAt
}

Authentication

Assessments

Jobs

Prompts

Admin

Architecture

Implementation Pattern

Step 1: Initiate Job

Step 2: Poll Job Status

Step 3: Handle Results

Retry Logic

Timeout Recommendations

Client-Side Timeout

Expected Durations

Best Practices

1. Exponential Backoff (Optional)

2. User Feedback

3. Error Handling

4. Cancellation

React Hook Example

Job Scoping

Database Schema

See Also

​Architecture

​Implementation Pattern

​Step 1: Initiate Job

​Step 2: Poll Job Status

​Step 3: Handle Results

​Retry Logic

​Timeout Recommendations

​Client-Side Timeout

​Expected Durations

​Best Practices

​1. Exponential Backoff (Optional)

​2. User Feedback

​3. Error Handling

​4. Cancellation

​React Hook Example

​Job Scoping

​Database Schema

​See Also

Architecture

Implementation Pattern

Step 1: Initiate Job

Step 2: Poll Job Status

Step 3: Handle Results

Retry Logic

Timeout Recommendations

Client-Side Timeout

Expected Durations

Best Practices

1. Exponential Backoff (Optional)

2. User Feedback

3. Error Handling

4. Cancellation

React Hook Example

Job Scoping

Database Schema

See Also